Should humanity give birth to a galactic civilization?

Do not attempt a literal interpretation, rather try to consider the gist of the matter, if possible.

I have a better idea. Please write posts such that they can be interpreted literally, so the gist follows naturally from the literal reading.

Let's reach the stars first and worry later about how many zillions of years of fun remain. If we eventually run out, then we can abort or wirehead. For now, it seems like the expected awesome of creating an intergalactic posthuman civilization is pretty high.

Even if we create unimaginably many posthumans having unimaginable posthuman fun, and then get into some bitter resource struggles as we approach the heat death of the universe, I think it will have been worth it.

This post is centered on a false dichotomy, to address its biggest flaw in reasoning. If we're at time t=0, and widespread misery occurs at time t=10^10, then solutions other than "Discontinue reproducing at t=0" exist. Practical concerns aside - as without practical concerns aside, there is no point in even talking about this - the appropriate solution would be to end reproduction at, say, t=10^9.6. This post arbitrarily says "Act now, or never" when, practically, we can't really act now, so any later time is equally feasible and otherwise simply better.

Most of these questions are beyond our current ability to answer. We can speculate and counter-speculate, but we don't know. The immediate barrier to understanding is that we do not know what pleasure and pain, happiness and suffering are, the way that we think we know what a star or a galaxy is.

We have a concept of matter. We have a concept of computation. We have a concept of goal-directed computation. So we can imagine a galaxy of machines, acting according to shared or conflicting utility functions, and constrained by competition and the death of the universe. But we do not know how that would or could feel; we don't even know that it needs to feel like anything at all. If we imagine the galaxy populated with people, that raises another problem - the possibility of the known range of human experience, including its worst dimensions, being realized many times over. That is a conundrum in itself. But the biggest unknown concerns the forms of experience, and the quality of life, of "godlike AIs" and other such hypothetical entities.

The present reality of the world is that humanity is reaching out for technological power in a thousand ways and in a thousand places. That is the reality that will issue either in catastrophe or in superintelligence. The idea of simply halting that process through cautionary persuasion is futile. To actually stop it, and not just slow it down, would require force. So I think the most constructive attitude towards these doubts about the further future is to see them as input to the process which will create superintelligence. If this superintelligence acts with even an approximation of humaneness, it will be sensitive to such issues, and if it really does embody something like the extrapolated volition of humanity, it will resolve them as we would wish to see them resolved.

Therefore, I propose that your title question - "Should humanity give birth to a galactic civilization?" - should be regarded as a benchmark of progress towards an exact concept of friendliness. A friendly AI should be able to answer that question, and explain its answer; and a formal strategy for friendly AI should be able to explain how its end product - the AI itself - would be capable of answering the question.

Updated the post to suit criticism.

Let's see, if I can't write a good post maybe I can tweak one to become good based on feedback.

These are very important questions which deserve to be addressed, and I hope this post isn't downvoted severely. However, at least one subset of them has been addressed already:

Further, although less dramatic, what if we succeed to transcendent, to become posthuman and find out that the universe does not contain enough fun for entities with mental attributes far exceeding those of baseline humanity? What if there isn't even enough fun for normal human beings to live up until an age of 150 and still have fun?

See the Fun Theory Sequence for discussion.

It is probably premature to ask such questions now. We have no idea how the world will look like in 10^20 years. And when I write no idea, I don't mean we have several theories from which we have to choose the right one, but still can't do that. I mean that (if human race doesn't get extinct soon and future will not turn out to be boringly same as the present or the recent history) we can't possibly imagine how would the world function, and even if told, we wouldn't understand. If there will be intelligent creatures in 10^20 years, they will certainly have emotions we don't possess, thoughts we can't fathom, values we would call perverse, if even the description in the language of emotions and values would make sense in that world.

Why should we care about the world we don't understand one bit? Trying to answer questions about such distant future puts us in the situaton of a Homo erectus evaluating the risks of inventing fire. Do we imagine any of ideas that a Homo erectus could invent would be even marginally valuable for us today? And given that we are no more than several hundert thousand years younger, flatworm would perhaps be a more fitting analogy than Homo erectus.

Caterpillars discussing the wisdom of flight.

We are looking at very long time scales here, so how wide should our scope be? If we use a very wide scope like this, we get issues, but if we widen it still further we might get even more. Suppose the extent of reality were unlimited, and that the scope of effect of an individual action were unlimited, so that if you do something it affects something, which affects something else, which affects something else, and so on, without limit. This doesn't necessarily need infinite time: We might imagine various cosmologies where the scope could be widened in other ways. Where would that leave the ethical value of any action we commit?

I will give an analogy, which we can call "Almond's Puppies" (That's a terrible name really, but it is too late now.)

Suppose we are standing at the end of two lines of boxes. Each line continues without end, and each box contains a puppy - so each line contains an infinity of puppies. You can choose to press a button to blow up the first box or another button to spare it. After you press the button, some mechanism, that you can't predict, will decide to blow up the second box or spare it, based on your decision, and then it will decide to blow up the third box or spare it, based on your decision, and so on. So you press that button, and either the first box is blown up or spared, and then boxes get blown up or spared right along the line, with no end to it.

You have to press a button to start one line off. You choose to press the button to spare the first puppy. Someone else chooses to press the button to blow up the first puppy. The issue now is: Did the other person do a bad thing? If so, why? Did he kill more puppies than you? Does the fact that he was nicer to the nearby puppies matter? Does it matter that the progress of the wave of puppy explosions along the line of boxes will take time, and at any instant of time, only a finite number of puppies will have been blown up, even though there is no end to it in the future?

If we are looking at distant future scenarios, we might ask if we are sure that reality is limited.

If this was the case, a true FAI will just kill us more painlessly than we could. And then go out and stop life evolved in other places from causing such suffering.

But this is nothing compared to the time from 10^20 to 10^100 years where possibly trillions of God-like entities will be slowly disabled due to a increasing lack of resources. This is comparable to suffering from Alzheimer's, just much worse, much longer and without any hope.

A different (more likely?) scenario is that the god-like entities will not gradually decline their resource usage--they'll store up energy reserves, then burn through them as efficiently as possible, then shut down. It will be really sad each time a god-like entity dies, but not necessarily painful.

Actually, if evolutionary pressures continue (i.e. no singleton) it seems fairly likely that usable resources will collapse suddenly, and resource starvation is relatively brief. Right now, we have an energy diet from the sun--it only releases so much energy at once. But future entities may try to break up stars to use their energy more efficiently (solar fusion is highly inefficient compared to possible levels).

Thanks for posting. Upvoted.

I have always had an uncomfortable feeling whenever I have been asked to include distant-future generations in my utilitarian moral considerations. Intuitively, I draw on my background in economics, and tell myself that the far-distant future should be discounted toward zero weight. But how do I justify the discounting morally? Let me try to sketch an argument.

I will claim that my primary moral responsibility is to the people around me. I also have a lesser responsibility to the next generation, and a responsibility lesser yet to the generation after that, and so on. A steep discount rate - 30% per generation or so. I will do my duty to the next generation, but in turn I expect the next generation to do its duty to the generation after that. After all, the next generation is in a far better position than me to forsee what problems the generation after that really faces. Their efforts will be much less likely than mine to be counterproductive.

If I were to spread my concern over too many generations, I would be shortchanging the next generation of their fair share of my concern. Far-future generations have plenty of predecessor generations to worry about their welfare. The next generation has only us. We mustn't shortchange them!

This argument is just a sketch, of course. I just invented it today. Feedback is welcome.

Is anyone going to propose this as an answer to (what some say is) the Fermi paradox?

what if it is more likely that maximizing utility not only fails but rather it turns out that the overall utility is minimized

If it does turn out that the overwhelmingly likely future is one of extreme negative utility, voluntary extinction (given some set of assumptions) IS maximizing utility.

Also, if the example really is as tangental as you're implying, it should probably not account for 95% of the text (and the title, and the links) in your post.

I cannot fathom the confusion that would lead to this question. Of course it's better for humanity to survive than to not survive. Of course it's better to go extinct in a million years than to go extinct now. The future is more wondrous and less scary than you imagine.

I think freedom should win when contemplating how to ethically shape the future. I don't have any direct evidence that posthumans in a post-Singularity universe will be "happy" throughout their lives in a way that we value, but you certainly don't have evidence to the contrary, either.

As long as neither of us know the exact outcome, I think the sensible thing to do is to maximize freedom, by trying to change technology and culture to unburden us and make us more capable. Then the future can decide for itself, instead of relying on you or I to worry about these things.

Also, how many people do you know who could honestly claim, "I wish I had never been born?" Although there are certainly some, I don't think there are very many, and life down here on Earth isn't even that great.

I added to the post:

To exemplify this let's assume there were 100 entities. At a certain point the universe will cease to provide enough resources to sustain 100 entities. So either the ruling FAI (friendly AI) is going to kill one entity or reduce the mental capabilities of all 100. This will continue until all of them are either killed or reduced to a shadow of their former self. This is a horrible process that will take a long time. I think you could call this torture until the end of the universe.

What if there isn't even enough fun for normal human beings to live up until an age of 150 and still have fun?

Really?

my intention is to inquire about the perception that it is our moral responsibility to minimize suffering

It's okay for us to cause more entities to exist, for a greater sum of suffering, provided that it's one of the better possible outcomes.

While intervening in a way that (with or without) consent inflicts a certain amount of additional net suffering on others (such as causing them to be created) is to be avoided all other things equal, it's justifiable if the net fun is increased by some multiple of the suffering (the requisite multiple depends on consent, on who gets the fun, i.e. if you're gaining fun by torturing another, the multiple may have to be huge).

I agree that we should consider the possibility of suffering. Suicide (by radical modification into something that is not suffering, or actual termination) seems like an easy solution.

I imagine some "artist" eventually creating a creature that is sentient, feels great pain, and eloquently insists that it does not want to be changed, or ended. Sick bastard. Or perhaps it would merely be programmed to elaborately fake great pain, to others' discomfort, while secretly reveling in it. I imagine technology would be able to tell the difference.

I added to the post. Please read the To clarify addendum. Thank you.

This is why Buddhism is dangerous.

What do y'all think about John Smart's thesis that an inward turn is more likely that the traditional script galactic colonization?

http://www.accelerating.org/articles/answeringfermiparadox.html

Rather wild read, but perhaps worth a thought. Would that alternative trajectory affect your opinion of the prospect, XiXiDu?

Interesting thoughts. I also haven't finished the fun sequence, so this may be malformed. The way I see it is this: You can explore and modify your environment for fun and profit (socializing counts here too), and you can modify your goals to get more fun and profit without changing your knowledge.

Future minds may simply have a "wirehead suicide contingency" they choose to abide by, by which, upon very, very strong evidence that they can have no more fun with their current goals, they could simply wirehead themselves. Plan it so that the value of just being alive and experiencing the slow end of the world goes up as other sources of fun diminish. (And leave in there a huge reward for discovering that you are wrong, just not motive to seek it out irrationally).

You would need a threshold of probability that life is going to suck forever from here on out, only after which the contingency was initiated.

Relevant literature: "Should This Be the Last Generation?" By PETER SINGER

• http://opinionator.blogs.nytimes.com/2010/06/06/should-this-be-the-last-generation/

I'd like to ask those people who downvote this post for their reasons. I thought this is a reasonable antiprediction to the claims made regarding the value of a future galactic civilisation. Based on economic and scientific evidence it is reasonable to assume that the better part of the future, namely the the time from 10^20 to 10^100 years (and beyond) will be undesirable.

If you spend money and resources on the altruistic effort of trying to give birth to this imaginative galactic civilisation, why don't you take into account the more distant and much larger part of the future that lacks any resources to sustain given civilisation? You are deliberately causing suffering here by putting short-term interests over those of the bigger part of the future.

First a few minor things I would like to get out there:

We are according to consensus which I do not dispute (since its well founded) slowly approaching heat-death. If I recall correctly we are supposed to approach maxentrhopy asymptotically. Can we with our current knowledge completley rule out the possibility of some kind computation machinery existing and waking up every now and then (at longer and longer intervals) in the wasteland universe to churn a few cycles of a simulated universe?

I don't quite see the difference between real and simulated torture in the context of a civilization as advanced as the one you are arguing against we let develop. So I'm not sure you are getting at by mentioning them as separate things.

You need to read up on fun theory. And if you disregard it, let me just point out that worrying about people not having fun is different concern than from assuming they will experience mental anguish at the prospect of suicide or a inevitable death. Actually not having fun can be neatly solved by suicide if you exhaust all other options as long as you aren't built to find it stressful committing to it.

Now assuming your overall argument has merit: My value function says its better to have loved and lost than not to have loved at all.

Humans may have radically different values once they are blown up to scale. Unless you get your finger first into the first AI's values, there will always be a nonzero fraction of agents who would wish to carry on even knowing it will increase total suffering, because they feel their values are worth suffering for. I am basically talking about practicality now: So what if you are right? The only way to do anything about it is to make sure your AI eliminates anything be it human or alien AI that can paperclip anything like beings capable of suffering. To do this in the long run (not just kill or sterilize all humans which is easy) properly you need to understand friendliness much better than we do now.

If you want to learn about friendliness you better try and learn to deceive agents with whom you might be able to work together to figure out more about it, especially concerning your motives. ;)

On balance I'm not too happy with the history of existence. As Douglas Adams wrote, "In the beginning the Universe was created. This has made a lot of people very angry and has been widely regarded as a bad move." I'd rather not be here myself, so I find the creation of other sentients a morally questionable act. On the other hand, artificial intelligence offers a theoretical way out of this mess. Worries about ennui strike me as deeply misguided. Oppression, frailty, and stupidity makes hanging out in this world unpleasant, not any lack of worthwhile pursuits. Believe me, I could kill a few millennia no problem. If Kurzweil's dreams of abundance (in every sense) come true, I won't be complaining.

Now, the notion of a negative but nonfatal Singularity deserves consideration. The way I typically see things, there's either death or Singularity in the long run and both are good. Indefinite life extension without revolutionary economic and social change would be a nightmare, though perhaps better at every individual point than the pain of aging.

Your concerns about the ultimate fate of the universe are intriguing but too distant to arouse much emotion from me. Who knows what will happen then? Such entities might travel to other universes or forge their own. I'll just say that judging by the present record, intelligence and suffering go together. Whether we can escape this remains to be seen.