Open Thread, February 15-29, 2012

It's not prescribed in Finland without a special permit from the authorities, and I don't want to take the risk of trying to obtain something that's considered an illegal drug.

I do use an alarm clock, but after going to bed at the right time for a couple of evenings, I start to wake up on my own, a little before the clock would sound. The alarm clock is just there as a backup, and to let me remain mostly-awake in bed for about 10-20 minutes longer before telling me to actually get up (as opposed to just getting awake).

ETA: I should specify that if I don't go to bed at the right time, I don't wake up naturally - well, I do, but so late that I'll feel groggy and generally inenergetic.

Suppose old person and child (perhaps better: young adult) would both gain 2 years, so we equalize payoff. What then? Why not be prioritarian at the margin of aggregate indifference?

You lose me the moment you introduce the moral premise. Why is it better for two people to each live a million years than one to live two million? This looks superficially the same sort of question as "Why is it better for two people to each have a million dollars than for one to have two million?", but in the latter scenario, one person has two million while the other has nothing. In the lifetimes case, there is no other person. The moral premise presupposes that nonexistent people deserve some of other peoples' existence in the same way that existing paupers deserve some of other peoples' wealth.

You may have an argument to that effect, but I didn't see it in my speed-run through your slides (nice graphic style, BTW, how do you do that?) or in your comment above. Your argument that we place value on future people only considers our desire to avoid calamities falling upon existent future people.

Diminishing returns for longer lifespans is only a problem to be tackled if it happens. The only diminishing returns I see around me for the lifespans we have result from decline in health, not excess of experience.

I find this argument incoherent, as I reject the idea of a person at the age of 1 being the same person as they are at the age of 800 - or for that manner, the idea of a person at the age of 400 being the same person as they are at the age of 401. In fact, I reject the idea of personal continuity in the first place, at least when looking at "fairness" at such an abstract level. I am not the same person as I was a minute ago, and indeed there are no persons at all, only experience-moments. Therefore there's no inherent difference in whether someone lives 800 years or ten people live 80 years. Both have 800 years worth of experience-moments.

I do recognize that "fairness" is still a useful abstraction on a societal level, as humans will experience feelings of resentment towards conditions which they perceive as unfair, as inequal outcomes are often associated with lower overall utility, and so forth. But even then, "fairness" is still just a theoretical fiction that's useful for maximizing utility, not something that would have actual moral relevance by itself.

As for the diminishing marginal returns argument, it seems inapplicable. If we're talking about the utility of a life (or a life-year), then the relevant variable would probably be something like happiness, but research on the topic has found age to be unrelated to happiness (see e.g. here), so each year seems to produce roughly the same amount of utility. Thus the marginal returns do not diminish.

Actually, that's only true if we ignore the resources needed to support a person. Childhood and old age are the two periods where people don't manage on their own, and need to be cared for by others. Thus, on a (utility)/(resources invested) basis, childhood and old age produce lower returns. Now life extension would eliminate age-related decline in health, so old people would cease to require more resources. And if people had fewer children, we'd need to invest fewer resources on them as well. So with life extension the marginal returns would be higher than with no life extension. Not only would the average life-year be as good as in the case with no life extension, we could support a larger population, so there would be many more life-years.

One could also make the argument that even if life extension wouldn't reduce the average amount of resources we'd need to support a person, it would still lead to increased population growth. Global trends currently show declining population growth all over the world. Developed countries will be the first ones to have their population drastically reduced (Japan's population began to decrease in 2005), but current projections seem to estimate that the developing world will follow eventually. Sans life extension, the future could easily be one of small populations and small families. With life extension, the future could still be one of small families, but it could be one of much larger populations as population growth would continue regardless. Instead of a planetary population of one billion people living to 80 each, we might have a planetary population of one hundred billion people living to 800 each. That would be no worse than no life extension on the fairness criteria, and much better on the experience-moments criteria.

I appreciated the level of thought you put into the argument, even though it does not actually convince me to oppose life extension. Thank you for writing (and prezi-ing) it, I look forward to more.

Basically, the hidden difference if you put me and 9 others behind a veil of ignorance and ask us to decide whether we each get 80 or one gets 800, is that in that case you have the presence of 10 people competing and trying to avoid being "killed" whereas in the choice between creating one 800 year old versus 10 80 year olds is conducted without an actual threat being posed to anyone.

While you can establish that the 10 people would anticipate with fear (and hence generate disutility) the prospect of being destroyed / prevented to live, that's not the same as establishing that 9 completely nonexistent people would generate the same disutility even if they never started to exist.

The main argument is that taking years from potential beings and adding them to existing ones is unjust, hence immoral. Given that, depending on the exact shape of the infinite universes scenario, life extension could be moral, amoral, or immoral.

If longer-lived people can reproduce and find new space more quickly than shorter lived people, life extension would be moral. (For example say more experienced people have more motive or ability to create new universes.) However all else being equal (for example, say the limit on reproduction is some unchangeable physical constant that says we cannot make black holes any faster than x, and we have already maxed that out), the fact that shorter lived people are dying and creating spaces for more kids makes that the more moral scenario.

While I agree that this is a flaw in the argument (longer lives can possibly result in more new kids born / new spaces opened than shorter ones), I don't think it is my true rejection of the argument overall, because it is not unreasonable to think the new spaces that can be opened is limited and/or cannot be increased by longer lives. I think the real problem is the idea that one can behave unjustly to a person whose existence is only potential, through the act of taking away their existence.

To me, the entire argument sounds like a rationalization for not signing up for cryo.

Signed,

Someone who has rationalized a reason for not signing up yet for cryo, and suspects that the real reason is laziness.

If infinite person years, then (so long as life is net positive) we have infinite utility, and I can't see obviously whether doling this out to a 'smaller' or 'larger' set of people (although both will have same cardinality) will matter. But anyway, I don't think anyone really thinks we can wring infinite amounts of life out of the universe.

Total life-time will have some upper bound. So in worlds where we are efficiently filling up lifespan, the choice is between more short-lived people or fewer long-lived people. In the real world for the foreseeable future, that won't quite apply - plausibly, there will be chunks of lifetime that can only be got at by extending your life, and couldn't be had by a future person, so you doing so doesn't deprive anyone else. However, that ain't plausible for an entire society (or a large enough group) extending their lives. Limiting case: if everyone made themselves immortal, they could only add people by increases in carrying capacity.

The most annoying part ist that I am unable to judge if it was a good or bad idea I forgot.

Anecdote: Discussing this with a particularly non-rational acquaintance, they remarked that I'm likely subconsciously discarding horrible ideas, and preventing myself from coming up with them again, and therefore the better for it.

But even if we're just worried about mugging, how could you possibly weight it enough? Even if paying once doomed me to spend the rest of my life paying $5 to muggers, the utility calculation still works out the same way.

My idea is as follows:

Mugger: Give me 5 dollars, or I'll torture 3^^^3 sentient people across the omniverse using my undetectable magical powers.
AI: If I make my decision on this and similar trades based on a decision process DP0 of comparing the disutility(3^^^3 torture) P(you're telling the truth) compared to the disutility(giving you 5 dollars), then even if you're telling the truth, a different malicious agent may then merely name a threat that involves 3^^^^3 tortures, and thus make me cause a vastly great amount of disutility in his service. Indeed there's no upper bound to the disutility such a hypothetical agent may claim will cause, and therefore surrendering to such demands mean a likewise unbounded exploitation potential. Therefore I will not* use the decision process DP0, and will instead utilize some different decision process (like "Never surrender to blackmail" or "Always demand proportional evidence before considering sufficiently extraordinary claims").

I agree with your first paragraph, but I'm not convinced of your second paragraph... at least, if you intend it as a rhetorical way of asserting that there is no possible way to weight the evidence properly. It's just another proposition; there's evidence for and against it.

I think we get confused here because we start with our bottom line already written.

I "know" that the EV of destroying my light cone is negative. But theory seems to indicate that, when assigning a confidence interval P1 to the statement "Destroying my future light cone will preserve 3^^^3 extra-universal people" (hereafter, statement S1), a well-calibrated inference engine might assign P1 such that the EV of destroying my light cone is positive. So I become anxious, and I try to alter the theory so that the resulting P1s are aligned with my pre-existing "knowledge" that the EV of destroying my light cone is negative.

Ultimately, I have to ask what I trust more: the "knowledge" produced by the poorly calibrated inference engine that is my brain, or the "knowledge" produced by the well-calibrated inference engine I built? If I trust the inference engine, then I should trust the inference engine.

Very true, but editing old PMs dosen't do that. Send a PM to yourself and then deleting it seems the most expedient solution. Thanks for the links however!

The last one, sort of. It usually takes the form of, "You don't want THAT to be in your dream log, do you? You'd better skip it just this once. It's okay, you'll write down the next one. That dream sucked anyway, and you're already forgetting it besides. Also don't you have better things to do?"

All, of course, with the low-level realization that I know all of this is bullshit but I obey it anyway.

Counterpoint: 48 Laws reads like cheap astrology.

"Why in the world would anyone [X]?" comes off as starting with a strong opinion that [X] is a bad idea, rather than actually asking for information about motives.

The high cost of divorce can make a lifetime commitment more robust. It also helps with taxes, visas and health care.

It seems nicer than eloping.

"Why wait?" is a perfectly reasonable question, but simply answering it "let's not!" probably doesn't yield the best expected value. (It might work perfectly fine. It'd probably work perfectly fine. But it seems likely to be slightly less conducive to everything being perfectly fine than some better-calibrated choice of timing.)

Try to avoid living together before marriage.

That seems like really dangerous advice to me. The article confirms my suspicion:

"We think that some couples who move in together without a clear commitment to marriage may wind up sliding into marriage partly because they are already cohabiting," Rhoades says. "It seems wise to talk about commitment and what living together might mean for the future of the relationship before moving in together, especially because cohabiting likely makes it harder to break up compared to dating."

The solution is not to avoid living together before marriage; the solution is to break up when you know you should.

Try to avoid living together before marriage..

Too late.

If I knew this relationship didn't have long-term potential, would I break it off?

I'm not sure which answer points to "Engage" here! I would guess "yes", since it allows you to reason "...and I'm still around, which means I believe it has long-term potential, which means we should get engaged". But "no" indicates attachment to the person and a willingness to make the relationship work even if it's rocky.

Received. Thanks a lot!

Indeed. There are no ideas of justice on exhibit other than human ones, so calling them "puny" seems like merely saying nasty things about reality.

What kind of experiments?

Mostly psychology. I'm particularly interested in experiments that would have political implications.

Why do you want to know about them in advance?

Because I want to be able to look at them and decide what kind of results would support a theory verses undermine it, before I (and the world) becomes biased by the actual results.

The ideas coming into your awareness are very strongly pre-filtered; creativity is far from random noise.

I agree but I don't think that changes my conclusions. In teaching humans to be more creative, they are taught to pay more attention for a longer time to at least some of the outlier ideas. Indeed, a lot of times I think the difference between the intellectually curious and creative people I like to interact with and the rest is that the rest have predecided a lot of things, turned their thresholds for "unreal" ideas coming in to consciousness up higher than I have turned mine. Maybe they are right more often than I am, but the real measure of why they do this is that their ancestors who outsurvived a lot of other people trying a lot of other things did that same level of filtering, and it resulted in winjning more wars, having more children that survived, killing more competitors, or some combination of these and other results that constitute selection pressures.

An AI in the process of FOOMing, which necessarily has the capacity to consider a lot more ideas in a lot more detail than we do, what makes you think that AI will constrain itself by the values it used to have? Unless you think we have the same values as the first self-replicating molecules that began life on earth, the FOOMing of Natural Intelligence (which has taken billions of years) has been accompanied by value changes.

The AI is not supposed to change it values, regardless of whether it is powerful enough to realize them. Values are not up for grabs. Once the AI has some values it either wins and reshapes reality according to them or loses.

A remarkably strong claim.

My initial reaction is that humanity's values have certainly changed over time. I think it would require some rather unattractive mental gymnastics to claim that people who beat their children for their own good and people who owned slaves and people who beat, killed, and/or raped either slaves or other people they had vanquished as their right "really" had the same values we currently have, but just hadn't really thought them through, or that our values applied in their world would have lead us to similar beliefs about right and wrong.

I had even thought my own values had changed over my lifetime. I'm not as sure of that, but what about that?

Certainly, it seems, as the human species has evolved its values have changed. Do chimpanzees and bonobos have different values than we do, or the same? If the same, I'd love to see your mental gymnastics to justify that, I would expect them to be ugly. If different, does this mean that our common ancestor has necessarily "lost," assuming its values were some intermediate between ours, chimps, and bonobos, and all of its descendants have different values than it had?

As I understand the word values, our values have changed over time, different groups of humans have some different values from each other, and if there is a "kernel" of common values in our species, that this kernel most likely differs from the kernel of values in homo neanderthalis or other sentient predecessors of modern homo sapiens.

So if NI (Natural Intelligence) in its evolution can change values (can it?) with generally broad consensus that "we" have not lost in this process, why would an AI be precluded from futzing with its values as it worked on self-modifying to increase its intelligence?

It counts as failure to preserve humanity. An AI that does that is probably unfriendly (barring the coercion by external powerful agents. Eliezer actually wrote a story about such scenario, without AIs though.)

Interesting. I think I may even agree with you. In that story each race would need to conclude that the other races are "unfriendly". So Eliezer has written a story in which all the NATURAL intelligences (except us of course) are "unfriendly," and in which a human would need to agree that from the point of view of the other intelligent races, human intelligence was "unfriendly."

Perhaps all intelligences are necessarily "unfriendly" to all other intelligences. This could even apply at the micro level, perhaps each human intelligence is "unfriendly" to all other human intelligences. This actually looks pretty real and pretty much like what happens in a world where survival is the only enforced value. Humans have the fascinating conundrum that even though we are unfriendly to the other humans, we have a much better chance of surviving and thriving by working with the other humans. The alliances and technical abilities and so on are, if not balance across all humans and all groups, at least balanced enough across many of them so that the result is a plethora of competing / cooperating intelligences where the jury is still out on who is the ultimate winner. Breeding in to us the ability (the value?) that "others" are our allies against "the enemies" clearly has resulted in collective efforts of cooperation that have produced quickly cascading production ability in our species. "We" worried about the Nazis FOOMing and winning, we worried the Soviets might FOOM and win. Our ancestors fought against every tribe that lived 5 miles away from them, before cultural evolution allowed them (us) to cooperate in groups of hundreds of millions.

So in Eliezer's story, 3 NI's have FOOMed and then finally run into each other. And they CANNOT resist getting up in each other's grills. And why not? what are the chances that the final intelligence IF only one is left will have been one which was shy about destroying potential competitors before they destroyed it?