Given a non-trivial population to start with, it will be possible to find people that will consent to copying given absolutely minimal (quite possibly none at all) assurances for what happens to their copy. The obvious cases would be egoists that have personal value systems that make them not identify with such copies; you could probably already find many of those today.

In the resulting low-wage environment, it will likewise be possible to find people who will consent to extensive modification/experimentation of their minds given minimal assurances for what happens afterwards (something on the order of "we guarantee you will not be left in abject pain" will likely suffice) if the alternative is starvation. Given this, why you do believe the idea of selection for donation-eagerness to be fanciful?

It seems like the benefit from this would reach some finite ceiling, ie. you get a constant benefit from not having to deal with these negative things and there's only so much pleasure you can get from luxurious surroundings.

Their physical appearance and surroundings would be what we'd see as very luxurious.

Only to the extent that this does not distract them from work. To the extent that it does, ems that care about such things would be outcompeted (out of existence, given a sufficiently competitive economy) by ones that are completely indifferent to them, and focus all their mental capacity on their job.

Hi Robin

What is significantly different between poor people and slaves? The poor have little means of travel, they must work for others often doing stuff they hate doing, just to get enough to survive. In many historical societies slaves often had better conditions and housing than many of the poor today.

How would you get security in such a system? How would anyone of wealth feel safe amongst those at the bottom of the distribution curve?

The sense of injustice is strong in humans - one of those secondary stabilising strategies that empower cooperation.

It is actually relatively easy to automate all the jobs that no-one wants to do, so that people only do what they want to do. In such a world, there is no need of money or markets.

There are actually of lot of geeks like me who love to automate processes (including the process of automation).

Market based thinking was a powerful tool in times of genuine scarcity. Now that we have the power to deliver universal abundance, market based thinking is the single greatest impediment to the delivery of universal security and universal abundance.

Because the "doing better" history is written by the victors. It's our values that are being used to judge the improvement. Further evolutionary change, if left to the same blind idiot god, is highly likely to leave our descendants with changed - and worse - values. So long as the value drift is slight and the competence keeps increasing, our descendants will live better lives. But if and when the value drift becomes large, that will reverse. That's why we've got to usurp the powers of the blind idiot god before it's too late.

Closely related: Scott Alexander's Meditations on Moloch.

Evolution tends to do a basically random walk exploration of the easily reached possibility space available to any specific life form. Given that it has to start from something very simple, initial exploration is towards greater complexity. Once a reasonable level of complexity is reached, the random walk is only slightly more likely to involve greater complexity, and is almost equally as likely to go back towards lesser complexity, in respect of any specific population. However, viewing the entire ecosystem of populations, there will be a general trajectory of expansion into new territory of possibility. The key thing to get is that in respect of any specific population or individual (when considering the population of behavioural memes within that individual), there is an almost equal likelihood of going back into territory already explored as there is of exploring new territory.

There is a view of evolution that is not commonly taught, that acknowledges the power of competition as a selection filter between variants, and also acknowledges that all major advances in complexity of systems are characterised by new levels of cooperation. And all cooperative strategies require attendant strategies to prevent invasion by "cheats". Each new level of complexity is a new level of cooperation.

There are many levels of attendant strategies that can and do speed evolution of subsets of any set of characters.

Evolution is an exceptionally complex set of systems within systems. At both the genetic and mimetic levels, evolution is a massively recursive process, with many levels of attendant strategies. Darwin is a good introduction, follow it with Axelrod, Maynard Smith, Wolfram; and there are many others worth reading - perhaps the best introduction is Richard Dawkins classic "Selfish Gene".

The one safe bet is that we'll be trying to maximize our future values, but in the emulated brains scenario, it's very hard to guess at what those values would be. It's easy to underestimate our present kneejerk egalitarianism: We all think that being a human on its own entitles you to continued existence. Some will accept an exception in the case of heinous murderers, but even this is controversial. A human being ceasing to exist for some preventable reason is not just generally considered a bad thing. It's one of the worst things.

Like most people, I don't expect that this value will be fully extended to emulated individuals. I do think it's worth having a discussion about what aspects of it might survive into the emulated minds future. Some of it surely will.

I've seen some (e.g. Marxists) argue that these fuzzy values questions just don't matter, because economic incentives will always trump them. But the way I see it, the society that finally produces the tech for emulated minds will be the wealthiest and most prosperous human society in history. Historical trends say that they will take the basic right to a comfortable human life even more seriously than we do now, and they will have the means to basically guarantee it for the ~9 billion humans. What is it that these future people will lack but want - something that emulated minds could give them - which will be judged to be more valuable than staying true to a deeply held ethical principle? Faster scientific progress, better entertainment, more security and more stuff? I know that this is not a perfect analogy, but consider that eugenic programs could now advance all of these goals, albeit slowly and inefficiently. So imagine how much faster and more promising eugenics would have to be before we resolve to just go for it despite our ethical misgivings? The trend I see is that the richer we get, the more repugnant it seems. In a richer world, a larger share of our priorities is overtly ethical. The rich people who turn brain scans into sentient emulations will be living in an intensely ethical society. Futurists must guess their ethical priorities, because these really will matter to outcomes.

I'll throw out two possibilities, chosen for brevity and not plausibility: 1. Emulations will be seen only as a means of human immortality, and de novo minds that are not one-to-one continuous with humans will simply not exist. 2. We'll develop strong intuitions that for programs, "he's dead" and "he's not running" are importantly different (cue parrot sketch).

Adaption executers, not fitness maximizers. Humans probably have specific hard-coded adaptations for the appreciation of some forms of art and play. It's entirely plausible that these are no longer adaptive in our world, and are now selected against, but that this has not been the case for long enough for them to be eliminated by evolution.

This would not make these adaptations particularly unusual in our world; modern humans do many other things that are clearly unadaptive from a genetic fitness perspective, like using contraceptives.

The Caplanian story about education is that a lot of what people are signalling is conformism, and it's inherently hard to move to a new better way of signalling conformism.

You mean besides "do a helluva lot of neuroscience"? A lot of our good qualities are subjective, in the most basic sense of that word. We don't even understand their physical basis well enough to predict whether (or which ones) significantly different physical systems will share. If you want to know whether robots can take your job, the question of whether machines can think is exactly as uninteresting as whether submarines can swim (h/t Dijkstra). If you want to know whether you can become a certain kind of robot, that's a whole 'nother story.

In a way happiness is ingrained into specific personality types. My neighbour - next flat - is amazingly happy even after she locked herself out and I tried to break in for her. That happiness can only be duplicated with good drugs. Then there is attitude. I was in India [not as a 5 * tourist either] and found they were content [a bit less than happy] with their lives which compared to ours was a big obvious difference. Anyway it's a moot point as the Scandinavians won that round globally the last time because - social democracy works and it is not socialism which a lot of the braindead insist it is. so will all this collapse in a sci fin Asimov type future? No. As cars replaced horse-cabbies and the underground trains created true mass transit happiness per se was-is not affected-effected. Nor to airoplanes instead of ancient clippers to travel across the seas. Personally I can't wait for the future. I even dream about it. At times. People adjust as kids to their surroundings and take it from there. Anthropologists, social scientists, historians, journalists and writers and even real scientists have shown us that we can be happy whether living in the Stone Age [Australian aborigines] or high tech astronauts and everything in between.