Eutopia is Scary

You can publish it; I promise we'll be fine.

I'm sure most of you would be fine, but the question I have to ask is "Will this drive more than 5% of my reading audience insane?"

Well, before I try to answer that, mind tabooing "soul sucking"? ie, what exactly do you mean by that?

Thanks.

The "corny pun" in this case being "Bayesian Conspiracy"?

postulating that talking about science in public is socially unacceptable, for the same reason that you don't tell someone aiming to see a movie whether the hero dies at the end. ... I started thinking that, well, maybe it really would be a good idea to get rid of all the textbooks, all they do is take the fun out of science.

Maybe they should exist, but shouldn't be thought of and written like textbooks. Maybe they should be like video game walkthroughs. You use them when you're stuck, as a last resort. Or, you just go dive right in, because you want to play optimally and make sure that you explore everything.

Of course, it's kind of hard to make learning many sciences into an experience that is like exploring a video game, because most people don't have the equipment on hand to, say, recreate the experiment that led Rutherford to hypothesize the existence of the atomic nucleus. On the other hand, you could make mathematics into a much more game-like experience, by simply presenting progressively harder problems and challenging students to come up with methods to solve them.

You may or may not be aware that mathematics in the Middle Ages was a highly competitive endeavor, with new problem-solving techniques, such as the general solution to the cubic equation, being carefully guarded trade secrets that mathematicians would use to challenge and one-up one another other.

Doug, what happens if I hand you a video game walkthrough of the rest of your life and tell you not to look at it unless you're stuck?

I mean, maybe you're the wrong person to ask this. And so am I, come to think. But I don't think it's something they do in Eutopia.

PS: It's easy for you to say "Show us your shocking future!"

Let's hear what sort of shocking development you think might result in a better future in which you would be out of place.

And by that I don't mean that you're economically obsoleted, if that's what you've already been raised to expect as a normal fate. I mean a genuinely arguably-better world that doesn't comfortably confirm all your political and moral beliefs - at least the ones you had before you asked the question.

If you're seriously considering the question, you'll probably be tempted to post your comment anonymously. Which you should feel free to do, of course.

I offered the example of "To spread science more effectively, hide the textbooks, taboo open discussion, keep the knowledge secret, surround its access with trial and ritual." That's bad enough to start with. What's yours?

Heh. A GameFAQs-style FAQ/Walkthrough for real life. What should I do with something like that?

Well, I actually am pretty much stuck right now, so the first thing I do is get myself out of my current jam (regarding employment and finances and such). Then I see if there is any section that covers vague general advice that seems relatively safe, and read that. Finally, I go lock it away in a safe deposit box at a bank and try to avoid using it again except in an emergency.

Taking the analogy a little too far, my game player ethics tells me that I should avoid looking up, say, future stock market returns or winning lottery numbers, because that would be cheating. Ideally, I'd simply want to use it to avoid blunders rather than to munchkin my way through all of life's puzzles and quests. (Did I just verb "munchkin"?)

On the other hand, it might be worthwhile to ctrl-F "cure for cancer" simply because, well, some things are worth doing a little cheating for. (Is there anything one can do in the physical world that could count as an exploit? Heck, if anything can be an exploit, the human brain sure seems like one, considering that humans have literally taken over the world in what, on the time scale of evolution, is like the blink of an eye.)

Drop back to the 16th century and one popular entertainment was setting a cat on fire. Ever see that in any moving picture, no matter how "lowbrow"?

That did happen in season 5 of The Wire.

Rob - that's because The Wire is more like real life than real life.

I can see the link to the Chronophone here Eliezer. What would Benny F have found most shocking about today? How can we extrapolate that forwards?

Surely the most scary changes will be in ethics and the way we think of the human condition and personal identity.

I'm currently most of the way through The Mind's I, and if Hofstadter's (very plausible) musings on identity are anything like accurate, we're going to have to start thinking very differently about who we are, and even whether that question has any real application. My shocking prediction for 100, 500, 1000 years' time? There won't be any individuals, any notion of 'I'.

The repercussions don't really need spelling out or analysing here, and I'm not going to try and predict how things will work. That's all I've got. Human history is a list of examples of our intuitions being exploded by our observations. Individual personal identity over time is an intuitive illusion, and one that'll become increasingly transparent, and less useful, as time goes by. This scares the living hell out of me - I can't think of any way I could possibly feel more out of place.

And I'm certainly not going to write any fiction set in that world.

I've been thinking about roughly this question a lot the past few weeks. My best guess is the end of sexual fidelity and/or self modification to remove sexual jealousy. Were I to be frozen and then thawed, and find that poly was now the norm I would honestly be disgusted and afraid. The kind of love that I had hoped and dreamed of would be effectively dead. None the less, I know that even with our current mind design there are people today who are poly and seem very happy with it. It seems at least plausible that without the complication of jealousy, romantic love could be that much more Fun.

I don't want this to happen, not at all, but if forced to make a guess, that would be mine.

Greg Egan's short story "The Hundred Light Year Diary" tells what happens when people are (basically) handed the walkthrough for their life.

It's well worth reading (along with the rest of the stories in Axiomatic - which include a bunch of technology that Elizer mentions on a regular basis, and the interesting effects they might have.

I will cite Robin Hanson for his vision of a world where human minds are frequently copied and deleted. It takes a certain sort of fortitude to post (in the same month) encouragement to sign up for cryonics and a prediction that minds uploaded from cryonics will be copied en masse, worked without rest at subsistence wages, then deleted to make way for the next wave of copies (possibly of the same mind with new training).

Mike Blume, you can check The Ethical Slut for the software hack on that. I recommend it to everyone as a book on communication in relationships, whether or not you want to be poly. If you and your partner can have a serious, healthy discussion about whether you should also sleep with other people, discussing whose turn it is to do the dishes should be easy.

Part of being thrown into a better future being scary is that (as you implied) some behaviors that you think of as completely normal will be strongly disapproved of and some behaviors that you think are revolting or pointless will be required.

The only way I can see "no public science" as workable would be if people's ability to do experiments and math was greatly enhanced, both of which strike me as good things.

I should toss out my own. The first that occurs to me is that utopia could be far more constrained rather than far freer. Most of us seem to have a vision of a universe where you can do whatever you want so long as it harms no one else, a kind of Nozickian utopia of utopias. We will all move to Permutation City or build Prime Intellect, and then your only restraint is that you cannot restrain others.

If freedom is only instrumentally useful, rather than morally fundamental, there is no reason for this given sufficient powers of prediction and control. If Asimov's supercomputers really can get around the Hayekian knowledge problem and perfect the economy, most arguments against command economies just went out the window. If I really do know better than you, with no epistemic issues, giving you more freedom is like letting a child play in traffic. If I can prove to you with mathematical certainty that decision X would make things worse, and you still choose X (objectively and by your preferences), we have just proved that you are not capable of handling freedom.

Telling someone where the mines are takes all the fun out of Minesweeper, but you should do it IRL if the town is on the edge of an old war zone. If someone has the walkthrough for my life, I may not consult it constantly, but I would like a pop-up confirmation window to appear whenever a decision will lead to "game over." I would also like more save points.

That parenthetical should have been after "worse," not the following clause.

It seems to me that you should take the surprising seductiveness of your imagined world that violated your abstract sensibilities as evidence that calls those sensibilities into question. I would have encouraged you to write the story, or at least write up a description of the story and what about it seemed seductive. I do think I have tried to describe how my best estimates of the future seem shocking to me, and that I would be out of place there in many ways.

The Orgasmimum may be the least scaring scenario. If that's so, we might get it.

Moin

http://en.wikipedia.org/wiki/Forever_War has some minor disturbing examples. Over time ppl first all turn gay (and ostracize heteros) later all become clones, and put the remaining old-humans on a nice colony while trying to ignore them.

What i could imagine as a future where i'm outer place would be: -way higher general intelligence -mankind becoming a singular, united mind while i'm kept out. -probably every future where uploads, and body switching are everyday events

Greetings,

Martin

I don't find this surprising at all, other than that it occurred to a consequentialist. Being a virtue ethicist and something of a Romantic, it seems to me that the best world will be one of great and terrible events, where a person has the chance to be truly and tragically heroic. And no, that doesn't sound comfortable to me, or a place where I'd particularly thrive.

Perhaps a benevolent singleton would cripple all means of transport faster than say horses and bicycles, so as to preserve/restore human intuitions and emotions relating to distance (far away lands and so on)?

Movies that were made in say the 40s or 50s, seem much more alien - to me - than modern movies allegedly set hundreds of years in the future, or in different universes.

They had different moral fashions. It is how the movies portray such moral fads that shows how different we've become. I haven't seen the f*d up remake of The Day The Earth Stood Still, but one of the principle plot elements of the first one is that some random stranger who Mrs Benson only met a few hours ago, gets to "babysit" her child unsupervised all day. Today, all anyone can think about a similar situation is "child molester."

What scares me is that there are moral fashions too. They're just as arbitrary, and just as invisible to most people. But they're much more dangerous. Fashion is mistaken for good design; moral fashion is mistaken for good. Dressing oddly gets you laughed at. Violating moral fashions can get you fired, ostracized, imprisoned, or even killed. http://www.paulgraham.com/say.html

@Eli "Ben Franklin, say" Franklin is probably the best person to come to the here - it's very well-known he wished to preserve his body in Madeira so he could be revived to see the future, yes? Plus if you've actually read any of his letters and other writing, you see how much more flexible his mind was than just anyone you have ever met. My impression is that he would find today less shocking than probably 75% of those who live now do.

"talking about science in public is socially unacceptable" This is already true in many places around the world and has been for several decades now, yes? I don't just mean Kansas.

@Robin " I would be out of place there in many ways" Considering "The Nerd Identity," (Matt Damon stars) don't you and Eli both rather revel in how out of place here you are in many ways? In light of that, wouldn't the future you have proposed in some ways also be more congenial to you?

Isn't the shock caused by the rigidity of identity, as Ben Jones implies? By re-thinking how identity functions, wouldn't the shock be far less? Altho' honestly I don't see why the idea frightens anyone, as it has been a core notion of both Hinduism and Buddhism for millenia - we should all have had time to integrate the concept apart from the dogma.

Virtual environments create possibilities for shock. The ability to torture a (non-sentient) simulated version of someone you hate, or engage in sexual activities that would be illegal in the real world come to mind.

Also what if, given the opportunity to live forever in eutopia, most minds freely choose the hardscrapple frontier? Even if the chances of death are significant?

Thom Blake:

I don't find this surprising at all, other than that it occurred to a consequentialist. Being a virtue ethicist and something of a Romantic, it seems to me that the best world will be one of great and terrible events, where a person has the chance to be truly and tragically heroic.

Every life a work of art! That sounds like my kind of future.

Change is disruptive, and if it is profound it can cause a strong resistence to it. Eutopia is scary, even if you could guarantee complete safety, people will resist it.

Zubon - This may be drifting off topic, but I'm sure I don't have to tell you that some girls (and probably some guys too) use poly as nothing more than a cheap, easy escape route from a relationship with which they've grown displeased. I had this done to me last summer, and it was quite simply the most miserable experience of my life. The depression and feelings of worthlessness with which it left me have only just begun to abate, and it will probably be some time before I can deal with the level of trust necessary for a vanilla relationship, let alone seriously consider choosing poly.

Zubon - Forgive my rudeness, there was totally supposed to be a "but thanks very much for the rec." in there somewhere.

Probably of interest: The Happiness Myth by Jennifer Michael Hecht. The title's not especially accurate. It's actually about what people have generally said through history about what leads to happiness, and how much of what we believe now on the subject is true.

Please publish this stuff! Or upload it as a free netbook if you're not feeling confident, haha.

I don't think you have anything to worry about. Humans don't go insane that easily. I know two people who have the kind of beliefs that stand in absolute opposition to the Singularity, to the point that they've told me they would rather die a slow painful death rather than live in a post-Singularity world, and I don't think they're lying. Even so, me telling them about the Singularity didn't drive them insane. I've gotten them both to acknowledge that such a world is a possibility, and while this hasn't lessened their profound disgust, it certainly hasn't made them nuts.

Anyway, you think that creating this imaginary world of yours has been a valuable experience for you, don't you? So why wouldn't reading about it be a valuable experience for the rest of us? Even if 5% of us go insane as a result, isn't it worth it the other 95% is enlightened?

It seems to me that you should take the surprising seductiveness of your imagined world that violated your abstract sensibilities as evidence that calls those sensibilities into question. I would have encouraged you to write the story...

Robin, I did and I did.

Steven, you're first to hit on one of the other rules that I'd added in (restoration of travel time / communication delay).

Dictionaries disagree with Ferris - e.g.:

"Happiness [...] antonym: sadness" - Encarta

"Boredom" makes a terrible opposite of "happiness". What is the opposite of boredom? Something interesting, to be sure, but many more things than just happiness fit that description.

restoration of travel time / communication delay

Communication delay as in IP over Avian Carriers? Next thing you know we'll be living in our EEA and have to walk 10 km uphill to school, both ways.

but the question I have to ask is "Will this drive more than 5% of my reading audience insane?"

So you're fine with that amount of collateral damage? :) Seriously, make a "mental stability" questionnaire and distribute your story under NDA to those who pass. Consider me interested too.

Why make science a secret, instead of inventing new worlds with new science for people to explore? Have you heard of "Theorycraft"? It's science applied to the World of Warcraft, and for some, Theorycraft is as much fun as the game it's based on.

Is there something special about the science of base-level reality that makes it especially fun to explore and discover? I think the answer is yes, but only if it hasn't already been explored and then covered up again and made into a game. It's the difference between a natural and an artificial challenge.

Allowing public dissemination of science is analogous to allowing overly fast increase in intelligence, in several respects.

First, in both cases you get some answers without enjoying the process of obtaining them. To solve a problem, you may learn an answer in a textbook, or find it using raw power of increased intelligence. Problems which you might've enjoyed are now spoiled, with answer either known or boringly trivial.

Second, greater knowledge or intelligence opens the ability to enjoy richer problems, so it might be a good idea to move from chimp-level problems to human-level problems even if it spoils the fun of solving chimp-level problems as a chimp. It might be a good idea to start from deeper background in science and better intelligence, preferably synchronized with each other, if still below the level where the difference becomes unclear.

And third, you are allowed or even encouraged to hurt yourself. If the path you choose as a result of being guided by external knowledge or intelligence leads in the wrong direction, that influence needs to be controlled. It looks like the optimal path should consist of a combination of local development by one's own power, and influence of superintelligent overseer that prevents heading in the wrong direction in the long term.

About the arguably good future that frightens.

Naively extrapolating from my thoughts about semantics of intelligence and economy of cognition, I suspect that it might be a good thing for people of the future to specialize and increasingly differ in their morality, while composing a community that as a whole robustly continues on the path set by the present human psychology. It doesn't look like a good thing even for some people to start liking "paperclipping", and maybe it isn't, but maybe it is.

Several steps further, if "person" remains a morally significant abstraction, people, while distinct, may become increasingly abstract and interleaved, so that the same process can be interpreted as a part of one person's thought and another person's different thought at once, with substrate interpreted as implementing individual subjective timelines distributed in space and time. One person as a high-level regularity in many other people's thoughts. It's more an implementation detail than morally significant picture though, like running a program in obfuscated binary code, with originally intended algorithm surprisingly managing to carry through.

One arguably plausible change, that is arguably an improvement (of sorts), and yet would be a shock could be if techniques for assessing individuals' abilities became much more accurate. We could wind up with a world with elments of GATTACA or Brave New World (albeit this alone wouldn't do the equivalent of bottle-farming deltas...). To an extent things like IQ already do this, but more precise, specific assessments (if human abilities are actually stable enough over time for this to be possible) would be a large blow to both traditional views of human equality and to individuals' dreams of the range of their possible futures. It would also be very politically incorrect, at least in some circles.

Maybe the scary part is that, in the end, the future really will turn out to be boring by our standards. It might turn out that getting rid of boredom requires introducing pain, and nobody wants that.

the question I have to ask is "Will this drive more than 5% of my reading audience insane?"

The "delete all the assumptions of childhood and sex and pedophilia" thing would be an easy gross-out with arguable merits ("I know Kung Fu!" style instant learning for full economic independence aged 3?), but I propose that general class of poke-the-taboo is a fourth, non useful direction, "modernarttopia". Suggestions should poke something the public isn't even aware is a taboo.

How about an economy based on the exchange and merging of mind states as currency? From the principle "I duplicate myself, we each own half as much" follows "we merge, I own as much as we both did" and perhaps "I split unevenly, my wealth splits unevenly". This seems like the ultimately most natural means of exchange. It's easy to visualize mind in 4D as a line segment now, a tree later, but this makes it into a DAG. At that point what would "identity" mean? (Could be very important and not at all the facile answer that "it would cease to mean anything").

the question I have to ask is "Will this drive more than 5% of my reading audience insane?" If you have to ask that question, publish.

Spoken like a man who's never actually driven any of his readers insane.

I propose that general class of poke-the-taboo is a fourth, non useful direction, "modernarttopia". Suggestions should poke something the public isn't even aware is a taboo.

Well put. Although it's okay to have a scenario that violates something the public is aware as a taboo, as long as the violation is surprising and has a surprising fun-theoretical reason. It's mostly the obvious that traps you.

"preserve/restore human intuitions and emotions relating to distance (far away lands and so on)"

Arguably Special Relativity already does this for us. Although I freely admit that a space opera is kind of the antithesis of a Weirdtopia.

On second thought, correction: relativity restoring far away lands, yes, preserving intuitions, no.

Probably the space you could visit at light speed in a given subjective time would be unreasonably large, depending on speedup and miniaturization.

"Let's hear what sort of shocking development you think might result in a better future in which you would be out of place."

I think what you are getting at is the difference between the timid shocking things teenagers think up and actual "thoughts that cannot be unthought", like when a child fully grasps death. It's hard but fun to try to knock down all of the pillars of your rationality and discover yourself still floating there. I wouldn't worry about driving people insane. You are more likely to get lynched first.

1) A "fun" one I lifted from Heinlein: monarchy wasn't too bad as stable governments go, and is relatively efficient. The biggest problems were succession and selection. So future kings & queens could be a) trained thoroughly in creche and b) scientifically cross-bred and culled like racehorses.

2) A stone-soup Eutopia, in which Eliezer has only one idea (science kept secret to encourage participation) but claims to have more so shocking he can't repeat them, and then encourages others to guess what they might be. After 6 months it collapses in a singularity of self-reference, or blossoms into a diseutopia so dire we all move to a collective farm and cut our tongues out.

3) Archiving and history is important to me. I've always thought that if I took a time machine FFWD 500 years, I might find that the natives had thrown all that stuff out because they had something better. (Such as time machines.)

4) Put all sociopaths on an island, give them material help, and let them work it out. I'm talking the real high-functioning sociopaths, the "Mask of Sanity" ones, the 0.5% that cannot be helped by anything we know. I suspect this would work out even better than Australia for improving the species, and the act of refining our means of identifying them (poor and rich!) will help our understanding of the mind tremendously. I'm actually unsure whether I'd rather live on the island or not.

Maybe the unhappiness in the world outweighs the happiness, and that contrary to our optimistic views there actually isn't a path to a better future with us in it, and human extinction is the only possible way to make the world a better place.

It's funny actually. I find myself reading this and the responses and feeling so incrediably...Well, stupid. If that's the right word.

I just feel that publishing this would do a lot of people good. Personally, I don't believe in insanity. I just accept it.

We are already living in such a world. Read Daniel Kolak's 'I am you' and realize that we are all God :P (I'm actually, no matter how bizarre it sounds, being completely serious).

Reviving the concept of sending children away to be trained by some kind of a strict brotherhood, the way medieval children would be sent to a monastery or fostered out to become a knight.

It's egalitarian -- those who send away their children can have them educated for free, and perhaps physically or chemically altered to improve their capacities. In exchange, the Brotherhood takes a lifetime of your work, your private life, and a large portion of whatever you earn. They own you, they decide where you live, they manage every instant of your time, every action results in the addition or deduction of points from your public profile, entertainment and non-procreative sex are strictly forbidden. Even the smallest details of life are brutally competitive; points are everything, so your work is pristine and your room is spotless and your physical fitness is superb. Punishment is ubiquitous, and luxuries rare, but that means that sharing a smuggled candy bar after you've started to heal up from your last whipping is impossibly wonderful. You have no choice at all -- but you also don't have the "choice" to fail, to procrastinate, to do anything but perform excellently and serve your Brotherhood.

It is scary how incredibly appealing I find this fantasy. It sounds much more fun -- literally fun -- than the present.

I think you've point in that - but that you take it too far. I don't have any precise study to point at, but from historical or anthropological books I read, and my own traveling experience, I got the impression that even broad culture clashes (like a tribe of native amazonian hunter/gatherer encountering the "civilized" world for the first time) are not as dreadful as you paint them. For example, Darwin explains in one his travel books how quickly the "savage" from some islands adapt to the modern life (of his time).

Some parts of our world is definitely scary for them, but even with such gap, not as much as it feels from your article. And it feels quite... barren to me to speak about "how scary the present would be for people from the past" without looking at situations which do exist and are very similar (isolated culture "stuck in the past" who encounter the "modern world").

Anyone with more anthropological background than myself could validate or invalidate this idea, or point me to a study on that topic ?

Socialists do believe in fun. So long as it is organised by Billy Bragg.

"Remember—boredom is the enemy, not some abstract 'failure.'"

Boredom is the mind-killer. Boredom is the little-death that brings total obliteration. I will face my boredom. I will permit it to pass over me and through me. And when it has gone past I will turn the inner eye to see its path. Where the boredom has gone there will be... all kinds of interesting shit, actually. Which I might never have noticed... had boredom not driven me to look.

Getting rid of textbooks, for example—postulating that talking about science in public is socially unacceptable, for the same reason that you don't tell someone aiming to see a movie whether the hero dies at the end. A world that had rejected my beloved concept of science as the public knowledge of humankind.

That's a pretty good chunk of the premise of Anathem, tho the people in that universe didn't do it for Fun exactly.

In the end, all that matters is choice. If you can always choose a safe and comforting option, nothing is scary (as long as you don’t have the itching sense of justice that makes you meddle in strangers’ business). If you can’t, it’s an authoritarian dystopia no matter how good it is on other accounts.