LessWrong 2.0 Reader

I took a quick look. I did not quite find this, I found other discussion of suns dying or being used as resources. Sharing as data.

In the song "Five Thousand Years" the lyrics talk about the sun dying in the next 5,000 years.

I don't quite know how things might change
I don't quite know what rules we'd break
Our present selves might think it strange
But there's so many lives at stake...

Entropy is bearin' down
But we got tricks to stick around.
And if we live to see the day
That yellow fades to red then grey,

We'll take a moment, one by one
Turn to face the dying sun
Bittersweetly wave goodbye--
The journey's only just begun...

In (Five thousand years)
(Whatcha want to do, whatcha wanna see, in another)
(Five million years)
(Where we want to go, who we want to be, in another)

Here's a reference to it as a battery, in the (fast, humerous, upbeat) song "The Great Transhumanist Future"

In the Great Transhumanist Future,
There are worlds all fair and bright,
We’ll be constrained by nothing but
The latency of light
When the hospitals are empty
And the sun’s a battery
Making it a breeze
To get outta deep freeze
To give humans wings
And some other things
In the Great Transhumanist Future.

I agree that not all decisions about the cosmos should be made on a majoritarian democratic way, but I don't see how replacing the Sun with artificial light can be done by market forces under normal property rights. I think you are currently would not be allowed to build a giant glass dome around someone's pot of land, and this feels at least that strong. 

I'm broadly sympathetic to having property rights and markets in the post-Singularity future, and probably the people will scope-sensitive and longtermist preferences will be able to buy out the future control of far-away things from the normal people who don't care about these too much. But these trades will almost certainly result the solar system being owned by a coalition of normal people, except if they start with basically zero capital. I don't know how you imagine the initial capital allocation to look like in your market-based post-Singularity world, but if the vast majority of the population doesn't have enough control to even save the Sun, then probably something went deeply wrong.

(This seems false and in as much as someone is willing to take bets that resolve after superintelligence, I would bet that most people do not care any appreciable amount about the actual sun existing by the time humanity is capable of actually providing a real alternative) 

I think it is an argument that they should not happen, but a very weak one, especially for things like this where the current public really hasn’t thought much about. 

Also, I think David is just wildly wrong here about what realistically would happen in 10,000 years for a society that could actually start using all of the suns energy. This would involve hundreds of generations of people each deciding to not grow, to not expand into the rest of the solar system, to pass up on enormous opportunities for joy and greatness and creation, out of sentimentality for a specific kind of attachment for the specific arrangement of our planet and solar system at this moment in time. This attachment is very much real, and worth something, but IMO obviously will not remotely outweigh the preferences and actions of the trillions of people who will all want to do more things and have more things (and since we are talking about gradual expansion, very much present in the conversation). 

it shows up in Solstice songs as a thing we want to do in the Great Transhumanist Future twenty years from now

Is this true?! (Do you have a link or something?)

Most decisions are not made democratically, and pointing out that a majoritarian vote is against a decision is no argument that they will not happen nor should not happen. This is true of the vast majority of resource allocation decisions such as how to divvy up physical materials.

You are putting words in people's mouths to accuse lots of people of wanting to round up the Amish and hauling them to extermination camps, and I am disappointed that you would resort to such accusations.

I agree that I don't viscerally feel the loss of the 200 galaxies, and maybe that's a deficiency. But I still find this position crazy. I feel this is a decent parallel dialogue:
Other person: "Here is a something I thought of that would increase health outcomes in the world by 0.00000004%."
Me: "But surely you realize that this measure is horrendously unpopular, and the only way to implement it is through a dictatorial world government."
Other person: "Well yes, I agree it's a hard dilemma, but on absolute terms, 0.00000004% of the world population is 3 people, so my intervention would save 3 lives. Think about how horrible tragedy every death is, I really feel there is a missing mood here when you don't even consider the upside of my proposal."

I do feel sad when the democratic governments of the world screw up policy in a big way according to my beliefs and values (as they often do). I still believe that democracy is the least bad form of government, but yes, I feel the temptation of how much better it would be if we were instead governed by perfect philosopher king who happens to be aligned with my values. But 0.00000004% is just a drop in the ocean.

Similarly, I believe that we should try to maintain a relatively democratic form of government at least in the early AI age (and then the people can slowly figure out if they find something better than democracy). And yes, I expect that democracies will do a lot of incredibly wasteful, stupid, and sometimes evil things by my lights, and I will sometimes wish if somehow I could have become the philosopher king. That's just how things always are. But leaving Earth alone really is chump change, and won't be among the top thousand things I disagree with democracies on. 

(Also, again, I think there will also be a lot of value in conservatism and caution, especially that we probably won't be able to fully trust our AIs in the most complicated issues. And I also think there is something intrinsically wrong about destroying Earth, I think that if you cut out the world's oldest tree to increase health outcomes by 0.00000004%, you are doing something wrong, and people have a good reason to distrust you.)

This point doesn't make sense to me. It sounds similar to saying "Most people don't like it when companies develop more dense housing in cities, therefore a good democracy will not have it" or "Most people don't like it when their horse-drawn carriages are replaced by cars, therefore a good democracy will not have it". 

The cost-benefit calculations on these things work out and it's good if most uninformed people who haven't spent much time on it are not able to get in the way of companies that are building goods and services in this regard. 

There are many many examples (e.g. GMOs, nuclear power, coal power, privatized toll roads, fracking, etc), and I expect if I researched for a few hours I would find even clearer examples for which it is currently consensus that it is a good idea, but at the time the majority disliked it.

More generally:

• People's mass preferences are sometimes dumb, and sometimes quite reasonable, and you should have a decision rule that distinguishes between the two, and when things are orders of magnitude more cost effective than other things, this is a good argument against arguments based on simple preference / aesthetics, and this comment does nothing to show that this isn't stupidity rather than wisdom.
• Just because a lot of people in a democracy disapproves of things does not mean that market forces shouldn't be able to disagree with them and be correct about that. Analogous to the Luddites who had little concept of how technological and economic progress lifts everyone out of poverty, most people today do not appreciate that future computational-life forms will be just as meaningful as the meat-based ones today, and should not sacrifice orders of magnitudes more life-years than will be lived on Earth^[1] for the difference between a big ball of plasma and something else that recreates the same quality of light.
• Majoritarian vote on everything is a terrible way to make decisions; most decisions should be given to as small a group as possible (ideally an individual) who is held accountable for the outcome being good, and is given the resources to make the decision well. We do it for political leaders due to the low levels trust and high levels of adversarial action, but this should not be extended to whether to take the sun apart for parts.

1. ^{^}

   I'd quickly guess that the energy difference supports ${10}^n$ life-years where $n$ is somewhere between $10$ and $30$.

Yes, I wanted to argue something like this.