I'm not sure the “poverty equilibrium” is real. Poverty varies a lot by country and time period, various policies in various places have helped with poverty, so UBI might help as well. Though I think other policies (like free healthcare, or fixing housing laws) might help more per dollar.

I think the main point of the essay might be wrong. It's not necessarily true that evolution will lead to a resurgence of high fertility. Yes, evolution is real, but it's also slow: it works on the scale of human lifetimes. Culture today evolves faster than that. It's possible that culture can keep adapting its fertility-lowering methods faster than humans can evolve defenses against them.

I think you're right, Georgism doesn't get passed because it goes against the interests of landowners who have overwhelming political influence. But if the actual problem we're trying to solve is high rents, maybe that doesn't require full Georgism? Maybe we just need to make construction legally easier. There's strong opposition to that too, but not as strong as literally all landowners.

It seems to me that land of the same quality as this can already be bought for cheaper in many places. The post says the new land could be more valuable because of better governance, but governance is an outcome of human politics, so it's orthogonal to old/new land. In Jules Verne's Propeller Island, a power conflict eventually leads to physical destruction of the island.

My impression is that Wikipedia was founded on an ideal of neutrality, but Gerard doesn't really believe in that ideal - he considers it harmful, a kind of shield for the status quo. That's a possible position, but I'm not sure how one can hold it and at the same time edit Wikipedia in good faith. Does anyone know how that can be justified?

anything that acts like us has our qualia

Well, a thing that acts like us in one particular situation (say, a thing that types "I'm conscious" in chat) clearly doesn't always have our qualia. Maybe you could say that a thing that acts like us in all possible situations must have our qualia? This is philosophically interesting! It makes a factual question (does the thing have qualia right now?) logically depend on a huge bundle of counterfactuals, most of which might never be realized. What if, during uploading, we insert a bug that changes our behavior in one of these counterfactuals - but then the upload never actually runs into that situation in the course of its life - does the upload still have the same qualia as the original person, in situations that do get realized? What if we insert quite many such bugs?

Moreover, what if we change the situations themselves? We can put the upload in circumstances that lead to more generic and less informative behavior: for example, give the upload a life where they're never asked to remember a particular childhood experience. Or just a short life, where they're never asked about anything much. Let's say the machine doing the uploading is aware of that, and allowed to optimize out parts that the person won't get to use. If there's a thought that you sometimes think, but it doesn't influence your I/O behavior, it can get optimized away; or if it has only a small influence on your behavior, a few bits' worth let's say, then it can be replaced with another thought that would cause the same few-bits effect. There's a whole spectrum of questionable things that people tend to ignore when they say "copy the neurons", "copy the I/O behavior" and stuff like that.

Since you conclude that both SIA and SSA are flawed because we know a lot about our parents, let's see if that works. Imagine a world where people spend their first years not knowing much about their parents, or about human reproduction. Suppose they each live in a kind of egg, and receive newscasts from outside only carrying information about the particular anthropic problem we want them to solve (e.g. "the world currently contains N people", or "there are two theories about astronomy" and so on). How should people solve anthropic problems under such conditions? Should they use SIA or SSA? Or should they still reject both, but based on some other argument, and the "we know a lot about our parents" argument was a red herring?

I think right of return might be close to the center of the problem. Imagine a Palestinian Arab who lived on the land before, and who agrees with Israel's system (democracy, civil rights and so on). According to an Israeli nationalist, should such a person be allowed to return and get Israeli citizenship?

I think that phrase is right "to zeroth order": one can imagine an agent with any preferences about population ethics. Then "to first order", I think the choice between average vs total utilitarianism does have a right answer (see link in my reply to Cleo). And then there are "second order" corrections like value of variety, which seem more subjective, but maybe there are right answers to be found about them as well.

It could be owned by the government and rented out for cheap, but only if the renter uses it as primary residence. Or it could be means-tested.

Yeah, this is an interesting proposal. It would make being a Russian soldier in Ukraine more attractive than being a Ukrainian soldier (who's stuck in the trenches) or a Ukrainian male of military age (who can't leave the country). Also, antiwar Russians would find that the fastest way to move to Europe is to enlist and defect (skipping the normal multi-year process of getting citizenship). Also, everyone along the way would want a cut: Russians would start paying money to recruitment offices when they enlist, and paying more money to Ukrainian soldiers when they defect. The economic incentives of the whole thing just get funnier and funnier as I think about it.

Yeah. I decided sometime ago that total utilitarianism is in some sense more "right" than average utilitarianism, because of some variations on the Sleeping Beauty problem. Now it seems the right next step is taking total utilitarianism and adding corrections for variety / consistency / other such things.

To me it actually feels plausible that identical people might value themselves less. If there are 100 identical people and 1 different person, the identical ones might prefer saving the different one rather than one of their own. But maybe not everyone has that intuition.

I think you make some good points. But this part stuck out to me a bit:

You are deciding where to build both a pharmacy and bakery (assume corresponding foot traffic uplift from the other is 0). Under the SQ, you build them next to each other such that you can maximally capture the land value increase of the other. Under LVT, you build them maximally apart to reduce your land value tax burden. The increased land value you obtained under SQ that was partially captured by you is now a cost being bourne out as inconvenience to the public under LVT.

It seems to me that in reality, the effect of business attracting customers to other nearby businesses (through foot traffic, more people moving into the area, incentivizing more transport nearby, etc) is very much nonzero and might cover the whole LVT increase.

That said, I agree that LVT is a nonobvious thing and might be off target from what we really want. The real problem is making sure there's enough low-income housing in cities, it's better to just solve that directly.

I think I just came up with a somewhat new idea on this, namely that some kinds of valuable experiences tend toward summing and others tend toward averaging.

Yeah, I see. Thinking more about this, they'd be right to mistrust this kind of offer. It feels like the only real solution is the hard one: making sure there's enough low-income housing in cities.

There are two things that are fixed: Land and attention.

I think that's a red herring though. The current scarcity of housing is far below the limit set by land, so I just call it artificial. There can be artificial scarcity of other necessities as well, all that's needed is that the providers of that necessity act as a cartel.

Yeah, maybe "diffusion" is a too optimistic view. A large part of improvements get captured by those who can create an artificial scarcity of some necessity - currently it's housing, in the past it used to be physical safety. And many other improvements happen as part of arms races, so on net they lead to more waste and no benefit - like improvements to advertising in a fixed size market, or improvements to weapons. I think these are the two main mechanisms that "eat" improvements and prevent us from having a 15-hour work week.

Here's a question I don't know the answer to. If there was a program offering a basic income to homeless people, enough to pay for housing+food+clothes+etc in a cheaper area of the US without having to find work, but to receive the money you have to actually move there and get housed - would most homeless people accept the offer and move? I ask because it seems like such a program could be pretty cheap. Or would most of them prefer to stay in cities even at the cost of being homeless? If so, what would be the main reasons?

That article seems to focus on recidivism rate, not incarceration rate or crime rate. But still your point is interesting and I decided to check. I compared the Wikipedia tables for incarceration rate and intentional homicide rate (the most objective proxy for crime rate that I can think of). It turns out if I sort countries by the ratio between the two, both Norway and the US are in the middle of the pack. Moreover, it seems there's no relation between these variables: the scatter plot looks like just a bunch of random points.

EDIT: thinking more about this, it's much more complex. Incarceration rate probably depends both on crime rate and on society's tolerance for crime (hidden variable), and crime rate probably depends both on incarceration rate and on society's custom for crime (hidden variable). So there are feedback mechanisms in both directions, with hidden variables that vary by country. Maybe someone can make a clear conclusion from this, but not me.

Since the US has the highest incarceration rate in the world, replacing an imprison-too-many-people system with a flog-too-many-people system doesn't seem like the most important thing to me. It seems more important to figure out how to punish fewer people in the first place.

A true superintelligence could wipe out humanity incredibly easily—but it could build a utopia nearly as easily. Even if it were almost entirely misaligned, just a sliver of human morality could make it decide to give humans a paradise beyond their wildest imaginings.

As long as the superintelligence's values don't contain any components that pull against components of human morality. But in case of almost-alignment there might indeed be some such components. Almost-alignment is where s-risks live.

Yeah, this agrees with my thinking so far. However, I think if you could research how to align AIs specifically to human flourishing (as opposed to things like obedience/interpretability/truthfulness, which defer to the user's values), that kind of work could be more helpful than most.

Yeah. The way the world works now, if technical alignment work is successful, it will just lead to AIs that are aligned to making money or winning wars. There need to be AIs aligned to human flourishing, but nobody wants to spend money training those. OpenAI was started for this purpose, but got taken over by money interests.

Yeah, I think your formulation is more correct than mine.

I think AIs will be able to do all cognitive labor for less resources than a human could survive on. In particular, "Scott Alexander quality writing" and "long term planning and coordination of programmers" - tasks that you assume will stay with humans - seem to me like tasks where AIs will surpass the best humans before the decade is out. Any "replacement industry" tasks can be taken up by AIs as well, because AI learning will keep getting better and more general. And it doesn't seem to matter whether demand satiates or grows: even fast-growing demand would be cheaper met by building more AIs than by using the same resources to feed humans.

(This is also why Ricardian comparative advantage won't apply. If the AI side has a choice of trading with humans for something, vs. spending the same resources on building AIs to produce the same thing cheaper, then the latter option is more profitable. So after a certain point in capability development, the only thing AIs and AI companies will want from us is our resources, like land; not our labor. The best analogy is enclosures in England.)

You point out several problems in the world: people have unhealthy lifestyles, nuclear power isn't used to its full potential, ecosystems are not protected, our social lives are not in accordance with human flourishing. Then you say all these problems could be solved by a "cooperation machine". But you don't seem to explain why these problems could be solved by the same "machine". Maybe they're all separate problems that need to be solved separately.

Maybe one exercise to try is holding off on proposing solutions. Can you discuss the problems in more detail, but without mentioning any solutions? Can you point out commonalities between the problems themselves? For example, "all these problems would be solved by a cooperation machine" wouldn't count, and "they happen because people are bad at cooperating" is too vague. I'm looking for something more like "these problems happen because people get angry", or "because people get deceived by politicians". Does that make sense? Can you give it a try?

It still seems to me that international cooperation isn't the right first step. If the US believes that AI is potentially world-ending, it should put its money where its mouth is, and first set up a national commission with the power to check AIs and AI training runs for safety, and ban them if needed. Then China will plausibly do the same as well, and from a cooperation of like-minded people in both countries' safety commissions we can maybe get an international commission. But if you skip this first step, then China's negotiators can reasonably say: why do you ask us for cooperation while you still continue AI development unchecked? This shows you don't really believe it's dangerous, and are just trying to gain an advantage.

I'm not sure what's the best solution in general. For this post specifically, maybe we could drop the narration?

I'm not sure international coordination is the right place to start. If the Chinese are working on a technology that will end humanity, that doesn't mean the US needs to work on the same technology. There's no point working on such technology. The US could just stop. That would send an important signal: "We believe this technology is so dangerous that nobody should develop it, so we're stopping work on it, and asking everyone else to stop as well." After that, the next step could be: "We believe that anyone else working on this technology is endangering humanity as well, so we'd like to negotiate with them on stopping, and we're prepared to act with force if negotiations fail."

Yeah, I think the narration doesn't catch up when I edit the post, and I've edited it a lot. Maybe there's a button to refresh it but I haven't found it. @habryka?

I think hoping for "pseudokindness" doesn't really work. You can have one-millionth care about a flower, but you'll still pave it over if you have more-than-one-millionth desire for a parking lot there. And if we're counting on AIs to have certain drives in tiny amounts, we shouldn't just talk about kindness, but also for example desire for justice (leading to punishment and s-risk). So putting our hopes on these one-millionths feels really risky.

(Edited because my previous reply was a bit off the mark.)

I don't think this scenario depends on government. If AI is better at all jobs and can make more efficient use of all resources, "AI does all jobs and uses all resources" is the efficient market outcome. All that's needed is that companies align their AIs to the company's money interest, and people use and adapt AI in the pursuit of money interest. Which is what's happening now.

A single AI taking dramatic transformative action seems less likely to me, because it'll have to take place in a world already planted thick with AI and near-AI following money interests.

I'm not sure what exactly you're proposing to transfer to the state.

• The right to charge rent on the land? But under Georgism the state already owns that right.

• The structure you built on the land? But that structure, without the land, is a depreciating asset. Chances are the next user of the land will just tear it down and build something else. So you might not get enough money for retirement by offering up the structure alone.

I think that response basically doesn't work. But when I started writing in more detail why it doesn't work, it morphed into a book review that I've wanted to write for the last couple years but always put it off. So thank you for finally making me write it!

Page 87:

The clusters can be built in the US, and we have to get our act together to make sure it happens in the US.

No, we have to make sure it doesn't happen anywhere.

Page 110:

What we want is to add side-constraints: don’t lie, don’t break the law, etc.

That's very not enough. A superintelligence will be much more economically powerful than humans. If it merely exhibits normal human levels of benevolence, truth-telling, law-obeying, money-seeking, power-seeking and so on, it will deprive humans of everything.

It's entirely legal to do jobs so cheaply that others can't compete, and to show people optimized messages to make them spend savings on consumption. A superintelligence merely doing these two things superhumanly well, staying within the law, is sufficient to deprive most people of everything. Moreover, the money incentives point to building superintelligences that will do exactly these things, while rushing to the market and spending the minimum on alignment.

Superintelligence requires super-benevolence. It must not be built for profit or for an arms race, it has to be built for good as the central goal. We've been saying this for decades. If AI researchers even now keep talking in terms like "add constraints to not break the law", we really are fucked.

Simulations; predictors (not necessarily perfect); amnesia; identical copies; players with aligned interests.

Why isn’t incremental progress at instilling human-like behavior into machines, incremental progress on AGI alignment?

It kind of is, but unfortunately treating others badly when you have lots of power is also part of human nature. And there's no real limit to how bad it could get, see the Belgian Congo for example.

An elected representative who’s term limit is coming up wouldn’t have the same incentives.

I think this proves too much. If elected representatives only follow self-interest, then democracy is pointless to begin with, because any representative once elected will simply obey the highest bidder. Democracy works to the extent that people vote for representatives who represent the people's interests, which do reach beyond the term limit.

Don't know about others, but to me feels like "wokeness" has faded from view a bit because economic inequality has become the main issue again. And I agree with that: making people less dependent on landlords and employers is indeed the main issue. (The AI issue is more urgent, but even the AI transition would feel safer to me if we had a system where people's livelihoods weren't tied to jobs or affording rent.)

If orchestral jobs for violinists are so scarce, what do you think about the options of branching out? Playing with a band, going into pop violin, etc.

I think Scott's original story described scissor statements a bit differently. The people reading them thought "hmm, this isn't controversial at all, this is just obviously true, maybe the scissor-statement-generator has a bug". And then other people read the same statement and said it was obviously false, and controversy resulted. Like the black and blue vs white and gold dress, or yanny/laurel. Maybe today's LLMs aren't yet smart enough to come up with new such statements.

EDIT: I think one possible reason why LLMs have trouble with this kind of question (and really with any question that requires coming up with specific interesting things) is that they have a bias toward generic. In my interactions with them at least, I keep having to constrain the question with specificity, and then the model will still try to give the most generic answer it can get away with.