LessWrong 2.0 Reader

There have been a lot of tricks I've used over the years, some of which I'm still using now, but many of which require some level of discipline. One requires basically none, has a huge upside (to me), and has been trivial for me to maintain for years: a "newsfeed eradicator" extension. I've never had the temptation to turn it off unless it really messes with the functionality of a website. 

It basically turns off the "front page" of whatever website you apply it to (e.g. reddit/twitter/youtube/facebook) so that you don't see anything when you enter the site and have to actually search for whatever you're interested in. And for youtube, you never see suggestions to the right of or at the end of a video.

Alcohol is also a drug. If Dalrymple really means "drugs" when he says "drugs", it would follow that he's advocating for prohibition to protect alcoholics from themselves.

We seem to have found a relatively tolerable equilibrium around alcohol where the substance is widely available, the majority of individuals who can enjoy it recreationally are free to do so, and yet it's legally just as intolerable for an intoxicated person to harm others as it would be for a sober person to take the same actions. Some individuals have addiction problems, and we have varyingly effective programs in place to help them deal with that, but ultimately the right of the majority to enjoy it responsibly (and the rights of the businesses to sell it to those who can use it responsibly) trump the "rights" of the minority to be protected from themselves by the government.

Maybe to get the same equilibrium around other drugs, we would need harsher punishments for the antisocial behaviors that we're actually trying to prevent by banning the drugs themselves. All I know is that anyone who unironically makes "ban the intoxicants" claims without considering what we can learn from our most widely accepted and normalized intoxicants is speaking on some level other than the literal and logical.

Agreed, but I will find a way.

Regarding Musk and Thiel, foremost they are billionaire capitalists, individuals who built enormous business empires. Even if we assume your thinking about the future is correct, we shouldn't assume that they have reproduced every step of it. You may simply be more advanced in your thinking about the future than they are. Their thought about the future crystallized in the 1980s, when they were young. Since then they have been preoccupied with building their empires. 

This raises the question, how do they see the future, and their relationship to it? I think Musk's life purpose is the colonization of Mars, so that humanity's fate isn't tied to what happens on Earth. Everything else is subordinate to that, and even robots and AI are just servants and companions for humanity in its quest for other worlds. As for Thiel, I have less sense of the gestalt of his business activities, but philosophically, the culture war seems very important to him. He may have a European sense of how self-absorbed cultural elites can narrow a nation's horizons, that drives his sponsorship of "heterodox" intellectuals outside the academy. 

If I'm right, the core of Musk's futurism is space colonization, and the core of Thiel's futurism is preserving an open society. They don't have the idea of an intelligence singularity whose outcome determines everything afterwards. In this regard, they're closer to e/acc than singularity thinking, because e/acc believes in a future that always remains open, uncertain, and pluralist, whereas singularity thinking tends towards a single apocalyptic moment in which superintelligence is achieved and irreversibly shapes the world. 

There are other reasons I can see why they would involve themselves in the culture war. They don't want a socialism that would interfere with their empires; they think (or may have thought until the last few years) that superintelligence is decades away; they see their culture war opponents as a threat to a free future (whether that is seen in e/acc or singularity terms), or even to the very existence of any kind of technological future society. 

But if I were to reduce it to one thing: they don't believe in models of the future according to which you get one thing right and then utopia follows, and they believe such thinking actually leads to totalitarian outcomes (where their definition of totalitarian may be, a techno-political order capable of preventing the building of a personal empire). Musk started OpenAI so Google wouldn't be the sole AI superpower; he was worried about centralization as such, not about whether they would get the value system right. Thiel gave up on MIRI's version of AI futurology years ago as a salvationist cult; I think he would actually prefer no AI to aligned AI, if the latter means alignment with a particular value system rather than alignment with what the user wants. 

Oops, you're right, fixed. That was just an accident.

When it comes to blind spots, we do have areas like medicine where we don't pay as a result of outcomes of medical treatment. That leads to silly things that when surgeons say that having 4k monitors will obviously improve the way they do surgery because it allows them to see details that they otherwise wouldn't, without anyone running a clinical trial that shows 4k monitors to be superior, they don't get adopted. 

Evidence-based medicine is a strong dogma [? · GW]that prevents market economies from making the medical provider that creates the best outcomes win. 

Thank you for this post. I agree this is important, and I'd like to see improved plans.

Three comments on such plans.

1: Technical research and work.

(I broadly agree with the technical directions listed deserving priority.)

I'd want these plans to explicitly consider the effects of AI R&D acceleration, as those are significant. The speedups vary based on how constrained projects are on labor vs. compute; those that are mostly bottle-necked on labor could be massively sped up. (For instance, evaluations seem primarily labor-constrained to me.)

The lower costs of labor have other implications as well, likely including security [LW · GW] (see also here [LW · GW]) and technical governance (making better verification methods technically feasible). 

2: The high-level strategy

If I were to now write a plan for two-to-three-year timelines, the high-level strategy I'd choose is:

Don't build generally vastly superhuman AIs. Use whatever technical methods we have now to control and align AIs which are less capable than that. Drastically speed up (technical) governance work with the AIs we have.^[1] Push for governments and companies to enforce the no-vastly-superhuman-AIs rule.

Others might have different strategies; I'd like these plans to discuss what the high-level strategy or aims are.

3: Organizational competence

Reasoning transparency and safety first culture are mentioned in the post (in Layer 2), but I'd further prioritize and plan organizational aspects, even when aiming for "the bare minimum". Beside the general importance of organizational competence, there are two specific reasons for this:

• If and when AI R&D acceleration is very fast, delays in information propagating to outsiders are more costly. That is: insofar as you want to keep external actors "in the loop" and contribute, you need to put more effort into communicating what is happening internally.
• Organizational competence and technical work are not fully at odds, as there are employees specialized in different things anyways.

(I think the responses to Evan Hubinger's request [LW · GW] for takes on what Anthropic should do differently has useful ideas for planning here.)

1. ^{^}

   Note: I'm not technically knowledgeable on the field.

>In other words, you don’t need reality to be i.i.d.; you simply need to structure your beliefs in a way that allows an “as if” i.i.d. interpretation.

I think I view exchangeability vs. iid slightly differently. In my view, the “independence” part of iid is just way too strong, and is not required in most of the places people scatter the acronym “iid”. 

For example, say you are catching fish in a lake, and you know only bass and carp live in the lake, and that there are a ton of fish in it, but not how many of each, and you’re trying to estimate the proportion of carp as you catch fish. 

When I catch a carp, the probability that my next catch is a carp goes up. So my probability is dependent on my previous catches - that’s why I can learn things about the proportion! If they were indeed independent, then I couldn’t learn anything. But happily, the correct requirement is not independence, but exchangeability, so I can still update my beliefs as I see more fish.

However, I may just be confused about “iid” as classicists use it, since I never properly learned classic statistics. Interested in what you think about the difference between the two in this example.

Great post! Probably worth noting explicitly that this one of de Finetti's theorems does not hold in the finite case. 

Ok, but you should leave the donation box up -- link now seems to not work?  I bet there would be at least several $K USD of donations from folks who didn't remember to do it in time.