LessWrong 2.0 Reader

It's true that risk alone isn't a good way to decide budgets. You're even more correct that convincing demands to spend money are something politicians learn to ignore out of necessity.

But while risk alone isn't a good way to decide budgets, you have to admit that lots of budget items have the purpose of addressing risk. For example, flood barriers address hurricane/typhoon rick. Structural upgrades address earthquake risk. Some preparations also address pandemic risk.

If you accept that some budget items are meant to address risk, shouldn't you also accept that the amount of spending should be somewhat proportional to the amount of risk? In that case, if the risk of NATO getting invaded is similar in amount to the rogue AGI risk, then the military spending to protect against invasion should be similar in amount to the spending to protect against rogue ASI.

I admit that politicians might not be rational enough to understand this, and there is a substantial probability this statement will fail. But it is still worth trying. The cost is a mere signature and the benefit may be avoiding a massive miscalculation.

Making this statement doesn't prevent others from making an even better statement. Many AI experts have signed multiple statements, e.g. the "Statement on AI Risk," and "Pause Giant AI Experiments." The most convincing statements naturally float to the top, so it is helpful to try adding something different now and then.

I just became a Patreon member and I suggest others do the same. 

Elizabeth has consistently done neglected, high-value impactful research on a wide variety of topics. 

Her research is often more thorough & honest & novel than that done by comparable academics or EA research institutions being paid orders of magnitude more. 

Maybe that's why people meditate – they enter a simple state of mind that emerges everywhere.

I don't think this would be a good letter. The military comparison is unhelpful; risk alone isn't a good way to decide budgets. Yet, half the statement is talking about the military. Additionally, call-to-action statements that involve "Spend money on this! If you don't, it'll be catastrophic!" are something that politicians hear on a constant basis, and they ignore most of them out of necessity.

In my opinion, a better statement would be something like: "Apocalyptic AI is being developed. This should be stopped, as soon as possible."

I don't know if you are going to address this, but if I were to write a sequence of posts on functionalism, I'd start with the problem that "computation" isn't very well defined, and hence functionalism isn't very well-defined, either. In practice it's often clear enough whether or not a system is computing something, but you're going to have a hard time giving a fully general, rigorous, universally applicable definition of what exactly a physical process has to do to count as computing something (and if so, what precisely it is computing). Similarly, your definition of the Practical CF inherits this problem because it's not at all clear what "capturing the dynamics of the brain on some coarse-grained level of abstraction" means. This problem is usually brushed over but imo that's where all the difficulty lies.

(Of course, many people think consciousness is inherently fuzzy, in which case associating it with similarly fuzzy concepts isn't a problem. But I'm assuming you're taking a realist point of view here and assume consciousness is well-defined, since otherwise there's not much of a question to answer. If consciousness is just an abstraction, functionalism becomes vacuously true as a descriptive statement.)

:)

If you want to try it meanwhile, check out https://github.com/MineDojo/Voyager

It will work only if I care for my observations, something like EDT. 

I assume salt water has lower side effects, so that seemed like a promising thing to check.

Why do you make that assumption? Besides the antiviral effect of it, I would expect salt water to drain H_2O from the oral mucosa. Do you think the effect is too small to matter? Do you think it's a desirable effect?

This is an important point. AI alignment/safety organizations take money as input and write very abstract papers as their output, which usually have no immediate applications. I agree it may appear very unproductive.

However, if we think from first principles, a lot of other things are like that. For instance, when you go to school, you study the works of Shakespeare, you learn to play the guitar, and you learn how Spanish pronouns work. These things appear to be a complete waste of time. If 50 million students in the US spend 1 hour a day on these kinds of activities, and each hour is valued at only $10, that's $180 billion/year.

But we know these things are not a waste of time, because in hindsight, when you study how students grow up, this work somehow helps them later in life.

Lots of things appear useless, but are valuable for reasons beyond the intuitive set of reasons we evolved to understand.

Studying the nucleus of atoms might appear like a useless curiosity, if you didn't know it'll lead to nuclear energy. There are no real world applications for a long time but suddenly there are enormous applications.

Pasteur's studies on fermentation might appear limited to modest winemaking improvements, but it led to the discovery of germ theory which saved countless lives.

The stone age people studying weird rocks may have discovered obsidian and copper. Those who studied the strange seeds that plants produce may have discovered agriculture.

We don't know how valuable this alignment work is. We should cope with this uncertainty probabilistically: if there is a 50% chance it will help us, the benefits per cost is halved, but that doesn't reduce ideal spending to zero.

I think it's a very visible example that right now is particularly often brought up. I'm not saying it's all there is to it but I think the fundamental visceral reaction to the very idea of self-mutilation is an important and often overlooked element of why some people would be put off by the concept. I actually think it's something that makes the whole thing a lot more understandable in what it comes from than the generic "well they're just bigoted and evil" stuff people come up with in extremely partisan arguments on the topics. These sort of psychological processes - the fact that we may first have a gut-level reaction, and only later rationalize it by constructing an ideological framework to justify why the things that repulses us are evil - are very well documented, and happen all over the place. Does not mean everyone who disagrees with me does so because of it (nor that everyone who agrees doesn't do it!) but it would be foolish to just pretend this never happens because it sounds a bit offensive to bring up in a debate. The entire concept of rationality is based around the awareness that yeah, we're constantly affected by cognitive biases like these, and separating the wheat from the chaff is hard work.

And by the way it's an excellent example of the reverse too. Just like people who are not dysphoric are put off by mutilation, people who are are put off by the feeling of having something grafted onto their bodies that doesn't belong. Which is sort of the flip side of it. Essentially we tend to have a mental image of our bodies and a strong aversion to that shape being altered or disturbed in some way (which makes all kinds of sense evolutionarily, really). Ironically enough, it's probably via the mechanism of empathy that someone can see someone else do something to their body that feels "wrong" and cringe/be grossed out on their behalf (if you think trans issues are controversial, consider the reactions some people can have even to things like piercings in particularly sensitive places).