LessWrong 2.0 Reader

Yes, architecture matters. We don't know what architectures are how likely to produce a rogue agent, but we have subjective expectations, and what a coincidence it would be that they should be the same in each case. For example, if an architecture easily solves a given task, it needs to be scaled up less and then I expect less opportunity for a mesa-optimizer to arise. Mixture of Experts is riskier if the chance that an expert of given size will go rogue is astronomically close to neither 0 nor 1; it is less risky if that chance scales too fast with size. Of course, it's a shaky assumption that splitting a large network into experts will render them unable to form an entire rogue. My object-level arguments might be wrong, but our risk mitigation strategies should not disregard the question of architecture entirely.

Are you saying that you have akrasia? Or that humans in general do?

But I actually agree with you. I just thought the imagery was funny.

So do you think we'll find a way to experimentally test for how much "soul" each human has? If you determine the fate of the cosmos, then you should be much more causally connected than some ineffectual fellow is. There should be a way to test the magnitude of causal connection. Will we truly be able to find the "protagonist"? 

It seems difficult to pin down Bossche's predictions as right or wrong.

Has his previous theory's been proven right?

His previous claim, as I understand it, is that mass vaccination will produce immune evading and more dangerous variants. I think the fact that the variants we've seen so far either emerged before widespread vaccination (alpha), or in places with low vaccination rates (the rest) is at least weak evidence against his claim.

Perhaps he would argue the low vaccination rates did produce those variants, even though the rates were low. But how could we tell?

Will his current theory's been proven right?

He says:

Omicron is likely to start out as a mild disease because short-lived, poorly functional anti-S antibodies (Abs) that resulted from previous asymptomatic infection (e.g., with another previously dominant variant) will no longer recognize Omicron. [...] However, the overall pattern of ‘mild’ disease would only prevail until Omicron becomes dominant and causes high infection rates. When this happens, short-lived, low affinity anti-S Abs will start to compete with innate Abs in an increasing part of the population as a direct result of the enhanced likelihood of re- exposure shortly after previous infection.

How would we distinguish this from the case where Omicron seems mild at the beginning because 90%+ of covid cases are mild, and then once the numbers are high enough we start seeing the smaller fraction of severe cases?

There is one prediction that looks almost tractable enough to (eventually) decide:

It is undeniable that mass vaccination will only drive the virus [...] to use alternate receptor domains on permissive cells. The fitness cost that may come with such a dramatic mutation is likely to be rewarded with enhanced pathogenicity. I am truly afraid that these dynamics will eventually allow for the natural selection of individuals with uncompromised innate immunity while eliminating those without it. While such natural selection would lead to an eradication of SARS-CoV-2 as innate immunity sterilizes the virus and blocks transmission [...] the price paid for ending the pandemic by virus eradication is not comparable to the one paid for by generating herd immunity and allowing the virus to enter an endemic state.

So I guess if we reach endemic state following mass vaccination (the general concensus is that eradication is basically impossible by now so I judge this extremely likely), then his theory will be proven wrong. And if many hundreds of millions die followed by eradication of the virus then he was right. Check back in a year or two?

The problem is that he can still claim half-credit if the virus uses an alternate receptor domain, but that doesn't lead to a high kill rate. And regardless we still won't know whether his counterfactual herd-immunity via natural immunity route would have avoided this.

The biggest disconnect is that this post is not a proposal for how to solve corrigibility. I'm just thinking about what corrigibility is/should be, and this seems like a shard of it—but only a shard. I'll edit the post to better communicate that. 

So, your points are good, but they run skew to what I was thinking about while writing the post.

Meta-comment:

I noticed that I found it very difficult to read through this post, even though I felt the content was important, because of the (deliberately) condescending style. I also noticed that I'm finding it difficult to take the ideas as seriously as I think I should, again due to the style. I did manage to read through it in the end, because I do think it's important, and I think I am mostly able to avoid letting the style influence my judgments. But I find it fascinating to watch my own reaction to the post, and I'm wondering if others have any (constructive) insights on this.

In general I I've noticed that I have a very hard time reading things that are written in a polemical, condescending, insulting, or ridiculing manner. This is particularly true of course if the target is a group / person / idea that I happen to like. But even if it's written by someone on "my side" I find I have a hard time getting myself to read it. There have been several times when I've been told I should really go read a certain book, blog, article, etc., and that it has important content I should know about, but I couldn't get myself to read the whole thing due to the polemical or insulting way in which it was written.

Similarly, as I noted above, I've noticed that I often have a hard time taking ideas as seriously as I probably should if they're written in a polemical / condescending / insulting / ridiculing style. I think maybe I tend to down-weight the credibility of anybody who writes like that, and by extension maybe I subconsciously down-weight the content? Maybe I'm subconsciously associating condescension (at least towards ideas / people I think of as worth taking seriously) with bias? Not sure.

I've heard from other people that they especially like polemical / condescending articles, and I imagine that it is effective / persuasive for a lot of readers. For all I know this is far and away the most effective way of writing this kind of thing. And even if not, Eliezer is perfectly within his rights to use whatever style he wants. Eliezer explicitly acknowledges the condescending-sounding tone of the article, but felt it was worth writing it that way anyway, and that's fine.

So to be clear: This is not at all a criticism of the way this post was written. I am simply curious about my own reaction to it, and I'm interested to hear what others think about that.

A few questions:

1. Am I unusual in this? Do other people here find it difficult to read polemical or condescending writing, and/or do you find that the style makes it difficult for you to take the content as seriously as you perhaps should?
2. Are there any studies you're aware of on how people react to polemical writing?
3. Are there some situations in which it actually does make sense to use the kind of intuitive heuristic I was using - i.e., if it's written in a polemical / insulting style then it's probably less credible? Or is this just a generally bad heuristic that I should try to get rid of entirely?
4. This is a topic I'm very interested in so I'd appreciate any other related comments or thoughts you might have.

I skip over those pragmatic problems because this post is not proposing a solution, but rather a measurement I find interesting. 

Immune response to the lipid nanoparticles themselves is very bad. It reduces the uptake of protein coding mRNA into the cells and diminishes vaccine efficacy. It also leads to most of the severe allergic reactions.

Spain.

That syntax is for the Markdown editor (enabled through your user settings). For the LW Docs editor, use >! to start your paragraph.

An alternate interpretation of the result is that Beta just didn't have much immune escape. That fact that it was crushed by Delta suggests that. The vaccine produced antibodies that were more tuned to Beta than those from the original vaccine, but the difference was slight, just a factor of 2. We won't know about OAS until we have a variant with real immune escape, which may well be Omicron. But I'm not worried about OAS because it's a lower order effect.