LessWrong 2.0 Reader

My (admittedly limited) knowledge of psychology and neurosciences suggests that this is not currently possible. Thankfully.

I feel like if you start seriously considering things that are themselves almost as bad as AI ruin in their implications in order to address potential AI ruin, you took a wrong turn somewhere.
If you can create a virus or something of the sort that makes people genuinely afraid of some vague abstract thing, you can make them scared of anything at all. Do I really need to spell it out how that would be abused? 
On the other hand, do you really need to go that far?
Launch media campaign and you can get most of the same results without making the world much more dystopian than it already is.
The main risk here is that it's easy to scare people so much that all of the research gets shut down. And I expect that to be the reason there's not much scare about it in media yet. As far as I remember, that's why most researcher in the field were at first reluctant to admit there's a risk at all.

What do we know about the nature of infection of vaccinated people, or re-infection of people who have recovered from Covid? It seems to me, this will have a big impact on Covid epidemiology in a mostly-vaccinated country. 

I have three mental models:

1. Immunocompromise: some people will never build up Covid resistance, no matter how much they get vaccinated or infected
2. Exposure: a person has resistance, but is exposed to Covid in a way that overwhelms them. High viral load, maybe, or their immune system is somehow having a bad day
3. Vaccination failure: for some reason the vaccination doesn't 'take'. If the sufferer catches Covid, they will build up more resistance and be better protected next time round

[I'm leaving out the impact of variants and the decline of immunity over time]

(1) means that we'll permanently have some segment of the population where Covid circulates, but could conceivably identify and protect those people

(2) is the same, but with less we can do in reaction.

(3) would imply things getting gradually better over time, as those people are exposed to endemic covid. Booster shots might help speed this along

Does anybody have a sense of which of these models (or something else) is closer to reality?

The fish-weight example was intuitive for me, but the temperature one wasn't. Slightly reformulating the thoughts in my head:

1. of course temperature measurement is local
2. that's what temperature is, I don't care about the many possible distributions, only about the current local sample. That's what's affecting things around me, not some hypothetical distribution that isn't instantiated right now.

Maybe you wanted to make a different point here, and I didn't get it?

Sample-mean is not distribution-mean.

This is my key-takeaway from this post, thank you for writing it.

I believe the upper right-hand corner of $a$ shouldn't be 1; even if both players are acting in each other's best interest, they are not acting in their own best interest. And alignment is about having both at the same time. The configuration of Prisoner's dilemma makes it impossible to make both players maximally satisfied at the same time, so I believe it cannot have maximal alignment for any strategy. 

Anyhow, your concept of alignment might involve altruism only, which is fair enough. In that case, Vanessa Kosoy has a similar proposal to mine, but not working with sums, which probably does exactly what you are looking for.

Getting alignment in the upper right-hand corner in the Prisoner's dilemma matrix to be 1 may be possible if we redefine $u(A,B)$ to $u(A,B)={max}_{u,v}{u}^T(A+B)v$, the best attainable payoff sum. But then zero-sum games will have maximal instead of minimal alignment! (This is one reason why I defined $u(A,B)={max}_{u,v}{u}^{T}Av+{max}_{u,v}{u}^{T}Bv$.) 
 

(Btw, the coefficient isn't symmetric; it's only symmetric for symmetric games. No alignment coefficient depending on the strategies can be symmetric, as the vectors can have different lengths.)






 

I'll take a shot at this. Let $A$ and $B$ be the sets of actions of Alice and Bob. Let $o_n:B\to \{1,...n\}$ (where 'n' means 'nice') be function that orders $B$ by how good the choices are for Alice, assuming that Alice gets to choose second. Similarly, let $o_s:B\to \{1,...,n\}$ (where 's' means 'selfish') be the function that orders $B$ by how good the choices are for Bob, assuming that Alice gets to choose second. Choose some function $\psi$ measuring similarity between two orderings of a finite set (should range over $[-1,1]$); the alignment of $B$ with $A$ is then $\psi (o_n,o_s)$.

Example: in the prisoner's dilemma, $B=\{c,d\}$, and $o_n$ orders $c>d$ whereas $o_s$ orders $d>c$. Hence $\psi (o_n,o_m)$ should be $-1$, i.e., Bob is maximally unaligned with Alice. Note that this makes it different from Mykhailo's answer which gives alignment $0.5$, i.e., medium aligned rather than maximally unaligned.

This seems like an improvement over correlation since it's not symmetrical. In the game where Alice and Bob both get to choose numbers $x,y\in \{1,2\}$ and Alice's utility function outputs $y+x$ whereas Bob's outputs $y-x$, Bob would be perfectly aligned with Alice (his $o_n$ and $o_s$ both order $2>1$) but Alice perfectly unaligned with Bob (her $o_n$ orders $1>2$ but her $o_s$ orders $2>1$).

I believe this metric meets criteria 1,3,4 you listed. It could be changed to be sensitive to players' decision theories by changing $o_s$ (for alignment from Bob to Alice) to be the order output by Bob's decision theory, but I think that would be a mistake. Suppose I build an AI that is more powerful than myself, and the game is such that we can both decide to steal some of the other's stuff. If the AI does this, it leads to -10 utils for me and +2 for it (otherwise 0/0); if I do it, it leads to -100 utils for me because the AI kills me in response (otherwise 0/0). This game is trivial: the AI will take my stuff and I'll do nothing. Also, the AI is maximally unaligned with me. Now suppose I become as powerful as the AI and my 'take AI's stuff' becomes -10 for AI, +2 for me. This makes the game a prisoner's dilemma. If we both run UDT or FDT, we would now cooperate. If $o_s$ is the ordering of the AI's decision theory, this would mean the AI is now aligned with me, which is odd since the only thing that changed is me getting more powerful. With the original proposal, the AI is still maximally unaligned with me. More abstractly, game theory assumes your actions have influence on the other player's rewards (else the game is trivial), so if you cooperate for game-theoretical reasons, this doesn't seem to capture what we mean by alignment.

Not sure what you mean by that being unverifiable? The question says:

This question resolves as the total number of nuclear weapons (fission or thermonuclear) reported to be possessed across all states on December 31, 2022. This includes deployed, reserve/ nondeployed, and retired (but still intact) warheads, and both strategic and nonstrategic weapons.

Resolution criteria will come from the Federation of American Scientists (FAS). If they cease publishing such numbers before resolution, resolution will come from the Arms Control Association or any other similar platform.

FAS update their estimates fairly regularly - here are their estimates as of May (that link is also provided earlier in the question text).

Though I do realise now that they're extremely unlikely to update their numbers on December 31 specifically, and maybe not even in December 2022 at all. I'll look into the best way to tweak the question in light of that. If that's what you meant, thanks for the feedback! 

(I do expect there'll be various minor issues like that, and we hope the community catches them quickly so we can tweak the questions to fix them. This was also one reason for showing some questions before they "open".)

[APPRENTICE] I am curious about "Agency", but I would love to know more about what kind of exercises/messages you have in mind. I am passionate about self improvement and growth. I do catch myself occasionally not accepting some rational argument for emotional reasons or not working on the most impactful problem. I would love to improve this. Happy to switch to private messages, but I thought that others might have similar questions.

Aaaand the video's gone.

Edit: Available at either

https://odysee.com/@ReelNews:8/How-to-Save-the-World-in-Three-Easy-Steps,-Robert-Malone,-Steve-Kirsch,---Bret-Weinstein:5

https://odysee.com/@GatheringOfEagles:9/how-to-save-the-world-ivermectin:b

It sounds like you think that actual AGI won't happen? And your argument is that we don't have a convincing roadmap for how to get there?