LessWrong 2.0 Reader

Sorry for writing two comments, but I'm really curious of some ground beliefs, because it feels a little silly to talk about specific policy proposals without mentioning the - in my mind - biggest reason not to vote for Trump: he tried to steal the last election.

More specifically, he said after the election that the election was rigged against him. He had a bunch of court cases which sought to prove that the election was rigged against him, and pretty much every case was proven false. Then when his vice president were to certify electors from the state, Trump wanted him to use fake electors, which were not sent from the states, to say that Trump won the election. If his plan would have worked, then Trump would have stolen the election. But Pence did not do that, because Pence didn't want to steal the election. He later said Trump "demanded that I choose between him and the constitution, [..] I chose the constitution". That's why the crowd was yelling "hang Mike Pence" during Jan 6.

Do you care about that?

You could instead pay me $10k now, with the understanding that I'll pay you $20k later in 2028 unless AGI has been achieved in which case I keep the money... but then why would I do that when I could just take out a loan for $10k at low interest rate?

We could set up the bet such that it would involve you losing/gaining no money in expectation under your views, whereas you would lose money in expectation with a loan? Also, note the bet I proposed above was about ASI as defined by Metaculus, not AGI.

This all assumes that the AI is an entity that is capable of being negotiated with.

You can't negotiate with a hurricane, an earthquake, a tsunami, an incoming asteroid, or a supernova. The picture I have always had of "AI doom" is a disaster of that sort, rather than rival beings with vastly superior abilities. That is also a possibility, but working out how ants might negotiate with humans looks like a small part of the ant survival problem.

Not quite following - your possibilities.
1. Alignment is almost impossible, then there is say 1e-20 chance we survive. Yes surviving worlds have luck and good alignment work etc. Perhaps you should work on alignment or still bednets if the odds really are that low.

2. Alignment is easy by default, but there is nothing like 0.999999 we survive, say 95% because AGI that is not TAI superintelligence could cause us to wipe ourselves out first, among other things. (This is a slow takeoff universe(s))

#2 has much more branches in total where we survive (not sure if that matters) and the difference between where things go well and badly is almost all about stopping ourself killing ourselves with non TAI related things. In this situation, shouldn't you be working on those things?

If you average 1,2 then you still get a lot of work on non-alignment related stuff.

I believe its somewhere closer to 50/50 and not so overdetermined one way or the other, but we are not considering that here.

I might go so far as to say it Pareto dominates most people’s agendas on importance and tractability. While being pretty neglected.

(Others had already written on these/related topics previously, e.g. https://www.lesswrong.com/posts/r3xwHzMmMf25peeHE/the-translucent-thoughts-hypotheses-and-their-implications [LW · GW], https://www.lesswrong.com/posts/FRRb6Gqem8k69ocbi/externalized-reasoning-oversight-a-research-direction-for [LW · GW], https://www.lesswrong.com/posts/fRSj2W4Fjje8rQWm9/thoughts-on-sharing-information-about-language-model#Accelerating_LM_agents_seems_neutral__or_maybe_positive_ [LW · GW], https://www.lesswrong.com/posts/dcoxvEhAfYcov2LA6/agentized-llms-will-change-the-alignment-landscape [LW · GW], https://www.lesswrong.com/posts/ogHr8SvGqg9pW5wsT/capabilities-and-alignment-of-llm-cognitive-architectures [LW · GW], https://intelligence.org/visible/.)   

I've made some attempts over the past year to (at least somewhat) raise the profile of this approach and its potential, especially when applied to automated safety research (often in conversation with / prompted by Ryan Greenblatt):

https://www.lesswrong.com/posts/HmQGHGCnvmpCNDBjc/current-ais-provide-nearly-no-data-relevant-to-agi-alignment#mcA57W6YK6a2TGaE2 [LW(p) · GW(p)]

https://www.lesswrong.com/posts/yQSmcfN4kA7rATHGK/many-arguments-for-ai-x-risk-are-wrong?commentId=KGPExCE8mvmZNQE8E [LW(p) · GW(p)] 

https://www.lesswrong.com/posts/yQSmcfN4kA7rATHGK/many-arguments-for-ai-x-risk-are-wrong?commentId=zm4zQYLBf9m8zNbnx [LW(p) · GW(p)] 

https://www.lesswrong.com/posts/yQSmcfN4kA7rATHGK/many-arguments-for-ai-x-risk-are-wrong?commentId=TDJFetyynQzDoykEL [LW(p) · GW(p)] 

I might write a high-level post at some point (which should hopefully help with the visibility of this kind of agenda more than the separate comments on various other [not-necessarily-that-related] posts).

This is not an obvious continuation of the prompt to me - maybe there are just a lot more examples of explicit refusal on the internet than there are in (e.g.) real life.

I'm definitely fine with not having Superman, but I'm willing to settle on him not intervening.

On a different note, I'd disagree that Superman, just by existing and being powerful, is a de facto ruler in any sense - he of course could be, but that would entail a tradeoff that he may not like (living an unburdened life).

I still think I'm right about this. Your conception (that not a genetically less smart sibling was born), is determined by quantum fluctuations. So if you believe that quantum fluctuations over the last 50 years make at most 2^-75 difference in the probability of alignment, that's an upper bound on how much a difference your life's work can make. While if you dedicate your life to buying bednets, it's pretty easily calculatable how many happy life-years do you save. So I still think it's incompatible to believe that the true quantum probability is astronomically low, but you can make enough difference that working on AI safety is clearly better than bednets.

Yeah I don’t want a Superman.

Thank you for your further clarifying response. I will not be continuing this discussion. Cheers.

Well the reason I didn't think of DEI statements and such is because that's not really something Trump talks about much, right? He mostly talks about immigration (cats and dogs!), Ukraine, inflation, etc. So I don't know much about it.

Also, I don't really see DEI statements as a restriction on "freedom of speech" or "freedom of conscience". If I understand correctly, it's that grants by NSF to institutions have to have a section explaining their DEI initiatives. And sure you can disagree if that should be a factor, and maybe you think it's stupid to have as a requirement, but just because they have different priorities/beliefs than you doesn't mean they're stifling freedom of speech.

Imagine if the government started caring a lot about AI and NSF required people to talk about their AI work/impact in every funding request. Would that be stupid? Yes, because a lot of important work isn't that relevant to AI. But this doesn't really impact freedom of speech, it just means they prioritize research differently than you would. The government thinks DEI should affect research grants. You don't. Someone will always feel left out when funding is allocated, and feel "forced" to act in some ways to receive funding. But I don't see that as a restriction on "freedom of speech".

When it comes to burning the flag, maybe it's a bad example. In my mind, burning the flag just means hating on America and I don't think that should be illegal. But some people see burning the flag like destroying something holy, so they treat that restriction differently than other restrictions. But it's still a more clear restriction than DEI statements. It's an actual law that restricts how you are allowed to express yourself in America.

But when it comes to infiltration of institutions, I find it worrying when Vance says:

Fire every single midlevel bureaucrat, every civil servant in the administrative state, replace them with our people.

What does he mean with "our people"? Would it be a restriction on freedom of speech to fire every civil servant and replace them with "our people"? I think it would. Surely he doesn't mean people who are critical of Trump as "our people".

Also, I think removing the right to abortion from millions of people is a much clearer restriction of liberty than DEI statements or banning flag-burning.