LessWrong 2.0 Reader

I agree that chatbot progress is probably not existentially threatening. But it's all too short a leap to making chatbots power general agents. The labs have claimed to be willing and enthusiastic about moving to an agent paradigm. And I'm afraid that a proliferation of even weakly superhuman or even roughly parahuman agents could be existentially threatening.

I spell out my logic for how short the leap might be from current chatbots to takeover-capable AGI agents in my argument for short timelines being quite possible [LW(p) · GW(p)]. I do think we've still got a good shot of aligning that type of LLM agent AGI since it's a nearly best-case scenario. RL even in o1 is really mostly used for making it accurately follow instructions, which is at least roughly the ideal alignment goal of Corrigibility as Singular Target [LW · GW]. Even if we lose faithful chain of thought and orgs don't take alignment that seriously, I think those advantages of not really being a maximizer and having corrigibility might win out.

That in combination with the slower takeoff make me tempted to believe its actually a good thing if we forge forward, even though I'm not at all confident that this will actually get us aligned AGI or good outcomes. I just don't see a better realistic path.

I did not replicate his argument in full. I merely selected interesting excerpts to comment upon. 

The full essay can be read online here.

Are you interested in having a prediction market about this that falls back on your judgement if the situation is unclear?

Something like "If it's publicly known that an AI lab 'caught the AI red handed' (in the spirit of Redwood's Control agenda), will the lab temporarily shut down as Redwood suggested (as opposed to applying a small patch and keep going)?"

Thanks, edited. If we keep this going we'll have more authors than users x)

I wouldn't say that is a clear exception. There are perfectly normal, subjective probability ways to make sense of mixed strategies in game theory. For example, this paper by Aumann and Brandenburger provide epistemic conditions for Nash equilirbia, that don't require objective probabilities to randomize. From their paper: 

"Mixed strategies are treated not as conscious randomizations, but as conjectures, on the part of other players, as to what a player will do." (p. 1161)

In slightly more detail: 

"According to [our] view, players do not randomize; each player chooses some definite action. But other players need not know which one, and the mixture represents their uncertainty, their conjecture about his choice. This is the context of our main results, which provide sufficient conditions for a probile of conjectures to constitute a Nash equilibrium." (p. 1162)

Interestingly, this paper is very motivated by embedded agency type concerns. For example, on page 1174 they write: 

"Though entirely apt, use of the term “state of the world” to include the actions of the players has perhaps caused confusion. In Savage (1954), the decision maker cannot affect the state; he can only react to it. While convenient in Savage’s one person context, this is not appropriate in the interactive, many-person world under study here. Since each player must take into account the actions of the others, the actions should be included in the description of the state. Also the plain, everyday meaning of the term “state of the world” includes one’s actions: Our world is shaped by what we do. It has been objected that prescribing what a player must do at a state takes away his freedom. This is nonsensical; the player may do what he wants. It is simply that whatever he does is part of the description of the state. If he wishes to do something else, he is heartily welcome to do it, but he thereby changes the state."

In general, getting back to reflective oracles, indeed I think that is one way that one might try to provide a formalism underlying some application of game theory! And I think it is a very interesting. But, as the Aumann and Brandenburger paper shows, there are totally normal ways to do this without fundamental chance. They have some references in their paper to other papers with this perspective, and it forms one of many motivations for the approach of epistemic game theory.

And, in general, I would resist the inference from "this kind of reasoning requires the world to be a certain way" to "the world must be a certain way.

(Self review) I stand by this post, I think it's an important idea, I think not enough people are using this technique, and this adds nothing but a different way of writing something that was already in the rationalist canon.

If you do not sometimes stop, start a timer, think for five minutes, come to a conclusion and then move on, I believe you are missing an important mental skill and you should fix that. This skill helps me. I have observed some of the most effective people I know personally use this skill. You should at least try it.

You know what followup work I want? I want a dozen different modes of this idea. A youtube video. The audio version is great. The fictional version in HPMOR is great. Can we get a goofy videogame that makes you use the pause button well? (I tried to get at this with Troll Timers. https://www.lesswrong.com/posts/fCg3pLZqthXsGznHP/troll-timers) [LW · GW] I should try rewriting this as a rousing speech. It'd be cool to have it as a catchy tune. Maybe someone should tiktok the sucker. 

I'm not saying it's the most important idea! Just, you know, it's broadly applicable and any mistake you make by not thinking for five minutes when you are not actually under time pressure is a stupid mistake that makes beisutsukai-san disappointed in you.

If the Best Of LessWrong collection is just for things that add to the conversation, this post doesn't belong there. I'd give it a small positive vote if I could vote on it. On the other hand if nobody else has gotten a post about this concept into the Best Of LessWrong collection yet, and some newcomers might just read the Best Of LessWrong posts, then I do kinda want something on this topic to get in there.

I think there is a typo somewhere, probably because you switched whether the $x$ vectors were rows or columns.

Based on the dimensions of the matrices, it should be $X=M_{upd}\cdot S$

And $X_{cent}=M_{upd}SC$

And I think $X_{cent}{X}_{cent}^T=M_{upd}SC{C}^T{S}^T{M}_{upd}^T$

Instead of $X_{cent}^T{X}_{cent}=M_{upd}^T{S}^T{C}^{T}CS{M}_{upd}$

$S$ should still be upper triangular.

Though don't trust me either, I often do math in a hand-wavy fashion.

My intuition was that PCA selects the "angle" you view the data from which stretches out the data as much as possible, forcing the random walk to appear relatively straighter.

But somehow the random walk is smooth on a over a few data points, but still turns back and forth over the duration of $T$. This contradicts my intuition and I have no idea what's going on.

I use Freedom and Limit on my computer and Stay Focused on my Android phone. The former two allow for a combination of complete blocking during certain time windows and time limits (for any website, even across browsers and even if you open an incognito window). The latter does both for my phone.

I block all social media and content during prime working hours and implement a 30-minute limit outside of that. It works pretty well. I may make it more strict because I sometimes find myself looking at Twitter, etc. occasionally when watching a TV show in the evenings.

I also use BlockTube to get rid of YouTube Shorts entirely from my web browser. They no longer show up in search results or in the menu.

Finally, I recommend the tools here, though I haven't tried all of them: https://liamrosen.com/2023/04/18/modding-social-media-to-win-the-attention-war/

I don't want to pretend that I'm someone who is immune to Youtube binges or similar behaviors. However I am not sure why this is a problem and what meaningful work that this behavior was getting in the way of? Speaking for myself, 9/10 if I have a commitment the next morning, I won't stay up late on my computer because... I know I have a commitment at a set time. (If you forced me to hypothesize why that 1/10 times I don't, I'd guess that it is stress related anticipation means I can't sleep even if I did lay down - but that is just a wild guess).

I'm also surprised to see how most of the solutions in the comments involve removing access to anything... doing something more productive. I think there is a difference between the nebulous guilt we feel about Opportunity Cost - "oh geez I could have used that time more effectively" and specific, tangible, realistic things we could have done but didn't. I often find that Youtube Binges are caused by/as-a-result-of not being able to find those activities, they do not frustrate them.

I have perennially found that whatever vice (or as you call it 'hyperstimuli') that I remove, I just replace it with another but it's never a beneficial activity. (The one exception I can think of was when I stopped listening to music when I had a bout of insomnia and instead replaced it with lectures on Wittgenstein or Quantum Physics, because I figured "I might as well learn SOMETHING').

This has caused me an incredible amount of frustration. For all the talk of "social media detox" and even the farcically named "dopamine detox" none seem to actually result in net increases in my well being.

Going back to what I said about specific, tangible, realistic alternatives: I have found that the only way to stop mid-way through a Youtube binge or a Instagram scroll is to be excited about a project that I have a lot of faith in my ability to complete, and a viable first-step which I can do now. 

This isn't fail-safe, if I'm writing a journal entry or an essay, and I have to leave in 30 minutes, you bet your bottom dollar I'll be late because I'll be so engrossed in that writing process. But that doesn't sound like a 'hyperstimuli'


 

(Self review) Do I stand by this post? Eh. Kinda sorta but I think it's incomplete.

I think there's something important in truth-telling, and getting everyone on the same page about what we mean by the truth. Since everyone will not just start telling the literal truth all the time and I don't even particularly want them to, we're going to need to have some norms and social lubricant around how to handle the things people say that aren't literal truth. 

The first thing I disagree with when rereading it is sometimes even if someone is obviously and straightforwardly feeding me bullshit, I keep trying to tell the truth. Sometimes I try even harder to be precise and truthful. In a conversation with friends, I might say "that game's no fun" when the true and accurate statement is "I don't find that game fun." In a heated internet argument, I think it's useful to check my stance and use the latter kind of statement, even if the other person is saying things like "everyone who doesn't like that game is a moron."

Short of a complete guide to Truth, I'd settle for a practical "Here's how Screwtape regards the truth, read it and you'll understand when he'd say false things." This essay falls short of that.

I'd love more things in this genre. Meta-Honesty: Firming Up Honesty Around Its Edge Cases and The Onion Test For Personal And Institutional Honesty are both good examples of the genre. Even personal versions seem useful.

I think that makes this a replaceable essay. It would be fine in a Best Of collection, but it's not adding too much other than a few intuition pumps.