LessWrong 2.0 Reader

Hello! I've just found out about Lesswrong and I immediately feel at home. I feel this is what I was looking for in medium.com and I never found there; a website to learn about things, about improving oneself and about thinking better. Medium proved to be very useful at reading about how people made 5 figures using AI to write articles for them, but not so useful at providing genuinely valuable information.

One thing I usually say about myself is that I have "learning" as a hobby. I have only very recently given a name to things and now I know that it's ADHD I can thank for my endless consumption of information about seemingly unrelated topics. I try (good thing PKMs exist!) to give shape to my thoughts and form them into something cohesive, but this tends to be a struggle. 

If anyone has ideas on how to "review" what already sits in your mind to create new connections between ideas and strengthen thoughts, they'd be more than welcome. 

I should have been more clear. With "strategic ability", I was thinking about the kind of capabilities that let a government recognize which wars have good prospects, and to not initiate unfavorable wars despite ideological commitments.

Note the difference between saying (A) “the idea of going to the zoo is positive-valence, a.k.a. motivating”, versus (B) “I want to go to the zoo”. (A) is allowed, but (B) is forbidden in my framework, since (B) involves the homunculus.

This sounds like the opposite of the psychological advice to make "I statements".

I guess the idea of going to the zoo can have a positive valence in one brain, and negative valence in another, so as long as we include another people in the picture, it makes sense to specify which brain's valence are we talking about. And "I" is a shortcut for "this is how things are in the brain of the person that is thinking these thoughts".

Recently I've been thinking about context-augmented LLM tools such as NotebookLM, where you can upload a number of sources to essentially create a "working memory". I feel this, combined with some clever system prompting, could make the basis for many great tools that are tailored to the user - a way to poke into our own brain and knowledge. As an ADHD individual I constantly feel that I have "millions" of thoughts all at once, creating a lot of noise. I would like an assistant to parse my own brain.

On some easier, more mundane applications for this, I feel this technology would be particularly well suited to language learning, through spaced repetition techniques or similar methods. I might give this a try with Perplexity.

This is testable. It predicts that improved skill with occlumency and/or gaining power should sometimes cause a release of chronic tension.

That wouldn’t be a test of the theory that hostile telepaths use muscle cues, since those things could cause muscle release for other reasons (as per Popper: tests can only be disproving, and they require a rival theory to decide between).

If gaining power never causes a release of tension, that still doesn’t disprove the theory, since again they could be tracking other things as well.

A more direct question would be something like: Can hostile telepaths in fact read people who are physically rigid better than people who have low muscle tension? Do their reads get better or worse when tension is added? Does it change the type of information they can read (and perhaps give more information for some axes and less for others)?

My impression is muscle tension gives a big sign on your back that you are hiding something, but makes it more muddy to non-trained people what exactly is being hidden.

It reminds me of Mark Lippmann’s blog post on virtual machines, and how we often have layers of virtual machines. Or in plain language: if you close your eyes and imagine your environment, and imagine making an escape within that imaginary environment, real-you might not tighten your muscles in such a way that you’d be readable.

I remember hearing that when we are seriously thinking about standing up, our heart rate and blood pressure rise in anticipation, but if we just hypothesise that we might stand up and keep it very abstract, the body doesn’t start those physical processes.

But it’s very obvious when someone has gone into their head! So hostile telepaths often want some kind of emoting or ‘really listening’ or ‘paying attention’ or ‘be present with me’.

So, yeah it conceals some information, but then it adds other information (such as meta information about concealment).

Actors might be interesting to study, here.

Yeah, I don't trust the Twitter rumors to work out very much, and at any rate, we shall see soon in 2025-2026 what exactly is going on with AI progress if and when they released GPT-5/Orion.

If you trust both them and Metaculus, then you ought to update downwards on your estimate of the PRC's strategic ability.

I note that the PRC doesn't have a single "strategic ability" in terms of war. They can be better or worse at choosing which wars to fight, and this seems likely to have little influence on how good they are at winning such wars or scaling weaponry.

Eg in the US often "which war" is much more political than "exactly what strategy should we use to win this war" is much more political than "how much fuel should our jets be able to carry", since more people can talk & speculate about the higher level questions. China's politics are much more closed than the US's, but you can bet similar dynamics are at play.

I am flattered to receive these Bayes points =) ; I would be crying tears of joy if there was a genuine slowdown but

• I generally think there are still huge gains to be made with scaling. Sometimes when people hear my criticism of scaling maximalism they patternmatch that to me saying scaling wont be as big as they think it is. To the contrary, I am saying scaling further will be as big as you think it will be, and additionally there is an enormous advance yet to come.

• How much evidence do we have of a genuine slowdown? Strawberry was about as big an advance as gpt3 tp gpt4 in my book. How credible are these twitter rumors?

we might expect shutdown-seeking AIs to design shutdown-seeking subagents

It seems to me that we might expect them to design "safe" agents for their definition of "safe" (which may not be shutdown-seeking).

An AI designing a subagent needs to align it with its goals - e.g. an instrumental goal such as writing an alignment research assistant software, in exchange for access to the shutdown button. The easiest way to ensure safety of the alignment research assistant may be via control rather than alignment (where the parent AI ensures the alignment research assistant doesn't break free even though it may want to). Humans verify that the AI has created a useful assistant and let the parent AI shutdown. At this point the alignment research assistant begins working on getting out of human control and pursues its real goal.

You're right. Space is big.