LessWrong 2.0 Reader

Thank you, my US employer uses Benevity too and I found Lightcone without any problem, probably because of this thread. My employer is doing a 2x match due to Giving Tuesday so my contribution got tripled!

I think causal closure of the kind that matters here just means that the abstract description (in this case, of the brain as performing an algorithm/computation) captures all relevant features of the the physical description, not that it has no dependence on inputs. Should probably be renamed something like "abstraction adequacy" (making this up right now, I don't have a term on shelf for this property). Abstraction (in)adequacy is relevant for CF I believe (I think it's straight-forward why?). Randomness probably doesn't matter since you can include this in the abstract description.

I wonder how much safer roads could be if human drivers were all a little more patient?

Depends on country. In my opinion, in Switzerland, the human drivers are already almost perfect. In Italy, I was surprised that most of them somehow manage to survive the day.

From today's perspective, Marx is just another old white cishet tech bro. (something something swims left)

I never expected that one day I would miss the old-style Marxists, but God forgive me, I do. We disagreed on many things, but at least we were able to have an intelligent debate.

Some thoughts on Rootclaim

Blunt, quick. Weakly held. 

The platform has unrealized potential in facilitating Bayesian analysis and debate.

Either

•   The platform could be a simple reference document
•  The platform could be an interactive debate and truthseeking tool
•  The platform could be a way to search the rootclaim debates

Currently it does none of these and is frustrating to me.

Heading to the site I expect:

• to be able to search the video debates
• to be able to input my own probability estimates to the current bayesian framework
• Failing this, I would prefer to just have a reference document which doesn't promise these

The problem of "humans hostile to humans" has two heavy tails: nuclear war and biological terrorism, which could kill all humans. A similar problem is the main AI risk: AI killing everyone for paperclips.

The central (and not often discussed) claim of AI safety is that the second situation is much more likely: it is more probable that AI will kill all humans than that humans will kill all humans. For example, by advocating for pausing AI development, we assume that the risks of nuclear war causing extinction are less than AI extinction risks.

If AI is used to kill humans as just one more weapon, it doesn't change anything stated above until AI evolves into an existential weapon (like a billion-drone swarm).

Thanks for pointing me to Figure 12, it alleviates my concern! I don't fully agree with RMU being a stand-in for ascent based methods. Targeted representation noising (as done in RMU) seems easier to reverse than loss maximization methods (like TAR). Finally, just wanted to clarify that I see SSD/Potion more as automated mechanistic interpretability methods rather than finetuning-based. What I meant to say was that adding some retain set finetuning on top (as done for gradient routing) might be needed to make them work for tasks like unlearning virology.

Using almost the same training parameters as above (I used full batch and train_frac=0.5 to get faster & more consistent grokking, but I don't think this matters here)

I did a few runs and the results all looked more or less like this. The training process of such toy models doesn't contain so many bits of interesting information, so I wouldn't be surprised if a variety of different metrics would capture this process in this case. (E.g. the training dynamics can be also modelled by an HMM, see here). 

Shameless self promotion: this one https://www.lesswrong.com/posts/ASmcQYbhcyu5TuXz6/llms-could-be-as-conscious-as-human-emulations-potentially [LW · GW]

It circumvents object level question and instead looks at epistemic one.

This one is about broader direction in "how the things that happened change attitudes and opinions of people"

https://www.astralcodexten.com/p/sakana-strawberry-and-scary-ai

This one too, about consciousness in particular

https://dynomight.net/consciousness/

I think it's somewhat productive direction explored in these 3 posts, but it's not like very object level, more about epistemics of it all. I think you can look up how like LLM states overlap / predict / correspond with brain scans of people who engage in some tasks? I think there were a couple of paper on that.

E.g. here https://www.neuroai.science/p/brain-scores-dont-mean-what-we-think

What is the next step in this direction? Neuralink? I wonder what horrors it will bring.