LessWrong 2.0 Reader

I think it's still valid to ask in the abstract whether RL is a particularly dangerous approach to training an AI system.

To test whether Drake’s circumvention of his short-term memory loss worked via the intended mechanism, I could ask my girlfriend in advance to prompt me once — and only once — to complete the long-term memory scene that I had been practicing. Then I could see if I have a memory of the scene after I fully regain my memory.

You should have her decide (and write down) what to encode in advance, so that you can check later not just if you remembered something, but if you successfully encoded it in such a way that you communicate what you intend to communicate to yourself.

(Since Drake managed to send a memory, but was only guessing about what it was intended to mean.)

I listened to part of “Processor clock speeds are not how fast AIs think”, but I was disappointed by the lack of a human narrator. I am not interested in machine readings; I would prefer to go read the article.

That's true; I misremembered that part when I wrote it. I'll just remove that.

The direction extracted using the same method will vary per layer, but this doesn’t mean that the correct feature direction varies that much, but rather that it cannot be extracted using a linear function of the activations at too early/late layers.

InDesign! Or anything on page layout / publishing / how to make a pretty and well formatted book

shut your phone off

Leave phones elsewhere, remove batteries, or faraday cage them if you're concerned about state-level actors:

https://slate.com/technology/2013/07/nsa-can-reportedly-track-cellphones-even-when-they-re-turned-off.html

Compared to what?

If you want an agentic system (and I think many humans do, because agents can get things done), you've got to give it goals somehow. RL is one way to do that. The question of whether that's less safe isn't meaningful without comparing it to another method of giving it goals.

The method I think is both safer and implementable is giving goals in natural language, to a system that primarily "thinks" in natural language. I think this is markedly safer than any RL proposal anyone has come up with so far. And there are some other options for specifying goals without using RL, each of which does seem safer to me:

Goals selected from learned knowledge: an alternative to RL alignment [AF · GW]

I do think being one syllable is pretty valuable. Although AI Org watch might be fine (kinda rolls off the tongue worse)