LessWrong 2.0 Reader

For example, a researcher I've been talking to, when asked what they would need to update, answered, "An AI takes control of a data center." This would be probably too late.

Very much to avoid, but I'm skeptical it 'would be probably too late' (especially if I assume humans are aware of the data center takeover); see e.g. from What if self-exfiltration succeeds?:

'Most likely the model won’t be able to compete on making more capable LLMs, so its capabilities will become stale over time and thus it will lose relative influence. Competing on the state of the art of LLMs is quite hard: the model would need to get access to a sufficiently large number of GPUs and it would need to have world-class machine learning skills. It would also mean that recursive self-improvement is already possible and could be done by the original model owner (as long as they have sufficient alignment techniques). The model could try fine-tuning itself to be smarter, but it’s not clear how to do this and the model would need to worry about currently unsolved alignment problems.'

A 10-year global pause would allow for a lot of person-years-equivalents of automated AI safety R&D. E.g. from Some thoughts on automating alignment research [LW · GW] (under some assumptions mentioned in the post): 'each month of lead that the leader started out with would correspond to 15,000 human researchers working for 15 months.' And for different assumptions the numbers could be [much] larger still: 'For a model trained with 1000x the compute, over the course of 4 rather than 12 months, you could 100x as many models in parallel.[9] You’d have 1.5 million researchers working for 15 months.'

This would probably obsolete all previous AI safety R&D.

Of course, this assumes you'd be able to use automated AI safety R&D safely and productively. I'm relatively optimistic that a world which would be willing to enforce a 10-year global pause would also invest enough in e.g. a mix of control and superalignment to do this.

There's also an entire literature of variations of [e.g. sparse or disentangled] autoencoders and different losses and priors that it might be worth looking at and that I suspect SAE interp people have barely explored; some of it literally decades-old. E.g. as a potential starting point https://lilianweng.github.io/posts/2018-08-12-vae/ and the citation trails to and from e.g. k-sparse autoencoders.

Trying to Disambiguate Different Questions about Whether Humans are Turing Machines

I often hear the sentiment that humans are Turing machines, and that this sets humans apart from other pieces of matter.

I've always found those statements a bit strange and confusing, so it seems worth it to tease apart what they could mean.

The question "is a human a Turing machine" is probably meant to convey "can a human mind execute arbitrary programs?", that is "are the languages the human brain emit at least recursively enumerable?", as opposed to e.g. context-free languages.

1. My first reaction is that humans are definitely not Turing machines, because we lack the infinite amount of memory the Turing machine has in form of an (idealized) tape. Indeed, in the Chomsky hierarchy human aren't even at the level of pushdown automata, instead we are nothing more than finite state automata. (I remember a professor pointing this out to us that all physical instantiations of computers are merely finite-state automata).
   1. Depending on one's interpretation of quantum mechanics, one might instead argue that we're at least nondeterministic finite automata or even Markov chains. However, every nondeterministic finite automaton can be transformed into a deterministic finite automaton, albeit at an exponential increase in the number of states, and Markov chains aren't more computationally powerful (e.g. they can't recognize Dyck languages, just as DFAs can't).
   2. It might be that Quantum finite automata are of interest, but I don't know enough about quantum physics to make a judgment call.
2. The above argument only applies if we regard humans as closed systems with clearly defined inputs and outputs. When probed, many proponents of the statement "humans are Turing machines" indeed fall back to a motte that in principle a human could execute every algorithm, given enough time, pen and paper.
   1. This seems true to me, assuming that the matter in universe does not have a limited amount of computation it can perform.
      1. In a finite universe we are logically isolated [LW(p) · GW(p)] from almost all computable strings, which seems pretty relevant.
      2. Another constraint is from computational complexity; should we treat things that are not polynomial-time computable as basically unknowable? Humans certainly can't solve NP-complete problems efficiently.
   2. I'm not sure this is a very useful notion.
      1. On the one hand, I'd argue that, by orchestrating the exactly right circumstances, a tulip could receive specific stimuli to grow in the right directions, knock the correct things over, lift other things up with its roots, create offspring that perform subcomputations &c to execute arbitrary programs. Conway's Game of Life certainly manages to! One might object that this is set up for the tulip to succeed, but we also put the human in a room with unlimited pens and papers.
      2. On the other hand, those circumstances would have to be very exact, much more so than with humans. But that again is a difference in degree, not in kind.

After all, I'm coming down with the following conclusion: Humans are certainly not Turing machines, however there might be a (much weaker) notion of generality that humans fulfill and other physical systems don't (or don't as much). But this notion of generality is purported to be stronger than the one of life:

$$\text{Turing-completeness}\Rightarrow \text{Context-sensitive}\Rightarrow \text{Context-free}\stackrel{?}{\Rightarrow }\text{Proposed-generality}\stackrel{?}{\Rightarrow }\text{Life}\stackrel{?}{\Rightarrow }\text{Finite-state automata}$$

I don't know of any formulation of such a criterion of generality, but would be interested in seeing it fleshed out.

Agree with the recommendation of using such websites.

I've found them to be very effective, especially for highly aversive medium-term tasks like applying for jobs, or finishing a project. Five times I wanted a thing to happen, but was really procrastinating on it. And five times pulling out the big guns (beeminder) got it done.

I haven't tried using them for long-term commitments, since my intuition is that using them is highly coercive towards subagents which then further entrench their opposition. So I've used these apps sparingly. Maybe I'll still give it a try.

I think these apps work best if really aversive tasks are set to a estimated-bare-minimum commitment, which in reality is probably the median amount of what one can get done.

Here's another tiny Windows Firefox bug report.

1. Expand any song.
2. Hit play.
3. Collapse the song.

The song keeps playing. Would've expected the music to stop when the video player wasn't visible.

This google search seems to turn up some interesting articles (like maybe this one, though I've just started reading it).

Paul [Christiano] called this “problems of the interior” somewhere

Since it's slightly hard to find: Paul references it here (ctrl+f for "interior") and links to this source (once again ctrl+f for "interior").  Paul also refers to it in this [AF · GW] post.  The term is actually "position of the interior" and apparently comes from military strategist Carl von Clausewitz.

Like, are you saying the mathematical model of true stochasticity (with some "one thing is real" formalization) is somehow incomplete or imprecise or wrong, because mathematics is deterministic?

Which model are you talking about here?

It makes sense to very legibly one-box even if Omega is a very far from perfect predictor. Make sure that Omega has lots of reliable information that predicts that you will one-box.

Then actually one-box, because you don't know what information Omega has about you that you aren't aware of. Successfully bamboozling Omega gets you an extra $1000, while unsuccessfully trying to bamboozle Omega loses you $999,000. If you can't be 99.9% sure that you will succeed then it's not worth trying.