LessWrong 2.0 Reader

I didn't say he wasn't overrated. I said he was capable of physics.

Did you read the linked post? Bohm, Aharonov, and Bell misunderstood EPR. Bohm's and Aharonov's formulation of the thought experiment is easier to "solve" but does not actually address EPR's concern, which is that mutual non-commutation of x-, y-, and z-spin implies hidden variables must not be superfluous. Again, EPR were fine with mutual non-commutation, and fine with entanglement. What they were pointing out was that the two postulates don't make sense in each other's presence.

That’s interesting! What would be some examples of axioms and theorems that describe a directed tree? 

Ok, probably a silly idea but... Maybe have some kind of competition for young people involving something like math / computer science / essay writing / puzzle solving / jailbreaking LLMs... Give a cash prize for the top three, and send books to a bunch of the runners up.

Sounds like something that would take a lot of organization effort, so somebody would need to be excited enough about this idea to want to spearhead it.

Bayesian decision system

Why would you assume AGI will use Byaesian decision system? Such system would be limited to known probabilities. Unknown probability = 0 probability is not intelligent (Hitchens's razor, Black swan theory, Fitch's paradox of knowability). Once you incorporate this, Orthogonality Thesis [? · GW] is no longer valid - it becomes obvious that every intelligent AI will only work in single direction (which is disastrous to humans). I know there is a huge gap between "unknown probabilities" and "existential risk", you can find more information in my posts and I am available to explain it verbally (calendly link below). Short teaser - it is possible that terminal value can also be discovered (not only assumed).

Yes!

Do the images load now?

I can only see the image of the 5-d random walk. The other images aren't rendering.

We also have 6k more copies (18k hard-cover books) left. We have no idea what to do with them. Suggestions are welcome.

Here's a map of Russian libraries that requested copies of HPMOR, and we've sent 2126 copies to:

Sending HPMOR to random libraries is cool, but I hope someone comes up with better ways of spending the books.

so we can control values by controlling data.

What do you mean? As in you would filter specific data from the posttraining step? What would you be trying to prevent the model from learning specifically?

We need to split "search" into more fine-grained concepts.

For example, "model has representation of the world and simulates counterfactual futures depending of its actions and selects action with the highest score over the future" is a one notion of search.

The other notion can be like this: imagine possible futures as a directed tree graph. This graph has set of axioms and derived theorems describing it. Some of the axioms/theorems are encoded in model. When model gets sensory input, it makes 2-3 inferences from combination of encoded theorems + input and selects action depending on the result of inference. While logically this situation is equivalent to some search over tree graph, mechanistically it looks like "bag of heuristics".