LessWrong 2.0 Reader

Meta: When should I used this rather the shortform? Do we really need both?

Huh, I thought that all amplification/distillation procedures were intended as a way to approximate HCH, which is itself a tree. Can you not meaningfully discuss "this amplification procedure is like an n-depth approximation of HCH at step x", for any amplification procedure?

For example, the internal structure of the distilled agent described in Christiano's paper is unlikely to look anything like a tree. However, my (potentially incorrect?) impression is that the agent's capabilities at step x are identical to an HCH tree of depth x if the underlying learning system is arbitrarily capable.

It's possible that I'm not understanding the difference between "depth", "tree-based" and "recursion" in this context

Huh, what would you recommend I do to reduce my uncertainty around meta-execution (e.g. "read x", "ask about it as a top level question", etc)?

I agree, the "big, vague things" are the bedrock of epistemology, the lens which helps you be more discerning and critical when you read anything else (e.g. those more "hands on" materials) and get more value out of it.

I think that the sequences could be rewritten to half the length and still retain the vast majority of the value, but oh well, this is where we are at the moment with core rationality literature.

HPMOR is especially fun if you've read the original HP.

Yeah, your goal is not always to deeply understand the material you're looking at, and the post only applies when it is.

Yes, great correction. I've modified the post to state that it only applies for Deep Things You Truly Want To Understand.

I'm often confronted with the difference you're describing but haven't ever articulated it as you just have.

I maybe think so? I'm not sure about meta-execution (as the comment thread above shows).

Agreed. The comment was meant to be referring to orgs that made particular sense as a monopoly)

"Depth" only applies to the canonical tree-based implementation of IDA. If you slot in other amplification or distillation procedures, then you won't necessarily have "depths" any more. You'll still have recursion, and that recursion will lead to more and more capability. Where it ends up depends on your initial agent and how good the amplification and distillation procedures are.

If you want to challenge a monopoly with a new org, there's likewise a particular burden to do a good job.

(This seems to depend on whether the job/project in question benefits from concentration.)