LessWrong 2.0 Reader

It seems like you're assuming people won't build AGI if they don't have reliable ways to control it, or else that sovereign (uncontrolled) AGI would be likely the be friendly to humanity. Both seem unlikely at this point, to me. It's hard to tell when your alignment plan is good enough, and humans are foolishly optimistic about new projects, so they'll probably build AGI with or without a solid alignment plan.

So I'd say any and all solutions to corrigibility/control should be published.

Also, almost any solution to alignment in general could probably be used for control as well. And probably would be. See

https://www.lesswrong.com/posts/7NvKrqoQgJkZJmcuD/instruction-following-agi-is-easier-and-more-likely-than [LW · GW]

And

https://www.lesswrong.com/posts/587AsXewhzcFBDesH/intent-alignment-as-a-stepping-stone-to-value-alignment [LW · GW]

pro-DEI-requirements in clinical trials, so that drug companies have to have a certain number of various groups in their trial population

If this is motivated by accounting for biological diversity that might translate into different responses to a drug, then this feels like a very non-central case of DEI at best and I would expect it's not what a majority of people think of when they hear/think "DEI".

Yeah, I also guess that something in this direction is plausibly right.

perhaps nothingness actually contains a superposition of all logically possible states, models, and systems, with their probability weighted by the inverse of their complexity.

I think the relevant question here is why we should expect their probability to be weighted by the inverse of their complexity. Is there any abstract theoretical argument for this? In other words, we need to find an a priori justification for this type of Ockham's razor.

Here is one such attempt: Any possible world can be described as a long logical conjunction of "basic" facts. By the principle of indifference, assume any basic fact has the same a priori probability (perhaps even probability 0.5, equal to its own negation), and that they are a priori independent. Then longer conjunctions will have lower probability. But longer conjunctions also describe more complex possible worlds. So simpler possible worlds are more likely.

Though it's not clear whether this really works. Any conjunction completely describing a possible world would also need to include a statement "... and no other basic facts are true", which is itself a quantified statement, not a basic fact. Otherwise all conjunctive descriptions of possible worlds would be equally long.

Nope, same position. In an ideal world he would be prevented from hurting animals (currently, paying people to torture them and strip their bodies of their flesh to bring to him). Current world would also be better without any animal torture payers / doers.

Also, it doesn't seem much like a clarification, versus a reframing. Previously they wrote "Animals suffer, but I don't much care"; now they wrote, "I don't kick dogs, but I have destroyed a wasp's nest in my garage"

(Later edit): Btw, I agree with your other comment that the "original comment has been so strongly disagree-voted because of the framing". In fact, even when people use that framing but say it's bad, it's often still controversial, because people don't like to think about what's being done to animals in direct terms. But it's presumably more controversial if someone uses a non-euphemistic framing, and then says they're okay with this.

Taking a stab at answering my own question; an almost-certainly non-exhaustive list:

• Would the results be applicable to deep-learning-based AGIs?^[1] If I think not, how can I be confident they couldn't be made applicable?

• Do the corrigibility results provide (indirect) insights into other aspects of engineering (rather than SGD'ing) AGIs?

• How much weight one gives to avoiding x-risks vs s-risks.^[2]

• Who actually needs to know of the results? Would sharing the results with the whole Internet lead to better outcomes than (e.g.) sharing the results with a smaller number of safety-conscious researchers? (What does the cost-benefit analysis look like? Did I even do one?)

• How optimistic (or pessimistic) one is about the common-good commitment [LW · GW] (or corruptibility) of the people who one thinks might end up wielding corrigible AGIs.

If compute is linear in space, then in the obvious way of doing things, you have your Nth kid in your $2^{N/3}$th year.

Regular LLMs can use chain-of-thought reasoning. He is speaking about generating chains of thought for systems that don't use them. E.g. AlphaGo, or diffusion models, or even an LLM in cases where it didn't use CoT but produced the answer immediately.

As an example, you ask an LLM a question, and it answers it without using CoT. Then you ask it to explain how it arrived at its answer. It will generate something for you that looks like a chain of thought. But since it wasn't literally using it while producing its original answer, this is just an after-the-fact rationalization. It is questionable whether such a post-hoc "chain of thought" reflects anything the model was actually doing internally when it originally came up with the answer. It could be pure confabulation.

I think that, perhaps, after reading this [LW(p) · GW(p)] clarifying comment from you, @quila [LW · GW] would change his perception of your position.

People do not want more than one child. Some don’t even want one. Two children is viewed as a stretch goal. 

The first Google result I get for "ideal family size in america" is a 2023 Gallup poll according to which 44% of Americans who had an opinion think two children is the idea family size, and 45% that three or more children is.

My current impression is that most people would prefer to have more children than they actually end up having, but there are various societal forces in play making this hard, so it'll take some time to develop the necessary social technologies to better align people's family sizes with their preferences.

Clearly ANNs are able to represent propositional content, but I haven't seen anything that makes me think that's the natural unit of analysis.

Well, we (humans) categorize our epistemic state largely in propositional terms, e.g. in beliefs and suppositions. We even routinely communicate by uttering "statements" -- which express propositions. So propositions are natural to us, which is why they are important for ANN interpretability.