LessWrong 2.0 Reader

Natural Latents [LW · GW]

Infra-Bayesianism [? · GW]

Poll for LW topics you'd like to be tutored in
(please use agree-react to indicate you'd personally like tutoring on a topic, I might reach out if/when I have a prototype)

Note: Hit cmd-f or ctrl-f (whatever normally opens search) to automatically expand all of the poll options below.

I guess refusing to use someone’s preferred pronouns is weak Bayesian evidence for wanting to have them killed, but the conclusion is so unlikely it’s probably not appropriate to raise it to the level of serious consideration.

What would be upsetting about being called "she"? I don't share your intuition. Whenever I imagine being misgendered (or am misgendered, e.g., on a voice call with a stranger), I don't feel any strong emotional reaction. To the point that I generally will not correct them.

I could imagine it being very upsetting if I am misgendered by someone who should know me well enough not to misgender me, or if someone purposefully misgenders me. But the misgendering specifically is not the main offense in these two cases.

Perhaps myself and ymeskhout are less tied to our gender identity than most?

Yeah, I admit a lot of the crux comes down to whether thinking whether your case is more the exception or the rule, and I admit that I think that your situation is more unusual compared to the case where you can locally verify something without having to execute the global plan.

I tend to agree far more with Paul Christiano than with John Wentworth on the delta of

But to address what it would mean for alignment to generalize more than capabilities, this would essentially mean it's easier to get an AI to value what you value without the failure modes of deceptive/pseudo/suboptimality alignment than it is to get an AI that actually executes on your values through capabilities in the real world.

(On the highest level I do not know my values and wouldn't hand over the full control of the future to any AI because I don't trust that I could tell good from bad, I think I'd mostly be confused about what it did.)

I admit that I both know a lot more about what exactly I value, and I also trust AIs to generalize more from values data than you do, for several reasons.

I also don't think it's useful to try and learn much about pronouns qua pronouns social battles over them. Using the pronoun people ask you to use has become a proxy for all sorts of other tolerant/benevolent attitudes towards that person and the way they want to live their life, and to an even greater extent, refusing to do that is a proxy for thinking they should be ignored, or possibly reviled, or possibly killed. 

I don't think everyone proxies it that way -- I know there are some people who are just old-fashioned, or passionate about prescriptive grammar, or have essentialist beliefs about gender but are libertarian about others' behavior. I think that if everyone had very high confidence that someone not using the pronouns they requested meant that at worst that person mildly disapproves of them but would still actively defend their civil + legal + human rights, there would probably be a lot less of the handwringing you mention, and we'd be able to learn a lot more about the fundamental intrinsic meaning of pronouns. 

to me "alignment tax" usually only refers to alignment methods that don't cost-effectively increase capabilities, so if 90% of alignment methods did cost effectively increase capabilities but 10% did not, i would still say there was an "alignment tax", just ignore the negatives.

Also, it's important to consider cost-effective capabilities rather than raw capabilities - if a lab knows of a way to increase capabilities more cost-effectively than alignment, using that money for alignment is a positive alignment tax

As I keep saying, deception is not some unique failure mode. Humans are constantly engaging in various forms of deception. It is all over the training data, and any reasonable set of next token predictions. There is no clear line between deception and not deception. And yes, to the extent that humans reward ‘deceptive’ responses, as they no doubt often will inadvertently do, the model will ‘learn’ deception.

https://www.lesswrong.com/posts/a392MCzsGXAZP5KaS/deceptive-ai-deceptively-aligned-ai [LW · GW] 

(found in the comments of this prediction market)

In the limit (what might be considered the ‘best imaginable case’), we might imagine researchers discovering an alignment technique that (A) was guaranteed to eliminate x-risk and (B) improve capabilities so clearly that they become competitively necessary for anyone attempting to build AGI. 

I feel like throughout this post, you are ignoring that agents, "in the limit", are (likely) provably taxed by having to be aligned to goals other than their own.  An agent with utility function "A" is definitely going to be less capable at achieving "A" if it is also aligned to utility function "B".   I respect that current LLM's not best described as having a singular consistent goal function, however, "in the limit" that is what they will be best described as.