LessWrong 2.0 Reader

The LessWrong Review [? · GW] runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2024. The top fifty or so posts are featured prominently on the site throughout the year. Will this post make the top fifty?

This seems contrary to how much of science works. I expect if people stopped talking publicly about what they're working on in alignment, we'd make much less progress, and capabilities would basically run business as usual.

The sort of reasoning you use here, and that my only response to it basically amounts to "well, no I think you're wrong. This proposal will slow down alignment too much" is why I think we need numbers to ground us.

You think that studying agency and infrabayesianism wont make small contributions to capabilities? Even just saying "agency" in the context of AI makes capabilities progress.

"So where do I privately share such research?" — good question! There is currently no infrastructure for this.

This is why I currently think you're completely wrong about this. There never will be an infrastructure for this. Privacy of communities isn't a solvable problem in general, as soon as your community is large enough to compete with the adversary, it's large enough and conspicuous enough that the adversary will pay attention to it and send in spies and extract leaks. If you make it compartmented enough to prevent leaks/weed out the spies, it wont have enough intellectual liveliness to solve the alignment problem.

There is nothing that makes differentially helping capabilities "fine if you're only differentially helping them a little bit".

If your acceptable lower limit for basically anything is zero you wont be allowed to do anything, really anything. You have to name some quantity of capabilities progress that's okay to do before you'll be allowed to talk about AI in a group setting.

"So where do I privately share such research?" — good question! There is currently no infrastructure for this.

I'd really like to have such a place, or even a standard policy how to do this.

I feel like the aintelope I'm working on has to secure it's stuff from scratch. Yes, it's early, but it is difficult to engineer security in later. You have to start with something. I'd really like to have a standard for AI Safety projects to follow or join.

People who have the ability to clarify in any meaningful way will not do so. You are in a biased environment where people who are most willing to publish, because they are most able to convince themselves their research is safe - eg, because they don't understand in detail how to reason about whether it is or not - are the ones who will do so. Ability to see far enough ahead would of course be expected to be rather rare, and most people who think they can tell the exact path ahead of time don't have the evidence to back their hunches, even if their hunches are correct, which unless they have a demonstrated track record they probably aren't. Therefore, whoever is making the most progress on real capabilities insights under the name of alignment will make their advancements and publish them, since they don't personally see how it's exfohaz. And it won't be apparent until afterwards that it was capabilities, not alignment.

So just don't publish anything, and do your work in private. Email it to anthropic when you know how to create a yellow node [LW · GW]. But for god's sake stop accidentally helping people create green nodes because you can't see five inches ahead. And don't send it to a capabilities team before it's able to guarantee moral alignment hard enough to make a red-proof yellow node!

when potentially ambiguous, I generally just say something like "I have a different model" or "I have different values"

Most importantly, I think the research published by people working on x-risk tends to overall help safety/alignment more than capabilities

It may be producing green nodes faster, but it seems on track to produce a red node before a yellow node. [LW · GW]

Obviously keep working, but stop talking where people who are trying to destroy the world can hear. If you're Neel Nanda and work for a company trying to destroy the world, consider not publishing anything else at all, and only publishing useless versions of your work, because your work being useful for resilient moral alignment depends on a long chain of things that publishing it makes near impossible.

EY may be too busy to respond, but you can probably feel pretty safe consulting with MIRI employees in general. Perhaps also Conjecture employees, and Redwood Research employees, if you read and agree with their views on safety. That at least gives you a wider net of people to potentially give you feedback.