LessWrong 2.0 Reader

Can you elaborate further on how Gato is proof that just supplementing the training data is sufficient? I looked on youtube and can't find any videos of task switching.

This story is moved me a lot, and I am giving it a substantial vote,

But... I do still really wish this line...

And they turn away and go back to their work—all except for one, who brushes past the grasshopper and whispers “Meet me outside at dusk and I’ll bring you food. We can preserve the law and still forgive the deviation.”

Came where it originally was located, significantly later in the post, after these sections:

The ants start to receive dozens of requests for food, then hundreds—and while many are fraudulent, enough are real that they are moved to act. In order to set incentives correctly, the ants decide to only give food to those who can prove that they lost their own food supplies through no fault of their own, and set up a system for vetting claims.

This works well for a time—but as fraudsters grow more sophisticated, the ants’ bureaucratic requirements grow more onerous. In order to meet them, other creatures start to deposit their food in large group storehouses which can handle the administrative overhead. But now the food supply is exposed to systemic risk if the leaders of those storehouses make poor decisions, whether from carelessness or greed.

One year several storehouses fail; in trying to fill the shortfall, the ants almost run out of food for themselves. To avoid that ever happening again, they set up stringent regulations and oversight of permissible storehouses, funded by taxes levied throughout the year. At first this takes only a small proportion of their labor—but as their regulatory apparatus inevitably grows, they need to oversee more and more aspects of the ecosystem, and are called upon to right more and more injustices. Eventually the ants—originally the most productive of all creatures—stop producing any food of their own, so busy are they in tending to the system they’ve created.

...

“And therefore, to reduce risks from centralization, and to limit our own power, we can’t give you any food”, the ants conclude. And they turn away and go back to their work, with a quiet sense of satisfaction that they’ve given such legible and defensible reasons for focusing on their own problems and keeping all the food for themselves.

In it's original positioning, the "We can preserve the law and still forgive the deviation" line felt like the climax of a mini-arc. Previously Richard had said he moved it for two reasons [LW(p) · GW(p)]:

Firstly, if the grasshopper was just unlucky, then there's no "deviation" to forgive—it makes sense only if the grasshopper was culpable. Secondly, the earlier parts are about individuals, and the latter parts are about systems—it felt more compelling to go straight from "centralized government" to "locust war" than going via an individual act of kindness.

But, what I find particularly meaningful about it is that even when giant systems have evolved that are turning grotesque with their burueacratic weight... and when maybe those systems are necessary or inevitable or something... individuals can still choose to just... do a nice thing, when they have local knowledge suggesting it's the right thing to do in this case. (And meanwhile, I had no problem backfilling and appropriate history for this particular snippet. I can imagine a grasshopper who slacked off a bit, and also got a bit unlucky, and also seems to have earnestly learned and will make better choices next time)

What do you think is the ideal use-case for steering? Or is it not needed

Note that this story has recieved a beautifully-animated video adaptation by Rational Animations!
 

Terrific!

Oops, yes.

What are the other describable or possible-though-indescribable hypotheses? If it's intuitive that there are no other hypotheses to start from — if the explanations have been reduced to some small number of all imaginable possibilities — that's a non-nothing sort of evidence which ought to be contended with at the very least, rather than scoffed at with 'you didn't see an epistemic polylemma therefor there's no evidence there was one'.

I'm generally favorable to see more of these published. Mainly, because I think these are going to end up being the basis for an industry audit standard and then a law. 

The more similar they are, they easier it will be for them to converge. 

I'm very happy to see Meta publish this. It's a meaningfully stronger commitment to avoiding deployment of dangerous capabilities than I expected them to make. Kudos to the people who pushed for companies to make these commitments and helped them do so.

One concern I have with the framework is that I think the "high" vs. "critical" risk thresholds may claim a distinction without a difference.

Deployments are high risk if they provide "significant uplift towards execution of a threat scenario (i.e. significantly enhances performance on key capabilities or tasks needed to produce a catastrophic outcome) but does not enable execution of any threat scenario that has been identified as potentially sufficient to produce a catastrophic outcome." They are critical risk if they "uniquely enable the execution of at least one of the threat scenarios that have been identified as potentially sufficient to produce a catastrophic outcome." The framework requires that threats be "net new," meaning "The outcome cannot currently be realized as described (i.e. at that scale / by that threat actor / for that cost) with existing tools and resources."

But what then is the difference between high risk and critical risk? Unless a threat scenario is currently impossible, any uplift towards achieving it more efficiently also "uniquely enables" it under a particular budget or set of constraints. For example, it is already possible for an attacker to create bio-weapons, as demonstrated by the anthrax attacks - so any cost reductions or time savings for any part of that process uniquely enable execution of that threat scenario within a given budget or timeframe. Thus it seems that no model can be classified as high risk if it provides uplift on an already-achievable threat scenario—instead, it must be classified as critical risk.​​​​​​​​​​​​​​​​

Does that logic hold? Am I missing something in my reading of the document? 

I think you (or @Adam Scholl [LW · GW]) need to argue why people won't be angry at you if you developed nuclear weapons, in a way which doesn't sound like "yes, what I built could have killed you, but it has an even higher chance of saving you!"

Otherwise, it's hard to criticize Anthropic for working on AI capabilities without considering whether their work is a net positive. It's hard to dismiss the net positive arguments as "idiosyncratic utilitarian BOTEC," when you accept "net positive" arguments regarding nuclear weapons.

Allegedly, people at Anthropic have compared themselves to Robert Oppenheimer. Maybe they know that one could argue they have blood on their hands, the same way one can argue that about Oppenheimer. But people aren't "rioting" against Oppenheimer.

I feel it's more useful to debate whether it is a net positive, since that at least has a small chance of convincing Anthropic or their employees.