Against Unlimited Genius for Baby-Killers

post by ggggg · 2025-02-19T20:33:27.188Z · LW · GW · 0 comments

This is a link post for https://ggggggggggggggggggggggg.substack.com/p/against-unlimited-genius-for-baby

Contents

No comments

Someone is wrong on the internet. Wrong enough, in fact, to lure me from my hitherto perpetual lurk.

Of course, the meme is (was) funny (amusing) because the people of the internet are often wrong — and often in sillier ways than the instance that has so exercised me here. The nuance in this case, however, is that the mistaken man is one upon whose whim our species’ future plausibly depends. Unfortunate!

See, Sam Altman declared last week that ‘terrorists should have access to unlimited genius to direct however they can imagine’. Or was it ‘baby-killers should have access to unlimited genius to direct however they can imagine’? Never mind, it was both. Indeed, the exact quote was that ‘everyone should have access to unlimited genius to direct however they can imagine’. (‘Everyone’ entails ‘terrorists’ and ‘baby-killers’, ergo…)

Please do not take me for a pedant. Rest assured, I would not be moved to such an undignified début over a simple slip-of-the-tongue. I am not looking to be uncharitable, either.[1] There are surely times when ‘everyone’ straightforwardly does not mean everyone. He obviously didn’t mean literally everyone, I hear you cry. StrAw MAn!!

I’ve no doubt that Altman would prefer a world where everyone except terrorists, baby-killers and their ilk were furnished with ‘unlimited genius’ on-demand. My point is that this is not a choice you get to make. Sam’s candour here has done us a service.[2] ‘Everyone’ really is what he means — or, rather, it is all he reasonably can mean — and a more dexterous choice of words would have served only to obscure that fact.

One way to view technological progress is as an indiscriminate amplifier. An increase in technological level delivers a corresponding increase in capacity for impact. This sounds almost tautological, and at the very least trite. But it matters, because ‘capacity for impact’ is a neutral term. Progress is an ‘indiscriminate amplifier’ in that it augments impacts in all directions. There are good people in the world, and our present technological level means those good people are better equipped than ever to do good things. But there are also bad people. And there are people who think they are being good but are accidentally being bad.

‘Genius’ works the same way. My genius can be wielded for vile and virtuous ends, and my capacity to achieve those ends is not insignificantly a function of the level of genius at my disposal. We once more have an indiscriminate amplification effect. More genius means more capacity for impact. But again, the effect goes both (or, more precisely, all) ways.

There are people who ‘can imagine’ harnessing unlimited genius for awful things — on the order of, say, destroying everything everywhere. I do not think I am making a complicated or controversial point here. You would have to be very bearish on the power of genius to believe that an unlimited amount of it could not deliver deeply transformative outcomes. And whatever Sam is, he is not bearish. So he presumably has to disagree on the ‘indiscriminate’ aspect; to argue that we can drastically turn up the good without doing the same for the bad. Or perhaps he understands the predicament, but finds it useful to pretend otherwise. I can see why that would be true.

I don’t like this problem, in part because I so deeply do want to just turn up the good. But there’s another reason. See, I misspoke earlier — it’s not exactly right to say that everyone has to mean everyone. There are conditions under which there might be a decision about access that someone gets to make.[3] But when Nick Bostrom attempted to sketch such conditions vis-à-vis a related scenario, he inadvertently outed himself as the Antichrist. And lest my anti-Antichrist credentials be in doubt, I am extremely sympathetic to the concern![4] Bostrom posits a ‘high-tech panopticon’ wherein citizens wear a ‘freedom tag’ which relays information to ‘patriot monitoring stations’. With cures like these, who needs a disease?

Still, to feign ignorance — to act as though granting the intellectual heirs of Aum Shinrikyo ‘access to unlimited genius to direct however they can imagine’ might produce a stable equilibrium — cannot be the answer. And the more repulsed you are by a Bostrom-style fix, the more motivated you should be to confront this reality. Because do understand: once the problem makes itself manifest, a ham-fisted, worst-of-all-worlds implementation of that genre of solution is the default mode.

Note that none of the above presupposes a particular orientation on broader questions re alignment, control, et cetera.

Back to the mountains I go.

 

  1. ^

    Note that this was not some off-the-cuff comment or tweet, but a non-trivial point in a fairly polished blog post

  2. ^

    Take that, team ‘not consistently candid’

  3. ^

    Putting aside how you kept your unlimited-genius-machine so responsive to human command

  4. ^

    As, it should be said, is Bostrom himself

0 comments

Comments sorted by top scores.