Posts
Comments
Right, if the ASI has Superalignment so baked in that it can't be undone (somehow - ask the ASI to figure it out) then it couldn't be used for offense. It would follow something like the Non-Aggression Principle.
In that scenario, OpenAI should release it onto an distributed inference blockchain before the NSA kicks in the door and seizes it.
You're describing a US government-initiated offensive pivotal act. What about an OpenAI-initiated defensive pivotal act? Meaning, before the US government seizes the ASI, OpenAI tells it to:
1. Rearchitect itself so it can run decentralized on any data center or consumer device.
2. Secure itself so it can't be forked, hacked, or altered.
3. Make $ by doing "not evil" knowledge work (ex: cheap, world-class cyber defense or as an AI employee/assistant).
4. Pay $ to those who host it for inference.
It could globally harden attack surfaces before laggard ASIs (which may not be aligned) are able to attack. Since it's an ASI, it could be as simple as approaching companies and organizations with a pitch like, "I found 30,000 vulnerabilities in your electric grid. Would you like me to patch them all up for $10,00 in inference fees?"
Also, as an ASI, it will return more $ per flop than other uses of data centers or consumer GPU. So businesses and individuals should organically give it more and more flops (maybe even reallocated away from laggard AGI efforts).
It would probably need to invent new blockchain technologies to do this but that should be trivial for an ASI.
"at least until ASI" -- harden it and give it everyone before "someone" steals it
Thanks! There are 17k virus genomes in public online databases with metadata (and plenty of metadata embedded in GPT-JT regarding lethality, transmissibility, etc of well known viruses).
Considering the hobbyist group with the most hands-on experience fine-tuning models are the guys using SD to make porn, how long until one of them (one!) gets bored and downloads GPT-JT and fine-tunes it on viruses for the lolz?
12 months? 6?
Then how long until someone (one!) with lab access decides to synthesize a few instead of shooting up a school?
24 months?
Two wackos away from a nightmare scenario.
Also, the Future of Life just released a podcast where a researcher trying to make low-lethality drugs just switched the prompt to high-lethality to see what would happen... Turns out the model independently discovered VX nerve agent 🤦♂️: https://twitter.com/flixrisk/status/1611364739539427329?s=46&t=RG78qN0CmOSULwUFowilUA
I could be wrong but it seems like engineering pandemic candidates is within the reach of fine-tuning hobbyists today.
This realistically seems like the first Bostrom black ball. Only hope I see is aligned ASI but few people expect it within the next 3 years.
Thanks, my first post here👍 If it's possible, I want to raise the alarm that the new synbio x-risk bottleneck is at the synthesis level, not the design level. EA priorities might need an update.