Memetic hazards of AGI architecture posts

I endorse not widely sharing info that could destroy the world (or accelerate us more than it helps ensure aligned AGI). Starting with smaller private discussion seems great to me.

My personal course of action has been to avoid this topic. I have specific thoughts about what current AI is not doing that bars it from becoming AGI, I have specific thoughts about what lines of research are most likely to successfully lead to AGI, I have arguments for these thoughts, and I've decided to keep them to myself. My reasoning is that if I'm right then sharing them marginally accelerates AGI development and if I'm wrong then whatever I say is likely neutral, so it's all downside, however I keep open the option to change my mind if I encounter some safety related work that hinges on these thoughts (and I've hinted at earlier versions of my thoughts in this space as part of previous safety writing to make the case for why I think certain things matter to building aligned AGI).

1. You're probably wrong. But who knows?
2. Remember that if you thought of it, somebody else has probably already thought of it, or something really close. Other people will keep thinking of it, even if you do happen to be the first.
3. Probably at least 95 percent of them will do nothing with the idea. They'll just say "Hmm, that seems like it'd work", and go on with their lives.
4. Eventually, one or more of them will probably do something more than think. They may publicize it, or they may try to start a public or secret effort to actually build it. Or they may drop it in casual conversation with somebody else who does those things. It may spread slowly by gossip until some random person lights a fire under it. It only takes one if they act effectively and are reasonably lucky.
5. Therefore, YOU ARE ON THE CLOCK, you do not know how long the clock will run, and you may get little or no warning when the clock runs out. Secrets have short shelf lives.
6. Any disclosure, public or private, is an irrevocable move, and should be considered carefully. But that does not mean it's a move you should never make. In fact it's probably the main move available to you. And you are still on the clock no matter what.
7. It is always good to get advice from others. You have to give them full information, or their advice will probably be wrong. Be careful whose advice you take. And each person you tell is another person who might do something you don't like.
8. In particular, it's not enough that they be useful, responsible, competent, and/or well-meaning. You may in many scenarios need to decide whether you think their idea of "aligned" matches yours, and whether you think their approach to getting there matches yours. "Alignment" is a very vague concept. "The AI Safety crowd" probably includes people who do not match you.
9. Actors you've never heard of can blow up your plans at any time, including by changing the timeline. "Competitors" with the same idea might not be the only people who can make moves that matter, nor the only people who might respond to your moves in ways that matter.
10. Whenever you make a move, try to think about how everybody relevant might respond, and how you and the others would respond to that, and so forth. It would probably help to make a list of actors or types of actors, known and unknown, and actually think about what they all might do. And remember that relative speeds matter a lot.
11. You will be stuck with some rules and heuristics like "try to tell this kind of people" or "try not to tell thak kind of people" or "always/never do X when/until Y". Simple rules like that are never right for all situations. They are last resorts. You shouldn't apply them if you can think out the specific, situational consequences instead. It's very easy to grab a cached rule and stop thinking.
12. At the same time, if you're about to violate a rule, you should run the situation against the original reason you originally adopted that rule, and be sure you're satisfied with why you're violating it.

Are you an engineer?

• If so, then just build the AGI yourself. Experimentation trumps talk. Actually building the thing will teach you more about the architecture than discussing it with other people. Building a scaled-down version is perfectly safe, provided you limit its compute (which happens by default) and don't put it in the path of any important systems (which also happens by default).
• If you can't build the AGI yourself then you're not at the top of the ML field which means your architecture is unlikely to be correct and you can publish it safely.