What I’ll be doing at MIRI

I have discussed with MIRI their decision to make their research non-disclosed-by-default and we agreed that my research agenda is a reasonable exception.

Small note: my view of MIRI's nondisclosed-by-default policy is that if all researchers involved with a research program think it should obviously be public then it should obviously be public, and that doesn't require a bunch of bureaucracy. I think this while simultaneously predicting that when researchers have a part of themselves that feels uncertain or uneasy about whether their research should be public, they will find that there are large benefits to instituting a nondisclosed-by-default policy. But the policy is there to enable researchers, not to annoy them and make them jump through hoops.

(Caveat: within ML, it's still rare for risk-based nondisclosure to be treated as a real option, and many social incentives favor publishing-by-default. I want to be very clear that within the context of those incentives, I expect many people to jump to "this seems obviously safe to me" when the evidence doesn't warrant it. I think it's important to facilitate an environment where it's not just OK-on-paper but also socially-hedonic to decide against publishing, and I think that these decisions often warrant serious thought. The aim of MIRI's disclosure policy is to remove undue pressures to make publication decisions prematurely, not to override researchers' considered conclusions.)

That you're working full time on research, have a stable salary, and are in a geographical location conducive to talking with a lot of other thoughtful people who think a lot about these topics, are all very valuable things, and I'm pleased to hear these things are happening for you :-)

On the subject of privacy, I was recently reading a friend's career plan, who was looking for jobs in AI alignment, and I wrote this:

Do not accept secrets lightly. If you accept one wrong secret, you will go the way of MIRI or Leverage or US government officials with a security clearance, where it’s just too much effort to communicate your thoughts with outsiders, for fear of accidentally letting out secrets. And you’re not allowed to tell people the true reasons for your beliefs. It’s not that nobody should work at MIRI or Leverage or get confidential info from the government. It’s that it raises the costs of participating in public discourse to be not worth it for almost everyone involved. 

It's really great to hear that you'll continue writing publicly, as I think the stuff you're doing is important and exciting and there are strong distributed benefits for the broader landscape of people working on AI alignment or who want to.

Also feel free to come downstairs and hang out with us in the LessWrong offices :-)

Congratulations! :) 

Do come visit our office in your basement sometimes.