Earning money with/for work in AI safety

Thanks! I hadn't come across the Foundational Research Institute yet.

Though, hmm, not plenty of experience? If there's talk about PhDs as an advantage, it sounds to me like they're looking for people with PhD-level experience. I'm far from that. But unless you say »oh well then maybe not«, I'll apply. Who knows what will come out of it.

Will your results ultimately take the form of blog posts such as those, or peer-reviewed publications, or something else?

I think FRI's research agenda is interesting and that they may very well work on important questions that hardly anyone else does, but I haven't yet supported them as I'm not certain about their ability to deliver actual results or the impact of their research, and find it a tad bit odd that it's supported by effective altruism organizations, since I don't see any demonstration of effectiveness so far. (No offence though, it looks promising.)

Thanks for your varied suggestions!

Actually I'm kind of more comfortable with MIRI math than with ML math, but the research group here is more interested in machine learning. If I recommended them to look into provability logic, they would get big eyes and say Whoa!, but no more. If, however, I do ML research in the direction of AI safety, they would get interested. (And they are getting interested, but (1) they can't switch their research too quickly and (2) I don't know enough Japanese and the students don't know enough English to make any kind of lunchtime or hallway conversation about AI safety possible.)

Thank you!

After graduating, why would you need to be based in Kagoshima?

I need to be based in Kagoshima for pretty strong personal reasons. Sorry for not providing details. If you really need them, I can tell you more via PM.

Ah, you write »after graduating«? Sorry for not providing that detail: research students in Japan are not working on a master's or PhD. They're just hanging around studying or doing research and hopefully learn something during that time.

Have you taken a look at the content on MIRI's to practice AI safety research?

Yes, I've read all of the agenda papers and some more.

Have you considered applying to visit AI safety researchers at MIRI or FHI? That would help you to figure out where your interests and theirs overlap, and to consider how you might contribute.

I applied for the MIRI Summer Fellows Programme, which I didn't get into by a small margin, and CFAR's Workshop on AI Safety Strategy, which I also didn't get into. They told me they might put me in the next one. That would definitely help me with my questions, but I thought it's better to start early, so I asked here.

If you're not eligible to visit for some reason, that might imply that you're further from being useful than you thought.

I am at the very beginning of learning ML and AI and therefore kind of far from being useful. I know this. But I'm quite good at maths and computer science and a range of other things, so I thought contributing to AI safety research shouldn't be too far to go. It will just take time. (Just as a master's programme would take time, for example.) The hard part is to get hold of money to sustain myself during that time.

I might be useful for other things than research directly, such as support software development, teaching, writing, outreach, organizing. I haven't done much teaching, outreach and organization, but I would be interested to try more.

I recently had an idea for an app that would make use of natural language processing and provide a service to businesses doing online marketing. I think there's a pretty good chance businesses would be willing to pay for this service, and after a quick Google search, I don't think there are companies doing anything similar yet. If you or anyone else interested in AI safety wants to hear more, feel free to send me a PM.

Not much going on as far as I know. What I know is the following:

• Naozumi Mitani has taught a course on Bostrom's Superintelligence and is »broadly pursuing the possible influence of AI on the future lives of humanity«. He's an associate professor of philosophy at Shinshu University (in Nagano).
• The Center for Applied Philosophy and Ethics at Kyoto University is also somehow interested in AI impacts.
• My supervisor is gradually getting interested, too. This is partly my influence, but also his own reading. For example, he found the Safely Interruptible Agents and Concrete Problems in AI Safety independently of me through Japanese websites. He's giving me chances to make presentations about AI safety for my fellow students and hopefully also for other professors.

Other than that I know of nobody and searching the web quickly, I didn't find out more. One problem here is that most students don't understand much English, so most of the AI safety literature is lost on them. The professors do know English, but I maybe they're usually not inclined or able to change their research focus.

It's a good sign that my supervisor finds AI safety articles through Japanese websites, though.

In the likely case that your marginal contribution to x-risk doesn't save the world

So you think that other people could contribute much more to x-risk, so I should go into areas where I can have a lot of impact? Otherwise, if everyone says »I'll only have a small impact on x-risk. I'll do something else.«, nobody would work on x-risk. Are you trying to get a better justification for work on x-risk out of me? At the moment I only have this: x-risk is pretty important, because we don't want to go extinct (I don't want humanity to go extinct or into some worse state than today). Not many people are working on x-risk. Therefore I do work on x-risk, so that there are more people working on it. Now you will tell me that I should start using numbers.

the fact that you won't consider leaving Kagoshima is an indication that you aren't as fully committed as you claim

What did I claim about my degree of commitment? And yes, I know that I would be more effective at improving the state of humanity if I didn't have certain preferences about family and such.

Anyway, thanks for pushing me towards quantitative reasoning.

Perhaps you'd do a lot more good with a slight reduction in shipping costs or tiny improvements in safety or enjoyment of some consumer product.

Perhaps you would also do more good by working in a slight increase in shipping costs.