Seeking information relevant to deciding whether to try to become an AI researcher and, if so, how.

Note that FAI research, AI research, and AGI research are three very different things.

Currently, FAI research is conducted solely by the Singularity Institute and researchers associated with them. Looking at SI's publications over the last few years, the FAI research program has more in common with philosophy and machine ethics than programming.

More traditional AI research, which is largely conducted at universities, consists mostly of programming and math and is conducted with the intent to solve specific problems. For the most part, AI researchers aren't trying to build general intelligence of the kind discussed on LW, and a lot of AI work is split into sub-fields like machine learning, planning, natural language processing, etc. (I'm an undergraduate intern for an AI research project at a US university. Feel free to PM with questions.)

AGI research mostly consists of stuff like OpenCog or Numenta, i.e. near-term projects that attempt to create general intelligence.

It's also worth remembering that AGI research isn't taken very seriously by some (most?) AI researchers, and the notion of FAI isn't even on their radar.

Another option: get finance credentials and work at a hedge fund. Finance relies a lot on math, which I infer you are pretty awesome at. If you do well, then you can make a lot of money. Beyond your living costs, donate your preferred percentage to charity and save much of the rest. Retire relatively early and use your savings to fund the philosophical projects you're interested in.

That's assuming best case scenario, of course. But it's a path I'm seriously looking into and I see some similarities between our situations. Hope this is of help. Let me know if you'd like more info.

I speak as someone who is about to enter their first year of grad school in AI; note that I have also already spent a year doing robotics and a year doing AI research, so I am slightly less naieve than the typical first-year grad student.

The first half of this comment will discuss my thoughts on the merits of AI / AGI / FAI research. The second half will discuss how to do AI research if you want to. They will be separated by a bunch of dashes (-------).

I personally chose to go into AI research for a few reasons, although I don't have a high degree of certainty that it's the right thing to do. But my reasons were --- high degree of aptitude and belief that FAI research uninformed by current state of the art in AI / ML is much more likely to fail (I also think that FAI is much more a social problem than a technical problem; but I'm not sure if that's relevant).

I am pretty skeptical of AGI projects in general, mostly based on an outside view and based on looking at a small number of them. Claims of near-term AI indicate a lack of appreciation for the difficulty of the problem (both engineering barriers that seem surmountable with enough man-hours, and theoretical barriers that require fundamentally new insights to get past). I don't want to say that it's impossible that there is some magical way around these, but to me it pattern-matches to amateur mathematicians proposing approaches to Fermat's Last Theorem. If you have some particular AGI project that you think is likely to work, I'm happy to look at it and make specific comments.

I am less skeptical of FAI research of the form done by e.g. Wei Dai, Vladimir Nesov, etc. I view it as being on the far theoretical end of what I see as the most interesting line of research within the AI community. I also think it's possible for such research to be conducted as essentially AI research, at one of the more philosophically inclined labs (most likely actually a computational cognitive science lab rather than an AI lab, such as Tom Griffiths, Noah Goodman, Josh Tenenbaum, or Todd Kemp).

Okay, so say you want to be an AI researcher. How do you go about doing this? It turns out that research is less about mathematical ability (although that is certainly helpful) and more about using your time effectively (both in terms of being able to work without constant deadlines and in terms of being able to choose high-value things to work on). There are also a ton of other important skills for research, which I don't have time to go into now.

If you want to work at a top university (which I highly recommend, if you are able to), then you should probably start by learning about the field and then doing some good research that you can point to when you apply. As an undergrad, I was able to do this by working in labs at my university, which might not be an option for you. The harder way is to read recent papers until you get a good idea, check to see if anyone else has already developed that idea, and if not, develop it yourself, write a paper, and submit it (although to get it accepted, you will probably also have to write it in the right format; most notably, introductions to papers are notoriously hard to write well). It also helps to be in contact with other good researchers, and to develop your own sense of a good research program that you can write about in your research statement when you apply. These are all also skills that are very important as a grad student, so developing them now will be helpful later.

Unfortunately, I have to go now, but if I left anything out feel free to ask about it and I can clarify.

Have you looked into theoretical computer science, for example, of the type Scott Aaronson is doing? It is mathematical enough for your liking, yet applied enough to avoid competition with the best and the brightest. It is also quite relevant to the AI research.

I understand that you are mainly interested in FAI research, but perhaps you can try out mainstream AI research (or something else; see Tetronian's comment) for a few months to see if AI research in general suits you? I mention this because I think you are likely able to find work in a nearby academic AI lab if you offer to do so at low or no wages (the standard undergrad rate where I am is $10/hr). Just contact the PI directly. Of course, the opportunity cost may be too large – I'd imagine that you'd have to permanently quit your current job to do this. (But it only costs sending a few emails to find out whether there are nearby AI groups who would take you in.)

Have you considered applying to a cognitive science program? It is an interdisciplinary field which overlaps with AI (which you are already considering), neuroscience (which you have a little experience with), and philosophy (which you seem to like).

Consider getting back into neuroscience!

AGI as a project is trying to make machines that can do what brains do. One great way to help that project is to study how the brains themselves work. Many many key ideas in AI come from ideas in neuroscience or psychology, and there are plenty of labs out there studying the brain with AI in mind.

Why am I telling you this? You claim that you'd like to be an AI researcher, but later you imply that you're new to computer programming. As mentioned in some of the comments, this is likely to present a large barrier to "pure" AI research in an academic CS department, corporation, etc. A computational psychology or neuroscience lab is likely to be much more forgiving for a newbie programmer (though you'll still have to learn). The major things that graduate programs in computational neuro look for are strong math skills, an interest in the mind/brain, and a bit of research experience. It sounds like you've got the first two covered, and the last can be gained by joining an academic lab as a research assistant on a temporary basis.

If you're considering giving academia another shot, its worth thinking about neuro and psych (and indeed linguistics and cognitive science, or a different/better philosophy program) as well as computer science and pure AI.

Good luck!

I'm in favor of you becoming an FAI researcher, possibly independent of SI, because you've done a decent job of disagreeing with them in the past and you seem to have relevant interests and aptitudes.

Obviously you'll have to support yourself or get someone to fund you. Getting a high income career, living frugally, and quitting after a while to do research full-time could be an interesting idea. One possibility would be to take periodic years-long "research vacations" in parts of the world with lower cost-of-living. You could buy some dogs for guaranteed companionship. Note: these are wild ideas; I have no idea if they would actually work.

The Republic of Georgia has the most lax visa laws (if you're from a developed country) of any country I looked at. You can basically stay in the country an entire year without any sort of visa, if I recall correctly. And they have a program for English teachers where they will pay for your plane ticket in and out of the country, give you housing and a stipend, etc.

http://www.footprintsrecruiting.com/teaching-in-georgia

I think the most important question here is whether you can deal with general issues related to being an academic. If your answer is "yes," the specific thing you want to study is less important, in my opinion (people often drastically change trajectories in their careers, especially after defending).

Why not do an MSc in computer science with a focus on AI? If you decide you like research, you can continue on down that path, if not, you will have a valuable credential that you can use to get a good job. Many subfields of AI are heavily neuroscience-inspired, so that part of your undergrad coursework will come in handy.

world-changing AI is close enough to be worth putting a lot of thought into seriously thinking about.

Probably true - but many feel the same way - just as many want to be rock stars.

When you talk about "AI research", are you specifically referring to narrow AI or strong AI? Because these two fields are very different things, with very different requirements and skillsets.

If you happen to be referring to Strong AI, aka Friendly AGI, check out "So you want to be a Seed AI Programmer", assuming you haven't already seen it. http://web.archive.org/web/20101227203946/http://www.acceleratingfuture.com/wiki/So_You_Want_To_Be_A_Seed_AI_Programmer

Related:

• Ben Goertzel's Sketch of an AGI curriculum
• Pei Wang's Suggested Education for Future AGI Researchers

I would suggest working on practical software, such as that which solves for best designs of transistors (gradually generalized), and perhaps for more optimal code for that. The AGI will have to out-foom such software to beat it, and probably won't, as the 'general==good' is just the halo effect (and the software self improvement is likely to be of very limited help for automated superior computer design). And if it won't outfoom such software, then we don't have the scenario of AGI massively outpowering the mankind, and the whole risk issue is a lot lower.

This prevents boat load of nasty scenarios including "the leader of FAI team is a psychopath and actually sets himself a dictator, legalizes rape, etc" which should be expected to have risk of at least 2..3% if the FAI team is to be technically successful (about 2..3% of people are psychopaths, and those folks have edge over normals when it comes to talking people into giving them money and/or control).