Research interests I don't currently have time to develop alone

That's an impressive list, congratulations on doing so much different work! I'm interested in doing anything related to decision theory.

I'd definitely be interested to talk more about many of these, especially anthropics and reduced impact / Oracle AI, and potentially collaborate. Lots of topics for future Oxford visits! :-)

Hope you'll get interest from others as well.

In descending order of utility:

• Problems with CEV: The Mahatma Armstrong argument will be part of my doctorate (either I get in Oxford or not), as you now know. So I'm researching it either way. Researching with you, besides an honor, would also be very utilitarian.

• Infinite ethics: I think we should account for the infinite causal link of events coming out of every action. After that, much of the remaining questions would be coming up with the right set of mathematical tools as to make the solution really simple and elegant, without having to deal with hyperreals, infinite shadows and whatnot all at the same time.

• Fermi paradox: I love it, it's a nest of many of the things I like and/or find important. However, I don't have many insights here. But there some things I find obvious which are de-emphasized in this area: (1) how strong Milan's arguments against expansion are and (2) how the many overlapping explanations add up to one big explanation. From a comment I sent you a while back: "For example, is possible that while the expansion-desiring civilizations are more common, those are also the ones who couldn't address many evolutionary shaped behaviors that lead to existential risks and hence they all go extinct. Life can be rare to the point there are only 10 intelligent life originating planets, and it can be the case only half have the desire of expand, of those only half achieve the technology, of those only half don't go extinct…. and this result in a very small probability that the one remaining civilization will colonize."

• Anthropic decision theory: I haven't got the time(or better, will power) to read properly your paper on it until this day. If I can convince myself there could be useful research if I read the paper, great. You can see some of my ideas on SSSA on my other comment I left you today.

Let's talk more about any of those points whenever you have the time. Specially, we should really talk about the Mahatma Armstrong thing, it seems to be a idea we stumble upon more or less at the same time. I wonder if you said it to me in February, and I remembered it again as mine later in March.

I expect to be quite busy with my Master's thesis, which is on a completely unrelated subject, but I would be interested in at least discussing, at possibly also co-authoring papers, on at least the following topics:

The social feasibility of reduced impact AI and Oracle AI. In the subsections of part 5.1. of Responses to Catastrophic AGI Risk, we argued that there would be a constant and strong incentive for anyone with an Oracle AI to turn it into an active one and give it more power, so they would be unlikely to be voluntarily reined in. If that argument is correct, then we would need regulation in order to do the reining in, but that has its own challenges.

The future of surveillance. I'm generally rather concerned with the negative sides of surveillance, but I also acknowledge that the current trend is a continual increase in the amount of surveillance, and we might just be forced to make the best out of it. There is a lot of bad stuff that currently goes on undetected that one could potentially avoid with more comprehensive surveillance, though it would also probably kill a lot completely harmless stuff. Also, there's the thing about being able to use it to stop x-risks and such. It seems like a comprehensive analysis of the exact function and (positive and negative) effects of privacy is something that we'd need in order to properly evaluate various surveillance scenarios.

The Turing test. I worked on a draft of a summary paper on measures of general intelligence last year, but we never got around finishing it. I e-mailed you with a link to it.

The problems with CEV / problems with total utilitarianism. I think that one of the most important questions relating to both CEV and utilitarianism would involve figuring out what exactly human preferences are. And it looks like psychology and neuroscience are starting to produce some very interesting hints about that. Luke posted about the possibility of humans having a hidden utility function, and in my review of Hirstein's Mindmelding book, I mentioned that

As the above discussion hopefully shows, some of our preferences are implicit in our automatic habits (the things that we show we value with our daily routines), some in the preprocessing of sensory data that our brains carry out (the things and ideas that are ”painted with” positive associations or feelings), and some in the configuration of our executive processes (the actions we actually end up doing in response to novel or conflicting situations). [...] This kind of a breakdown seems like very promising material for some neuroscience-aware philosopher to tackle in an attempt to figure out just what exactly preferences are; maybe someone has already done so.

I'm interested in working on the total utilitarianism project.

I won't have much free time in the near future, so I can't promise quick progress on anything, but I'm interested in doing research related to many of these, particularly infinite ethics problems. I was under the impression that the cake or death problem was solved; if there is still a cake-or-death-related open problem, I might be interested in tackling that too.

Oracle AI and the (non-)differences between tool AIs and agents

Instrumental AGI is something that I am working on. There might be value in collaborating. Specifically I am interested in practical boxing mechanisms informed by real-world AGI designs and the fundamental limitations of finite computational substrates.

My informed prior belief is that boxed AI is not a fundamentally hard problem: it fully maps onto computer science and security problems which have already been solved in other contexts. Further, all of the arguments I have seen against boxing suffer from either invalid premises or flawed reasoning. Still, there's much to be done in validating (or disproving) my prior assumptions. Since I am actively working to create AGI, it would be nice to get some answers before we need them. Collaboration with a philosopher on some of the more fundamental epistemic issues might be a good idea.

I have considerable interest in the future of surveillance.

However I strongly value individual autonomy so I tend to view mass surveillance as evil. Looking at your links I rather suspect we are on the different sides of barricades.

Which direction would you like to develop in within these concept spaces?

Hi Stuart, I'm not sure if this post is still active but I've only recently come across your work and I'd like to help. At the moment I'm particularly interested in the Control problem, the Fermi paradox and the simulation hypothesis, but I'm sure a chat with you would spur my interest in other directions too. Would be great if you could get in touch so maybe we can figure out if I can be of any help.