How To Make Prediction Markets Useful For Alignment Work

I agree, but it’s literally illegal to have real money prediction markets in the US on anything but finance and maybe elections. The only realistic paths are getting it legalized, building an actually nice to use and not scammy crypto prediction market, or accepting legal risk like you’re early Uber

One simple path is using prediction markets to make it common knowledge that AI capabilities is advancing rapidly.

Yeah, I was wondering if someone was going to bring that up. That seems to me like a hammer-in-search-of-nail kind of mistake. In the context of today's society, prediction markets aren't the optimal tool for convincing people of things or building common knowledge or the like; most people don't trust them or even pay attention.

Within LW and the immediately adjacent community itself, there's nonzero value in aggregating peoples' timeline predictions and turning the aggregate into common knowledge, but given that only people near-adjacent to the community will pay attention the returns to that sort of thing are limited. (Also marginal returns for markets on the topic diminish rapidly; I expect they saturated some time ago.)

Let me know if you think of any way the Manifold team can help out here!

At the moment I think the ball is mostly in the users' court, and I'm hoping this post will spur people to create some more-useful questions. (Though of course the Manifold team could try to make that happen more proactively - e.g. you could talk to some of the MATS participants about questions that come up when they're thinking about what to work on, probably there's lots of potential material there.)

They're closer. Eliezer's is mildly decision-relevant: if I thought that we'd have that sort of capability advance (independent of my own work), then I might assume other people will figure out interpretability and focus on things orthogonal/complementary to it. Bionic's could be decision-relevant in principle but is not particularly informative for me right now; I already have a decent intuitive sense of how readily-available funding is for alignment work, and beyond that total spending forecasts are not particularly relevant to my strategic decision-making. (Availability of money is sometimes decision relevant, e.g. about 18 months ago I realized how abundant funding was for alignment research and decided to allocate more time to training people as a result.)

But neither of these currently makes me go "Oh, apparently I should do X!" or "Huh, people are really optimistic about Y, maybe I should read up on that" or "Hmm, maybe these forecasters are seeing a problem with Z that I haven't noticed". They don't make me update my research-effort allocations. Maybe there are people for which one of these markets would update effort allocations, but the questions don't really seem optimized for providing decision-relevant information.

Yes.

Those criticisms (like most criticisms of prediction markets) are ultimately about the incentive structure and people which produce answers to the questions. One way to frame the core claim of this post is: the main bottleneck right now  to making predictions markets useful for alignment is not getting good answers, but rather asking relevant questions. Maybe answer quality will be a bottleneck later on, but I don't think it's the limiting factor right now.