If there was a millennium equivalent prize for AI alignment, what would the problems be?

post by Yair Halberstadt (yair-halberstadt) · 2022-06-09T16:56:10.788Z · LW · GW · 2 comments

This is a question post.

Contents

  Answers
    5 Lorenzo Rex
None
2 comments

If someone were to create one or a number of huge (e.g 1 to 10 million dollar) prizes for solving important problems related to AI alignment, it might have a decent chance of focusing a lot of researchers on the topic, which might help advance progress towards a solution/s, as well as make the field more respectable.

In order to avoid bad feelings, attract serious academics, and get meaningful results, the problems have to be clearly defined and have unambiguous solutions - something like "solve AI alignment" won't cut it.

The problems wouldn't have to be complete solutions to AI alignment. Instead they should be initial explorations on which further research and new branches of mathematics/computer science can be built. We want people to win those prizes, and quickly, so we can iterate with further prizes and attract more momentum.

So what problems can you think of that are concrete, achievable, and would represent useful milestones towards AI alignment?

Answers

answer by lorepieri (Lorenzo Rex) · 2022-06-10T22:02:03.099Z · LW(p) · GW(p)

The 1 million prize problem should be "clearly define the AI alignement problem". I'm not even joking, actually understanding the problem and enstablising that there is a problem in the first place may give us hints to the solution.

↑ comment by Esben Kran (esben-kran) · 2022-06-19T16:16:52.163Z · LW(p) · GW(p)

Yes, defining the challenge also seems to get us 90%< there already. A Millennium prize has a possibility of being too vague compared to the others.

2 comments

Comments sorted by top scores.

comment by Evan R. Murphy · 2022-06-10T02:33:45.961Z · LW(p) · GW(p)

Good question. This is also very similar to one of the questions in Holden's Important, actionable questions for the most important century [EA · GW], a list of AI alignment and strategy questions which I generally think should be receiving more attention:
 

What’s an AI alignment result or product that would make sense to offer a $1 billion prize for?

I think longtermist funders would be excited to fund and launch such a prize if it were well-designed. I’d expect scoping the prize (to prevent spurious wins but also give maximum clarity as to the goals), promoting the prize, giving guidance to entrants, and judging entries to be a lot of work, so I’d be most excited to do it with a backdrop of having done the hard intellectual work to figure out what’s worth rewarding.

More detail on what it would look like to work on this sort of question, how it could matter, and who’s working on it today

Replies from: lorenzo-rex
↑ comment by lorepieri (lorenzo-rex) · 2022-06-10T22:13:35.526Z · LW(p) · GW(p)

Nice link, thanks for sharing.