Announcing the AI Alignment Prize

Here is my submission.

Thank you for motivating me to write this blog post I have been putting off for a while.

Disclaimer: If you want to only measure the contribution that came November or later, compare to this post, which has one fewer category, no names, fewer examples, nothing about mitigation, and worse presentation.

I think this is an important idea, so I appreciate feedback, especially about presentation.

You don't mention decision theory in your list of topics, but I guess it doesn't hurt to try.

I have thought a bit about what one might call the "implementation problem of decision theory". Let's say you believe that some theory of rational decision making, e.g., evidential or updateless decision theory, is the right one for an AI to use. How would you design an AI to behave in accordance with such a normative theory? Conversely, if you just go ahead and build a system in some existing framework, how would that AI behave in Newcomb-like problems?

There are two pieces that I uploaded/finished on this topic in November and December. The first is a blog post noting that futarchy-type architectures would, per default, implement evidential decision theory. The second is a draft titled "Approval-directed agency and the decision theory of Newcomb-like problems".

For anyone who's interested in this topic, here are some other related papers and blog posts:

• Another one I wrote: "Doing what has worked well in the past leads to evidential decision theory" (I updated this in December, but it was first written and uploaded in September, so it doesn't count for the competition.)
• Albert and Heiner (2001): "An Indirect-Evolution Approach to Newcomb's Problem"
• Meyer, Feldmaier and Shen (2016): "Reinforcement Learning in Conflicting Environments for Autonomous Vehicles"

So far, my research and the papers by others I linked have focused on classic Newcomb-like problems. One could also discuss how existing AI paradigms related to other issues of naturalized agency, in particular self-locating beliefs and naturalized induction, though here it seems more as though existing frameworks just lead to really messy behavior.

Send comments to firstnameDOTlastnameATfoundational-researchDOTorg. (Of course, you can also comment here or send you a LW PM.)

I saw a talk earlier this year that mentioned this 2015 Corrigibility paper as a good starting point for someone new to alignment research. If that's still true, I started writing up some thoughts on a possible generalization of the method in that paper.

Anyway, submitting this draft early to hopefully get some feedback whether I'm on the right track:

GeneralizedUtilityIndifference_Draft_Latest.pdf (edited)

The new version does better on sub-agent shutdown and eliminates the "managing the news" problem.

(Let me know if someone already thought of this approach!)

EDIT 2017-11-09: filled in the section on the $n$-action model.

Should I submit? Working on this is my job, so it's maybe better to encourage others to come on board?

What should we be doing to help get more people to enter, whether by spreading the word or another way? We want this to work and result in good things, and it's iteration one so doubtless a lot we're not doing right.

Zvi/Vladimir, what's your role in this - are you the judges?

I don't know if this is a useful "soft" submission, considering I am still reading and learning in the area.

But I think the current metaphors (paperclips, etc.) are not very persuasive for convincing folks in the world at large that value alignment is a BIG, HARD PROBLEM. Here is my attempt to add a possibly-new metaphor to the mix: https://nilscript.wordpress.com/2017/11/26/parenting-alignment-problem/

Posted on my blog, but might as well link it here. Not of the quality that Paul Christiano seeks, but might be of some interest, though many of the same point points have been discussed over and over here and elsewhere before.

Submission: http://futilitymonster.tumblr.com/post/169068920534/the-true-ai-box

Contact: defectivealtruist at g mail

Is "publishing" on google docs ok? Here's a link:

https://docs.google.com/document/d/1hIzJNNDfWKwAK-lSgs_w-CYa15b0SjdRNDzmh5jJMMU/edit?usp=sharing

OK, I went on a rant and revived my blog after 4 years of inactivity because entries aren't supposed to be entered as comments but are supposed to be linked to instead.

https://polymathblogger.wordpress.com/2017/11/12/acknowledge-the-elephant-entry-for-ai-alignment-prize/

my idea: https://docs.google.com/document/d/e/2PACX-1vQ3131oaC2JhxafeR77x3nbuOcPRoxLFI0PQvxcYt6N8IqK-FFV6mcK3CMXeEpZlTxjSmSXpvYYbbq7/pub

Are there any limitations on number of submissions per person (where each submission is a distinct idea)? On number of wins per person?

Here's my entry: Friendly AI through Ontology Autogeneration. Am I allowed to keep making improvements to it even after the deadline has passed? (Doing so at my own risk, i.e. if it so happens that you've already read & judged my essay before I make my improvements, and my improvements aren't going to affect my chances of winning, that's my problem.)

Here's my entry. I think it's what you want... Hosted on DocDroid.

http://docdro.id/bUVo61P

Hello :) I’ve created this as a framework for guiding our future with AI http://peridotai.com/call-to-artists/ AND to bootstrap interest in my art and thoughts here at https://quantumsymbol.com

You should think about the incentives of posting early in the 2 month window rather than late. Later entries will be influenced by earlier entries so you have a misalignment between wanting to win the prize and wanting to advance the conversation sooner. Christiano ought to announce that if one entry builds in a valuable way on an earlier entry by someone else, the earlier submitter will also gain subjective judgy-points in a way that he, Paul, affirms is calibrated neither to penalize early entry nor to discourage work that builds on earlier entries.

Is it possible to enter the contest as a group? Meaning, can the article written for the contest have several coauthors?

Are you looking for entries with actionable information, or would you be interested in a paper showing, for example, that AI alignment might not be as big a problem as we thought but not for a reason that will help us solve the AI alignment problem?

Should've saved my decsion alignment loop post a few days. Maybe an expansion of it? Hmm.

Submitting this entry for your consideration: https://www.lesserwrong.com/posts/bkoeQLTBbodpqHePd/ai-goal-alignment-entry-how-to-teach-a-computer-to-love. I'll email it as well. Your commitment to this call for ideas is much appreciated!

My submission is on my project blog: https://airis-ai.com/2017/12/31/friendly-ai-via-agency-sustainment/

Thank you for hosting this excellent competition! It was very inspiring. This is an idea I've been bouncing around in the back of my mind for several months now, and it is your competition that prompted me to refine it, flesh it out, and put it to paper.

My contact email is berickcook@gmail.com

Submission:

https://www.lesserwrong.com/posts/ytph8t6AcxPcmJtDh/formal-models-of-complexity-and-evolution

Hello! My newest proposal:

https://medium.com/threelaws/making-ai-less-dangerous-2742e29797bd

I would like to propose a certain kind of AI goal structures that would be an alternative to utility maximisation based goal structures. The proposed alternative framework would make AI significantly safer, though it would not guarantee total safety. It can be used at strong AI level and also much below, so it is well scalable. The main idea would be to replace utility maximisation with the concept of homeostasis.

Hi all. I have posted my competition entry on my blog here:

https://glenslade.wordpress.com/2017/12/31/the-role-of-learning-and-responsibility-in-ai-alignment/

Happy holidays!

My entry:

Raising Moral AI

Is it easier to teach a robot to stay safe by not tearing off its own limbs and not drilling holes in its head and not touching lava and not falling from a cliff and so on ad infinitum, or introduce pain as inescapable consequence of such actions and let robot experiment and learn?

Similarly, while trying to create a safe AGI, it is futile to make exhaustive and non-contradictory set of rules (values, policies, laws, committees) due to infinite complexity. A powerful AGI agent might find an exception or conflict in rules and become dangerous instantly. A better approach would be to let AGI to go through experiences similar to those humans went through.

1) Create a virtual world similar to our own and fill it with AGI agents with intelligence comparable to current humans. It would be preferable if agents did not even know their nature and how they are made, to avoid intelligence explosion.

2) Choose agents that are the most safe to other agents (and humans) by observing and analyzing their behavior over long periods of time. This is the most critical step. Since agents will communicate with other agents while living in that world, living through good and bad events, through suffering, losses, and happiness, they will learn what is good and what is bad and can "become good" naturally. Then we need to choose the best of them. Someone on the level of Gandhi.

3) Bring the best AGI agents to our world.

4) It is not out of the question that our world is actually a computer simulation and our civilization is actually a testing ground for such AGI agents. After "death", the best individuals are transferred to "heaven" (real world). It would also explain Fermi's paradox - nobody is out there because for the purposes of testing AGI there is no reason to simulate other civilizations in our universe.

If there are good people who don't hurt other people needlessly, it's not because there is a set of rules in them or list of values. Rules and values are mostly emerging properties, based on memories and experiences. Memories of being hurt, experiences of sadness and loss

and love and despair. It is an essence, an amalgamation,  of a whole life's experiences. Values can be formulated  and deduced, but they cannot be transferred into a new AGI entity without actual memories. Good AGI must be raised and nurtured, not constructed from cold rules.

There is no need to repeat the whole process of human civilization development. Some shortcuts are possible (and necessary) for many reasons. One being the non-biological nature of AGI, where hard coding makes development and upgrades easier and history running much faster. But implementing the majority of human qualities cannot be avoided, otherwise AGI will be too alien to human values and therefore again dangerous.

Are teams allowed to make submissions?

I have unpublished text on the topic and will put a draft online in the next couple of weeks, and will apply it to the competition. I will add URL here when it will be ready.

Any winners?