Is machine learning in a period of multiple discovery?

Anecdotally, it feels as though we have entered a period of multiple discovery in machine learning, with numerous individuals coming up with very similar ideas.

Logically, this can be expected when more people pursue the same low-hanging fruit. Imagine orchards in full bloom with a crowd of hungry gatherers. Initially, everyone targets the nearest fruit. Exploring a new scientific frontier can feel somewhat similar. When reading the history books on the Enlightenment, I get a similar impression.

If we are indeed in a period of multiple discovery, we should not simply go after the nearest prize; it will soon be claimed. Your time is better spent looking further afield or exploring broader horizons.

Is any of this backed by empirical evidence? No! I have simply plotted Wikipedia's list of multiple discoveries. It shows multiple discoveries increasing with population, I don't see any distinct periods, so it's inconclusive.

Draft: A cartoonishly simplified overview of some technical AI alignment efforts

Epistemic status: excessive lossy compression applied

How are people actually trying to make friendly AI? Here are few simplified examples

LessWrong has some great [LW · GW] technical [LW · GW] and critical [LW · GW] overviews [LW · GW] of alignment agendas, but for many readers they take too long to read.

But first, what is the problem we are trying to solve? I would say that we want to program AI's that have our values (or better). And we want those values to persist, even when the AI gets smarter, even after long spans of subjective time, even when encountering situations humans have never encountered before.

This is hard because we don't know how to do any of those things! So how do people propose to solve this/

Here's my attempt at cartoonishly simplified explanations of technical alignment efforts:

• Let's make AI solve it [LW · GW]

  • Just make a friendly dumb AI and that will make a smarter friendly AI, and so on - Iterated Amplification [? · GW]
  • Just make AI's debate [? · GW] each other, so that the truth comes out when we look at both sides of the argument
  • Just have the AI follow a constitution [? · GW]
  • Just make a friendly automatic alignment researcher - Superalignment

• Let's have lots of AI's interacting:

  • Just have multiple focused AIs competing in various roles, kind of like a corporation - Drexler's The Open Agency Model [AF · GW]
  • Just use dumb AI's to build an understandable world simulation, then train a smart AI in that simulation so that we can verify that it's aligned - DavidAD's plan [AF · GW]
  • Just have the AI learn and respect other beings' boundaries - Boundaries/Membranes [LW(p) · GW(p)]

• Let's make sure the goal is good [LW · GW]

  • Just design the AI to want to help humans but be maximally uncertain about how to do that, so it constantly seeks guidance - CIRL [LW(p) · GW(p)]
  • Just make a lazy AI that is satisfied after doing "enough" - Mild Optimization [? · GW]

• Let's build tools that will let us control smarter AI's

  • Just read their minds - ELK
  • Just edit their minds - Activation Steering

• Let's understand more

  • What it means to be an agent - agent foundations

Some historic proposals sounded promising but seem to have been abandoned fow now, I include this to show how hard the problem is:

• Just keep the AI in a sandbox environment where it can't cause any real-world harm while we continue to align and train it - AI Boxing [LW · GW]
• Just make an oracle AI [? · GW] that only talks, not acts
• simulate a human thinking for a very long time about the alignment problem, and use whatever solution they write down - TODO Another proposal by Christiano (2012):
• Do what we would wish for if we knew more, thought faster, were more the people we wished we were - coherent extrapolated volition (CEV) [? · GW]

I've left out the many debates over the proposals. I'm afraid that you need to dig much deeper to judge which methods will work. If you want to know more, just follow the links below.

If you dislike this: please help me make it better by contributing better summaries, and I'll be pleased to include them.

If you would like to know more, I recommend these overviews:

• 2023 - Shallow review of live agendas in alignment & safety [LW · GW] - I've taken heavily from this post which has one sentence summaries as well as much, much more
• 2022 A newcomer’s guide to the technical AI safety field [AF · GW]
• 2023 - A Brief Overview of AI Safety/Alignment Orgs, Fields, Researchers, and Resources for ML Researchers [LW · GW]
• 2023 - The Genie in the Bottle: An Introduction to AI Alignment and Risk [LW · GW]
• 2022 - On how various plans miss the hard bits of the alignment challenge [LW · GW]
• 2022 - (My understanding of) What Everyone in Technical Alignment is Doing and Why [LW · GW]

If we regularise human values, what would happen?

We want AI to take up human values, in a way that generalizes out of distribution better than our capabilities. In other words, if our AI meets friendly aliens, we want its gun skills to fail before its morals.

Human values - as expressed by actions - seem messy, inconsistent, and hypocritical. If we forced an AI to regularise them, would it be wiser, more consistent, and simply good? Or would it oversimplify and make a brittle set of values.

I would predict that a simplified and more consistent form of human values would extrapolate better to a new situation. But if they are too simplified, they will be brittle and fail to generalize or perform how we want. I expect there to be quite a lot of simplification that can be performed initially, and I expect it to be interesting and helpful.

Perhaps we could even remove the parts of human values that are to an individual's advantage. Removing self-preservation, self advantage, then ensuring consistency would lead to an interesting set of values.

It would be an interesting experimental direction. Perhaps an early experiment could make LLM answers conditional on a latent space which encodes values. It would be much like the image generators that are conditional on style. As a bonus, you would have this nice regular space of values where you can examine and play with.

We should limit the intelligence gap between machines and humans to, say, 150%. I think control will always fail for a big enough gap. The size of the gap we can manage will depend on how good our alignment and control tools turn out to be.

The strategy here is to limit machine intelligence to be C% move than the smartest human (I_H). That way, the smartest AI (I_AI) will be a function of 1) how good our alignment/control tools are and 2) how smart we can increase humans.

I_AI = I_H * C