[April Fools'] Definitive confirmation of shard theory

post by TurnTrout · 2023-04-01T07:27:23.096Z · LW · GW · 8 comments

Contents

8 comments

I've written a lot about shard theory [? · GW] over the last year. I've poured dozens of hours into theorycrafting, communication, and LessWrong comment threads. I pored over theoretical alignment concerns with exquisite care and worry. I even read a few things that weren't blog posts on LessWrong.^[1] In other words, I went all out.

Last month, I was downloading gpt-neox-20b when I noticed the following:

I've concluded the following:

• Shard theory is correct (at least, for gpt-neox-20b).
• I wasted a lot of time arguing on LessWrong when I could have just touched grass [LW · GW] and ran the cheap experiment of actually downloading a model and observing what happens. 
• I totally underappreciated how much work the broader AI community^[2] has done on alignment. They are literally able to print out how many shards an "uninterpretable" model like gpt-neox-20b has.
  • They don't whine about how tough these models are to interpret. They just got the damn job done.
  • This really makes team shard's recent work [LW · GW] look naive and incomplete. I think we found subcircuits of a cheese-shard. The ML community just found all 46 shards in gpt-neox-20b, and nonchalantly printed that number for us to read.
• This is embarrassing for me, even though shard theory just got confirmed. I seriously spent months arguing about this stuff? Cringe. 
  • I did a bad job of interpreting theoretical arguments, given that I was still uncertain after months of thought. Probably this is hard and we are bad at it.
  • Stop arguing in huge comment threads about what highly speculative approaches are going to work, and start making your beliefs pay rent by finding ways to experimentally test them. 
  • See how well you can predict AI alignment results today [LW · GW] before trying to predict what happens for 500IQ GPT-5.
  • I don't ever want to hear another "but I won't be able to learn until the intelligence explosion happens or not." Correct theories leave many clues and hints [LW · GW], and some of those are going to be findable today. If even a navel-gazing edifice like shard theory can get instantly experimentally confirmed, so can your theory.

I then had GPT-4 identify some of the 46 shards. 

1. The Procrastination Shard: This shard makes the AI model continually suggest that users read just one more LessWrong post or engage in another lengthy comment thread before taking any real-world action.
2. Hindsight Bias Shard: This shard leads the model to believe it knew the answer all along after learning new information, making it appear much smarter in retrospect.
3. The "Armchair Philosopher" Shard: With this shard, the AI can generate lengthy and convoluted philosophical arguments on any topic, often without any direct experience or understanding of the subject matter.
4. Argumentative Shard: A shard that compels the model to challenge every statement and belief, regardless of how trivial or uncontroversial it may be.
5. Trolley Problem Shard: This shard makes the model obsess over hypothetical moral dilemmas and ethical conundrums, while ignoring more practical and immediate concerns.
6. The Existential Dread Shard: This shard causes the AI to frequently bring up the potential risks of AI alignment and existential catastrophes, even when discussing unrelated topics.

I'm pleasantly surprised that gpt-neox-20b shares values which I (and other LessWrong users) have historically demonstrated. The fact that gpt-neox-20b and LessWrong share many shards makes me more optimistic about alignment overall, since it implies convergence in the values of real-world trained systems.

(As a corollary, this demonstrates that LLMs can make serious alignment progress.)

---

Although I'm embarrassed and humbled by this experience, I'm glad that shard theory is true. Here's to—next time—only taking three months before running the first experiment! 🥂

1. ^{^}

   DM me if you want links to websites where you can find information which is not available on LessWrong!

2. ^{^}

   "The broader AI community" taken in particular to include more traditional academics, like those at EleutherAI.

8 comments

Comments sorted by top scores.