Contra Yudkowsky on 2-4-6 Game Difficulty Explanations

post by Josh Hickman (josh-hickman) · 2024-09-08T16:13:33.187Z · LW · GW · 1 comments

This is a link post for https://xn--2r8hmb.ws/blog/2-4-6/

Contents

  Background
  The Core Insight
  A Practical Approach
  The Tactical Insight
  Why People Struggle with the 2-4-6 Game
  Why This Stuck In My Craw
None
1 comment

To be clear from the outset: I don't believe Eliezer Yudkowsky would make any of the mistakes I describe in this post when playing 2-4-6-alike games. I've performed well in novel pattern-finding challenges, but not necessarily better than I imagine Yudkowsky would. I'm confident he correctly understands and would emphatically agree with everything I'm about to say. My aim here is to be pedantically precise about how this idea is presented to the public because I believe the fundamental difficulty people face when tackling the 2-4-6 game is both simple to describe correctly and valuable to consider.

Background

Let's review some required reading [? · GW]. In the 2-4-6 game, participants are asked to characterize a pattern that groups 3-number sequences into two buckets. Both Yudkowsky's explanation and his fictional representation emphasize the importance of seeking both "no" and "yes" answers. While this advice is correct, I find it to be an incomplete description of how to approach the task effectively. By addressing what I see as a "type error" in the typical thinking about this game, we can clarify how to play it well enough that even reasonably inexperienced beginners could perform competently.

The Core Insight

The error lies in thinking that you want to get an equal mix of "yes" and "no" answers per se. In reality, every answer provides (at most) one bit of information, and your goal should be to use that information to split the space *of explanations* in half as efficiently as possible. You're not looking for sets of numbers; you're searching for rules. The time to stop is when the information you're getting from the judgments ceases to surprise you, despite still being eager for additional bits about the discriminator.

A Practical Approach

Here's a good rule of thumb for playing the 2-4-6 game:

1. Think of 4 different explanations that are compatible with everything you've observed so far.
2. Use those to construct a triplet that would yield a "yes" for two explanations and a "no" for the other two.
3. Ask about this triplet.
4. Reject the incompatible explanations and continue the process.

The Tactical Insight

Even framing the search as being for both "no"-ish and "yes"-ish answers misses the most crucial tactical insight. In this game (which is essentially a variant of "Guess Who?"), you don't want to ask overly specific questions like "Is the number always even, greater than 10, and divisible by 3?" You'll get a yes-or-no answer, but it will, on average, provide much less than one bit of information.

This tactical insight—the idea that when learning, you should be maximizing information gain—addresses many other flaws in approach. Without it, you'll likely be outperformed by even a moderately skilled player who understands this principle.

Why People Struggle with the 2-4-6 Game

My theory for why people often perform poorly at the 2-4-6 game is that they're not optimizing for information gain. Instead, they're using heuristics that help them seem reasonable to others, but are ineffective in games like this. I believe this explanation aligns more closely with reality than either "confirmation bias" (which doesn't adequately capture that this is a standard strategic failure rather than a cognitive bias) or "positive bias."

Why This Stuck In My Craw

When I think about safe exploration (as an AI safety topic), it seems "theory-blind" in a similar way to the novice 2-4-6 players. We want to find a good classifier/policy, but often avoid consideration of classifiers/policies as options to weigh, instead focusing on specific tests/actions. I want to continue thinking about how to build a system that naturally phrases such problems so that (as an example) value of information questions are just a special case of a broader optimization. I think using a more detailed description of ideal play in examples like this could help our collective vocabulary in that endeavor.

1 comments

Comments sorted by top scores.