AlphaGo Zero and the Foom Debate

I think it is interesting to look at the information efficiency. So we can ask how much better it is at absorbing information compared to humans civilization. So if we gave it and humanity a new problem how many trials/tests would we expect it to need compared to humans.

I'm guessing serious go players (who may have contributed to human knowledge) play on average of 2 games per day? Over a 60 year lifespan. Gives 44k games per person over their life time. Around 60 million go players currently today. I'm assuming 120 million over the lifespan of humanity, because of the rapid growth in our population. This gives roughly 5.25 x 10 ^12 games.

Alpha Go played 5 million games against itself. So it is roughly 1 million times more efficient than historical civilization (at learning Go).

Number are very much back of the envelope kind of thing. If anyone knows more about the likely number of games an average person plays I would be interested.

Caveats:

If we were both starting to learn a game from this point in history, we would expect humans to be more efficient at information extraction than we were historically as we had bad communication between them and they knowledge might have to be re-learnt many times.

Go might be an unusually bad game for transferring knowledge gained between people. We could look at people learning go and see how much they improved by playing the game vs by reading strategies/openings. We could look at other important tasks (like physics and programming) and see how much people improved by doing the activity vs reading about the activity.

Edit: So we should be really worried if new variant of Alpha Go comes out that only need 44k games to get as good as it did.

This might not be the actually important or central questions, but given my background, it's where my brain keeps going...

I need to read up more about the techniques used, but they are clearly very general and basic. Given that, what other games or problems would we expect this system to master, versus ones that it would not master?

Go is a strange mix of a super hard problem and a strangely easy problem.

The game takes two hundred moves over a 19x19 board where large portions of that board might constitute not-obviously-wrong moves at any given time, often with sharp discontinuities in value where a slight adjustment changes groups from alive to dead or vice versa. Thus, using pure ply search and considering all possibilities is a non-starter and you need to have some way of figuring out what moves to consider at all. Doing anything with raw calculation doesn't work.

On the other hand, Go has some things that make it a good target for this algorithm and for ML in general. Go is well-defined and bounded. Go has a limited number of possible moves - yes it's very large, but you can know all of them and see what your ML function thinks of them for at least 1 ply. Go is complete information and deterministic. Go has a well-defined victory condition and no in-between goals (which AlphaGo exploits quite well). Go has no state - if you know the board and number of captured stones you don't care about how you got there, which goes with complete information since you don't need a prior on hidden information.

This means that Go has a lot of nice properties that allow you to use such a simple architecture. What would happen with other games?

This should work on Chess, with almost no modifications, so I'm wondering why they haven't done that, and whether we'll see it in a week. Would it crush the best computer chess programs out there? Wouldn't shock me if it did. Another thing I'd want to see is if it's better in the middle/end game versus the opening and by how much. I'd assume that AlphaChess would be basically perfect in the endgame and super strong in the middle game, whereas the opening is where we'd find out if past knowledge is actually worth a damn at all. Note that even if AlphaChess crushes everything after three days, we might learn interesting things by looking at it when it's only worked for less time than that and isn't yet crushing existing programs (even if that's a pretty short window, you can find it by continuously testing it against Stockfish or whatever between cycles). Similarly, I bet we'd get some interesting results if we saw what partially trained versions of AlphaGo-Zero do well versus poorly in Go, as well.

One of the nice properties of such games is that playing as if you are playing against an expert is basically correct, whereas in games with hidden information that is no longer a safe assumption. Would that prevent the system from being able to train against itself without getting exploited by strategies it had trained itself not to use? Could you get it to play Hold 'em this way without getting exploited?

In some senses I'm super impressed and scared with DeepMind smashing Go this badly, while in other ways it seems cool but not scary.

I've become substantially more confident in the past two years about Robin's position on the differences between humans and chimps.

Henrich's book The Secret Of Our Success presents a model in which social learning is the key advantage that caused humans to diverge from other apes. This model is more effective than any other model I've seen at explaining when human intelligence evolved.

To summarize chapter 2: Three year old humans score slightly worse than three year old chimpanzees on most subsets of IQ-like tests. "Social learning" is the only category where the humans outperform chimpanzees. Adult humans outperform chimpanzees on many but not all categories of IQ-like tests.

If humans have an important improvement in general intelligence over chimpanzees, why are its effects hard to observe at age 3?

Promoted to Featured for incorporating new and relevant evidence into the Foom Debate.

"However, if even with the human special sauce we're to expect AGI capabilities to be slow, domain-specific, and requiring feed-in from a big market ecology, then the situation we see without human-equivalent generality special sauce should not look like this. "

Actually, most of what humans know is from other humans. This is not unrelated to generality: you cannot deduce facts about China from your experiences if you exclude what you have heard and read from other humans. Any general AI will be in the same situation. Most of what they know will have to come from the testimony of human beings. This is unlike the situation with Go, where everything is already available by considering the board and the moves.

(Obviously I meant if you are not Chinese and have not been in China.)