NES-game playing AI [video link and AI-boxing-related comment]

Also, the behaviour at the very end is very much "oh we didn't expect that to happen forever when we wrote that utility function"

Tom's Mario page.

Tom's blog.

He promises to post more videos when he gets back from Japan.

I LOLed, for real, at the end of the video.

Wow. First thought: who is this guy who submits a really cool scientific result to a thing like SIGBOVIK? He could have sent this thing to a real conference! It's a thing no one has ever tried!

Then I checked out his website. The academic one. And the others.

Well, short description: "Superhero of Productivity". The list of stuff he created doesn't fit on his site. Sites. Also, see this remark of his,

One of the best things about grad school was that if you get your work done then you get to do other stuff too.

(I'm also at CS grad school, am happy if I have time to sleep, and my only productive output is... LW comments... does that count?)

This thing looks more and more relevant as I think about it. What it does is not just optimizing an objective function in a weird and unexpected way, but actually learning it in all its complicatedness from observed human behavior.

Would it be an overestimation to call this a FAI research paper?

I would like to see more discussion of this sort of "AI boxing".

Typically the implicit assumption I've seen is that "The AI is in a box where a limited communication channel enables it to learn about the real universe and have conversations with its designers", which I'm convinced is unacceptably fragile.

I think a version like "The AI is in a box where the laws of physics are little more than a video game, it doesn't get any direct information about the real world, and we just watch its behavior in the game world to make inferences about it" might be more interesting. Occam's Razor isn't designed to be proof against overwhelming deception like this, and so it might not be too hard to push any credence the AI gives to "this world is an illusion generated within a much more complex outer world" down to negligible values.

I thought that the most interesting observation was in the paper and not in video, namely the use of pause in Bubble Bobble. As in Tetris, it often found that if it paused, it was happier about the results than if it didn't, but each step when paused let it try again to search for good moves and since it wasn't trapped like in Tetris, it eventually found them.

I would be interested in seeing the learning process for pacman further. I guess the algorithm just ran for a couple of iterations. Also, could we run an experiment with more complicated games, like doom? There also is an obvious way to count, namely the number of killed enemies. Chess? Maybe even some poker?

I will have to read that paper.

New video by Tom7! NES AI Learnfun & Playfun, ep. 2: Zelda, Punch-Out, stocks, etc.