[Link] Word-vector based DL system achieves human parity in verbal IQ tests

I'm assuming that the math section of these tests was already figured out long ago, and is currently pretty damn close to superhuman levels. Can somebody more familiar with these tests suggest which sorts of intelligence tests AI researchers might work toward mastering in the future to get even closer to human like intelligence?

Just to pimp my school, Georgia Tech offers a free course through Udacity in Knowledge-Based AI that involves programming an agent to take the Raven's progressive matrices test. I never took the course, but I wanna say from hearing other students that somewhere around 80 is the current state of the art (that's not an IQ and I'm not sure how to translate a Raven's score to an IQ).

I'm not sure I'm clear on the AI/AIG distinction. Wouldn't an AI need to be able to apply its intelligence to novel situations to be "intelligent" at all, therefore making its intelligence "general" by definition? Watson winning Jeopardy! was a testament to software engineering, but Watson was programmed specifically to play Jeopardy!. If, without modification, it could go on to dominate Settlers of Catan then we might want to start worrying.

I guess it's natural that QI tests would be chosen. They are objective and feature a logic a computer can, at least theoretically, recreate or approximate convincingly. Plus a lot of people conflate IQ with intelligence, which helps on the marketing side. (Aside: if there is one place the mind excels, it's getting more out than it started with--like miraculously remembering something otherwise forgotten (in some case seemingly never learned) at just the right moment. Word-vector embeddings and other fancy relational strategies seem to need way more going in--data-wise--than the chuck back out, making them crude and brute force by comparison.)

"These types of IQ tests measure specific abilities which are correlated with general intelligence in humans, but these specific abilities are only a small subset of the systems/abilities required for general intelligence, and probably rely on a smallish subset of the brain's circuitry."

Sounds like you're tinting your statement with a society-of-mindish perspective...would you say that's fair? There are even stronger reasons to question the importance of such results. Rather than exercising some specific dedicated IQ-test circuitry in the brain, it could just as well be that, in humans, IQ scores reflect how well-tuned your cognitive machinery is by some general measure much like an athlete's hundred-meter sprint time reflects his overall fitness as well as white muscle development in his legs. He would not achieve a low sprint time were it not for the proper functioning of his very complex, variegated biology. That one can build a simple robot that achieves faster sprint times says very little about that robot's potential for gymnastics or martial arts. For a robot to perform well on the sprint time just does not require anything like the solution to everything that evolution had to solve for human athletes to perform as well as they do.

Arithmetic ability would be another example of a metric which is nowhere near "agi-complete" since it's solvable by a relatively straightforward procedure.

In that case the solution of special "sub-tasks" of intelligence such as IQ puzzles would seem to me pretty uninformative. I'm reminded of the difference between Harry Foundalis' Phaeco which learns how to represent visual structures independently vs. earlier work in Bongard problems which 'cheated' by pre-encoding the images as logical objects.

I suppose my view on this is somewhat an inversion of what I see as the norm: others dismiss IQ tests as meaningless for human beings but find them significant metrics for machine intelligence.