How to measure optimisation power

I discussed a third group's attempt: http://lesswrong.com/lw/42t/aixistyle_iq_tests/

Thanks for the links, but I don't understand your objections - when measuring the optimisation power of a system or an AI, costs such as cycles, memory use, etc... are already implicitly included in Eliezer's measure. If the AI spends all it's time calculating without ever achieving anything, or if it has too little memory to complete any calculations, it will achieve an optimisation of zero.

Related to Legg's work: Ben Goertzel's paper "Toward a Formal Characterization of Real-World General Intelligence "

Abstract:

Two new formal definitions of intelligence are presented, the ”pragmatic general intelligence” and ”efficient pragmatic general intelligence.” Largely inspired by Legg and Hutter’s formal definition of ”universal intelligence,” the goal of these definitions is to capture a notion of general intelligence that more closely models that possessed by humans and practical AI systems, which combine an element of universality with a certain degree of specialization to particular environments and goals. Pragmatic general intelligence measures the capability of an agent to achieve goals in environments, relative to prior distributions over goal and environment space. Efficient pragmatic general intelligences measures this same capability, but normalized by the amount of computational resources utilized in the course of the goal-achievement. A methodology is described for estimating these theoretical quantities based on observations of a real biological or artificial system operating in a real environment. Finally, a measure of the ”degree of generality” of an intelligent system is presented, allowing a rigorous distinction between ”general AI” and ”narrow AI.”

That's exactly it. Just specifying the problem to be solved specifies the relationship between the value of a solution, and its percentile ranking, although you may not know this relationship without fully solving the problem. If all solutions have value between 0 and 1 (e.g. you're trying to maximize your chances of succeeding at something) and half of all solutions have value at least 0.99 (so that it takes 1 bit of optimization power to get one of these) then an extra 100 bits of optimization power won't do much. It's not that the AI you ask to solve the problem isn't good enough. It's that the problem is inherently easy to find approximately optimal solutions to.

Since rigorous elicitation of agent's own goal is one of the great unsolved problems in FAI theory, a definition of optimization power might be somewhat FAI-theory-complete...

I would suggest GAI-theory-complete rather than FAI-theory-complete. FAI requires several additional considerations that go beyond that required for analysis of or implementation of an optimization algorithm with a goal system.

Could you evaluate against all possible goals and see which scores the highest in terms of optimization power?

Immediate problems with my suggestion:

• It's not very practical. There are a lot of possible functions to consider.
• It would tend to lead to the conclusion that everyone's goal is to ensure the laws of physics are being followed, and that they're succeeding perfectly.

An optimizer could be trying to maximize entropy on Earth.

The objective function that measures the value of the solution should be a part of the problem: it's part of figuring out what counts as a solution in the first place. Maybe in some cases the solution is binary: if you're solving an equation, you either get a root or you don't.

In some cases, the value of solution is complicated to assess: how do you trade off a cure for cancer that fails in 10% of all cases, versus a cure for cancer that gives the patient a permanent headache? But either way you need an objective function to tell the AI (or the human) what a "cure for cancer" is; possibly your intuitive understanding of this is incomplete, but that's another problem.

Edit: Your objection does have some merit, though, because you could have two different utility functions that yield the same optimization problem. For instance, you could be playing the stock market to optimize $$ or log($$), and the ranking of solutions would be the same (although expected values would be thrown off, but that's another issue); however, the concept of a solution that's "a tiny bit better" is different.