The Importance of Goodhart's Law

There are no generic solutions to bridging the gap between G and G*, but the body of knowledge of theory of constraints is a very good starting point for formulating better measures for corporates.

A good example from my own history of doing this is when I worked for an ISP and persuaded them to eliminate "cases closed" as a performance measurement for customer service and tech support people, because it was causing email-based cases to be closed without any actual investigation. People would email back and create a new case, and then a rep would get credit for closing that one without investigation either.

The replacement metric was one I derived via the Theory of Constraints, inspired by Goldratt's "throughput-dollar-days" measurement. The replacement metric was "customer-satisfaction-waiting-hours" - a measurement of collective work-in-progress inventory at the team level, and a measurement of priority at the ticket level.

I also made it impossible to truly "close" a case - you could say, "I think this is done", but the customer could still email into it and it would jump right back to its old place in the queue, due to the accumulated "satisfaction waiting hours" on the ticket.

Of course, the toughest part in some ways was educating new service managers that, no, you can't have a measurement of cases closed on a per-rep basis. Instead, you're going to have to actually pay attention to a rep's work in order to know if they're doing the job. (Of course, the system I developed also had ways to make it easy to see what people are working on, not only at the managerial but the team level - peer pressure is a useful co-ordination tool, if done right.)

I have no idea how well the system fared since I left the company, since it's entirely possible they found programmers since then to give them new metrics that would f**k it up, although I did design the database in such a way as to make it as close to impossible as I could manage. ;-)

Anyway, the theory of constraints positively rocks for business performance optimization, and its Thinking Processes are generally useful tools for any rationalist. They were also a big inspiration for me developing other thinking processes and ultimately mindhacking techniques, in that they showed that it's possible to think systematically even about some of the vaguest and most ill-defined problems imaginable, rigorously hone in on key leverage points, resolve conflicts between goals, and generally overcome our brains' processing limitations for analysis and planning.

[Edit to add: the Wikipedia page on thinking processes doesn't really show why a rationalist would be interested in the processes; it's useful to know that a key element of the processes are something called the "categories of legitimate reservation", which have to do with logical proof and well-formedness of argument. They are a key part of constructing and critiquing the semantic maps that are created by the thinking processes.

For example, ToC's conflict resolution method effectively maps out certain implicit assumptions in a conflict, and then invites you to logically disprove these assumptions in order to break the conflict. (That is, if you can find a circumstance where one of those assumptions is false, then the conflict will no longer exist under that circumstance - and you have a potential way out of your dilemma.)

So, in short, ToC thinking processes are mostly about constructing past, present, or future semantic maps of a situation, and applying systematic logic to validating (or invalidating) the maps' well-formedness, as a way of solving problems, creating plans, etc. Very core rationalist stuff, from an instrumental-rationality POV.]

I am reminded of one of Dijkstra's sayings:

To this very day we have organizations that measure "programmer productivity" by the "number of lines of code produced per month"; this number can, indeed, be counted, but they are booking it on the wrong side of the ledger, for we should talk about "the number of lines of code spent".

So, in short: incentives can have unintented consequences, as the incentives influence whatever you want to influence with them.

There are a lot of examples of this in e.g. Dan Ariely's book and Freakonomics.

But the best example must be the bizarre 1994 footbal (soccer) match between Barbados and Grenada. Barbados needed to win with a two goal difference.

The special incentive here was that any goal scored in the extra time would count double. Now, shortly before the end of the regular time, it was 2-1 for Barbados. Imagine what happened...

(edit: added the note about the two-goal difference, thanks Hook)

Goodhart's Law starts some other way. It's not quite right to say:

Superiors want an undefined goal G.

Mathematically speaking, the problem can't be that G is undefined. If G were really undefined in any absolute sense, then superiors would be indifferent to all possible outcomes, or would choose their utility function literally at random. That rarely happens.

Instead, the problem could be that G is difficult to articulate. It is "undefined" only in the sense that people have had trouble coming up with an explicit verbal definition for it. i know what I want and how to get it, but I don't know how to communicate that want to you ex ante. For example, maybe I want you (the night shift manager) to page me (the owner) whenever there's a decision to make that could affect whether our business keeps a client, but I've never taken any business classes and don't quite have the vocab to say that, so instead I say to only page me if it's "important." "Important" is vague, but "important' is just a map, and the map is not the territory.

Alternatively, the problem could be that G is difficult to commit to. I can define my goal in words just fine today, but I know (or you suspect) that later I will be tempted to evaluate you by some other criterion. For example, I would like to give a raise to whichever police officer does the most to keep his beat safe, and, as a thoughtful and experienced police chief, I know exactly what the difference is between a safe neighborhood and an unsafe neighborhood, and I'm happy to explain it to anyone who's interested. As one of my employees, though, you can't verify that I'm actually rewarding people for making neighborhoods safe, and not, say, giving raises to people who bring in the most money for drug busts, or who artificially lower their crime statistics, or who give me a kickback. It might make more sense for me to just announce that I'll pay people based on hours worked and complaints lodged, because that announcement is more verifiable, and thus more credible, so at least I'll be viewed as evenhanded.

Finally, as you've already pointed out, the problem could be that G is difficult or expensive to measure. Alternative measures of GDP that take into account factors like health, leisure, and environmental quality have gotten pretty good about specifying what health is, and it's easy enough to pass laws that commit agencies to valuing health in a particular way, but it's expensive to measure health, especially in any broad sense. A physical is $60; an exercise fitness exam is another $45; an STD test runs about $20; a battery of prophylactic tests for cancer and heart disease and so on is another $100 or so; a mental health exam is another $80, and then you multiply all that by the size of a valid random sample and we're talking real money. In my opinion, it would be money very, very well spent, but one can understand why GDP - which can be measured just by asking the IRS for a copy of its tax receipts - is such a popular metric. It's cheap to use.

CEV, until designed and defined properly, is just a black box that everyone universally agrees is 'good', but has little else in term of defining features.

The fact that students who are motivated to get good scores in exams very often get better scores than students who are genuinely interested in the subject is probably also an application of Goodhart's Law?

Goodhart's law seems very applicable to natural selection: the Blind Idiot God wants creatures to have higher fitness (G), and so creates targets that are correlated with fitness in the ancestral habitat (e.g., pleasure-seeking and pain-avoidance (G*)). Once you get creatures that are self-aware (us), they figure out G-star, and start optimizing for that instead of G.

Relevant.

In software development, this is (or ought to be) known as the Mini-Van Law.

Example:

The Big Mac Index has been used to compare prices across countries, as we have noted before. Argentina currently has very high prices due to a combination of inflation and a strong economy, and this shows up glaringly in the Big Mac Index.

Tyler Cowen reports (translating a Spanish original) that the Argentinian government has persuaded McDonalds to lower the price of the Big Mac (relative to other McDonalds items, and relative to competing hamburgers), so that Brazil’s Big Mac Index becomes more competitive.

http://www.statschat.org.nz/2011/11/16/goodharts-law-and-brazilian-hamburgers/

In other words, the real price of the Big Mac rose nearly twice as much as the official statistics were willing to admit, in Argentina of course. That’s not right, so the government sprang into action. The minister of the commerce department “persuaded” McDonald’s to price the Big Mac at $16, while other sandwiches at the chain are in the $21 to $23 range.

The outlets now keep the Big Mac well-hidden

http://marginalrevolution.com/marginalrevolution/2011/11/sentences-to-ponder-32.html

Getting back to trying to propose practical mitigation strategies for goodhart's law, I propose a fairly simple solution: Choose a G*, evaluate performance based on it, but KEEP IT SECRET. This of course wouldn't really work for national scale, GDP-esque kind of situations, but for corporate management situations it seems like it could work well enough. If only upper management knows what G* is, it becomes impossible to optimize for it, and everyone has to just keep working under the assumption they're being evaluated on G.

Taking it a step further, to hedge against employees eventually figuring out G* and surreptitiously optimizing for it, you could have a bounty on guessing G* - the first employee who figures out what the mystery metric G* really is gets a prize, and as soon as it's claimed, you switch to using G**

Pretty much every trick in organization design or management can be thought of as a partial solution to this problem. Listing "anarchy" or "absolute authority" explicitly on a short list of solutions is therefore a bit misleading.

At work a large part of my job involves choosing G , and I can report that Goodhart's Law is very powerful and readily observable.
Further : rational players in the workspace know full-well that management desire G, and the G is not well-correlated with G, but nonethelss if they are rewarded on G*, then that's what they will focus on.

The best solution - in my experience - is mentioned in the post: the balanced scorecard. Define several measures G1 G2 G3 and G4 that are normally correlated with G. The correlation is then more persistent : if all four measures improve it is likely that G will improve.

G1 G2 G3 G4 may be presented as simulaneous measures, or if setting four measures in one go is too confusing for people trying to prioritise (the frwer the measures the more powerful) they can be sequential. IE If you hope to improve G over 2 years, then measure G1 for two quarters, then switch the measurement to G2 for the next two and so on. (obviously you don't tell people in advance). NB this approach can eb effective, but will make you very unpopular.

Goodhart's Law is a very nice corollary to the Snafu Principle: Communication is impossible in a hierarchy.

Temple Grandin has written about the importance of finding relevant, measurable standards-- the example she gives is the number of cattle falling down on the way to slaughter. Not falling down means the genes, food, lighting, walking surface etc. are all good enough.

Thing to check: Do measures used as targets for policy always become completely useless, or do they sometimes become increasingly less useful, but not totally useless? Does culture matter? I suspect that the amount of judgement which people are allowed to mix into the system varies a lot.

She seems to believe that thinking visually will be more likely to produce such standards than (as most people do) verbally.

I'm not convinced of that-- dog and cat show standards are an example of well-defined visual standards not producing reliably good results.

I have no idea if it's even possible to have that good a standard for financial markets.

I've heard "you can't manage what you can't measure", but I think "you can't manage what you can't perceive" is better. Is it possible to generalize the idea of the king traveling incognito to see how the kingdom is doing?

We laugh at such ridiculous stories, because our societies are generally much better run than Soviet Russia.

That's not even true, when you measure results Soviet Union was ran about as well as any other country. They were just ran differently.

Anyway, it doesn't even seem mathematically obvious to me that optimizing for G* will reduce correlation between G and G*.

Let G=market value of nails, G*=number of nails. Once G* target is introduced everyone switches from making medium nails to making tiny nails, but correlation between market value of nails and number of nails among different factories is still very very high.

There is some loss of value from optimizing for G* as opposed to optimizing for G, but no incentive system is perfect, and incentive systems have costs, so simple incentive scheme might still be better than having to pay the cost of more accurate but far more complicated scheme.

1) Examples of G* should be given a cost-benefit analysis. Yeah, scammers and parasites exist, but societies that use money still seem to better off than societies that try to get rid of it.

2) It's unclear to me why you list CEV as one of the solutions. We use money to allocate limited resources. If magic nano-AI appears and resources become unlimited, why keep score at all? If it doesn't and resources stay limited, how does CEV help you distribute bread, and would you really like it to replace money? (I wouldn't. No caring daddies for me, please.)

I noticed this tendency in British running of hospitals, schools and police forces. The gov got hooked on the idea of targets and not on medicine, education and public order.

I wish there were some examples (other than the Soviet nails) ... if I had some better idea of what G and G* might actually represent, I'd be able to more easily get my head around the rest of the post.

I suggest editing a "summary break" into this post to create a "continue reading" link on the frontpage. It's the 6th button from the left atop the editing interface.

Diversity of an ecosystem is a way to reduce the impact of Goodhart's law. If different universities would use very different G* for their hiring decisions it would be harder for young researchers to optimize for any particular G*.

I don't see how hierarchical rule is a solution. Hierarchy requires the people at the top of the hierarchy to give order to people at the bottom to achieve certain outcomes and measure whether those outcomes are achieved.

Often a goal set is not based on a single set of arguments justifying it, but because it is a good compromise point between multiple arguments, motivations or interest groups. For example human rights formulations don't perfectly fulfill any groups desires (utilitarians, egalitarians, deontological groups, religious motivations etc.) but are a point of overlap between their goal sets (both utilitarians and deontologists both think torture and murder are generally bad). Similarly with GDP, economic growth is a shared interest of several groups in society.

So some instances of goodhearts law may be an observation that particular sets of goals are not being perfectly fulfilled.

body of knowledge of theory of constraints is a very good starting point for formulating better measures for corporates

I've had some interest in TOC, could you please expand on how it works to get G* closer to G?

Generally I've found TOC to be some really interesting semi-scientific stuff mixed with a ton of self promotion by goldratt.

See also, Campbell's Law

One method is to have no G*. Tell people some of the things you'll be looking at, but don't give them specific targets or tell them exactly how you will be judging G.

This is the method currently in use to assess the quality of research in departments of universities in the UK. Every department that wants to be assessed must supply certain very detailed information (e.g. for each member of staff declared as doing research, a list of their recent publications, grants held, awards received, etc.). The actual assessment is carried out behind closed doors. They give general guidelines about their criteria, but the process is one of "expert review, informed by indicators where appropriate".

This is done every few years. The data requested changes every time, and sometimes even the name of the exercise. As government funding depends on the outcome, every research-active department is desperate to get a good rating.

The same issue came up many years ago in the work of H Edwards Deming in quality control.

• Any data used to reward and/or punish people will become useless for managing the organization.

I like this article a lot. My solution (borrowed from Nassim Taleb) would be skin in the game. Any potential outcome resulting from the actions of the agent should be also affecting the agent.

Interesting: Left Anarchist, Right Libertarian, and Distributist ideals are fundamentally the same. While Right-Libertarians pay a form of lip service to the idea of hierarchical corporate capitalism, scratch them a bit and you find they long for SV startups or farmers on the American Frontier, as presented in books like The Moon Is A Harsh Mistress - family businesses or small, egalitarian workgroups like the 3 guys who founded YouTube. And Left-Anarchism and Disrtributism are pretty much the same, the difference is LA putting a mainly socially liberal sauce on it, while Distributism putting a socially reactionary medievalist-catholic sauce on it (medieval artisans loosely cooperating in guilds being their ideal).

From this angle, a lot of different people want a small-business world, apparently we can have both socially liberal and socially reactionary versions of it, we just don't know how to deal with the Economies of Scale. But apparently if we could figure that out, it would be an economic model usable by many different people. Distributist Catholics could worship the idea of the Family (as a productive economic unit), Left-Anarchist would have their non-hiearchy, and Right-Libertarians could have people get off their lawn.

CEV isn't even an idea yet. It's a collection of phrases stringing together undefined terms. What isn't undefined is self-contradictory or incoherent. I very much doubt that it is possible of being refined into anything coherent and consistent; it has contradictions built into its core.

I think the Lucas Critique is actually more general than Goodhart's Law. Goodhart is a specific form of Lucas!

It's true that GDP is not identical to national welfare. And you can come up with anecdotes where some welfare measure isn't fully captured by GDP (both positive and negative).

But GDP is useful, because it is very hard to game. The examples in your "fetishism" link are very weak. Unlike the nails example, where we can all agree that the factory made the wrong choice for society, it is far from clear that the GDP examples resulted in the wrong policy, even if GDP is only an approximation for welfare.

GDP is not a good example of Goodhart's Law. It's nothing at all like the (broken) correlation between inflation and unemployment, which varies widely depending on policy choices.