True numbers and fake numbers

I think there is a tale to tell about the consumer surplus and it goes like this.

Alice loves widgets. She would pay $100 for a widget. She goes on line and finds Bob offering widgets for sale for $100. Err, that is not really what she had in mind. She imagined paying $30 for a widget, and feeling $70 better off as a consequence. She emails Bob: How about $90?

Bob feels like giving up altogether. It takes him ten hours to hand craft a widget and the minimum wage where he lives is $10 an hour. He was offering widgets for $150. $100 is the absolute minimum. Bob replies: No.

While Alice is deciding whether to pay $100 for a widget that is only worth $100 to her, Carol puts the finishing touches to her widget making machine. At the press of a button Carol can produce a widget for only $10. She activates her website, offering widgets for $40. Alice orders one at once.

How would Eve the economist like to analyse this? She would like to identify a consumer surplus of 100 - 40 = 60 dollars, and a producer surplus of 40 - 10 = 30 dollars, for a total gain from trade of 60 + 30 = 90 dollars. But before she can do this she has to telephone Alice and Carol and find out the secret numbers, $100 and $10. Only the market price of $40 is overt.

Alice thinks Eve is spying for Carol. If Carol learns that Alice is willing to pay $100, she will up the price to $80. So Alice bullshits Eve: Yeh, I'm regretting my purchase, I've rushed to buy a widget, but what's it worth really? $35. I've over paid.

Carol thinks Eve is spying for Alice. If Alice learns that they only cost $10 to make, then she will bargain Carol down to $20. Carol bullshits Eve: Currently they cost me $45 to make, but if I can grow volumes I'll get a bulk discount on raw materials and I hope to be making them for $35 and be in profit by 2016.

Eve realises that she isn't going to be able to get the numbers she needs, so she values the trade at its market price and declares GDP to be $40. It is what economist do. It is the desperate expedient to which the opacity of business has reduced them.

Now for the twist in the tale. Carol presses the button on her widget making machine, which catches fire and is destroyed. Carol gives up widget making. Alice buys from Bob for $100. Neither is happy with the deal; the total of consumer surplus and producer surplus is zero. Alice is thinking that she would have been happier spending her $100 eating out. Bob is thinking that he would have had a nicer time earning his $100 waiting tables for 10 hours.

Eve revises her GDP estimate. She has committed herself to market prices, so it is up 150% at $100. Err, that is not what is supposed to happen. Vital machinery is lost in a fire, prices soar and goods are produced by tedious manual labour, the economy has gone to shit, producing no surplus instead of producing a $90 surplus. But Eve's figures make this look good.

I agree that there is a problem with the consumer surplus. It is too hard to discover. But the market price is actually irrelevant. Going with the number you can get, even though it doesn't relate to what you want to know is another kind of fake, in some ways worse.

Disclaimer: I'm not an economist. Corrections welcomed.

Some of your fake numbers fall out of the common practice of shoehorning a partial order into the number line. Suppose you have some quality Foo relative to which you can compare, in a somewhat well-defined manner in at least some cases. For example, Foo = desirable: X is definitely more desirable to me than Y, or Foo = complex: software project A is definitely more complex than software project B. It's not necessarily the case that any X and Y are comparable. It's then tempting to invent a numerical notion of Foo-ness, and assign numerical values of Foo-ness to all things in such a way that your intuitive Foo-wise comparisons hold. The values turn out to be essentially arbitrary on their own, only their relative order is important.

(In mathematical terms, you have a finite (in practice) partially ordered set which you can always order-embed into the integers; if the set is dynamically growing, it's even more convenient to order-embed it into the reals so you can always find intermediate values between existing ones).

After this process, you end up with a linear order, so any X and Y are always comparable. It's easy to forget that this may not have been the case in your intuition when you started out, because these new comparisons do not contradict any older comparisons that you held. If you had no firm opinion on comparative utility of eating ice-cream versus solving a crossword, it doesn't seem a huge travesty that both activities now get a specific utility value and one of them outranks the other.

The advantages of this process are that Foo-ness is now a specific thing that can be estimated, used in calculations, reasoned about more flexibly, etc. The disadvantages, as you describe, are that the numbers are "fake", they're really just psychologically convenient markers in a linear order; and the enforced linearity may mislead you into oversimplifying the phenomenon and failing to investigate why the real partial order is the way it is.

Assuming you broadly subscribe to the notion of "true numbers" and "fake numbers", how do you classify the following?

Food calories [pollid:595]

The position of an object's centre of mass [pollid:596]

The equilibrium price [pollid:597]

A population's carrying capacity [pollid:598]

The population mean [pollid:599]

Another example of a fake number is "complexity" or "maintainability" in software engineering.

Yet another is "productivity". In fact, most of software engineering consists of discussions of fake numbers. :/ This article (pdf) discusses that rather nicely.

Another example of a fake number is "complexity" or "maintainability" in software engineering. Sure, people have proposed different methods of measuring it. But if they were measuring a true number, I'd expect them to agree to the 3rd decimal place, which they don't

Why is such precision required for something to count as a 'measurable quantity'? Depending on how you do the measurements, measurements of (e.g.) prices don't always agree to two decimal places, let alone three.

But we should probably try harder to find measurable components of "intelligence", "rationality", "productivity" and other such things, because we'd be better at improving them if we had true numbers in our hands.

Sure, though IQ is already a triumph of psychometrics, and Stanovich is working on the first RQ (rationality quotient) test.

Let me throw in what might be a useful term: "unobservable".

Take, for example, the standard deviation of a time series. We can certainly make estimates of it, but the actual volatility is unobservable directly, we can only see its effects. A large chunk of statistics is, in fact, dedicated to making estimates of unobservable quantities and figuring out whether these estimates are any good.

Another useful term is "well-defined". For example, look at inflation. Inflation in general (defined as "change in prices", more or less) is not well-defined and different people can (and do) propose various ways to quantify it. But if you take one specific measure, say in the US a particular CPI and define it as a number that comes out of specific procedure that the BLS performs every month, then it becomes well-defined.

I suspect this distinction between "real" and "fake" numbers is blurrier than you are describing.

Consider voltage in classical physics. Differences in voltages are a real measurable quantity. But the "absolute" voltage at a given point is a mathematical fiction.

Or consider Kolmogorov complexity. It's only defined once you fix a specific Turing machine (which researchers rarely bother to do.) And even then, it's not decidable. Is that a real number or a fake number?

So. Entropy: real or fake number?

... even if you don't know about quantum mechanics?

To make it worse, multiplicity!

I instantly disliked your terminology of true vs fake. If I understand it correctly, you are making a distinction between widely agreed upon quantification procedures (like measuring weight) and those which are are either not well defined or contentious (cheeseburger surplus). You do not seem to stipulate that the output of these procedures be useful in any way, but maybe it is implied? Anyway, I would call these "metrics", not "numbers".

If I were to try to quantify the "trueness" of these "numbers", I would look into repeatability of the quantification procedures and their robustness to small deviations (an equivalent of well-posedness in mathematics) and, separately, their acceptance level, i.e. the fraction of the experts in the area who use the same procedure. This gives you at least two separate dimensions, and I am sure there are others I overlooked. Additionally, only useful metrics are worth putting an effort in, though I am not sure how to quantify usefulness in a repeatable, acceptable and useful way (trying to be reflectively consistent here).

In summary, I would first identify which metrics would be useful, then figure out several robust ways to design them, then work on the acceptance level.

I believe the best example of 'fake numbers' may be the measurement of IQ. The problem of this sort of fake numbers is that it is not certain to tell whether IQ really represents our true intellectual being but people still use it to be judgmental or even to justify their study not knowing when to stop to regard it as a simple reference.

Fake numbers seem to prevail in our professional life as companies do quantify people's labor thanks to technology. They might be good estimates but that kind of numerical fixation affects people's mind tremendously so that the moment the numbers are revealed it now controls the people. It won't stay as a mere measurement reflecting the phenomena it gathered.

An example of a "fake number" is utility. I've never seen a concrete utility value used anywhere, though I always hear about nice mathematical laws that it must obey.

I'm sure that in AI research many programs have been written around a specific well-defined utility function. Or, by "utility" you mean utility for a human? The "complexity of value" thesis is that the latter is very hard to define / measure. I'm not sure it makes it a "bad" concept.

An example of a "true number" is mass. We can measure the mass of a person or a car, and we use these values in engineering all the time. An example of a "fake number" is utility. I've never seen a concrete utility value used anywhere, though I always hear about nice mathematical laws that it must obey.

It is interesting that you choose mass as your prototypical "true" number. You say we can "measure" the mass of a person or car. This is true in the sense that we have a complex physical model of reality, and in one of the most superficial levels of this model (Newtonian mechanics) there exist some abstract numbers which characterise the motions of "objects" in response to "forces". So "measuring" mass seems to only mean that we collect some data, fit this Newtonian model to that data, and extract relatively precise values for this parameter we call "mass".

Most of your examples of "fake" numbers seem to be to be definable in exactly analogous terms. Your main gripe seems to be that different people try to use the same word to describe parameters in different models, or perhaps that there do not even exist mathematical models for some of them; do you agree? To use a fun phrase I saw recently, the problem is that we are wasting time with "linguistic warfare" when we should be busy building better models?

I don't think that dividing reality into true numbers and fake numbers is very useful.

I think it's more useful to ask yourself whether you can measure something for reasonable effort that's a good predictor of some outcome you care about. You should also ask whether it continues to be a good predictor if you optimise towards it.

Discussing whether IQ is the real measure of intelligence is irrelevant. The important question is whether it predicts performance for tasks you care about.

Apart seeking for quantities that are good predictors it also makes sense to seek for quantities that are easy to measure. If you have a way to gather a lot of data cheaply and you can afterwards analyse what you can predict with it. That's one of the reasons I always try to talk someone into doing the work of turning Anki review data in daily cognitive measurement scores. I don't care whether the cognitive measurement from the Anki data is more of less real than IQ.

If I get it calculated for every day without having to spend additional time, that's very valuable. IQ tests are relatively expensive because they take time and if you take the same test multiple time you train it in a way that make sit useless. We can also analyse whether it provides a good predictors for other outcomes we care about.

Another example: financial engineers treating correlation of housing prices as if they were they were concrete and fixed.

I think utility is a real number, it just happens to be a persons assigned value to a certain thing. If I assign 100 to something, then the number really is 100 as far as I am concerned - the fact that there is no instrument to measure this clouds the issue. It is similar to survey results - none of the numbers are absolutely measurable, as each is the persons answer to that question at that time.