What's So Bad About Ad-Hoc Mathematical Definitions?

Shannon mutual information doesn't really capture my intuitions either. Take a random number X, and a cryptographically strong hash function. Calculate hash(X) and hash(X+1). 

Now these variables share lots of mutual information. But if I just delete X, there is no way an agent with limited compute can find or exploit the link. I think mutual information gives false positives, where Pearson info gave false negatives.

So Pearson Correlation => Actual info => Shannon mutual info. 

So one potential lesson is to keep track of which direction your formalisms deviate from reality in. Are they intended to have no false positives, or no false negatives. Some mathematical approximations, like polynomial time = runnable in practice, fail in both directions but are still useful when not being goodhearted too much.

This is related to something I never quite figured out in my cognitive-function-of-categorization quest. How do we quantify how good a category is at "carving reality at the joints"?

Your first guess would be "mutual information between the category-label and the features you care about" (as suggested in the Job parable in April 2019's "Where to Draw the Boundaries?" [LW · GW]), but that actually turns out to be wrong, because information theory has no way to give you "partial credit" for getting close to the right answer, which we want. Learning whether a number between 1 and 10 inclusive is even or odd gives you the same amount of information (1 bit) as learning whether it's over or under 5½, but if you need to make a decision whose goodness depends continuously on the magnitude of the number, then the high/low category system is useful and the even/odd system is not: we care about putting probability-mass "close" to the right answer, not just assigning more probability to the exact answer.

In January 2021's "Unnatural Categories Are Optimized for Deception" [LW · GW], I ended up going with "minimize expected squared error (given some metric on the space of features you care about)", which seems to work, but I didn't have a principled justification for that choice, other than it solving my partial-credit problem and it being traditional. (Why not the absolute error? Why not exponentiate this feature and then, &c.?)

Another possibility might have been to do something with the Wasserstein metric, which reportedly fixes the problem of information theory not being able to award "partial credit". (The logarithmic score is the special case of the Kullback–Leibler divergence when the first distribution assigns Probability One to the actual answer, so if there's some sense in which Wasserstein generalizes Kullback–Leibler for partial credit, then maybe that's what I want.)

My intuition doesn't seem adequate to determine which (or something else) formalization captures the true nature of category-goodness, to which other ideas are a mere proxy.

Curated. It's easy enough to say that getting your definitions correct is important; this post helps far beyond that by actually offering gears-level advice on how to get your definitions/metrics correct.

Pushing symbols around (or the metaphorical equivalent) might be necessary for building intuitions. Incidentally, they call novice chess players woodpushers.

When I teach piano improv, I show my students a way of making decent-sounding notes that takes about 30 seconds to explain. And then I have them play that way for a long time, only occasionally adding complexity - a new voicing, a different chord order.

Likewise, to train a neural network, you let it make blind guesses, then tell it how it did, until it gets very good at finding the right answer. There seems to be a lot of value in messing around as a form of training.

I get the sense that in your model, blind calculation is a distraction from intuitive problem solving. In my model, blind calculation builds intuition for the problem. In both our models, intuitive problem-solving must be followed by proof, which is the step that Pearson skipped.

I really appreciate this post.

I've thought for a long time that people on this website have a bias to just throw down some maths without asking about the philosophical assumptions behind it. And then once they've produced something beautiful, loss aversion makes them reluctant to question it too deeply.

(This is why I've focused my attention on Agent Meta-Foundations [LW · GW]).

The VARIANCE of a random variable seems like one of those ad hoc metrics. I would be very happy for someone to come along and explain why I'm wrong on this. If you want to measure, as Wikipedia says, "how far a set of numbers is spread out from their average value," why use E[ (X - mean)^2 ] instead of E[ |X - mean| ], or more generally E[ |X - mean|^p ]? The best answer I know of is that E[ (X - mean)^2 ] is easier to calculate than those other ones.

This post reminds me of non-standard analysis.

The story goes like this: in the beginning, Leibniz and Newton developed calculus using infinitesimals, which were intuitive but had no rigorous foundation (which is to say, ad-hoc). Then, the $ϵ-\delta$ calculus was developed, which meant using limits instead, and they had a rigorous foundation. Then, despite much wailing and gnashing of teeth from students ever after, infinitesimals were abandoned for centuries. Suddenly came the 1960s, when Abraham Robinson provided a rigorous formalism for infinitesimals, which gave birth to non-standard (using infinitesimals) analysis to be contrasted with standard (using $ϵ-\delta$) analysis.

So now there is continuous low-grade background fight going on where people do work in non-standard analysis but then have to convert it into standard analysis to get published, and the non-standards say theirs is intuitive and the standards say theirs is formally just as powerful and everyone knows it already so doing it any other way is stupid.

The way this relates to the post is claims like the following, about why non-standard analysis can generate proofs that standard analysis (probably) can't:

The reason for this confusion is that the set of principles which are accepted by current mathematics, namely ZFC, is much stronger than the set of principles which are actually used in mathematical practice. It has been observed (see [F] and [S]) that almost all results in classical mathematics use methods available in second order arithmetic with appropriate comprehension and choice axiom schemes. This suggests that mathematical practice usually takes place in a conservative extension of some system of second order arithmetic, and that it is difficult to use the higher levels of sets. In this paper we shall consider systems of nonstandard analysis consisting of second order nonstandard arithmetic with saturation principles (which are frequently used in practice in nonstandard arguments). We shall prove that nonstandard analysis (i.e. second order nonstandard arithmetic) with the -saturation axiom scheme has the same strength as third order arithmetic. This shows that in principle there are theorems which can be proved with nonstandard analysis but cannot be proved by the usual standard methods. The problem of finding a specific and mathematically natural example of such a theorem remains open. However, there are several results, particularly in probability theory, whose only known proofs are nonstandard arguments which depend on saturation principles; see, for example, the monograph [Ke]. Experience suggests that it is easier to work with nonstandard objects at a lower level than with sets at a higher level. This underlies the success of nonstandard methods in discovering new results. To sum up, nonstandard analysis still takes place within ZFC, but in practice it uses a larger portion of full ZFC than is used in standard mathematical proofs.

Emphasis mine. This is from On the Strength of Nonstandard Analysis, a 1986 paper by C Ward Henson and H Jerome Keisler. I found the paper and this part of the quote from a StackOverflow answer. Note: you will probably have to use sci-hub unless you have Cambridge access; the find-free-papers browser tools seem to mismatch this with another later paper by Keisler with an almost identical title.

I now treat this as a pretty solid heuristic: when it comes to methods or models, when people say it is intuitive, they mean that it chunks at least some stuff at a lower level of abstraction. Another math case with similar flavor of claim is Hestenes' Geometric Algebra, which mostly does it by putting the geometric structure at the foundation, which allows humans to use their pretty-good geometric intuition throughout. This pays out by tackling some questions previously reserved for QM with classical methods, among other neat tricks.

For the record I do not know how to do non-standard analysis even a little; I only ever knew what it was because it gets a footnote in electrical engineering as "that thing that let us figure out how to convert between continuous and discrete time."

I liked this post. It partially answers the question "why is it useful for selection theorems like VNM, Cox to take in several intuitions and prove that something is uniquely specified?" It seems that

• Ad-hoc mathematical definitions result from underconstrained sets of intuitions, and are not useful because they are but one in a large space of possible definitions
• Good definitions result from perfectly constrained sets of intuitions. In addition to the uniqueness, you have all intuition required to uniquely specify a definition, and so you probably have captured every important intuition

Good definitions/theorems can also tell us about our intuitions:

• Impossibility theorems result from overconstrained sets of intuitions, and are often required to tell you that your intuitions are wrong
• Very good definitions (e.g. those in fundamental theorems) result from consistent overconstrained sets of intuitions, and give additional evidence that our intuitions are consistent

Here's an example to show where Pearson's approach goes wrong:

A Q comes in two types: an X or a Y.

• An X comes in two types: A1 or B2.
• A Y comes in two types: A2 or B1.

You need to reveal the A/B- and 1/2-data for your Qs, while keeping a secret whether the Q is an X or a Y.

The A vs. B and 1 vs. 2 properties of a Q are uncorrelated with whether it's an X or a Y. You can reveal one or the other, but not both, while keeping the secret. If you reveal both, even though neither piece of information is correlated with the secret identity, you reveal the secret.

Intuition can give us a rough sense of how much a constraint idea limits the size of the search space, and how likely it is to be valid.

The right way to go is to brainstorm a large number of possible constraint ideas, and use intuition to prioritize them for proof and eventual rollout of a solution.

The wrong way to go is to jump straight from idea to proof, or even worse straight from idea to execution, as happened in Pearson's case. What leads us down this false path? Mathematical symbols, famous names, being a tall person with a deep voice, you know the drill.

The key challenge here is to come up with a set of intuitive arguments which uniquely specify a particular definition/metric, exactly like a set of equations can uniquely specify a solution.

For another perspective on this challenge: given the math, can you infer the intuitive concept it's formalizing [LW · GW]?

Meta: I like the use of the Recap and In Short sections of the post, which in my opinion did an excellent job of summarizing and then testing whether I followed the post correctly. By this I mean that if the In Short section wasn't relatively clear, than I missed something important about the post.

The key challenge here is to come up with a set of intuitive arguments which uniquely specify a particular definition/metric, exactly like a set of equations can uniquely specify a solution. If our arguments have “many solutions”, then there’s little reason to expect that the ad-hoc “solution” we chose actually corresponds to our intuitive concept.

Maybe I'm missing something in the post, but why is this the case? Isn't it arbitrary to suppose that only one possible metric exists that fully 'solves' the problem?

Delightful post.

Another, hypothetical example of ad-hoc definitions (maybe this is wrong, I'm not an hydraulics engineer): Suppose we are trying to design lubricants for a bearing. We might try to quantify the "slippyness" of a liquid by pouring it on a sloped table of standard length and timing how long it takes to flow off the table. What we really want is viscosity, which (for a Newtonian fluid) is invariant to shear rate and lots of other things. But "slippyness" is not a good abstraction, because it depends on viscosity, cohesion and adhesion forces, etc.

We can tell slippyness is not uniquely constrained because when we change parameters like pour rate, table slope, length, and composition, there is no simple law that relates these different slippyness measures. Force of a Newtonian fluid in rheometer is linear in shear rate.

Later, we could observe that viscosity applies to liquids and gases, whereas slippyness only applies to liquids, giving us a little more evidence that viscosity is a better abstraction. When designing the bearing, matching the viscosity of the lubricant to the bearing forces proves better than using slippyness.

Is there a reference on the events with the Bell labs ? I can imagine some scenarii where the military transmits some information an can sort of engineer what the adverse party can read (for example Eve can read the power supply of some device, Alice must then add sufficient noise on the power supply), but none seems realistic in the context. 

Sorry if this is a stupid question, but is it true that p(X,Y)^2 has the degrees of freedom as you described? If X=Y is a uniform variable on [0,1] then p(X,Y)^2 = 1 but P(f(X),g(Y))^2 =/= 1 for (most) non-linear f and g. 

In other words, I thought Pearson correlation is specifically for linear relationships so its variant under non-linear transformations. 

Something that bothers me about the Shannon entropy is that we know that it's not the most fundamental type of entropy there is, since the von Neumann entropy is more fundamental.

A question I don't have a great answer for: How could Shannon have noticed (a priori) that it was even possible that there was a more fundamental notion of entropy?