Why are probabilities represented as real numbers instead of rational numbers?
post by Yaakov T (jazmt) · 2022-10-27T11:23:39.300Z · LW · GW · No commentsThis is a question post.
Contents
Answers 13 tailcalled 9 JBlack 2 benjamincosman None No comments
I have started going through Jaynes’ book on probability. In Chapter 1 (pg 17) he lists the basic desiderata for the theory. The first desiderata is that “Degrees of plausibility are represented by the real numbers”. I understand why we want to use numbers, and why we want continuity, but why do we specifically want to use real numbers instead of rational numbers?
Answers
If you throw a dart to uniformly hit the interval [-1, 1] x [-1, 1], what is the probability that it will hit within the unit disk?
Real numbers are complete under all sorts of important properties such as integration of continuous functions over compact areas. If you don't have this closure, the math becomes impractical.
↑ comment by Yaakov T (jazmt) · 2022-10-30T18:12:11.004Z · LW(p) · GW(p)
Cool. So in principle we could just as well use the rationals from the standpoint of scientific inference. But we use the reals because it makes the math easier. Thank you.
Replies from: tailcalled↑ comment by tailcalled · 2022-10-30T18:39:37.729Z · LW(p) · GW(p)
I should probably also mention that if you actually used rationals, the way you would do it (when running into the tough integrals) would be by just phrasing everything in terms of bounded approximations, which is basically just unrolling a construction of the real numbers. So you might as well just use real numbers.
You can't get continuity without real numbers. Technically the required property is completeness, but the intuition is the same. For example, suppose you have some idea that some random number is "equally likely" in the range from 0 to 1 in some sense. You want to get a number that expresses the plausibility that its square is less than 1/2. You'll find that every rational number is either too large, or too small.
One of the intuitions of continuity is that if you have a continuous curve that goes from too small to too large, there must be a point somewhere in the middle where it is just right. If you apply this requirement to rational numbers (via the Dedekind Cut construction), you get the real numbers. Real numbers are the smallest extension of the rational numbers in which you have this sort of continuity, and so if you want a mathematical theory of plausibilities that allows rational values and continuity then you need real numbers.
↑ comment by Yaakov T (jazmt) · 2022-10-30T18:09:53.235Z · LW(p) · GW(p)
Thank you.
I am a little confused. I was working with a definition of continuity mentioned here https://mathworld.wolfram.com/RationalNumber.html : "It is always possible to find another rational number between any two members of the set of rationals. Therefore, rather counterintuitively, the rational numbers are a continuous set, but at the same time countable."
I understand that Rationals aren't complete, and my question is why this is important for scientific inference. In other words, are we using the reals only because it makes the math easier, or is there a concrete example of inference which completeness helps with?
Specifically, since the context Jaynes is interested in is designing a (hypothetical) robot's brain, and in order to achieve that we need to associate degrees of plausibility with a physical state, I don't see why that entails the property of completeness which you mentioned? In fact, we mostly use digital and not analog computers, which use rational approximations for the reals. What does this system of reasoning lack?
Replies from: JBlack↑ comment by JBlack · 2022-10-31T06:03:49.102Z · LW(p) · GW(p)
The word used for the property referred to in the Wolfram article really should be dense, not continuous. The set of rationals is dense, but incomplete and totally disconnected.
The main property lacking is exactly what I stated earlier: for some perfectly reasonable questions, rationals only allow you to work with approximations that you can prove are always wrong. That's mathematically very undesirable. It's much better to have a theory in which you can prove that there exists a correct result, and then if you only care about rational approximations you can just find a nearby rational and accept the error.
I don't know if there are 'natural' events with irrational probabilities, but I think we can construct an artificial one? Roughly, let X be the event that a uniformly-random number in [0,10) is < pi; then P(X)=pi/10.* So if you were forced to give X a rational probability, what probability would you choose?
*One can't actually produce a uniformly-random real number, but here's a real-world procedure that should be equivalent to X: start generating an infinite string of digits v_i one at a time using a d10. For each digit, if it is less than the i'th digit of pi return YES, if it is greater return NO, if it is equal continue to rolling the next digit. So e.g. a roll of 2 is an immediate YES; a roll of 3 -> 1 -> 7 is a NO. This procedure takes unbounded time, but with probability 1 it terminates eventually.
No comments
Comments sorted by top scores.