Book Club Update and Chapter 1

Is there a PDF version that doesn't have the text looking faded and hard-to-read? Even zoomed in to page width on a 22" monitor it is not pleasant to look at. Or maybe there is some easy way to convert it (with all formatting intact) into a nicer looking font? I tried copy+pasting into Word but it appears to have not been properly OCRed and there are a lot of errors.

Guess they're different every time. There are more pairs of different compounds from a selection group than pairs of same ones. (For any given compound, there are 9 matches and 10 non-matches.)

Probability that both compounds are A = P(1st is A)P(2nd is A) = (1/2)(9/19) = 0.24

Probability that both are B = 0.24

Probability that both are same = 0.47

Probability that they are different = 0.53

Conclusion: Always predict they are different.

More detail in the protocol would be helpful. For example, do you get to repeatedly use the same bottle of the set? If so, I can do the following for a 20 trial set: Pick bottle 1. Now run through the other 19 bottles and guess for each that it is different from bottle 1. I'll be correct 10 out of 19 trials. This method generalizes in a fairly obvious fashion although it isn't clear to me if one is going to do n trials whether this is actually the optimal procedure for maximizing how often you are correct. I suspect that one can do better but it isn't clear to me how.

This sounds like a great applied exercise for Chapter 3 on elementary sampling theory. ;)

Because only MoR is addictive enough to keep people reading and fannishly discussing it. There have been attempts to read/blog through many relevant texts like Jaynes or Dennett's Freedom Evolves, but inevitably, people lack the Conscientiousness to finish them. (There is a lesson here for academics: propaganda works.)

There is a separate DST column for daylight saving time adjustment. 1 in the DST column means the time zone needs to be adjusted with daylight saving time

Oops.

Speaking of Chapter 1, it seems essential to point out another point that may be unclear on superficial reading.

The author introduces the notion of a reasoning "robot" that maintains a consistent set of "plausibility" values (probabilities) according to a small set of rules.

To a modern reader, it may make the impression that the author here suggests some practical algorithm or implementation of some artificial intelligence that uses Bayesian inference as a reasoning process.

I think, this misses the point completely. First: it is clear that maintaining such a system of probability values even for a set of simply Boolean formulas (consistently!) amounts to solving SAT problems and therefore computationally infeasible in general.

Rather, the author's purpose of introducing the "robot" was to avoid the misconception that plausibility desiderata are some subjective, inaccurate notions that depend on some hidden features of the human mind. So by detaching the inference rule from the human mind and using a idealized "robot", the author wants to argue that these axioms and their consequences can and should be studied mathematically and independently from all other features and aspects of human thinking and rationality.

So here the objective was not to build some intelligence, rather study an abstract and computationally unconstrained version of intelligence obeying the above principles alone.

Such an AI will never be realized in practice (due to inherent complexity limitations, and here I don't just speak about P!=NP !), Still, if we can prove what this theoretical AI will have to do in certain specific situations, then we can learn important lessons about the above principles, or even guide our decisions by that insights we gained from that study.

I never thought about the connection between logic and probability before, though now it seems obvious. I've read a few introductory logic texts and deductive reasoning always seemed a bit pointless to me (in RL premises are usually inferred from something). -

To draw from a literary example, Sherlock Holmes use of the phrase "deduce" always seemed a bit deceptive. You can say "that color of dirt exists only in spot x in London. Therefore, that Londoner must have come in contact with spot x if I see that dirt on his trouser knee." This is presented as a deduction, but really, the premises are induced and he assumes some things about how people travel.

It seems more likely that we make inferences, not deductions, but convince ourselves that the premises must be true, without bothering to put real information about likelihood into the reasoning. An induction is still a logical statement, but I like the idea of using probability to quantify it.

Along with the distinction between causal and logical connections, when considering the conditional premise of the syllogisms (if A then B), Jaynes warns us to distinguish between those conditional statements of a purely formal character (the material conditional ) and those which assert a logical connection.

It seems to me that the weak syllogisms only "do work" when the conditional premise is true due to a logical connection between antecedent and consequent. If no such connection exists, or rather, if our mind cannot establish such a connection, then the plausibility of the antecedent doesn't change upon learning the consequent.

For example, "if the garbage can is green then frogs are amphibians" is true since frogs are amphibians, but this fact about frogs does not increase (or decrease) the probability that the garbage can is green since presumably, most of us don't see a connection between the two propositions.

At some point in learning logic, I think I kind of lost touch with the common language use of conditionals as asserting connections. I like that Jaynes reminds us of the distinction.

What do you make of Jaynes' observation that plausible inference is concerned with logical connections, and must be carefully distinguished from physical causation?

His example of the rain at 10:30 implying clouds at 10:15 with any physical causation going in the other direction is clear. And I appreciate his polemic that limiting yourself to reasoning based upon physical cause and effect is dull and impractical. He was a physicist and the ideal of physicists is to discover previously unknown natural laws of cause and effect, which made him a bit eccentric within his own community and so we get the tone in there of pleading. It is a minor distraction in the midst of great material.

57 participants should make for a sustained critical mass even with heavy attrition.

It occurs to me that Jaynes is missing a desideratum that I might have included. I can't decide if it's completely trivial, or if perhaps it's covered implicitly in his consistency rule 3c; I expect it will become clear as the discussion becomes more formal -- and of course, he did promise that the rules given would turn out to be sufficient. To wit:

• The robot should not assign plausibilities arbitrarily. If the robot has plausibilities for propositions A and B such that the plausibility of A is independent of the plausibility of B, and the plausibility of A is updated, then the degree of plausibility for B should remain constant barring other updates.

One more thing. The footnote on page 12 wonders: Does it follow that AND and NOT (or NAND alone) are sufficient to write any computer program?

Isn't this trivial? Since AND and NOT can together be composed to represent any logic function, and a logic function can be interpreted as a function from some number of bits (the truth values of the variable propositions) to one result bit, it follows that we can write programs with AND and NOT that make any bits in our computer an arbitrary function of any of the other bits. Is there some complication I'm missing?

(Edited slightly for clarity.)

• Compare Jaynes' framing of probability theory with your previous conceptions of "probability". What are the differences?

Probability is a 1-to-1 mapping of plausibilities onto real numbers, as opposed to an objective thing waiting to be discovered, mind-independently.

• What do you make of Jaynes' observation that plausible inference is concerned with logical connections, and must be carefully distinguished from physical causation?

It seems quite reasonable. His storm cloud analogy works quite well.

I was particularly impressed with the "Comments" section after the chapter.

• Can you think of further desiderata for plausible inference, or find issues with the one Jaynes lays out?

I find desideratum 1) to be poorly motivated, and a bit problematic. This is urged upon us in Chapter 1 mainly by considerations of convenience: a reasoning robot can't calculate without numbers. But just because a calculator can't calculate without numbers doesn't seem a sufficient justification to assume those numbers exist, i.e., that a full and coherent mapping from statements to plausibilities exists. This doesn't seem the kind of thing we can assume is possible, it's the kind of thing we need to investigate to see if it's possible.

This of course will depend on what class of statements we'll allow into our language. I can see two ways forward on this: 1) we can assume that we have language of statements for which desideratum 1) is true. But then we need to understand what restrictions we've placed on the kinds of statements that can have numerical plausibilities. Or 2) We can pick a language that we want to use to talk about the world, and then investigate whether desideratum 1) can be satisfied by that language. I don't see that this issue is touched on in Chapter 1.

There is further discussion of this in Appendix C; will this be discussed in connection with Chapter 1, or at some later time in the sequence? For example, in Appendix C, it turns out that desideratum 1 subdivides into two other axioms: transitivity, and universal comparability. The first one makes sense, but the second one doesn't seem as compelling to me.

Jaynes discusses a "tricky point" with regard to the difference between the everyday >meaning of the verb "imply" and its logical meaning; are there other differences between >the formal language of logic and everyday language?

In formal logic, the disjunction "or" is inclusive -- "A or B" is true if A and B are true. In everyday language, typically "or" is exclusive -- "A or B" is meant to exclude the possibility that A and B are both true.

People haven't had extensive live discussions yet. I think we're currently collecting preferred times on the Google Docs spreadsheet and will decide on the weekdays and times of day when we've gotten more of those.

I'd guess that what he's getting at is that the first statement is merely reporting an observation while the second is making a claim about the state of the world and the entities within it. Of course such claims are somewhat implicit in the first to the extent that they are implicit in language but it is mainly reporting observations without explicitly tying them to claims about the world. The second statement is more explicitly calling out the existence of a thing called noise and a thing called a room and saying that the latter contains the former.

Yes, see the latest post - it hasn't been promoted (yet?), so if you're only getting the front page RSS feed you might have missed it.

A different question about 1-8. I was able to figure out how he got A!B = !B (where ! is bar) but using the Boolean identities he provides, I couldn't get to B!A = !A. Can anyone enlighten me on this?

I'm not sure there is a transformation intended... Proposition (1-9) appears by itself in a section that discusses "implication" (it introduces => as a shorthand for A=AB) and does not appear to follow from (1-8).