Probability is Subjectively Objective

Hi, I think that due to the nature of your site you would be interested in my blog at

www.orble.com/the-compassionate-cynic

please link me on technocrati however that works ;)

Description:

"The Compassionate Cynic is an entity which documents its notes and observations via bit streams directly to the blog in this link."

Is not subjective objectivity the highest degree of objectivity possible for a human being?

Objective truth does exist, but people can only perceive it with their own perception filters. And, perhaps, AIs with the perception filters of their makers.

I have to decide what is truth as best I can, and may choose to assert a truth even though every one else denies it, eg Galileo. It is to my advantage to seek to make my perception filters as little distorting as possible, but I doubt I could ever achieve that completely.

there is no such thing as a probability that isn't in any mind.

Hmm. Doesn't quantum mechanics (especially if we're forgetting about MWI) give us genuine, objective probabilities?

Probability assigned to a belief is estimated according to a goal that says "high Bayesian score is good". It has a particular optimization target, and so it doesn't run away from it. It is an "objective" fact about this optimization process that it tries to have a good Bayesian score. The probability that it produces is within its mind, and in this sense can be said to be "subjective", but it is no more subjective than my decision to take an apple from the table, that also happens in my mind and is targeted on a goal of eating an apple. There is no point in assuming a state of mind just for the sake of reflecting the territory, if it doesn't contribute to the goal. In this case, the goal happens to include the state of mind itself.

To answer Allan Crossman and "I never saw Jaynes spell out explicitly":

For some sixty years it has appeared to many physicists that probability plays a fundamentally different role in quantum theory than it does in statistical mechanics and analysis of measurement errors. It is a commonly heard statement that probabilities calculated within a pure state have a different character than the probabilities with which different pure states appear in a mixture, or density matrix. As Pauli put it, the former represents "Eine prinzipielle Unbestimmtheit, nicht nur Unbekanntheit". But this viewpoint leads to so many paradoxes and mysteries that we explore the consequences of the unified view, that all probability signifies only incomplete human information.

This is from Probability In Quantum Theory (1999). Jaynes seems to be ignoring animals and AIs.

Probability isn't only used as an expression of a person's own subjective uncertainty when predicting the future. It is also used when making factual statements about the past. If a coin was flipped yesterday and came up heads 60% of the time, then it may have been a fair coin which happened to come up heads 60% of the time, or it may have been a trick, biased, coin, whose bias caused it to come up heads 60% of the time. To say that a coin is biased is to make a statement about probability. As Wikipedia explains:

In probability theory and statistics, a sequence of independent Bernoulli trials with probability 1/2 of success on each trial is metaphorically called a fair coin. One for which the probability is not 1/2 is called a biased or unfair coin.

So a statement about probability can enter into a factual claim about the causes of past events.

The distinction here may be quite simple: an objective Bayesian accepts Bayes' Theorem, a subjective one does not. After all, Bayes' Theorem posits that repeated adjustments of priors based on new posterioris from the latest observations will asymptotically converge on the "true" probability distribution. That is only meaningful if one believes in an objective, "true" probability distribution (and of course assuming that certain necessary conditions hold regarding the underlying distribution and its dimensionality).

All true.

When E.Yudkowsky's foe agrees with something he says, you can be sure its correct ;) Of course, the answers are in the materials on ontology I posted on the everything-list months ago.

As to where an algebraic relation such as '2+3' exists, it exists in the same place as the objective value archetypes of course- it's a global feature of the Tegmark multiverse - a relation between all the possible worlds.

Probabilities, on the other hand, are not a global feature of the Tegmark multiverse, but are computed in the individual minds existing in QM branches. That's the difference.

Allan, QM is completely deterministic. All that actually exists is the QM wave function, which has a completely deterministic evolution. Again, the probabilities only exist in the minds of observers viewing specific 'cross-sections' of the multiverse.

In which case, it is good that your mind does not mention people's opinions, and only thinks of the lottery balls; the ideal computation makes no mention of the lottery balls

no mention of the people's opinions?

Allan: I would think that one of the reasons Eliezer is such a strong believer in MWI is that it does change the interpretation of the probability of the wavefunction from being objective back to being subjectively objective.

Barkley Rosser, I think the difference between objective Bayesians and subjective Bayesians has more to do with how they treat prior distributions than how they view asymptotic convergence.

I'm personally not an objective Bayesian by your definition -- I don't think there are stable, true probability distributions "out there." Nevertheless, I do find the asymptotic convergence theorems meaningful. In my view, asymptotic convergence gets you to the most informative distribution conditional on some state of information, but that state of information need not be maximal, i.e., deterministic for observables. When the best you can do is some frequency distribution over observables, it's because you've left out essential details, not because the observables are random.

pdf, fixed.

Re: a statement about probability can enter into a factual claim about the causes of past events.

Not under the view we are discussing. Did you read the referenced Probability is in the Mind page?

The idea is that uncertainty is a psychological phenomenon. It exists in the mind, not in the world.

Not under the view we are discussing.

That was my point.

In order to criticise Jaynes' perspective, you should first understand it. It is not clear to me that you have done that.

Jaynes' perspective on the historical behaviour of biased coins would make no mention of probability - unless he was talking about the history of the expectations of some observer with partial information about the situation. Do you see anything wrong with that?

Reality is that which, when you stop believing in it, doesn't go away.

This is false, of course; with sufficiently advanced technology you could build a machine that read out your mind state and caused Earth to disappear once it determined you no longer believed in Earth. Doesn't mean Earth was never real.

Jaynes' perspective on the historical behaviour of biased coins would make no mention of probability - unless he was talking about the history of the expectations of some observer with partial information about the situation. Do you see anything wrong with that?

I see nothing wrong with that. Similarly, if someone mentions only the atoms in my body, and never mentions me, there is nothing wrong with that. However, I am also there.

What I have pointed out is that seemingly unproblematic statements can indeed be made of the sort that I described. That Jaynes himself makes no such statements says nothing one way or another about this. There are different possible responses, including:

1) It might be shown that certain classes of factual statements about history, including the one I gave, are in fact in some sense relative, may incorporate a tacit perspective and therefore may be in that sense subjective. An example of such a statement might be a statement that an object is "at rest" rather than "in motion". This statement tacitly presupposes a frame of reference, and so is in that sense not fully objective.

2) It might be shown that there was something wrong about the sort of statement that I gave as an example.

A couple of comments on a very big topic.

1. There is a fascinating history of ideas about the objective contents of thought, starting with the Austrians Brentano and Meinong, running through analytical philosophy (Russell and Moore) and phenomenology (Husserl and Heidegger), and also through evolutionary epistemology (Popper and Munz). http://www.the-rathouse.com/EvenMoreAustrianProgram/EMAThreeAustrianStrands.html

2. On the Bayesian appraisal of theories, with reference to the Duhem problem, it seems that Bayes gives a good result when there is only one major theory in the picture but is not so good when help is most needed, that is to judge between two serious rival theories. http://www.the-rathouse.com/Theses/Duhem-QuineInBayesNewExperimentalism.html

Re: seemingly unproblematic statements can indeed be made of the sort that I described.

The example appears to be that of the accursèd frequentists - the view that Jaynes spent much of his academic life crusading against.

If such ideas seem unproblematic to you, that's fine - but be aware that others find the notion of real word probabilities to be unsupported by evidence, and contrary to Occam's razor.

If such ideas seem unproblematic to you

It is the example that seems on the face of it unproblematic. I am open to either (a) a demonstration that it is compatible with subjectivism[*], or (b) a demonstration that it is problematic. I am open to either one. Or to something else. In any case, I don't adhere to frequentism.

[*] (I made no firm claim that it is not compatible with subjectivism - you are the one who rejected the compatibility - my own purpose was only to raise the question since it seems on the face of it hard to square with subjectivism, not to answer the question definitively.)

steven: This is false, of course; with sufficiently advanced technology you could build a machine that read out your mind state and caused Earth to disappear once it determined you no longer believed in Earth. Doesn't mean Earth was never real.

There is no clear separation between the mind and the territory. The structure of the mind is instrumental to optimization process. When you change the beliefs, change the state of the mind, you are in fact performing an action on the territory, that is instrumental to the goal. Establishing specific state (process) of the territory is the final goal, so in changing the mind, you are moving the territory along your course. Reality doesn't go away, but it moves a step closer to the attractor, and if it doesn't, you are not being rational.

I thought we had agreed that the historical behaviour of coins was "compatible with subjectivism":

Jaynes' perspective on the historical behaviour of biased coins would make no mention of probability - unless he was talking about the history of the expectations of some observer with partial information about the situation.

An observer with one set of partial information might have predicted a coin would come up heads 50% of the time. An observer with another set of partial information might have predicted a coin would come up heads 60% of the time. That the predictions of these observers differs illustrates the subjective element of probability estimates.

However, it is their (partial) ignorance of the state of the world that leads to such assessments. With complete information (and a big computer) an observer would know which way the coin would land - and would find probabilities irrelevant. The probabilities arise from ignorance and lack of computing power - properties of observers, not properties of the observed.

With complete information (and a big computer) an observer would know which way the coin would land - and would find probabilities irrelevant.

But this is true of most everyday observations. We observe events on a level far removed from the subatomic level. With complete information and infinite computing power an observer would would find all or virtually all ordinary human-level observations irrelevant. But irrelevancy to such an observer is not the same thing as non-reality. For example, the existence of elephants would be irrelevant to an observer who has complete information on the subatomic level and sufficient computing power to deal with it. But it does not follow that elephants do not exist. Do you think it follows that elephants do not exist?

The probabilities arise from ignorance and lack of computing power - properties of observers, not properties of the observed.

The concept of an elephant could with equal reason be said to arise from ignorance and lack of computing power. I can certainly understand that a thought such as, "the elephant likes peanuts, therefore it will accept this peanut" is a much easier thought to entertain than a thought that infallibly tracks every subatomic particle in its body and in the environment around it. So, certainly, the concept of an elephant is a wonderful shortcut. But I'm not so sure about getting from this to the conclusion that elephants (like probability) are subjective. Do you think that elephants are subjective?

Re: Do you think it follows that elephants do not exist?

Elephants are not properties of physics any more than probabilities are.

The concept of an elephant is subjective - as are all concepts.

The atoms composing an elephant are real enough.

AFAICS, nobody ever claimed that probabilites "do not exist". The idea is that uncertainty is a psychological phenomenon, not that it is non-existent.

I hope that clarifies things.

Elephants are not properties of physics any more than probabilities are. The concept of an elephant is subjective - as are all concepts.

If you are indeed agreeing with the parallel I have set up between probability and elephants and if this is not just your own personal view, then perhaps the subjectivist theory of probability should more properly be called the subjectivist theory of pretty much everything that populates our familiar world. Anyway, I think I can agree that probability is as subjective and as psychological and as non-physical and as existing in the mind and not in the world as an elephant or, say, an exploding nuclear bomb - another item that populates our familiar world.

We don't have people claiming that elephants are a concept in fundamental physics. However, people do claim that about probability. Thus the focus on probability and not on elephants.

Cyan,

OK, I grant your point. However, assuming that there is some "subjectively real" probability distribution that the Bayes' Theorem process will converge is a mighty strong assumption.

Barkley Rosser, it's a strong assumption in principle, but in practice, humans seem to be pretty good at obtaining enough information to put in the model such that the posterior does in fact converge to some point in the parameter space.

Cyan,

Why should there be convergence to some such point when there is no underlying "true" distribution, either subjective or objective? Are you counting on herding by people? It is useful to keep in mind the conditions under which even in classical stats, Bayes' Theorem does not hold, for example when the underlying distribution is not continuous or if it is infinite dimensional. In the former case convergence can be to a cycle of bouncing back and forth between the various disconnected portions of the distribution. This can happen, presumably in a looser purely subjective world, with even a multi-modal distribution.

Barkley Rosser, what I have in mind is a reality which in principle predictable given enough information. So there is a "true" distribution -- it's conditional on information which specifies the state of the world exactly, so it's a delta function at whatever the observables actually turn out to be. Now, there exists unbounded sequences of bits which don't settle down to any particular relative frequency over the long run, and likewise, there is no guarantee that any particular sequence of observed data will lead to my posterior distribution getting closer and closer to one particular point in parameter space -- if my model doesn't at least partially account for the information which determines what values the observables take. Then I wave my hands and say, "That doesn't seem to happen a lot in practical applications, or at least, when it does happen we humans don't publish until we've improved the model to the point of usefulness."

I didn't follow your point about a distribution for which Bayes' Theorem doesn't hold. Are you describing a joint probability distribution for which Bayes' Theorem doesn't hold, or are you talking about a Bayesian modeling problem in which Bayes estimators are inconsistent a la Diaconis and Freedman, or do you mean something else again?

Diaconis and Freedman.

Barkley Rosser, there definitely is something a little hinky going on in those infinite dimensional model spaces. I don't have the background in measure theory to really grok that stuff, so I just thank my lucky stars that other people have proven the consistency of Dirichlet process mixture models and Gaussian process models.

Gotta have that continuous support too, which is the real key to converging on a cycle rather than a point.

In the fuzzier world of not a definite for-real underlying distribution, I note that multiple equilibria or basins in dynamical systems can give the multi-modality that within a herding framework can lead to some sort of cycle in bouncing back and forth between the dominant states.

"Probability exists in your mind: if you're ignorant of a phenomenon, that's an attribute of you, not an attribute of the phenomenon."

Eliezer, a small point here: if QM is true, then the universe and phenomena within the same are inherently probabilistic, are they not?

Great post as usual.

Re: if QM is true, then the universe and phenomena within the same are inherently probabilistic

Not really - e.g. see:

Q13 Is many-worlds a deterministic theory? Yes, many-worlds is a deterministic theory [...]

• http://www.hedweb.com/everett/everett.htm

Beside the point, but you can calculate arbitrary digits of pi with the formula explained in this article.

I was one of the people who suggested the term h-right before. I'm not great with mathematical logic, and I followed the proof only with difficulty, but I think I understand it and I think my objections remain. I think Eliezer has a brilliant theory of morality and that it accords with all my personal beliefs, but I still don't understand where it stops being relativist.

I agree that some human assumptions like induction and Occam's Razor have to be used partly as their own justification. But an ultimate justification of a belief has to include a reason for choosing it out of a belief-space.

For example, after recursive justification hits bottom, I keep Occam and induction because I suspect they reflect the way the universe really works. I can't prove it without using them. But we already know there are some things that are true but can't be proven. I think one of those things is that reality really does work on inductive and Occamian principles. So I can choose these two beliefs out of belief-space by saying they correspond to reality.

Some other starting assumptions ground out differently. Clarence Darrow once said something like "I hate spinach, and I'm glad I hate it, because if I liked it I'd eat it, and I don't want to eat it because I hate it." He's was making a mistake somewhere! If his belief is "spinach is bad", it probably grounds out in some evolutionary reason like insufficient energy for the EEA. But that doesn't justify his current statement "spinach is bad". His real reason for saying "spinach is bad" is that he dislikes it. You can only choose "spinach is bad" out of belief-space based on Clarence Darrow's opinions.

One possible definition of "absolute" vs. "relative": a belief is absolutely true if people pick it out of belief-space based on correspondence to reality; if people pick it out of belief-space based on other considerations, it is true relative to those considerations.

"2+2=4" is absolutely true, because it's true in the system PA, and I pick PA out of belief-space because it does better than, say, self-PA would in corresponding to arithmetic in the real world. "Carrots taste bad" is relatively true, because it's true in the system "Yvain's Opinions" and I pick "Yvain's Opinions" out of belief-space only because I'm Yvain.

When Eliezer say X is "right", he means X satisfies a certain complex calculation. That complex calculation is chosen out of all the possible complex-calculations in complex-calculation space because it's the one that matches what humans believe.

This does, technically, create a theory of morality that doesn't explicitly reference humans. Just like intelligent design theory doesn't explicitly reference God or Christianity. But most people believe that intelligent design should be judged as a Christian theory, because being a Christian is the only reason anyone would ever select it out of belief-space. Likewise, Eliezer's system of morality should be judged as a human morality, because being a human is the only reason anyone would ever select it out of belief-space.

That's why I think Eliezer's system is relative. I admit it's not directly relative, in that Eliezer isn't directly picking "Don't murder" out of belief-space every time he wonders about murder, based only on human opinion. But if I understand correctly, he's referring the question to another layer, and then basing that layer on human opinion.

An umpire whose procedure for making tough calls is "Do whatever benefits the Yankees" isn't very fair. A second umpire whose procedure is "Always follow the rules in Rulebook X" and writes in Rulebook X "Do whatever benefits the Yankees" may be following a rulebook, but he is still just as far from objectivity as the last guy was.

I think the second umpire's call is "correct" relative to Rulebook X, but I don't think the call is absolutely correct.

...yeah, this was supposed to go in the new article, and I was just checking something in this one and accidentally posted it here. Please ignore embarrassed

"Reality is that which, when you stop believing in it, doesn't go away." -- Philip K. Dick

I have to comment on the irony of this quote. Philip K. Dick's novels are almost all extended riffs on the idea that there is no reality; or that reality is unknowable or irrelevant (eg. The Man in the High Castle, Flow My Tears the Policemen Said, Do Androids Dream of Electric Sheep? (Blade Runner), We Can Remember It for You Wholesale (Total Recall)). And Philip K. Dick was unable to distinguish reality from fantasy in everyday life.

But it also makes a deep philosophical point as well, which I never saw Jaynes spell out explicitly, but I think he would have approved: there is no such thing as a probability that isn't in any mind. Any mind that takes in evidence and outputs probability estimates of the next event, remember, can be viewed as a prior - so there is no probability without priors/minds.

Is this why you believe so strongly in Many-Worlds? To avoid mind-free, objective, quantum-mechanical probabilities?

I felt all the way through this post like it was confronting a difficult philosophical problem head-on, and that if I kept reading a little more it would reveal a solution; but I never saw a solution. It described the problem very well; but if it was intended to move beyond that, then I missed it.

When you judge those alternate minds, you'll do so using your own mind - your own beliefs about the universe - your own posterior that came out of your own prior, your own posterior probability assignments P(X|A,B,C,...,Q1).

Hehe, "your own posterior".

[strangles her inner 12-year-old]

I am almost convinced, honestly. I was leaning towards a frequentest view, but I'm realizing now -- as pointed out here by a fellow community member -- that some of my statements are similar if not identical to the conclusion here:

Jaynes certainly believed very firmly that probability was in the mind... But Jaynes also didn't think that this implied a license to make up whatever priors you liked. There was only one correct prior distribution to use, given your state of partial information at the start of the problem.

is pretty similar to :

Bayesian reasoning is the field which tells us the optimal probability to assign to a proposition given the rest of our information, but that that is the optimal probability given the rest of our information is a fact about the world.

(When I say "our" there I mean each of us as individuals, not our collective knowledge. )

I'll give my view; I think I agree with EY; I'll be as short as I can. We have a standard deck of playing cards. It is a fact about them that 1/4 of them are hearts. Not just to me or someone else; it is a fact that the universe keeps track of: 1/4 of them are hearts. Two agents A and B are placing bets on the next suit to come out. They both know that 1/4 of the cards are hearts, and that is all that A knows, but B also knows that 8 out of 10 of the top 10 cards are hearts. So B bets that "The top card is a Heart." and A bets that "~The top card is a heart.". Before hand they argue, and B says "you know I think there's an 80% chance that I'll win." B says "Are you crazy? Everyone knows that 75% of the cards in a deck aren't hearts, the chances are 75% in my favor."

Now, of all the possible states that satisfy A's knowledge, which is the statement "75% of the cards in a deck aren't hearts.", exactly 75% of them satisfy "~The top card is a heart." So it is no wonder that A ascribes a 75% probability in his favor. A's beliefs constrain A's expected experiences, but not to one line; they constrain the possible worlds A thinks it might be in. B's knowledge is the same. Of the possible words that satisfy "8 out of 10 of the top 10 cards are hearts." 80% of them satisfy "The top card is a heart." So, duh, B concludes that it has an 80% chance of winning. Using this simple setup it becomes clear to me that the more knowledge you have about the state of the deck, the more useful the probability you ascribe will be. This is because the more knowledge you have about the state of the deck, the more you constrict the space of possible worlds that you as an agent think you might be in. A third agent that knows every detail about the deck of course ascribes no non-1 or non-0 probability to any statement about the deck, since it know what possible world its in.

So then I would say that the probability a perfect reasoner P, gives to a statement S, is the fraction of possible worlds that satisfy S that also satisfy the rest of the statements that P holds. If P has no ignorance, then he has one possible world, and that fraction is always 1 or 0. And I would go as far as to say that this is a decent explanation of what a probability is. It's a propositional attitude had by an agent; it has a value from 0 to 1, and represents the fraction of states that the agent thinks it might find it self in that satisfy the given proposition. This to me doesn't seem to be inconsistent with the view expressed in this post, but I'm not sure of this.

My view does suggest that "P(a) = such and such" is a claim. Its just that its a claim about the possible worlds that an agent can consistently expect to find him/herself in given the rest of its beliefs. An agent can be wrong about a probability it ascribes. Suppose an agent R, which has the same knowledge as B, but insists that the probability of "The top card is a heart." is 99%. Well, R is wrong, R is wrong about the fraction of worlds that satisfy R's knowledge base that also satisfy "The top card is a heart."

In conclusion, I will risk the hypothesis that: "P(a|b) = x" is true if and only if x of the possible worlds that satisfy b, also satisfy a. But of course, there are no possible worlds without uncertainty, and there is no uncertainty without the ignorance of an agent in a determined world.

Probability is in the Mind

Now, can't I be a philosophical frequentest and a subjective bayesian? Just because probability theory models subjective beliefs does not mean that it doesn't model frequencies; in fact, if some body told me that bayes doesn't model frequencies I'm pretty sure I could prove them wrong much more easily than someone who said that probabilities don't model degrees of belief.

But there is no contradiction in saying that the komolgorov probability function models both degrees of beliefs and actual frequencies.

(edit)

In fact it seems to me that komolgorov plainly does model frequency since it models odds, and odds model frequencies by a simple conversion. In fact, degrees of belief seem to model frequency as well. Using the thought experiment of frequencies of worlds you think you might find yourself in makes it simple to see how at least some degrees of belief can be seen as frequencies in and of themselves. In this thought experiment we treat probability as a measure of the worlds you think you might find yourself in; if you think that "there are ten cards and eight of them are blue", then in 4/5s of the the worlds where "there are ten cards and eight of them are blue" holds, "the top card is blue" also holds. So you rightfully assign a 80% probability to the top card being blue.

Where the frequentest makes an error is in thinking that probabilities are then out there in the world treated as degrees of belief. What I mean by this is that they take the step from frequencies being in the world to uncertainties being in the world. This is a mistake, but I think that it is not central to the philosophical doctrine of frequent-ism. All that some frequentests claim is that probability models frequency and this is plainly true. And it is also true that there are frequencies in the world. Real frequencies independent of our minds. These are not probabilities, because there are no probabilities anywhere. Not even in minds.

Probability is not degree of subjective belief, probability is a class of automatized functions. These automized function model a great deal of things, measure theory, euclidean geometry, the constraints of rational beliefs, set cardinality, frequencies, etc. and the list can go on and on. Probability is a mathematical tool. And it is isomorphic to many important features of rationality and science, perhaps the most important being subjective degree of belief. But to argue that probability is subjective degree of belief just because it models degree of belief seems as silly to me as arguing that probability is frequency just because it models frequency. Why not the probability position of measure. Measure theory is isomorphic to probability. Why not say that probability is measure? Add that to the debating line.

I think the position to take towards probability is a properly Hofstadter-ish-ian formalism. Where the true statements about probability are simply the statements which are formed when you interpret the theorems of probability theory. Whatever else probability may be able to talk about truthfully it does so through isomorphism.

"Perform the experiment a hundred times, and—" Okay, let's talk about the ten trillionth digit of pi, then. Single-shot problem, no "long run" you can measure.

And there goes my belief in any kind of probability as a phenomenon. I don't know what the ten trillionth digit of pi is, but I know the algorithm which generates it, and it never involves a die roll or coin flip of any kind. And if the universe is to be lawful, it doesn't roll dice either. There is no probability. To say there was is to say the ten trillionth digit of pi might somehow have come out differently. And that would be unlawful.

"Reality is that which, when you stop believing in it, doesn't go away."

By that definition, stuff like the value of the US dollar aren't real.

Somewhere out in mind design space, there's a mind with any possible prior; but that doesn't mean that you'll say, "All priors are created equal."

The corrected phrase may be: "All unentangled priors are created equal."

I think this is the reason why a distinction between subjective and objective probability is needed.

If we figure out how to build GAI, we could build several with different priors, release them into the universe, and see which ones do better. If we give them all the same metric to optimize, they will all agree on which of them did better, thus determining one prior that is the best one to have for this universe.