Principles of Disagreement

Wait, so did you or did you not say "Be wary of the temptation to make exceptions"?

But dammit, wavefunctions don't collapse!

Sorry to distract from your main point, but it is quite a distance from there to "therefore, there are Many Worlds". And for that matter, you have definitely not addressed the notion of collapse in all its possible forms. The idea of universe-wide collapse caused by observation is definitely outlandish, solipsistic in fact, and also leaves "consciousness", "observation", or "measurement" as an unanalysed residue. However, that's not the only way to introduce discontinuity into wavefunction evolution. One might have a jump (they won't be collapses, if you think in terms of state vectors) merely with respect to a particular subspace. The actual history of states of the universe might be a complicated agglomeration of quantum states, with spacelike joins provided by the tensor product, and timelike joins by a semilocal unitary evolution.

Furthermore, there have always been people who held that the wavefunction is only a representation of one's knowledge, and that collapse does not refer to an actual physical process. What you should say as a neo-rationalist is that those people should not be content with an incomplete description of the world, and that something like Minimum Description Length should be used to select between possible complete theories when there is nothing better, and you should leave it at that.

I suppose that the larger context is that in the case of seed AI, we need to get it right the first time, and therefore it would be helpful to have an example of rationality which doesn't just consist of "run the experiment and see what happens", in order to establish that it is possible to think about things in advance. But this is a bad example! Alas, I cannot call upon any intersubjective, third-party validation of this claim, but I do consider this (quantum ontology), if not my one true expertise, certainly a core specialization. And so I oppose my intuition to yours, and say that any valid completion of the Many Worlds idea is either going to have a simpler one-world variant, or, at best, it will be a theory of multiple self-contained worlds - not the splitting, half-merged, or otherwise interdependent worlds of standard MWI rhetoric. This is not a casual judgement; I could give a lengthy account of all the little reasons which lead up to it. If we had a showdown at the rationality dojo, I think it would come out the winner. But first of all, I think you should just ask yourself whether it makes sense to say "Many Worlds is the leading candidate" when all you have really shown is that a particular form of collapse theory is not.

Particle masses?? Definitely go with Gell-Mann.

That taxi anecdote is a blazing star of dorkiness in an otherwise mundane universe. I applaud you, sir.

Let me try to paraphrase:

I don't trust anyone as much as I trust myself to be competent and really try hard to get things right. I can see that if I followed any rule I can imagine to disagree less with people, based on their qualifications as seen by third parties, there would be cases where I'd have to agree more with claims that to me are obviously wrong. So I don't follow any rule, I just do what seems right, and damn it, doing that is what seems right to me - and it seems to work!

"A primary reason I don't always update when people disagree with me, is that I don't think they're taking that disagreement with the extraordinary gravity that would be required, on both sides, for two people to trust each other in an Aumann cage match."

The problem with this is that there is at least a small probability that they are taking it with that gravity. Given this small probability, it is still necessary to update, although a smaller amount. This was my point in the previous post. The fact that you should update when you are surprised by someone's opinion doesn't depend on Aumann's theorem: it simply follows from Bayes's rule, if you accept that there is at least some tiny chance that he holds his opinion for good reasons, and therefore that there is at least some tiny chance that the truth is the cause of his opinion. For if there is, then the probability that he would state that opinion, given that it is true, is greater than the probability that he would state the opinion, given that it is false. Therefore you should update your opinion, allowing his opinion to be more likely to be true than you previously thought it.

Then again, there are some topics that seem to turn even the most brilliant minds to mush. (These seems to be the same topics for which, incidentally, there could be no epistemically relevant "authority.") Hence, your disagreement with Aumann out-of-hand, for instance.

Thus I'd think some kind of topic-sensitivity is lurking in your rule as well.

Eliezer,

I also think that considering the particular topics is helpful here. In the math book, you were pretty confident the statement was wrong once you discovered a clear formal proof, because essentially there's nothing more to be said.

On the interpretation of quantum mechanics, since you believe we have almost all the relevant data we'll ever have (save for observed superpositions of larger and larger objects) and the full criteria to decide between these hypotheses given that information, you again think that disagreement is unfounded.

(I suggest you make an exception in your analysis for Scott Aaronson et al, whose view as I understand it is that progress in his research is more important than holding the Best Justified Interpretation at all times, if the different interpretations don't have consequences for that research; so he uses whatever one seems most helpful at the moment. This is more like asking a different valid question than getting the wrong answer to a question.)

But on the prospects for General AI in the next century, well, there's all sort of data you don't yet have that would greatly help, and others might have it; and updating according to Bayes on that data is intractable without significant assumptions. I think that explains your willingness to hear out Daniel Dennett (albeit with some skepticism).

Finally, I think that when it comes to religion you may be implicitly using the same second-order evaluation I've come around to. I still ascribe a nonzero chance to my old religion being true—I didn't find a knockdown logical flaw or something completely impossible in my experience of the world. I just came to the conclusion I didn't have a specific reason to believe it above others.

However, I'd refuse to give any such religion serious consideration from now on unless it became more than 50% probable to my current self, because taking up a serious religion changes one's very practice of rationality by making doubt a disvalue. Spending too much thought on a religion can get you stuck there, and it was hard enough leaving the first time around. That's a second-order phenomenon different from the others: taking the Copenhagen interpretation for a hypothesis doesn't strongly prevent you from discarding it later.

My best probability of finding the truth lies in the space of nonreligious answers instead of within any particular religion, so I can't let myself get drawn in. So I do form an object-level bias against religion (akin to your outright dismissal of Aumann), but it's one I think is justified on a meta-level.

I laughed out loud at the taxi story!

Scott said:

Particle masses?? Definitely go with Gell-Mann.

But Scott, the fact that you say that, means that if I saw that you were still arguing particle masses with Gell-Mann, the most probable explanation in my mind would be some kind of drastically important modern update that Gell-Mann was bizarrely refusing to take into account. Such an event is not beyond probability.

Gell-Mann is a Nobel laureate and you are not (yet), but you know that, so you would disagree with Gell-Mann lot more reluctantly than he would disagree with you. To the extent that I trust you to know this, your disagreement with Gell-Mann sends a much stronger signal than Gell-Mann's disagreement with you. It doesn't cancel out exactly, because I don't trust you perfectly; if I trusted you very little, it would cancel out almost not at all.

But you can see how, in the limit of disagreement between ideal Bayesians, the direction of disagreement on each succeeding round would be perfectly unforeseeable.

Eliezer: Yeah, I understand. I was making a sort of meta-joke, that you shouldn't trust me over Gell-Mann about particle physics even after accounting for the fact that I say that and would be correspondingly reluctant to disagree...

"To the best of your ability to say at this point, what would have been your initial probability that the bill was yours?" I said. "Fifteen percent," said Nick. "I would have said twenty percent," I said. So we split it $8.57 / $11.43, and went happily on our way, guilt-free. (this is a 3 : 4 ratio) Now for this to have been fair, you must both have walked away with a 3/7 probability that the money belonged to Nick.

This algorithm definitely feels like the wrong answer. Taking this ratio couldn't possibly be the correct way for both of you to update your beliefs.

Why?

Well, because the correct answer really ought to be invariant under swapping. If you and Nick Bostrom had traded opinions, Nick's answer would have been eighty percent, your answer would have been eighty-five percent, and you would have split the twenty in a 16:17 ratio. You would also have ended up with 16:17 if you had instead asked "What would have been your initial probability that the bill was not yours?"

It's the wrong answer because if X is the proposition that the bill belongs to Eliezer then setting your mutual belief in X to: f(X) = p_E(X) / (p_E(X) + p_N(~X)) doesn't look pretty and symetric. Not only that, but f(X)+f(~X) != 1. What you did only appeared symetric because X made reference to the parties involved. Granted, when propositions do that, people tend to be biased in their own favor, but taking that into account would be solving a different problem.

Now I haven't read Aumann's paper, so I don't know the correct answer, if he presents one. But if I had to come up with an answer just intuitively it would be: p(X) = (p_E(X)+p_N(X))/2

One (oversimplified) possible set of assumptions that could lead to this answer would be:

"Well, presuming we saw the same things, one of our memories must have made a serious mistake. Now if we ignore the possibility that we both made mistake, and we know nothing about the relative frequences at which we both make mistakes like this one, it seems reasonable to assume that, given that one of us made a mistake the principle of indifference would suggest that the chances are half that I was the one who reasoned properly and half that you are the one who reasoned properly. So, provided that we think mistakes aren't more or less likely for either of us depending on the truth of the proposition, averaging our probability estimates makes sense."

Now of course, real mistakes are continuous, and more seriously, different evidence was observed by both parties, so I don't think averaging is the correct answer in all cases.

However, that said, this formula gives p(X) = 21/40, and thus your fair share is $10.50. While I won't say this with authority because I wasn't there and I don't know what types of mistakes people are likely to make, and how likely each of you was to see and remember what, you owe Nick Bostrom ninety-seven cents.

Hm... yes, I can't say your formula is obviously right, but mine is obviously inconsistent. I guess I owe Nick ninety-seven cents.

If they're both about equally likely to reason as well, I'd say Eliezer's portion should be p * $20, where ln(p/(1-p))=(1.0*ln(0.2/0.8)+1.0*ln(0.85/0.15))/(1.0+1.0)=0.174 ==> p=0.543. That's $10.87, and he owes NB merely fifty-six cents.

Amusingly, if it's mere coincidence that the actual split was 3:4 and in fact they split according to this scheme, then the implication is that we are trusting Eliezer's estimate 86.4% as much as NB's.

Re: "Gell-Mann is a strong proponent of many-worlds" - see also:

"Gell-Mann does not like the "many worlds" interpretation."

• http://en.wikiversity.org/wiki/Talk:Many-worlds_interpretation

He seems to prefer "consistent histories":

http://en.wikipedia.org/wiki/Consistent_histories

The Taxi anecdote is ultra-geeky - I like that! ;-)

Also, once again I accidentally commented on Eliezers last entry, silly me!

Ooo I love this game! How many inconsistencies can you get in a taxi...

2008: "Fifteen percent," said Nick. "I would have said twenty percent," I said.

2004: He named the probability he thought it was his (20%), I named the probability I thought it was mine (15%)

Any more? :)

We're all flawed and should bear that in mind in disagreements, even when the mind says it's sure.

RI, I've never claimed my memory for past events is reliable. In fact, it often seems to me that my mind has no grip on arbitrary facts - facts that could just as easily have been "the other way around".

In 2004 the event in question was only a year ago, so the 2004 memory is much more likely to be correct.

I tend to just avoid identity fetishes, symmetry fetishes, and structural fetishes. Structural fetishes bite me from time to time but only when I'm feeling extra geeky and I'm trying to reduce everything down to Jack's magic bean.

Probability and provability are not one and the same.

The dependency of both collapse and many world on the design of the experiment makes me very fidgetty. Also the fact that QM keeps dodging the question of what goes on at the filter/splitter/polarizer bugs the bejeezus out of me. You would think one the multilapse twin theories could create the probabilities from subspaces existing at the decision boundary. That would impress me.

I still haven't heard why your counterpart has to be you. Why not one of the rats of NIMH?

And since there are multiple entanglements wouldn't it be necessary to make every part, the polarizer, the detector, and the photon superpose?

I always try to attack a problem from where it tickles. Can a macromechanical system be built such that without any modification of the photon substitute or the polarizer substitute the results would match the micro scale results?

If so then just shine a light on that sucker while it does its business.

I've come to the conclusion that:

#1 the use of amplitudes and complex numbers is really an unholy marriage between logic and arithmetic. The structure of multiplying by imaginary numbers and adding resembles logical arguments rather than a mysterious quantum recipe.

#2 the use of complex numbers implies the encoding of two interdependent variables into one construct like the way you can replace i with t in RC circuit equations. Soon as you do that you get a time dependent value for the voltage.

i by definition when squared transforms one variable (or member of a vector/quaternion/octonion) into its neighboring term without the usual fuss of inverting equations. just square and subtract or reduce to restore the form you are seeking.

I don't think Scott Aaronson ever did an Enron commercial.

I tend to just avoid identity fetishes, symmetry fetishes, and structural fetishes. Structural fetishes bite me from time to time but only when I'm feeling extra geeky and I'm trying to reduce everything down to Jack's magic bean.

Probability and provability are not one and the same.

The dependency of both collapse and many world on the design of the experiment makes me very fidgetty. Also the fact that QM keeps dodging the question of what goes on at the filter/splitter/polarizer bugs the bejeezus out of me. You would think one the multilapse twin theories could create the probabilities from subspaces existing at the decision boundary. That would impress me.

I still haven't heard why your counterpart has to be you. Why not one of the rats of NIMH?

And since there are multiple entanglements wouldn't it be necessary to make every part, the polarizer, the detector, and the photon superpose?

I always try to attack a problem from where it tickles. Can a macromechanical system be built such that without any modification of the photon substitute or the polarizer substitute the results would match the micro scale results?

If so then just shine a light on that sucker while it does its business.

I've come to the conclusion that:

#1 the use of amplitudes and complex numbers is really an unholy marriage between logic and arithmetic. The structure of multiplying by imaginary numbers and adding resembles logical arguments rather than a mysterious quantum recipe.

#2 the use of complex numbers implies the encoding of two interdependent variables into one construct like the way you can replace i with t in RC circuit equations. Soon as you do that you get a time dependent value for the voltage.

i by definition when squared transforms one variable (or member of a vector/quaternion/octonion) into its neighboring term without the usual fuss of inverting equations. just square and subtract or reduce to restore the form you are seeking.

Here's one simple rule which I hope you're following, and to which I suspect you should give more weight: If you are already aware that many apparently intelligent people disagree about an issue, hearing another person's conclusions about that issue should have little effect on your beliefs about that issue. If you hear an intelligent person disagree with a belief for which you weren't aware of intelligent disagreement, give more weight to that person's belief than your instincts suggest. I'd give more weight to your opinions if it were clear that you relied heavily on rules such as this.

I object to your advice that "if any of these reasonable-sounding principles turn out not to work, don't use them." The evidence about what works for investing is sufficiently noisy that it's dangerous for investors to think they can get anything resembling conclusive evidence about what works. I'd suggest instead advising them to "assume these rules are necessary but not sufficient for success."