How should Eliezer and Nick's extra $20 be split

[EDITED substantially from initial state to fix some serious errors.]

If we try to do this properly with Bayes, here's how it goes.Your odds ratio Pr(N) : Pr(E) presumably starts at 1:1 since the initial situation is symmetrical. Then it needs to be multiplied by Pr(N says 15% | N) : Pr(N says 15% | E) and then by Pr(E says 20% | N) | Pr(E says 20% | E).

Continuing to assume that we have no outside knowledge that would distinguish N from E (in reality we might; they're quite different people; this might also change our prior odds ratio), we'd better have a single function giving Pr(say p | was yours) : Pr(say p | not yours), and then our posterior odds ratio is f(15%) / f(20%).

(And then Pr(E) is f(20%) / (f(15%) + f(20%)) and Eliezer should get that fraction of the $20.)

I don't see any strong grounds for singling out one choice of f as The Right One. At least for comparably "round" choices of p, f should be increasing. It shouldn't go all the way to 0:1 at p=0 or 1:0 at p=1, but for people as clueful about probabilities as E and N it should be pretty close at those extremes.

But, still, where does f come from? Presumably you have some idea of how carefully you counted your money (and some idea of how the other guy counted, which complicates matters a bit more), and the more carefully you counted (and weren't aware that the $20 was yours) the more likely you are to give a low value of p, and also the more likely it is that the money really isn't yours.

Toy model #1: there's a continuum of counting procedures, each of which puts in an extra $20 with probability q for some 0 <= q <= 1, and you know in hindsight which you employed, and you give that value of q as your estimate. But that isn't actually the right value of q; the right value depends on what the other person is expected to have done. We should think better of both N and E than to model them this way.

If you do this, though, what happens is that if the prior probability of error for each party is g (for "goof") then f(q) = q(1-g) : (1-q)g, and then f(q1) / f(q2) = q1(1-q2) : (1-q1)q2. This is the same as option 4 in the article.

Toy model #2: there's a continuum of counting procedures, as before, and which you use is chosen according to some pdf, the same pdf for both parties, and you know which procedure you used and what the pdf is. Then your Pr($20 is mine | q) is Pr(I goofed and the other guy didn't | q, exactly one $20 extra) = q Pr(other guy didn't goof) / (q Pr(other guy didn't goof) + (1-q) Pr(other guy goofed)) so if the overall goof probability is g (this is all you need to know; the whole pdf is unnecessary) then for a given value of q, the probability you quote will be q(1-g) / [q(1-g)+(1-q)g]. Which means (scribble, scribble) that if you quote a probability p then your q is (necessarily, exactly) gp / ((1-g)-(1-2g)p). Which means, unless I'm confused which I might be because it's after 1am local time and I know I already made one mistake in the first version of this comment, that when you say p1 and he says p2 the odds ratio is gp1 / ((1-g)-(1-2g)p1) : gp2 / ((1-g)-(1-2g)p2) = p1 ((1-g)-(1-2g)p2) : p2 ((1-g)-(1-2g)p1).

This depends on g, which is an unknown parameter. When g->0 we recover the same answer as in toy model #1. (When errors are very rare, the probability that the $20 is yours is basically the same as the probability that you goofed.) When g->1 the posterior odds approach 1:1. (When non-errors are very rare, Pr(N goofed) and Pr(E goofed) are both very close to 1, so their ratio is very close to 1, so once we know there was exactly one error it's about equally likely to be either's.) As g increases, the odds ratio becomes monotonically less unequal.

I don't see any obvious principled way of estimating g, beyond the trivial observations that it shouldn't be too small since an error happened in this case and it can't be too large since N and E were both surprised by it.

If, like Coscott, you feel that when p2=1-p1 (i.e., the two parties agree on the probability that the money belongs to either of them) the posterior odds should be the same as you'd get from either (i.e., you should divide the money in the "obvious" proportions), then this is achieved for g=1/2 and for no other choice of g. For g=1/2, the posterior odds are simply p1 : p2. That's the original Yudkowsky solution, #1 in the article, with which shminux agrees in comments here.

If, IIRC like someone in the original discussion, you feel that replacing p1,p2 with 1-p1,1-p2 should have the same effect as replacing them with p2,p1 -- i.e., if one party says "20% N" and the other says "15% E" it doesn't matter which is which -- then you must take either g=0 (reproducing answer 4) or g=1 (always splitting equally).

What's wrong with toy model #2, aside from that annoying free parameter? A few things. Firstly, in reality people tend to be overconfident. (Maybe not smart bias-aware probability-fluent people talking explicitly about probabilities, but I wouldn't bet on it.) This amounts to saying that we should move q1 and q2 towards 50% somewhat before doing the calculation, which will make the posterior odds less unequal. Exactly how much is anyone's guess; it depends on our guess of the calibration curves for people like N and E in situations like this.

Secondly, you won't really know your own procedure's q exactly. You'll have some uncertain estimate of it. If my scribblings are right then symmetrical uncertainty in q doesn't actually change the posterior odds. Your uncertainty about q won't really be symmetrical, even after accounting for miscalibration -- e.g., if you think q=0.01 then you might be 0.02 too low but not 0.02 too high -- but for moderate values like 0.15 or 0.20 symmetry's probably a harmless assumption.

Thirdly, whatever your "internal" estimate of q it'll get rounded somewhat, hence the nice round 15% and 20% figures N and E gave. This is probably also a fairly symmetrical affair for probabilities in this range, so again it probably doesn't make much difference to the posterior odds.

On the basis of all of which, I'll say:

• Answer #4 in the article seems like an upper bound on how unequally the money can reasonably be distributed. If you either adopt the overoptimistic model 1, or take g to be tiny in model 2, and don't allow for any overconfidence, then you get this answer.
• There doesn't seem to be any very obvious way to choose that free parameter g.
  • If you take the two parties' estimates of the probability that they goofed as indicative and take g=(p1+p2)/2, which really isn't a very principled choice but never mind, then in this case you get a division $20 = $8.35 + $11.65, just barely less unequal than the g=0 solution.
  • If you take g=0 you get answer #4 again. This is your only choice if you want only the two probability assignments to matter, and not who offers which.
  • If you take g=1/2 you get answer #1. This is your only choice if you want to divide the money in the ratio p:1-p when both parties agree that Pr(money is N's) = p.
  • If you take g=1 you get equal division regardless of quoted probabilities.
• Marcello's answer (#2 in the article) is always less unequal than #4 and is nice and easy to calculate. It might be a good practical compromise for those not quite pragmatic enough to adopt the obvious "meh" solution I proposed elsewhere in comments.

I wish to disclaim my prior reply as long-ago and made-up-on-the-spot.

When I got to the end of the anecdote, I immediately assumed that Eliezer ended up taking the entire twenty dollars; it wasn't until I read a little further that I realized we were discussing other options. I can justify this split by saying that since Eliezer appears to be more deserving of the twenty dollars, there's no reason for Nick to receive a penny of it.

Another possible line of reasoning: there's an extra twenty dollars there, and two people whom it would be reasonable to give it to, each of them perfectly willing to let the other have it, and so the money should go to whoever wants it more (or perhaps whoever's less well off to begin with, or whoever's younger).

And another: Eliezer is more deserving of it, and so we should prefer to give it to Eliezer, but Nick's expected amount-of-money-that-was-his is 0.15 * $20, so we should give Eliezer as much as possible while still giving Nick at least that amount, i.e. Eliezer should get 85% and Nick should get 15%.

Or just give it to the driver.

I don't see any reason that the probability-assigning function g would be identical to the money-assigning function f, because the amount of money Eliezer should get is not necessarily proportional to the probability that the twenty originally belonged to Eliezer. If we pretend that the algorithms given in the article are probability-assigning algorithms, then 4 looks the most compelling, 1 also looks reasonable, I don't really understand 3, and 2 looks problematic (if Nick had said 15% and Eliezer had said 0%, then the correct adjustment seems to be to say that it's Nick's, with 100% probability).

My preferred solution: "Meh, 15% and 20% are pretty close. Call it $10 each?" "OK." Done.

I interpreted the story as EY says "20% odds it's mine" and NB says "15% odds it's mine", so the first approximation is to renormalize the odds to add to 100% and split the bill as 20/(20+15)= and 15/(20+15) respectively. Anything more involved requires extra information and progressively involved assumptions. For example, what can you conclude about the calibration level of each one? Did NB actually mean 1/7 when he said 15%? Is EY prone to over/under-estimating the reliability of his memory? And practical questions: how cumbersome would it be to split the $20 given the change available?

So the real exchange probably went something like this: "Here, I'll take the twenty and give you a ten and a dollar bill... Oh, here is a couple of quarters, too". "Keep the quarters, I hate coins." 'Done."

Your concluding question does not seem to be relevant, and the calculation depends heavily on how you assigned p and q to begin with. Was there shared evidence? What other alternatives were considered? I can easily imagine that in some circumstances the combined probability could be lower than both p and q, because both heavily rely on the same piece of evidence and not seeing that the two hypotheses were equivalent weakens the value of this evidence (what else did you miss?).

I'm surprised nobody has yet written that the appropriate way for them to split it in this case is $10 each, because the transaction cost of working out something else in more detail and then making the appropriate change is greater than the difference between $10 and whatever the appropriate answer is.

Another relevant figure is EY's estimate that the bill was Nick's, and vice versa. We do not yet have enough data to solve the problem. If EY said that he also had a 20% probability that the bill was Nick's and Nick had a 15% probability that the bill was EY's, they should split it 50/50.

Also, if Nick had said it was 1% chance it was his and EY had said it was 3% it was his (they had both checked their wallets earlier, say), would it make sense to split it 3:1 in favor of EY, or anything close to that? I figure that in that case the probability that it was already in the cab and they should split it 50/50 as a windfall begins to dominate. Such a notion does not seem to have entered any of the above calculations.

As promised, here is my analysis.

First of all, I think that the g and f functions should be the same. My reason is that to be completely satisfied with who gets the 20, both people should update their probabilities to the same value, so f(p,q) seems like the probability that EY and NB should both walk away with saying is the probability that EY deserved the 20. Since EY and NB trust each other, this should be the same if there is a single person with both beliefs learning they are equivalent. Some of the features I want in f are easier to justify in the g example, which might have caused some of my mistakes.

I think this question is inherently about how much we treat p and q as coming from independent sources of information. If we say that the sources are independent, then #4 is the only reasonable answer. However, the dependency of the evidence is not known.

I think f should have the following properties:

A: f(p,p)=p -- If we wanted f to be a general rule for how to take two probabilities and output a probability that is an agreement of the two, then there is a danger in setting g(p,p) to anything other than p. In that we can repeatedly apply our rule and get different answers. We update p and q to g(p,q) and g(p,q), but then these are our new probabilities, so we will update them both to g(g(p,q),g(p,q)). Our new answer is not consistent under reflection. Therefore, I think g(p,p) should be p, and this makes me believe that f(p,p) should also be p. (Maybe I am working by analogy, but I still believe this.)

B: f(p,q)=1-f(1-q,1-p) -- This is just saying that the answer is symmetric with respect to swapping EY and NB.

C: f(p,q)=f(q,p) -- This is saying that the answer is symmetric with respect to swapping who says what. This property seems really necessary in the g problem.

D: f(1,q)=1 -- If EY knows he put the 20 in, he should get it back. In the g problem, if A is a theorem, and we learn B if and only if A, then we can prove B also.

E: p1>=p2, then f(p1,q)>=f(p2,q) -- If EY is more sure he should get the money, he shouldn't get less money.

F: f(f(p,q),f(r,s))=f(f(p,r),f(q,s)) -- This relation doesn't mean much for f, but for g, it is saying that the order in which we learn conjectures are equivalent shouldn't change the final answer.

G: f is continuous -- A small change in probability shouldn't have a huge effect on the decision.

I think that most people would agree with B, C, and E, but looking at the comments, A and D are more controversial. F doesn't make any sense unless you believe that f=g. I am not sure how people will feel about G. Notice that if B, C, and D together imply that f(1,0) is not defined, because it would have to be both 1 and 0, I think this is okay. You can hardly expect EY and NB to continue trusting each other after something like this, but it is necessary to say to be mathematically correct.

Now, to critique the proposed solutions.

1 Violates C and D (I didn't check F)

2 Violates D

4 Violates A

3 Does not violate any of my features.

I did not do a good job of describing where #3 came from, so let me do better. #3 chooses the unique value f, such that if we wanted to update EYs probability to f and update NBs probability to f, this would take the same amount of evidence. It satisfies A because if they are both already the same, then it doesn't take any evidence. It satisfies D, because no finite amount of evidence can bring you back from certainty.

We do not have enough information to say that #3 is the unique solution. If we were to try to, it would look like roughly like this:

If we think about the problem by looking at p =log(p/1-p). Then #3 just finds the f =(p +q )/2. E and A together tell us that f should be somewhere between p and q but it is not immediately clear that the arithmetic mean is the best compromise. However, I believe this implies that we can apply some monotone function h, such that h(f ) is the always the arithmetic mean of h(p ) and h(q ). B tells us that this monotone function must be an odd function (h(-x)=-h(x)).

From here, #3 assumes that h(x)=x, but if we were to take h(x)=x^3 for example, we would still meet all of our properties. We have freedom from the fact that we can weigh probabilities with different distances from 1/2 differently.

How about if the two do probability updates, Aumann agreement style, until their estimates agree? (Maybe this is equivalent to your method 3; I don't recall how the math works out.)

EY believes he deserves the money with probability p.

NB believes he deserves the money with probability q.

The following rules would, I think, be considered fair by most people:

• If p=q, f(p,q) should be 1/2.

• If p=0, f(p,q) should be 0 (EY gets nothing).

• If q=0, f(p,q) should be 1 (EY gets everything).

The simplest rational function obeying these conditions is f(p,q) = p/(p+q)

For 4 could you explain where that formula comes from? I thought I understood how to apply Bayes' theorem, but I'm getting stuck on what P(B) or P(B|A) should be (where A is: $20 is Eliezer's, and B is: Nick says 85%).

Should

Step 1: Before anything else, let's get the normative question out of the way. What exactly is the goal here...i.e, what are we maximizing for?

"This bill clearly belongs to exactly one of the two us; I think it is 80% likely to be yours and 20% likely to be mine, and you think it is 85% likely to be mine and 15% likely to be yours. In order to fairly divide it, we weigh our beliefs equally (by symmetry), and divide it according to the ratio 80+15:20+85 you:me; that's 95:105, or 47.5:52.5. That's a 52.5% chance that I get the bill, and a 47.5% chance that you get it." (Same expected values as 2, from equivalent math)

I don't think there's a general case for the function g. Consider the case where hypotheses B, C, and D are mutually exclusive. Proposition A is equivalent to "not C" but B is only known to be mutually exclusive with C. On discovering that D is false, A becomes equivalent to B, but there is also potentially new information about C.

Variant 1: In a departure from Monty Hall; I put a bean under one of the cups B,C,D after rolling a d10. (Time a) Then I show you what is under cup D. There is not a bean. (Time b)

If the odds ratio at time a is 2:4:4, then the odds at time b are 2:4:0. Proposition A went from 6/10 likely to 1/3 likely, while B went from 2/10 to 1/3.

Variant 2: I roll the die; if the result is prime, I put the bean under cup d. Otherwise, if the result is greater than five, I put the bean under cup c. Otherwise I put the bean under cup b. At time a, the odds are 2:4:4. Then I tell you that the number I rolled is a perfect square. You should update to 2:1:0

In both cases, you updated that B and not-C were equivalent, but in the latter case you gained new information about B and C. In the general case, if there is no new information about B and C, then the ratio B:C should remain constant; I intuit that that will probably mean a fairly simple transformation.

My intuition is #1, but a variant on #2 would be to use the geometric mean instead - so f = sqrt(pq). This has the desirable (but common) features that:

p=q => f=p

But unfortunately

p = 1, q = 0 => f=0

Are either of them well calibrated enough for the difference between 15% and 20% to be meaningful?

I'm GreedyAlgorithm, and here is some more discussion: https://rhollerith3.jottit.com/

Hal Finney's answer here which Richard Hollerith seems to endorse is #4.

Reposting:

I got the same answer as Marcello by assuming that each of them should get the same expected utility out of the split.

Say that Nick keeps x and Eliezer keep y. Then the expected utility for Nick is

0.85 x − 0.15 ($20 − x),

while the expected utility for Eliezer is

0.8 y − 0.2 ($20 − y).

Setting these equal to each other, and using x + y = $20, yields that Nick should keep x = $9.50, leaving y = $10.50 for Eliezer.

Method 3 chooses the unique f such that updating p to f and updating q to f require the same amount of information.

Before reading GreedyAlgorithm's post, I decided independently that I support method 3, although I may find another answer I like better. Methods 1 and 2 I do not like, because if you let p=1 and q=1/2, they do not give all the money to EY. Method 4 I do not like, because I think that f(p,p) should equal p. However, I have no argument for 3 other than the fact that it feels right, and it meets all the criteria I can think of in degenerate cases.

There's something really off with the formatting here - some of the paragraphs have overlapping lines

My first reaction to the second question is to consider the case in which p + q = 1. Then, the answer is clearly that g(p, q) = p + q. I suspect that this is incomplete, and that further relevant information needs to be specified for the answer to be well-defined.