Posts
Comments
Your post reads a bit strangely.
At first, I thought you were arguing that AGI might be used by some extremists to wipe out most of humanity for some evil and/or stupid reason. Which does seem like a real risk.
Then you went on to point out that someone who thought that was likely might wipe out most of humanity (not including themselves) as a simple survival strategy, since otherwise someone else will wipe them out (along with most other people). As you note, this requires a high level of unconcern for normal moral considerations, which one would think very few people would countenance.
Now comes the strange part... You argue that actually maybe many people would be willing to wipe out most of humanity to save themselves, because... wiping out most of humanity sounds like a pretty good idea!
I'm glad that in the end you seem to still oppose wiping out most of humanity, but I think you have some factual misconceptions about this, and correcting them is a necessary first step to thinking of how to address the problem.
Concerning climate change, you write: "In the absence of any significant technological developments, sober current trajectory predictions seem to me to range from 'human extinction' to 'catastrophic, but survivable'".
No. Those are not "sober" predictions. They are alarmist claptrap with no scientific basis. You have been lied to. Without getting into details, you might want to contemplate that global temperatures were probably higher than today during the "Holocene Climatic Optimum" around 8000 years ago. That was the time when civilization developed. And temperatures were significantly higher in the previous interglacial, around 120,000 years ago. And the reference point for supposedly-disastrous global warming to come is "pre-industrial" time, which was in the "little ice age", when low temperatures were causing significant hardship. Now, I know that the standard alarmist response is that it's the rate of change that matters. But things changed pretty quickly at the end of the last ice age, so this is hardly unprecedented. And you shouldn't believe the claims made about rates of change in any case - actual science on this question has stagnated for decades, with remarkably little progress being made on reducing the large uncertainty about how much warming CO2 actually causes.
Next, you say that the modern economy is relatively humane "under conditions of growth, which, under current conditions, depends on a growing population and rising consumption. Under stagnant or deflationary conditions it can be expected to become more cutthroat, violent, undemocratic and unjust."
Certainly, history teaches that a social turn towards violence is quite possible. We haven't transcended human nature. But the idea that continual growth is needed to keep the economy from deteriorating just has no basis in fact. Capitalist economies can operate perfectly fine without growth. Of course, there's no guarantee that the economy will be allowed to operate fine. There have been many disastrous economic policies in the past. Again, human nature is still with us, and is complicated. Nobody knows whether social degeneration into poverty and tyranny is more likely with growth or without growth.
Finally, the idea that a world with a small population will be some sort of utopia is also quite disconnected from reality. That wasn't the way things were historically. And even if it was, it woudn't be stable, since population will grow if there's plenty of food, no disease, no violence, etc.
So, I think your first step should be to realize that wiping out most of humanity would not be a good thing. At all. That should make it a lot easier to convince other people not to do it.
It probably doesn't matter, but I wonder why you used the name "Sam" and then referred to this person as "she". The name "Sam" is much more common for men than for women. So this kicks the text a bit "out of distribution", which might affect things. In the worst case, the model might think that "Sam" and "she" refer to different people.
There are in fact many universities that have both "research faculty" and "teaching faculty". Being research faculty has higher prestige, but nowadays it can be the case that teaching faculty have almost the same job security as research faculty. (This is for permanent teaching faculty, sessional instructors have very low job security.)
In my experience, the teaching faculty often do have a greater enthusiasm for teaching than most research faculty, and also often get better student evaluations. I think it's generally a good idea to have such teaching faculty.
However, my experience has been that there are some attitudinal differences that indicate that letting the teaching faculty have full control of the teaching aspect of the university's mission isn't a good idea.
One such is a tendency for teaching faculty to start to see the smooth running of the undergraduate program as an end in itself. Research faculty are more likely to have an ideological commitment to the advancement of knowledge, even if promoting that is not as convenient.
A couple anecdotes (from my being research faculty at a highly-rated university):
At one point, there was a surge in enrollment in CS. Students enrolled in CS programs found it hard to take all the courses they needed, since they were full. This led some teaching faculty to propose that CS courses (after first year) no longer be open to students in any other department, seeing as such students don't need CS courses to fulfill their degree requirements. Seems logical: students need to smoothly check off degree requirements and graduate. The little matter that knowledge of CS is crucial to cutting-edge research in many important fields like biology and physics seemed less important...
Another time, I somewhat unusually taught an undergrad course a bit outside my area, which I didn't teach again the next year. I put all the assignments I gave out, with solutions, on my web page. The teaching faculty instructor the next year asked me to take this down, worrying that students might find answers to future assigned questions on my web page. I pointed out that these were all my own original questions, not from the textbook, and asked whether he also wanted the library to remove from circulation all the books on this topic...
Also, some textbooks written by teaching faculty seem more oriented towards moving students through standard material than teaching them what is actually important.
Nevertheless, it is true that many research faculty are not very good at teaching, and often not much interested either. A comment I once got on a course evaluation was "there's nothing stupid about this course". I wonder what other experiences this student had had that made that notable!
These ideas weren't unfamiliar to Hinton. For example, see the following paper on "Holographic Reduced Representations" by a PhD student of his from 1991: https://www.ijcai.org/Proceedings/91-1/Papers/006.pdf
The logic seems to be:
- If we do a 1750 year simulation assuming yearly fresh water additions 80 times the current greenland ice melt rate, we see AMOC collapse.
- Before this simulated collapse, the value of something that we think could be an indicator changes.
- That indicator has already changed.
- So collapse of the AMOC is imminent.
Regarding (1), I think one can assume that if there was any way of getting their simulation engine to produce an AMOC collapse in less than 1750 years, they would have showed that. So, to produce any sort of alarming result, they have to admit that their simulation is flawed, so they can say that collapse might in reality occur much sooner. But then, if the simulation is so flawed, why would one think that the simulation's indicator has any meaning?
They do claim that the indicator isn't affected by the simulation's flaws, but without having detailed knowledge to assess this myself, I don't see any strong reason to believe them. It seems very much like a paper that sets out to show what they want to show.
From the paper:
Under increasing freshwater forcing, we find a gradual decrease (Fig. 1A) in the AMOC strength (see Materials and Methods). Natural variability dominates the AMOC strength in the first 400 years; however, after model year 800, a clear negative trend appears because of the increasing freshwater forcing. Then, after 1750 years of model integration, we find an abrupt AMOC collapse
Given that the current inter-glacial period would be expect to last only on the order of some thousands of years more, this collapse in 1750 years seems a bit academic.
I don't get it.
Apparently, the idea is that this sort of game tells us something useful about AI safety.
But I don't get it.
You obviously knew that you were not unleashing a probably-malign superintelligence on the the world by letting Ra out. So how does your letting Ra out in this game say anything about how you would behave if you did think that (at least initially)?
So I don't get it.
And if this does say something useful about AI safety, why is it against the rules to tell us how Ra won?
I don't get it.
Interesting. I hadn't heard of the Child Born on Tuesday Problem. I think it's actually quite relevant to Sleeping Beauty, but I won't go into that here...
Both problems (your 1 and 2) aren't well-defined, however. The problem is that in real life we do not magically acquire knowledge that the world is in some subset of states, with the single exception of the state of our direct sense perceptions. One could decide to assume a uniform distribution over possible ways in which the information we are supposedly given actually arrives by way of sense perceptions, but uniform distributions are rather arbitrary (and will often depend on arbitrary aspects of how the problem is formulated).
Here's a boys/girls puzzle I came up with to illustrate the issue:
A couple you've just met invite you over to dinner, saying "come by around 5pm, and we can talk for a while before our three kids come home from school at 6pm".
You arrive at the appointed time, and are invited into the house. Walking down the hall, your host points to three closed doors and says, "those are the kids' bedrooms". You stumble a bit when passing one of these doors, and accidentally push the door open. There you see a dresser with a jewelry box, and a bed on which a dress has been laid out. "Ah", you think to yourself, "I see that at least one of their three kids is a girl".
Your hosts sit you down in the kitchen, and leave you there while they go off to get goodies from the stores in the basement. While they're away, you notice a letter from the principal of the local school tacked up on the refrigerator. "Dear Parent", it begins, "Each year at this time, I write to all parents, such as yourself, who have a boy or boys in the school, asking you to volunteer your time to help the boys' hockey team..." "Umm", you think, "I see that they have at least one boy as well".
That, of course, leaves only two possibilities: Either they have two boys and one girl, or two girls and one boy. What are the probabilities of these two possibilities?
The symmetrical summaries of what is learned are intentionally misleading (it's supposed to be a puzzle, after all). The way in which you learned they have at least one girl is not the same as the way you learned that they have at least one boy. And that matters.
You may think the difference between "the card is an Ace" and "JeffJo says the card is an Ace" is just a quibble. But this is actually a very common source of error.
Consider the infamous "Linda" problem, in which researchers claim that most people are irrational because they think "Linda is a bank teller" is less likely than "Linda is a bank teller and active in the feminist movement". When you think most people are this blatantly wrong, you maybe need to consider that you might be the one who's confused...
Actually, there is no answer to the problem as stated. The reason is that the evidence I (who drew the card) have is not "the card is an Ace", but rather "JeffJo said the card is an Ace". Even if I believe that JeffJo never lies, this is not enough to produce a probability for the card being the Ace of Spades. I would need to also consider my prior probability that JeffJo would say this conditional on it being the Ace of Space, the Ace of Hearts, the Ace of Diamonds, or the Ace of Clubs. Perhaps I believe the JeffJo would never say the card is an Ace if it is a Space. In that case, the right answer is 0.
However, I agree that a "reward structure" is not required, unless possible rewards are somehow related to my beliefs about what JeffJo might do.
For example, I can assess my probability that the store down the street has ice cream sundaes for sale when I want one, and decide that the probability is 3/4. If I then change my mind and decide that I don't want an ice cream sundae after all, that should not change my probability that one is available.
Except, you know, that's exactly what I do with Full Non-indexical Conditioning, but you don't like the answer.
Philosophy is full of issues where lots of people think they're just doing the "obvious thing", except these people come to different conclusions.
The wording may be bad, but I think the second interpretation is what is intended. Otherwise the discussion often seen of "How might your beliefs change if after awakening you were told it is Monday?" would make no sense, since your actual first awakening is always on Monday (though you may experience what feels like a first awakening on Tuesday).
I'm not sure what you're saying here.
Certainly an objective outside observer who is somehow allowed to ask the question, "Has somebody received a green ball?" and receives the answer "yes" has learned nothing, since that was guaranteed to be the case from the beginning. And if this outside observer were somehow allowed to override the participants' decisions, and wished to act in their interest, this outside observer would enforce that they do not take the bet.
But the problem setup does not include such an outside objective observer with power to override the participants' decisions. The actual decisions are all made by individual participants. So where do the differing perspectives come from?
Perhaps of relevance (or perhaps not): If an objective outside observer is allowed to ask the question, "Has somebody with blonde hair, six-foot-two-inches tall, with a mole on the left cheek, barefoot, wearing a red shirt and blue jeans, with a ring on their left hand, and a bruise on their right thumb received a green ball?", which description they know fits exactly one participant, and receives the answer "yes", the correct action for this outside observer, if they wish to act in the interests of the participants, is to enforce that the bet is taken.
Yes, this is the right view.
In real life we never know for sure that coin tosses are independent and unbiased. If we flip a coin 50 times and get 50 heads, we are not actually surprised at the level of an event with 1 in 2 to the -50 probability (about 1 in 10 to the -15). We are instead surprised at the level of our subjective probability that the coin is grossly biased (for example, it might have a head on both sides), which is likely much greater than that.
But in any case, it is not rare for rare events to occur, for the simple reason that the total probability of a set of mutually-exclusive rare events need not be low. That is the case with 50 coin tosses that we do assume are unbiased and independent. Any given result is very rare, but of course the total probability for all possible results is one. There's nothing puzzling about this.
Trying to avoid rare events by choosing a restrictive sigma algebra is not a viable approach. In the sigma algebra for 50 coin tosses, we would surely want to include events for "1st toss is a head", "2nd toss is a head", ..., "50th toss is a head", which are all not rare, and are the sort of event one might want to refer to in practice. But sigma algebras are closed under complement and intersection, so if these events are in the sigma algebra, then so are all the events like "1st toss is a head, 2nd toss is a tail, 3rd toss is a head, ..., 50th toss is a tail", which all have probability 1 in 20 to the -50.
Well, for starters, I'm not sure that Ape in the coat disagrees with my statements above. The disagreement may lie elsewhere, in some idea that it's not the probability of the urn with 18 green balls being chosen that is relevant, but something else that I'm not clear on. If so, it would be helpful if Ape in the coat would confirm agreement with my statement above, so we could progress onwards to the actual disagreement.
If Ape in the coat does disagree with my statement above, then I really do think that that is in the same category as people who think the "Twin Paradox" disproves special relativity, or that quantum mechanics can't possibly be true because it's too weird. And not in the sense of thinking that these well-established physical theories might break down in some extreme situation not yet tested experimentally - the probability calculation above is of a completely mundane sort entirely analogous to numerous practical applications of probability theory. Denying it is like saying that electrical engineers don't understand how resistors work, or that civil engineers are wrong about how to calculate stresses in bridges.
But then... Why are you expecting point (2) to follow?
I'm not clear on your step (1). The new taxes would decrease the amount of money people have to spend, but this would be exactly balanced by an increase in money available to spend due to people no longer using their money to buy government bonds. The people with more money to spend may not be the same as the people with less money to spend, so there could be second-order effects if this shifts which goods or services money gets spent on, but how seems hard to predict. There is also the issue that some government bonds are bought by foreigners - but then, foreigners can buy exported goods too...
... priors are not actually necessary when working with Bayesian updates specifically. You can just work entirely with likelihood ratios...
I think that here you're missing the most important use of priors.
Your prior probabilities for various models may not be too important, partly because it's very easy to look at the likelihood ratios for models and see what influence those priors have on the final posterior probabilities of the various models.
The much more important, and difficult, issue is what priors to use on parameters within each model.
Almost all models are not going to fix every aspect of reality that could affect what you observe. So there are unknowns within each model. Some unknown parameters may be common to all models; some may be unique to a particular model (making no sense in the context of a different model). For parameters of both types, you need to specify prior distributions in order to be able to compute the probability of the observations given the model, and hence the model likelihood ratios.
Here's a made-up example (about a subject of which I know nothing, so it may be laughably unrealistic). Suppose you have three models about how US intelligence agencies are trying to influence AI development. M0 is that these agencies are not doing anything to influence AI development. M1 is that they are trying to speed it up. M2 is that they are trying to slow it down. Your observations are about how fast AI development is proceeding at some organizations such as OpenAI and Meta.
For all three models, there are common unknown parameters describing how fast AI progresses at an average organization without intelligence agency intervention, and how much variation there is between organizations in their rate of progress. For M1 and M2, there are also parameters describing how much the agencies can influence progress (eg, via secret subsidies, or covert cyber attacks on AI compute infrastructure), and how much variation there is in the agencies' ability to influence different organizations.
Suppose you see that AI progress at OpenAI is swift, but progress at Meta is slow. How does that affect the likelihood ratios among M0, M1, and M2?
It depends on your priors for the unknown model parameters. If you think it unlikely that such large variation in progress would happen with no intelligence agency intervention, but that there could easily be large variation in how much these agencies can affect development at different organizations, then you should update to giving higher probability to M1 or M2, and lower probability to M0. If you also thought the slow progress at Meta was normal, you should furthermore update to giving M1 higher probability relative to M2, explaining the fast progress at OpenAI by assistance from the agencies. On the other hand, if you think that large variation in progress at different organizations is likely even without intelligence agency intervention, then your observations don't tell you much about whether M0, M1, or M2 is true.
Actually, of course, you are uncertain about all these parameters, so you have prior distributions for them rather than definite beliefs, with the likelihoods for M0, M1, and M2 being obtained by integrating over these priors. These likelihoods can be very sensitive to what your priors for these model parameters are, in ways that may not be obvious.
I think these examples may not illustrate what you intend. They seem to me like examples of governments justifying policies based on second-order effects, while actually doing things for their first-order effects.
Taxing addictive substances like tobacco and alcohol makes sense from a government's perspective precisely because they have low elasticity of demand (ie, the taxes won't reduce consumption much). A special tax on something that people will readily stop consuming when the price rises won't raise much money. Also, taxing items with low elasticity of demand is more "economically efficient", in the technical sense that what is consumed doesn't change much, with the tax being close to a pure transfer of wealth. (See also gasoline taxes.)
Government spending is often corrupt, sometimes in the legal sense, and more often in the political sense of rewarding supporters for no good policy reason. This corruption is more easily justified when mumbo-jumbo economic beliefs say it's for the common good.
The first-order effect of mandatory education is that young people are confined to school buildings during the day, not that they learn anything inherently valuable. This seems like it's the primary intended effect. The idea that government schooling is better for economic growth than whatever non-mandatory activities kids/parents would otherwise choose seems dubious, though of course it's a good talking point when justifying the policy.
So I guess it depends on what you mean by "people support". These second-order justifications presumably appeal to some people, or they wouldn't be worthwhile propaganda. But I'm not convinced that they are the reasons more powerful people support these policies.
If forced to choose between "prices increased a lot, and are still increasing a lot" and "prices increased a lot, but have now stabilized", the correct answer is the first. A more accurate answer would be "prices increased a lot, and are still increasing, but at a bit slower pace", but it wasn't an option.
Good to know that most people in the US are clear on this, even if your president isn't.
I'm having a hard time making sense of what you're arguing here:
The problem was about dynamic inconsistency in beliefs, while you are talking about a solution to dynamic inconsistency in actions.
I don't see any inconsistency in beliefs. Initially, everyone thinks that the probability that the urn with 18 green balls is chosen is 1/2. After someone picks a green ball, they revise this probability to 9/10, which is not an inconsistency, since they have new evidence, so of course they may change their belief. This revision of belief should be totally uncontroversial. If you think a person who picks a green ball shouldn't revise their probability in this way then you are abandoning the whole apparatus of probability theory developed over the last 250 years. The correct probability is 9/10. Really. It is.
I take the whole point of the problem to be about whether people who for good reason initially agreed on some action, conditional on the future event of picking a green ball, will change their mind once that event actually occurs - despite that event having been completely anticipated (as a possibility) when they thought about the problem beforehand. If they do, that would seem like an inconsistency. What is controversial is the decision theory aspect of the problem, not the beliefs.
Your assumption that people act independently from each other, which was not part of the original problem, - it was even explicitly mentioned that people have enough time to discuss the problem and come to a coordinated solution, before the experiment started, - allowed you to ignore this nuance.
As I explain above, the whole point of the problem is whether or not people might change their minds about whether or not to take the bet after seeing that they picked a green ball, despite the prior coordination. If you build into the problem description that they aren't allowed to change their minds, then I don't know what you think you're doing.
My only guess would be that you are focusing not on the belief that the urn with 18 green balls was chosen, but rather on the belief in the proposition "it would be good (in expectation) if everyone with a green ball takes the bet". Initially, it is rational to think that it would not be good for everyone to take the bet. But someone who picks a green ball should give probability 9/10 to the proposition that the urn with 18 balls was chosen, and therefore also to the proposition that everyone taking the bet would result in a gain, not a loss, and one can work out that the expected gain is also positive. So they will also think "if I could, I would force everyone with a green ball to take the bet". Now, the experimental setup is such that they can't force everyone with a green ball to take the bet, so this is of no practical importance. But one might nevertheless think that there is an inconsistency.
But there actually is no inconsistency. Seeing that you picked a green ball is relevant evidence, that rightly changes your belief in whether it would be good for everyone to take the bet. And in this situation, if you found some way to cheat and force everyone to take the bet (and had no moral qualms about doing so), that would in fact be the correct action, producing an expected reward of 5.6, rather than zero.
The charity sees just what I say above - an average loss of 22.8 if you vote "no", and an average loss of only 20 if you vote "yes".
An intuition for this is to recognize that you don't always get to vote "no" and have it count - sometimes, you get a red ball. So never taking the bet, for zero loss, is not an option available to you. When you vote "no", your action counts only when you have a green ball, which is nine times more likely when your "no" vote does damage than when it does good.
I think you're just making a mistake here. Consider the situation where you are virtually certain that everyone else will vote "yes" (for some reason we needn't consider - it's a possible situation). What should you do?
You can distinguish four possibilities for what happens:
- Heads, and you are one of the 18 people who draw a green ball. Probability is(1/2)(18/20)=9/20.
- Heads, and you are one of the 2 people who draw a red ball. Probability is (1/2)(2/20)=1/20.
- Tails, and you are one of the 2 people who draw a green ball. Probability is (1/2)(2/20)=1/20.
- Tails, and you are one of the 18 people who draw a red ball. Probability is (1/2)(18/20)=9/20.
If you always vote "no", the expected reward will be 0(9/20)+12(1/20)+0(1/20)+(-52)(9/20)=-22.8. If you always vote "yes", the expected reward will be 12(9/20)+12(1/20)+(-52)(1/20)+(-52)(9/20)=-20, which isn't as bad as the -22.8 when you vote "no". So you should vote "yes".
OK. Perhaps the crucial indication of your view is instead your statement that 'On the level of our regular day to day perception "I" seems to be a simple, indivisible concept. But this intuition isn't applicable to probability theory.'
So rather than the same person having more than one probability for an event, instead "I" am actually more than one person, a different person for each decision I might make?
In actual reality, there is no ambiguity about who "I" am, since in any probability statement you can replace "I" by 'The person known as "Radford Neal", who has brown eyes, was born in..., etc., etc.' All real people have characteristics that uniquely identify them. (At least to themselves; other people may not be sure whether "Homer" was really a single person or not.)
But if for some reason you believe that the other people with green balls will take the bet with probability 0.99, then you should take the bet yourself. The expected reward from doing so is , with E as defined above, which is +3.96. Why do you think this would be wrong?
Of course, you would need some unusually strong evidence to think the other people will take the bet with probability 0.99, but it doesn't seem inconceivable.
I don't understand how you think I've changed the problem. The $50 penalty for not all doing the same thing was an extra "helpful" feature that Eliezer mentioned in a parenthetical comment, under the apparent belief that it makes the problem easier for causal decision theory. I don't take it to be part of the basic problem description. And Eliezer seems to have endorsed getting rid of the anthropic aspect, as I have done.
And if one does have the $50 penalty, there's no problem analysing the problem the way I do. The penalty makes it even more disadvantageous to take the bet when the others are unlikely to take it, as per the prior agreement.
I suspect that you just want to outlaw the whole concept that the different people with green balls might do different things. But in actual reality, different people may do different things.
I've just put up this post, before having read your comment:
I think my conclusion is similar to yours above, but I consider randomized strategies in more detail, for both this problem and its variation with negated rewards.
I'll be interested to have a look at your framework.
On the contrary, I think there is no norm against board members criticizing corporate direction.
I think it is accepted that a member of the board of a for-profit corporation might publicly say that they think the corporation's X division should be shut down, in order to concentrate investment in the Y division, since they think the future market for Y will be greater than for X, even though the rest of the board disagrees. This might be done to get shareholders on-side for this change of direction.
For a non-profit, criticism regarding whether the corporation is fulfilling its mandate is similarly acceptable. The idea that board members should resign if they think the corporation is not abiding by its mission is ridiculous - that would just lead to the corporation departing even more from its mission.
Compare with members of a legislative body. Legislators routinely say they disagree with the majority of the body, and nobody thinks the right move if they are on the losing side of a vote is to resign.
And, a member of the miltary who believes that they have been ordered to commit a war crime is not supposed to resign in protest (assuming that is even possible), allowing the crime to be committed. They are supposed to disobey the order.
The mathematics of a latent variable model expresses the probabilities, p(x), for observations x as marginal probabilities integrating over unobserved z. That is, p(x) = integral over z of p(x,z), where p(x,z) is typically written as p(z)p(x|z).
It's certainly correct that nothing in this formulation says anything about whether z captures the "causes" of x.
However, I think it sometimes is usefully seen that way. Your presentation would be clearer if you started with one or more examples of what you see as typical models, in which you argue that z isn't usefully seen as causing x.
I'd take a typical vision model to be one in which z represents the position, orientation, and velocity of some object, at some time, and x is the pixel values from a video camera at some location at that time. Here, it does seem quite useful to view z as the cause of x. In particular, the physical situation is such that z at a future time is predictable from z now (assuming no forces act on the object), but x at a future time is not predictable from x now (both because x may not provide complete knowledge of position and orientation, and because x doesn't include the velocity).
This is the opposite of what you seem to assume - that x now cause x in the future, but that this is not true for the "summary" z. But this seems to miss a crucial feature of all real applications - we don't observe the entire state of the world. One big reason to have an unobserved z is to better represent the most important features of the world, which are not entirely inferrable from x. Looking at x at several times may help infer z, and to the extend we can't, we can represent our uncertainty about z and use this to know how uncertain our predictions are. (In contrast, we are never uncertain about x - it's just that x isn't the whole world.)
OK. My views now are not far from those of some time ago, expressed at https://glizen.com/radfordneal/res-bayes-ex.html
With regard to machine learning, for many problems of small to moderate size, some Bayesian methods, such as those based on neural networks or mixture models that I've worked on, are not just theoretically attractive, but also practically superior to the alternatives.
This is not the case for large-scale image or language models, for which any close approximation to true Bayesian inference is very difficult computationally.
However, I think Bayesian considerations have nevertheless provided more insight than frequentism in this context. My results from 30 years ago showing that infinitely-wide neural networks with appropriate priors work well without overfitting have been a better guide to what works than the rather absurd discussions by some frequentist statisticians of that time about how one should test whether a network with three hidden units is sufficient, or whether instead the data justifies adding a fourth hidden unit. Though as commented above, recent large-scale models are really more a success of empirical trial-and-error than of any statistical theory.
One can also look at Vapnik's frequentist theory of structural risk minimization from around the same time period. This was widely seen as justifying use of support vector machines (though as far as I can tell, there is no actual formal justification), which were once quite popular for practical applications. But SVMs are not so popular now, being perhaps superceded by the mathematically-related Bayesian method of Gaussian process regression, whose use in ML was inspired by my work on infinitely-wide neural networks. (Other methods like boosted decision trees may also be more popular now.)
One reason that thinking about Bayesian methods can be fruitful is that they involve a feedback process:
- Think about what model is appropriate for your problem, and what prior for its parameters is appropriate. These should capture your prior beliefs.
- Gather data.
- Figure out some computational method to get the posterior, and predictions based on it.
- Check whether the posterior and/or predictions make sense, compared to your subjective posterior (informally combining prior and data). Perhaps also look at performance on a validation set, which is not necessary in Bayesian theory, but is a good idea in practice given human fallibility and computational limitations.
- You can also try proving theoretical properties of the prior and/or posterior implied by (1), or of the computational method of step (3), and see whether they are what you were hoping for.
- If the result doesn't seem acceptable, go back to (1) and/or (3).
Prior beliefs are crucial here. There's a tension between what works and what seems like the right prior. When these seem to conflict, you may gain better understanding of why the original prior didn't really capture your beliefs, or you may realize that your computational methods are inadequate.
So, for instance, infinitely wide neural networks with independent finite-variance priors on the weights converge to Gaussian processes, with no correlations between different outputs. This works reasonably well, but isn't what many people were hoping and expecting - no "hidden features" learned about the input. And non-Bayesian neural networks sometimes perform better than the corresponding Gaussian process.
Solution: Don't use finite-variance priors. As I recommended 30 years ago. With infinite-variance priors, the infinite-width limit is a non-Gaussian stable process, in which individual units can capture significant hidden features.
OK. I think we may agree on the technical points. The issue may be with the use of the word "Bayesian".
Me: But they aren't guaranteed to eventually get a Bayesian to think the null hypothesis is likely to be false, when it is actually true.
You: Importantly, this is false! This statement is wrong if you have only one hypothesis rather than two.
I'm correct, by the usual definition of "Bayesian", as someone who does inference by combining likelihood and prior. Bayesians always have more than one hypothesis (outside trivial situations where everything is known with certainty), with priors over them. In the example I gave, one can find a b such that the likelihood ratio with 0.5 is large, but the set of such b values will likely have low prior probability, so the Bayesian probably isn't fooled. In contrast, a frequentist "pure significance test" does involve only one explicit hypothesis, though the choice of test statistic must in practice embody some implicit notion of what the alternative might be.
Beyond this, I'm not really interested in debating to what extent Yudkowsky did or did not understand all nuances of this problem.
If I do an experiment, you generally don't know the precise alternate hypothesis in advance -- you want to test if the coin is fair, but you don't know precisely what bias it will have if it's unfair.
Yes. But as far as I can see this isn't of any particular importance to this discussion. Why do you think it is?
If we fix the two alternate hypotheses in advance, and if I have to report all data, then I'm reduced to only hacking by choosing the experiment that maximizes the chance of luckily passing your threshold via fluke. This is unlikely, as you say, so it's a weak form of "hacking". But this is also what I'm reduced to in the frequentist world! Bayesianism doesn't actually help. They key was (a) you forced me to disclose all data, and (b) we picked the alternate hypothesis in advance instead of only having a null hypothesis.
Actually, a frequentist can just keep collecting more data until they get p<0.05, then declare the null hypothesis to be rejected. No lying or suppression of data required. They can always do this, even if the null hypothesis is true: After collecting data points, they have a 0.05 chance of seeing p<0.05. If they don't, they then collect more data points, where is big enough that whatever happened with the first data points makes little difference to the p-value, so there's still about a 0.05 chance that p<0.05. If that doesn't produce a rejection, they collect more data points, and so on until they manage to get p<0.05, which is guaranteed to happen eventually with probability 1.
But they aren't guaranteed to eventually get a Bayesian to think the null hypothesis is likely to be false, when it is actually true.
I am saying that Yudkowsky is just plain wrong here, because omitting info is not the same as outright lying.
This is silly. Obviously, Yudkowsky isn't going to go off on a tangent about all the ways people can lie indirectly, and how a Bayesian ought to account for such possibilities - that's not the topic. In a scientific paper, it is implicit that all relevant information must be disclosed - not doing so is lying. Similarly, a scientific journal must ethically publish papers based on quality, not conclusion. They're lying if they don't. As for authors just not submitting papers with undesirable conclusions - well, that's a known phenomenon, that one should account for, along with the possibility that a cosmic ray has flipped a bit in the memory of the computer that you used for data analysis, and the possibility that you misremembered something about one of the studies, and a million other possibilities that one can't possibly discuss in every blog post.
This is never the scenario, though. It is very easy to tell that the coin is not 90% biased no matter what statistics you use.
You misunderstand. H is some hypothesis, not necessarily about coins. Your goal is to convince the Bayesian that H is true with probability greater than 0.9. This has nothing to do with whether some coin lands heads with probability greater than 0.9.
I can get a lot of mileage out of designing my experiment very carefully to target that specific threshold (though of course I can never guarantee success, so I have to try multiple colors of jelly beans until I succeed).
I don't think so, except, as I mentioned, that you obviously will do an experiment that could conceivably give evidence meeting the threshold - I suppose that you can think about exactly which experiment is best very carefully, but that isn't going to lead to anyone making wrong conclusions.
The person evaluating the evidence knows that you're going to try multiple colors. A frequentist would handle this with some sort of p-value correction. A Bayesian handles this by a small prior probability of the drug working, which may partly be based on the knowledge that if drugs of this class (set of colors) had a high probability of working, there would probably already be evidence of this. But this has nothing to do with the point about the stopping rule for coin flips not affecting the likelihood ratio, and hence the Bayesian conclusion, whereas it does affect the p-value.
If you say that you are reporting all your observations, but actually report only a favourable subset of them, and the Bayesian for some reason assigns low probability to you deceiving them in this way, when actually you are deceiving them, then the Bayesian will come to the wrong conclusion. I don't think this is surprising or controversial.
But I don't see how the Bayesian comes to a wrong conclusion if you truthfully report all your observations, even if they are taken according to some scheme that produces a distribution of likelihood ratios that is supposedly favourable to you. The distribution doesn't matter. Only the observed likelihood ratio matters.
For example, suppose you want to convince the Bayesian that H is true with probability greater than 0.9. Some experiments may never produce data giving a likelihood ratio extreme enough to produce such a high probability. So you don't do such an experiment, and instead do one that could conceivably produce an extreme likelihood ratio. But it probably won't, if H is not actually true. If it does produce strong evidence for H, the Bayesian is right to think that H is probably true, regardless of your motivations (as long as you truthfully report all the data).
I think that various "pro-fertility" people have a variety of motivations.
But "more people are better" ought to be a belief of everyone, whether pro-fertility or not. It's an "other things being equal" statement, of course - more people at no cost or other tradeoff is good. One can believe that and still think that less people would be a good idea in the current situation. But if you don't think more people are good when there's no tradeoff, I don't see what moral view you can have other than nihilism or some form of extreme egoism.
BTW: I'm not ruling out an expansive definition of "people" - maybe gorillas are people, maybe some alien species are, maybe some AIs would be - but I think that's outside the scope of the current discussion.
Integrals of the likelihood function aren't really meaningful, even if normalized so the integral is one over the whole range. This is because the result depends on the arbitrary choice of parameterization - eg, whether you parameterize a probability by p in [0,1], or by log(p) in [-oo,0]. In Bayesian inference, one always integrates the likelihood only after multiplying by the prior, which can be seen as a specification of how the integration is to be done.
I think you've got his pretty much figured out. But you may be missing an additional subtlety.
You say "Bayesian likelihood ratios really do only depend on the probability each hypothesis assigned only to the information that you received". Which could be interpreted as saying that the "likelihood function" is the probability assigned to the information received, seen as a function of f. But the likelihood function is actually not a function at all, but rather an equivalence class of functions of f that differ only by an overall positive scale factor.
You can see how this matters when comparing a report of an observation of 6 flips, such as HHHHHT, versus a report that gives only the number of tails, which is 1 in this case. The probability of HHHHHT as a function of f is , but the probability of 1 tail is , which is not the same function, but is in the same equivalence class, since it differs only by an overall factor of 6. Of course, this overall scale factor cancels out when looking at likelihood ratios for different values of f.
Your factual claims here seem at least somewhat reasonable. Naively extrapolating sub-replacement fertility centuries into the future is silly. Our wealthy civilization ought to be capable of finding some way of coping with increased elder care. The current number of humans may perhaps be more than is optimal.
But your moral view is atrocious. Human extinction would be bad - very bad. Because humans are the most interesting species to arise so far. Because human striving is properly focused on the future, not just hedonistic pleasures of the moment. Because there may be a meaning to human existence that we can so far only dimly perceive.
And more humans are better, if they come at no (or small) cost to the quality of life of the existing humans. Human lives have intrinsic value, and every additional life adds value.
I do wonder if your moral views have biased your factual assessments as well.
I tested this with ChatGPT4 just now. It mostly got my specific questions right, but its commentary reveals some substantial misconceptions, such as that a flight from Ecuador to Sri Lanka would first pass over the Atlantic Ocean and then over Brazil.
Here it is:
User
If I flew by the most direct route from Ecuador to Sri Lanka, would I pass over Egypt?
ChatGPT
If you were to fly the most direct route (a great circle route) from Ecuador to Sri Lanka, you would not pass over Egypt.
Here's a general idea of what the path would look like:
- Starting from Ecuador, the route would take you eastward over the Atlantic Ocean.
- You'd then pass over parts of Brazil in South America.
- Continuing over the South Atlantic Ocean, you'd cross over southern Africa, likely missing most of the major countries to the south.
- From there, you'd head out over the Indian Ocean, aiming towards Sri Lanka.
The path would largely pass south of the equator and far to the south of Egypt. Remember, the shortest path between two points on a sphere (like Earth) isn't necessarily intuitive when looking at typical flat maps, but becomes clearer when visualizing on a globe or using a great circle route tool.
User
Would I pass over Somalia?
ChatGPT
If you fly the most direct route (a great circle route) from Ecuador to Sri Lanka, you would indeed come closer to the eastern coast of Africa, which includes Somalia. After crossing the Atlantic Ocean and part of southern Africa, the route could take you over or near the northern part of Madagascar and then across the Indian Ocean towards Sri Lanka.
Given this trajectory, it's possible that you might pass over or near the southern part of Somalia or its adjacent waters. However, the exact trajectory can vary based on specific starting and ending points, as well as any airspace restrictions or other navigational considerations.
User
Would I pass over Columbia?
ChatGPT
Let's clarify the countries in question:
- Ecuador is in the northwestern part of South America.
- Colombia (with an "o", not "u") shares a border with Ecuador to its north.
If you're flying a direct route (great circle route) from Ecuador to Sri Lanka, the trajectory would first take you eastward, likely across the northern part of Brazil. Given the proximity of Colombia to Ecuador, especially if your departure point in Ecuador is in the northern part of the country or near the border, it's very plausible that your flight could pass over southern Colombia or skirt its southern border.
So, to answer your question: Yes, it's possible that you would pass over parts of Colombia, depending on your exact departure point in Ecuador.
User
Would I pass over South Africa?
ChatGPT
If you were flying the most direct route (a great circle route) from Ecuador to Sri Lanka, the trajectory would most likely not pass directly over South Africa.
After departing from Ecuador, the route would head eastward over the Atlantic Ocean, then cross over parts of South America (likely Brazil), continue across the South Atlantic, and then head towards the southern regions of the African continent. However, given the positions of Ecuador and Sri Lanka on the globe, the route would likely take you over or near the southeastern parts of Africa, such as Mozambique or the adjacent oceanic regions, rather than South Africa itself.
To summarize, while the flight might come close to South Africa or its adjacent waters, it would not necessarily pass directly over South Africa.
Well, given that the text of the US constitution seems to clearly state that all powers not explicitly granted to the federal government belong to the states (or the people), I don't see how "power is devolved to the states from the federal government". It seems that the states don't need to wait for the federal government to "devolve" power to them in order to do something. As indeed we saw recently with respect to covid policy.
You could argue that the federal government "lets" the states do this, in the sense that the federal government has more guns than the states, and hence could stop them it it wanted to. But this would be naive. These guns are operated by people, whose loyalty to the federal government if there were a conflict would not be automatic.
the constitution is quite clear that power is devolved to the states from the federal government, and not that the federal government is granted power at the behest of the states
The 10th amendment to the US constitution says:
The powers not delegated to the United States by the Constitution, nor prohibited by it to the States, are reserved to the States respectively, or to the people.
which sounds like the opposite of what you say. Of course, practice may be different.
Can you tell me why?
It think if we encountered aliens who were apparently not hostile, but presumably strange, and likely disgusting or disturbing in some ways, there would be three groups (likely overlapping) of people opposed to wiping them out:
- Those who see wiping them out as morally wrong.
- Those who see wiping them out as imprudent - we might fail, and then they wipe us out, or other aliens now see us as dangerous, and wipe us out.
- Those who see wiping them out as not profitable - better to trade with them.
There would also be three groups in favour of wiping them out:
- Those who see wiping them out as morally good - better if the universe doesn't have such disgusting beings.
- Those who see wiping them out as the prudent thing to do - wipe them out before they change their mind and do that to us.
- Those who see wiping them out as profitable - then we can grab their resources.
I think it's clear that people with all these view will exist, in non-negligible numbers. I think there's at least a 5% chance that the "don't wipe them out" people prevail.
Subgroups of our species are also actively wiping out other subgroups of our species they don't like.
Yes, but that's not how interactions between groups of humans always turn out.
We didn't really wipe out the Neanderthals (assuming we even were a factor, rather than climate, disease, etc.), seeing as they are among our ancestors.
We are a species that has evolved in competition with other species. Yet, I think there is at least a 5% chance that if we encountered an intelligent alien species that we wouldn't try to wipe them out (unless they were trying to wipe us out).
Biological evolution of us and aliens would in itself be a commonality, that might produce some common values, whereas there need be no common values with an AI created by a much different process and not successfully aligned.
Perhaps of relevance:
One problem I have with Diamond's theory is that I doubt that there is anything for it to explain. The Americas and Eurasia/Africa were essentially isolated from each other for about 15,000 years. In 1500 AD, the Americas were roughly 3500 years less advanced than Eurasia/Africa. That seems well within the random variation one would expect between two isolated instances of human cultural development over a 15,000 year time span. If you think there is still some remaining indication that the Americas were disadvantaged, the fact that the Americas are about half the size of Eurasia/Africa seems like a sufficient explanation.
Perhaps you could give the definition you would use for the word "probability".
I define it as one's personal degree of belief in a proposition, at the time the judgement of probability is being made. It has meaning only in so far it is (or may be) used to make a decision, or is part of a general world model that is itself meaningful. (For example, we might assign a probability to Jupiter having a solid core, even though that makes no difference to anything we plan to do, because that proposition is part of an overall theory of physics that is meaningful.)
Frequentist ideas about probability being related to the proportion of times that an event occurs in repetitions of a scenario are not part of this definition, so the question of what denominator to use does not arise. (Looking at frequentist concepts can sometimes be a useful sanity check on whether probability judgements make sense, but if there's some conflict between frequentist and Bayesian results, the solution is to re-examine the Bayesian results, to see if you made a mistake, or to understand why the frequentist results don't actually contradict the Bayesian result.)
If you make the right probability judgements, you are supposed to make the right decision, if you correctly apply decision theory. And Beauty does make the right decision in all the Sleeping Beauty scenarios if she judges that P(Heads)=1/3 when woken before Wednesday. She doesn't make the right decision if she judges that P(Heads)=1/2. I emphasize that this is so for all the scenarios. Beauty doesn't have to ask herself, "what denominator should I be using?". P(Heads)=1/3 gives the right answer every time.
Another very useful property of probability judgements is that they can be used for multiple decisions, without change. Suppose, for example, that in the GWYD or GRYL scenarios, in addition to trying not to die, Beauty is also interested in muffins.
Specifically, she knows from the start that whenever she wakes up there will be a plate of freshly-baked muffins on her side table, purchased from the cafe down the road. She knows this cafe well, and in particular knows that (a) their muffins are always very delicious, and (b) on Tuesdays, but not Mondays, the person who bakes the muffins adds an ingredient that gives her a stomach ache 10 minutes after eating a muffin. Balancing these utilities, she decides to eat the muffins if the probability of it being Tuesday is less than 30%. If Beauty is a Thirder, she will judge the probability of Tuesday to be 1/3, and refrain from eating the muffins, but if Beauty is a Halfer, she will (I think, trying to pretend I'm a halfer) think the probability of Tuesday is 1/4, and eat the muffins.
The point here is not so much which decision is correct (though of course I think the Thirder decision is right), but that whatever the right decision is, it shouldn't depend on whether Beauty is in the GWYD or GRYL scenario. She shouldn't be considering "denominators".
I think we actually have two quantities:
"Quobability" - The frequency of correct guesses made divided by the total number of guesses made.
"Srobability" - The frequency of trials in which the correct guess was made, divided by the number of trials.
Quabability is 1/3, Scrobability is 1/2. "Probability" is (I think) an under-precise term that could mean either of the two.
I suspect that the real problem isn't with the word "probability", but rather the word "guess". In everyday usage, we use "guess" when the aim is to guess correctly. But the aim here is to not die.
Suppose we rephrase the GRYL scenario to say that Beauty at each awakening takes one of two actions - "action H" or "action T". If the coin lands Heads, and Beauty takes action H the one time she is woken, then she lives (if she instead takes action T, she dies). If the coin lands Tails, and Beauty takes action T at least one of the two times she is woken, then she lives (if she takes action H both times, she dies).
Having eliminated the word "guess", why would one think that Beauty's use of the strategy of randomly taking action H or action T with equal probabilities implies that she must have P(Heads)=1/2? As I've shown above, that strategy is actually only compatible with her belief being that P(Heads)=1/3.
Note that in general, the "action space" for a decision theory problem need not be the same as the "state space". One might, for example, have some uncertain information about what day of the week it is (7 possibilities) and on that basis decide whether to order pepperoni, anchovy, or ham pizza (3 possibilities). (You know that different people, with different skills, usually make the pizza on different days.) So if for some reason you randomized your choice of action, it would certainly not say anything directly about your probabilities for the different days of the week.
By "GWYL" do you actually mean "GRYL" (ie, Guess Right You Live)?
One could argue that if the coin is flicked and comes up tails then we have both "Tails&Monday" and "Tails&Tuesday" as both being correct, sequentially.
Yes, it is a commonplace occurrence that "Today is Monday" and "Today is Tuesday" can both be true, on different days. This doesn't ordinarily prevent people from assigning probabilities to statements like "Today is Monday", when they happen to not remember for sure whether it is Monday or not now. And the situation is the same for Beauty - it is either Monday or Tuesday, she doesn't know which, but she could find out if she just left the room and asked some passerby. Whether it is Monday or Tuesday is an aspect of the external world, which one normally regards as objectively existing regardless of one's knowledge of it.
All this is painfully obvious. I think it's not obvious to you because you don't accept that Beauty is a human being, not a python program. Note also that Beauty's experience on Monday is not the same as on Tuesday (if she is woken). Actual human beings don't have exactly the same experiences on two different days, even if the have memory issues. The problem setup specifies only that these differences aren't informative about whether it's Monday or Tuesday.
What do you mean by "probability".
I'm using "probability" in the subjective Bayesian sense of "degree of belief". Since the question in the Sleeping Beauty problem is what probability of Heads should Beauty have when awoken, I can't see how any other interpretation would address the question asked. Note that these subjective degree-of-belief probabilities are intended to be a useful guide to decision-making. If they lead one to make clearly bad decisions, they must be wrong.
Consider two very extreme cases of the sleeping beauty game: - Guess wrong and you die! (GWYD) - Guess right and you live! (GRYL)
If we look at the GWYD and GRYL scenarios you describe, we can, using an "outside" view, see what the optimal strategies are, based on how frequently Beauty survives in repeated instances of the problem. To see whether Beauty's subjective probability of Heads should be 1/2 or 1/3, we can ask whether after working out an optimal strategy beforehand, Beauty will change her mind after waking up and judging that P(Heads) is either 1/2 or 1/3, and then using that to decide whether to follow the original strategy or not. If Beauty's judgement of P(Heads) leads to her abandoning the optimal strategy, there must be something wrong with that P(Heads).
For GWYD, both the strategy of deterministically guessing Heads on all awakenings and the strategy of deterministically guessing Tails on all awakenings will give a survival probability of 1/2, which is optimal (I'll omit the proof of this).
Suppose Beauty decides ahead of time to deterministically guess Tails. Will she change her mind and guess Heads instead when she wakes up?
Suppose that Beauty thinks that P(Heads)=1/3 upon wakening. She will then think that if she guesses Heads, her probability of surviving is P(Heads)=1/3. If instead, she guesses Tails, she thinks her probability of surviving is P(Tails & on other wakening she also guesses Tails), which is 2/3 if she is sure to follow the original plan in her other wakening, and is greater than 1/3 as long as she's more likely than not to follow the original plan on her other wakening. So unless Beauty thinks she will do something perverse on her other wakening, she should think that following the original plan and guessing Tails is her best action.
Now suppose that Beauty thinks that P(Heads)=1/2 upon wakening. She will then think that if she guesses Heads, her probability of surviving is P(Heads)=1/2. If instead, she guesses Tails, she thinks her probability of surviving is P(Tails & on other wakening she also guesses Tails), which is 1/2 if she is sure to follow the original plan in her other wakening, and less than 1/2 if there is any chance that on her other wakening she doesn't follow the original plan. Since Beauty is a human being, who at least once in a while does something strange or mistaken, the probability that she won't follow the plan on her other wakening is surely not zero, so Beauty will judge her survival probability to be greater if she guesses Heads than if she guesses Tails, and abandon the original plan of guessing Tails.
Now, if Beauty thinks P(Heads)=1/2 and then always reasons in this way on wakening, then things turn out OK - despite having originally planned to always guess Tails, she actually always guesses Heads. But if there is a non-negligible chance that she follows the original plan without thinking much, she will end up dead more than half the time.
So if Beauty thinks P(Heads)=1/3, she does the right thing, but if Beauty thinks P(Heads)=1/2, she maybe does the right thing, but not really reliably.
Turning now to the GRYL scenario, we need to consider randomized strategies. Taking an outside view, suppose that Beauty follows the strategy of guessing heads with probability h, independently each time she wakes. Then the probability that she survives is S=(1/2)h+(1/2)(1-h*h) - that is, the probability the coin lands Heads (1/2) times the probability she guesses Heads, plus the probability that the coin lands Tails time the probability that she doesn't guess Heads on both awakenings. The derivative of S with respect to h is (1/2)-h, which is zero when h=1/2, and one can verify this gives a maximum for S of 5/8 (better than the survival probability when deterministically guessing either Heads of Tails).
This matches what you concluded. However, Beauty guessing Heads with probability 1/2 does not imply that Beauty thinks P(Heads)=1/2. The "guess" here is not made in an attempt to guess the coin flip correctly, but rather in an attempt to not die. The mechanism for how the guess influences whether Beauty dies is crucial.
We can see this by seeing what Beauty will do after waking if she thinks that P(Heads)=1/2. She knows that her original strategy is to randomly guess, with Heads and Tails both having probability 1/2. But she can change her mind if this seems advisable. She will think that if she guesses Tails, her probability of survival will be P(Tails)=1/2 (she won't survive if the coin landed Heads, because this will be her only (wrong) guess, and she will definitely survive if the coin landed Tails, regardless what she does on her other awakening). She will also compute that if she guesses Heads, she will survive with probability P(Heads)+P(Tails & she guesses Tails on her other wakening). Since P(Heads)=1/2, this will be greater than 1/2, since the second term surely is not exactly zero (it will be 1/4 if she follows the original strategy on her other awakening). So she will think that guessing Heads gives her a strictly greater chance of surviving than guessing Tails, and so will just guess Heads rather than following the original plan of guessing randomly.
The end result, if she reasons this way each awakening, is that she always guesses Heads, and hence survives with probability 1/2 rather than 5/8.
Now lets see what Beauty does if after wakening she thinks that P(Heads)=1/3. She will think that if she guesses Tails, her probability of survival will be P(Tails)=2/3. She will think that if she guesses Heads, her probability of survival will be P(Heads)+P(Tails & she guesses Tails on her other wakening). If she thinks she will follow the original strategy on her other wakening, the this is (1/3)+(2/3)*(1/2)=2/3. Since she computes her probability of survival to be the same whether she guesses Heads or Tails, she has no reason to depart from the original strategy of guessing Heads or Tails randomly. (And this reinforces her assumption that on another wakening she would follow the original strategy.)
So if Beauty is a Thirder, she lives with probability 5/8, but if she is a Halfer, she lives with lower probability of 1/2.
If the Tuesday bet is considered to be "you can take the bet, but it will replace the one you may or may not have given on a previous day if their was one", then things line up to half again.
I can think of two interpretations of the setup you're describing here, but for both interpretations, Beauty does the right thing only if she thinks Heads has probability 1/3, not 1/2.
Note that depending on the context, a probability of 1/2 for something does not necessarily lead one to bet on it at 1:1 odds. For instance, if based on almost no knowledge of baseball, you were to assign probability 1/2 to the Yankees winning their next game, it would be imprudent to offer to bet anyone that they will win, at 1:1 odds. It's likely that the only people to take you up on this bet are the ones who have better knowledge, that leads them to think that the Yankees will not win. But even after having decided not to offer this bet, you should still think the probability of the Yankees winning is 1/2, assuming you have not obtained any new information.
I'll assume that Beauty is always given the option of betting on Heads at 1:1 odds. From an outside view, it is clear that whatever she does, her expected gain is zero, so we hope that that is also her conclusion when analyzing the situation after each awakening. Of course, by shifting the payoffs when betting on Heads a bit (eg, win $0.90, lose $1) we can make betting on Heads either clearly desirable or clearly undesirable, and we'd hope that Beauty correctly decides in such cases.
So... first interpretation: At each awakening, Beauty can bet on Heads at 1:1 odds (winning $1 or losing $1), or decide not to bet. If she decides to bet on Tuesday, any bet she may have made on Monday is cancelled, and if she decides not to bet on Tuesday, any bet she made on Monday is also cancelled. In other words, what she does on Monday is irrelevant if the coin landed Tails, since she will in that case be woken on Tuesday and whatever she decides then will replace whatever decision she made on Monday. (Of course, if the coin landed Heads, her decision on Monday does take effect, since she isn't woken on Tuesday.)
Beauty's computation on awakening of the expected gain from betting on Heads (winning $1 or losing $1) can be written as follows:
P(Heads&Monday)*1 + P(Tails&Monday)*0 + P(Tails&Tuesday)*(-1)
The 0 gain from betting on Monday when the coin landed Tails is because in that situation her decision is irrelevant. The expected gain from not betting is of course zero.
If Beauty thinks that P(Heads)=1/3, and hence P(Heads&Monday)=1/3, P(Tails&Monday)=1/3, and P(Tails&Tuesday)=1/3, and the formula above evaluates to zero, as we would hope. Furthermore, if the payoffs deviate from 1:1, Beauty will decide to bet or not in the correct way.
If Beauty instead thinks that P(Heads)=1/2, then P(Heads&Monday)=1/2. Trying to guess what a Halfer would think, I'll also assume that she thinks that P(Tails&Monday)=1/4 and P(Tails&Tuesday)=1/4. (Those who deny that these probabilities should sum to one, or maintain that the concept of probability is inapplicable to Beauty's situation, are beyond the bounds of rational discussion.) If we plug these into the formula above, we get that Beauty thinks her expected gain from betting on Heads is $0.25, which is contrary to the outside view that it is zero. Furthermore, if we shift the payoffs a little bit, so a win betting on Heads gives $0.90 and a loss betting on Heads still costs $1, then Beauty will compute the expected gain from betting on Heads to be $0.45 minus $0.25, which is $0.20$, so she's still enthusiastic about betting on Heads. But the outside view is that with these payoffs the expected gain is minus $0.05, so betting on Heads is a losing proposition.
Now... second interpretation: At each awakening, Beauty can bet or decide not to bet. If she decides to bet on Tuesday, any bet she may have made on Monday is cancelled, but if she decides not to bet on Tuesday, a bet she made on Monday is still in effect. In other words, she makes a bet if she decides to bet on either Monday or Tuesday, but if she decides to bet on both Monday and Tuesday, it only counts as one bet.
This one is more subtle. I answered essentially the same question in a comment on this blog post, so I'll just copy the relevant portion below, showing how a Thirder will do the right thing (with other than 1:1 payoffs). Note that "invest" means essentially the same as "bet":
- - -
First, to find the optimal strategy in this situation where investing when Heads yields -60 for Beauty, and investing when Tails yields +40 for Beauty, but only once if Beauty decides to invest on both wakenings, we need to consider random strategies in which with probability q, Beauty does not invest, and with probability (1-q) she does invest.
Beauty’s expected gain if she follows such a strategy is (1/2)40(1-q^2)-(1/2)60(1-q). Taking the derivative, we get -40q+30, and solving for this being zero gives q=3/4 as the optimal value. So Beauty should invest with probability 1/4, which works out to giving her an average return of (1/2)40(7/16)-(1/2)60(1/4)=5/4.
That’s all when working out the strategy beforehand. But what if Beauty re-thinks her strategy when she is woken? Will she change her mind, making this strategy inconsistent?
Well, she will think that with probability 1/3, the coin landed Heads, and the investment will lose 60, and with probability 2/3, the coin landed Tails, in which case the investment will gain 40, but only if she didn’t/doesn’t invest in her other wakening. If she follows the strategy worked out above, in her other wakening, she doesn’t invest with probability 3/4, and if Beauty thinks things over with that assumption, she will see her expected gain as (2/3)(3/4)40-(1/3)60=0. With a zero expected gain, Beauty would seem indifferent to investing or not investing, and so has no reason to depart from the randomized strategy worked out beforehand. We might hope that this reasoning would lead to a positive recommendation to randomize, not just acquiescence to doing that, but in Bayesian decision theory randomization is never required, so if there’s a problem here it’s with Bayesian decision theory, not with the Sleeping Beauty problem in particular.
- - -
On the other hand, if Beauty thinks the probability of Heads is 1/2, then she will think that investing has expected return of (1/2)(-60) + (1/2)40 or less (the 40 assumes she didn't/doesn't invest on the other awakening, if she did/does it would be 0). Since this is negative, she will change her mind and not invest, which is a mistake.