Help with Bayesian priors

Let:
E = Alexander the Great actually existed
C = A coin is minted with the face of Alexander the Great
~A = not A

You want to know P(E|C), which by Bayes is

P(C|E)P(E)/P(C)

which by partition of unity is

P(C|E)P(E) / (P(C|E)P(E) + P(C|~E)P(~E))

So you need only to assess 1 (existed, had a coin) and 3 (not existed, had a coin), where P(E) is your prior probability that Alexander existed. There's no need to do complicated categorical reasoning.

I think that P(C|E) is pretty close to 1, I would suspect that any great emperor worth his salt would have minted a coin with his head on it.
So the crucial point becomes P(C|~E), as it should be: what is the probability that a coin would be minted with a mythological figurehead? That is what is to be analyzed and that is what will drive the probability of his existence.
Let's say that we know that it's definitely possible that Alexander the Great existed, and it's also definitely possible that he did not exist, but we know nothing else (that is, coins are the first evidence we want to introduce). This would put P(E) = P(~E) = 1/2. Leaving P(C|~E) unknown, we get:

P(E|C) = P(C|E)P(E) / (P(C|E)P(E) + P(C|~E)P(~E)) =>
P(E|C) = .5 / .5 + x .5 =>
P(E|C) = 1 / (1+ x)

All this is coherent with common sense: if the ancients were happily minting mythic figures all the times (x close to 1), this means that P(E|C) = .5 = P(E), and coins cannot be used as an argument for Alexander's existence, while on the other hand if almost never happened (x close to 0), then a coin becomes a sure sign of his historicity, with intermediate values (say x = .3) increasing the odds by a small or a great percentage.
Notice one thing of my model: I've assumed that P(C|E) is close to 1, and this means that a coin can only increase the probability of Alexander existing, however slightly, but never decrease it. This wouldn't have been true if there was a possibility that a great emperor did not had a coin minted with his face, which would be your case n° 2.

I'm pretty sure that you're actually asking some deep questions right now.

I'm not all that well-versed in epistemology or probability theory, but when you write:

A friend in academia suggested that this touches on a problem with Bayes priors that has not been settled.

I think this is a reference to the problem of priors.

I think 'a problem with Bayes priors that has not been settled' kind of understates the significance.

And:

The first issue is that there are infinite people who never existed and did not have a coin made. If I narrow it to historic figures who turned out not to exist and did not have a coin made it becomes possible but also becomes subjective as to whether someone actually thought they existed. For example, did people believe the Minotaur existed? Perhaps I should choose another filter instead of historic figure, like humans that existed. But picking and choosing the category is again so subjective. Someone may also argue that woman inequality back then was so great that the data should only look at men, as a woman’s chance of being portrayed on a coin was skewed in a way that isn’t applicable to men.

I believe this is referred to as the reference class problem. It seems that in a Bayesian framework, the reference class problem is something of a subproblem of the problem of priors. It seems that you're only trying to define a suitable reference class in the first place because you're trying to produce a reasonable prior.

It's my understanding that one approach to the problem of priors has been to come up with a 'universal' prior, a prior which is reasonable to adopt before making any observations of the world. One example is Solomonoff's algorithmic probability. It seems however than even this may not be a satisfactory solution, because this prior defies our intuition in some ways. For example, humans might find it intuitive to assign nonzero prior probability to uncomputable hypotheses (e.g. our physics involves hypercomputation), but algorithmic probability only assigns nonzero probability to computable hypotheses, an agent with this prior will never be able to have credence in uncomputable hypotheses. Another problem is that, with this prior, hypotheses are penalized for their complexity, but utility can grow far more quickly than complexity. Increasing the number of happy people in a program from 1000 people to 1,000,000 people seems to increase its utility a lot without increasing its complexity by much. Taking this up to larger and larger numbers that become difficult to intuitively comprehend, it may be that such a prior would result in agents whose decision-making is dominated by very improbable outcomes with very high utilities. We can also ask if this apparently absurd result is a point against Solomonoff induction or if it's a point against how humans think, but if we humans are thinking the right way, we still don't know what it is that's going right inside of our heads and how it compares to Solomonoff induction.

For any other readers, sorry if I'm mistaken on any of this, it is quite technical and I haven't really studied it. Do correct me if I've made a mistake.

Back to my point, I think that you accidentally hit upon a problem that doesn't seem to take too many prerequisites to initially run into, and that, after a bit of squinting, turns out to be way harder than it seems it should be at first glance, given the small number of prerequisites necessary to realize that the problem exists. Personally, I would consider the task of providing a complete answer to your questions an open research problem.

Stanford Encyclopedia of Philosophy's entry on Bayesian epistemology might be some good reading for this.

I think reading Tedlocks book about Superforcasting could help you because it talks about how his Superforcasters made their decisions and outperformed the CIA analysts.

References to books or other online resources or even somewhere else I should be posting this kind of question would all be gratefully received.

How about http://stats.stackexchange.com/?

With the Alexander the Great issue, you not only have the question of the probability that someone would have a coin minted and so on, but also the probability that he actually did have a coin minted with his face on it, given that someone says he did. And you cannot assign that to 100% even if you have seen the coin yourself, because it may not have been intended to be the face of Alexander the Great, but someone else.

And basically all of those probabilities are subjective, and cannot be made objective. So your project might work for an individual to map out their own beliefs, allowing them to assign subjective probabilities to things they consider priors. But it cannot work for a community, because people will disagree with the priors you assign, and so there will be no reason for them to agree with the probabilities your program comes up with.

I'm late to this debate; it's been well analyzed on the meta level but I want to add something on the object level. The question of 'were coins of Alexander minted?', which you want as your input data, sounds like a very simple one, but it may not be. The analysis of ancient coins in large part assumes a known history and dating.

1. The coins called 'Alexander coins' today didn't carry the image of Alexander, but of Herakles. They had Alexander's name on them, and sometimes the title 'King', but that's just two words; not strong evidence of who exactly the coin-minters thought Alexander was and what he was supposed to have accomplished. So you have to separate 'some Greek King named Alexander existed' at one extreme, from 'he did what's commonly ascribed to him and is the person we think of as Alexander the Great' at the other.
2. These coins were minted in many countries for hundreds of years, becoming something of a common coinage. People probably kept minting them because they were a standard coin, not because they really wanted to honor Alexander 300 years after his death. So the existence of most coins isn't independent evidence.
3. When comparing coins minted of other people, you need to factor in the the different amounts of coins minted due to different technology and economy, the differential survival of the coins over time, and the probability of us finding them.
4. How do we identify a coin as being Alexandrian? Usually by matching it to other coins we've already identified. How do we identify the first Alexandrian coin we've ever seen? By noticing the name Alexander on it. What's the probability coins were minted of Alexander, perhaps bearing his likeness (and not that of Herakles), but without his name? Putting the name of a king on a coin (whether or not he was pictured) became common (but not universal) somewhat after Alexander.
5. There were five kings of ancient Macedonia named Alexander, the third of them being the Great. Not to mention some non-Macedonian Alexanders. Coins with the name Alexander don't specify which one is meant, or even 'of Macedonia'. (A minority was actually minted in Macedonia, but if lots of other countries were minting coins of a foreign monarch, who's to say Macedonia didn't as well?) So the dating of the coins has to be precise enough (to within 50 years or so), which can be really hard (by the time you've found a coin who knows where it's been?)

Clarification: I don't seriously doubt the historicity of Alexander, but I'm also not versed in the subject, so I'm just going by expert opinion. My point is more that coins are really complex and/or weak as a source of evidence, and handling P(Alexander existed|Coins of Alexander) is really hard.

But picking and choosing the category is again so subjective.

No. Use all information available. What problem are you actually looking to analyze? What information do you have?

Someone may also argue that woman inequality back then was so great that the data should only look at men, as a woman’s chance of being portrayed on a coin was skewed in a way that isn’t applicable to men.

That may be some useful information to include. Willfully ignoring relevant information, or not seeing how to use some information that seems like it may be relevant does not mean that the problem is "subjective", it means that we are often lazy and confused. And that's fine.

Include what you can transform into meaningful probabilities.

That thinking is hard is not a problem unique to bayesian methods.

I don't recall Jaynes discussing it much. Anyone?

For him, I think the reference class is always the context of your problem. Use all information available.

A brief google for "jaynes reference class" turned up his paper on The Well Posed Problem.

http://bayes.wustl.edu/etj/articles/well.pdf

He analyzes the Bertrand Paradox, and finds that in the real world, the mathematical "paradox" is resolved by identifying the transformation group (and thereby prior) that in reality is applicable.

My take on this is that "non-informative priors" and "principle of indifference" are huge misnomers. Priors are assertions of information, of transformation groups or equivalence classes believed appropriate to the problem. If your prior is "gee I don't know and don't care", then you're just making shit up.

Yeah, the estimates will always be subjective to an extent, but whether you choose historic figure, or all humans and fictional characters that ever existed or whatever, shouldn't make huge differences to your results, because, in Bayes' formula, the ratio P(C|E)/P(C) ¹ should always be roughly the same, regardless of filter.

¹ C: coin exists
E: person existed