Naming the Highest Virtue of Epistemic Rationality

The general reason Eliezer advocates not naming the highest virtue (as I understand it) is that there may be some type of problem for which bayesian updating (and the scoring rule referred to) yields the wrong answer. This idea sounds rather improbable to me, but there is a non-negligible probability that bayes will yield a wrong answer on some question. Not naming the virtue is supposed to be a reminder that if bayes ever gives the wrong answer, we go with the right answer, not bayes.

There is a major issue with the proposed scoring - it is underspecified. In particular, in the definition:

for B in your beliefs that's true

How do we determine if B is in an agent's set of beliefs? We cannot only consider the beliefs that are currently running through the agent's mind, because we'd end up with at most a few. We need a definition of what "B is in your beliefs" means. However, it is very difficult to specify all of an agent's beliefs - humans don't walk around carrying a well-defined sack of beliefs with probabilities attached.

Less importantly, the linearity in the sum can be exploited. For example, I can easily get myself to believe the following sequence of statements in Peano arithmetic:
1=1
2=2
3=3
...

This will give me a favorable score with minimal effort. At least in this case, the proposed scoring is orthogonal to measuring epistemic rationality.

It is a deductive fact that no other scoring rule could possibly give: Score(P(A|B)) + Score(P(B)) = Score(P(A&B))

What assumptions are you using here? In the simplest form this is obviously false. Simply let the Score function always be zero. Moreover, any Score function that satisfies this identity can be scaled by any number and still satisfy it. So not only does your log work but a log to any other base will work. Also if I believe in the axiom of choice (I think I need choice here to get a transcendence basis. Can someone more foundationally oriented comment if this is correct?), then there is a function f on the positive reals such that f(ab)=f(a)+f(b) and f(x) is not equal to a constant times log x. So one could just as well use that f as your score function.

I think your statement is true if you want your score function to be continuous and normalized so that one has Score(1/2)= -1.

On a completely different note, the repeated references to the Sequences come off as a bit off. You assume a high degree of detailed familiarity with the various essays that is unjustified. For example, the quote from the Twelve Virtues sounded familiar, but I almost certainly would not have been able to place it. Moreover, the way these quotes are used almost feels like you are quoting religious proof texts or writing a highschool English essay rather than actually using them in a useful way.

Why don't we start treating the sum of log_2 of the probability — conditional on every available piece of information — you assign to every true sentence, as the best measure of your epistemic success?

Wait. Perhaps I'm misunderstanding something here, but how are we going to decide what a true sentence is independently of all of our available information?

You misspelled Rationality in the subject.

Purely mathematical statements are great, if you're an ALU. If you're running on repurposed, buggy hardware, you might need something more.

Can't you get the maximum score of 0 by simply setting P(b) = 1 for all beliefs b, regardless of the truth value of b? False beliefs have to be penalized if you want

a score of 0 [to] represent each of your beliefs being completely and perfectly accurate.

A point of math: log(a) < 0 when 0<a<1, so your proposed measure is negative and gets smaller and smaller the more "epistemically successful" you are. Is this really your intention? To clarify a bit, sum of logs = log of product, and product of probabilities tends to zero as you pile on more of them (cf. the Conjunction fallacy)

Something about this seems wrong because it doesn't take into account the significance of propositions the agent has as beliefs. What use is having a high proportion of true to false beliefs if they are about things that don't matter?