Should you go with your best guess?: Against precise Bayesianism and related views

(I'll reply to the point about arbitrariness in another comment.)

I think it's generally helpful for conceptual clarity to analyze epistemics separately from ethics and decision theory. E.g., it's not just EV maximization w.r.t. non-robust credences that I take issue with, it's any decision rule built on top of non-robust credences. And I worry that without more careful justification, "[consequentialist] EV-maximizing within a more narrow "domain", ignoring the effects outside of that "domain"" is pretty unmotivated / just kinda looking under the streetlight. And how do you pick the domain?

(Depends on the details, though. If it turns out that EV-maximizing w.r.t. impartial consequentialism is always sensitive to non-robust credences (in your framing), I'm sympathetic to "EV-maximizing w.r.t. those you personally care about, subject to various deontological side constraints etc." as a response. Because “those you personally care about” isn’t an arbitrary domain, it’s, well, those you personally care about. The moral motivation for focusing on that domain is qualitatively different from the motivation for impartial consequentialism.)

So I'm hesitant to endorse your formulation. But maybe for most practical purposes this isn't a big deal, I'm not sure yet.

it seems pretty arbitrary to me where you draw the boundary between a credence that you include in your representor vs. not. (Like: What degree of justification is enough? We'll always have the problem of induction to provide some degree of arbitrariness.)

To spell out how I’m thinking of credence-setting: Given some information, we apply different (vague) non-pragmatic principles [LW · GW] we endorse — fit with evidence, Occam’s razor, deference, etc.

Epistemic arbitrariness means making choices in your credence-setting that add something beyond these principles. (Contrast this with mere “formalization arbitrariness”, the sort discussed in the part of the post [LW · GW] about vagueness.)

I don’t think the problem of induction forces us to be epistemically arbitrary. Occam’s razor (perhaps an imprecise version!) favors priors that penalize a hypothesis like “the mechanisms that made the sun rise every day in the past suddenly change tomorrow”. This seems to give us grounds for having prior credences narrower than (0, 1), even if there’s some unavoidable formalization arbitrariness. (We can endorse the principle underlying Occam’s razor, “give more weight to hypotheses that posit fewer entities”, without a circular justification like “Occam’s razor worked well in the past”. Admittedly, I don’t feel super satisfied with / unconfused about Occam’s razor, but it’s not just an ad hoc thing.)

By contrast, pinning down a single determinate credence (in the cases discussed in this post) seems to require favoring epistemic weights for no reason. Or at best, a very weak reason that IMO is clearly outweighed by a principle of suspending judgment. So this seems more arbitrary to me than indeterminate credences, since it injects epistemic arbitrariness on top of formalization arbitrariness.

Also, my sense is that many people are making decisions based on similar intuitions as the ones you have (albeit with much less of a formal argument for how this can be represented or why it's reasonable). In particular, my impression is that people who are are uncompelled by longtermism (despite being compelled by some type of scope-sensitive consequentialism) are often driven by an aversion to very non-robust EV-estimates.

You seem to be underestimating how pervasive / universal this critique is - essentially every environment is more complex than we are

I agree it's pretty pervasive, but the impression I've gotten from my (admittedly limited) sense of how infra-Bayesianism works is:

The "more complex than we are" condition for indeterminacy doesn't tell us much about when, if ever, our credences ought to capture indeterminacy in how we weigh up considerations/evidence — which is a problem for us independent of non-realizability. For example, I'd be surprised if many/most infra-Bayesians would endorse suspending judgment in the motivating example in this post, if they haven't yet considered the kinds of arguments I survey. This matters for how decision-relevant indeterminacy is for altruistic prioritization.

I'm also not aware of the infra-Bayesian literature addressing the "practical hallmarks" I discuss, though I might have missed something.

(The Solomonoff induction part is a bit above my pay grade, will think more about it.)

mostly problems with logical omnisicence not being satisfied

I'm not sure, given the "Indeterminate priors" section. But assuming that's true, what implication are you drawing from that? (The indeterminacy for us doesn't go away just because we think logically omniscient agents wouldn't have this indeterminacy.)

the arbitrariness of the prior is just a fact of life

The arbitrariness of a precise prior is a fact of life. This doesn't imply we shouldn't reduce this arbitrariness [LW · GW] by having indeterminate priors.