Should you go with your best guess?: Against precise Bayesianism and related views
post by Anthony DiGiovanni (antimonyanthony) · 2025-01-27T20:25:26.809Z · LW · GW · 8 commentsContents
8 comments
8 comments
Comments sorted by top scores.
comment by Davidmanheim · 2025-01-28T11:48:53.277Z · LW(p) · GW(p)
I’m not merely saying that agents shouldn’t have precise credences when modeling environments more complex than themselves
You seem to be underestimating how pervasive / universal this critique is - essentially every environment is more complex than we are, at the very least when we're embedded agents, or other humans are involved. So I'm not sure where your criticism (which I agree with) is doing more than the basic argument is in a very strong way - it just seems to be stating it more clearly.
The problem is that Kolmogorov complexity depends on the language in which algorithms are described. Whatever you want to say about invariances with respect to the description language, this has the following unfortunate consequence for agents making decisions on the basis of finite amounts of data: For any finite sequence of observations, we can always find a silly-looking language in which the length of the shortest program outputting those observations is much lower than that in a natural-looking language (but which makes wildly different predictions of future data).
Far less confident here, but I think this isn't correct as a mater of practice. Conceptually, Solomonoff doesn't say "pick an arbitrary language once you've seen the data and then do the math" it says "pick an arbitrary language before you've seen any data and then do the math." And if we need to implement the silly looking language, there is a complexity penalty to doing that, one that's going to be similarly large regardless of what baseline we choose, and we can determine how large it is in reducing the language to some other language. (And I may be wrong, but picking a language cleverly should not means that Kolmogorov complexity will change something requiring NP programs to encode into something that P programs can encode, so this criticism seems weak anyways outside of toy examples.)
Replies from: antimonyanthony↑ comment by Anthony DiGiovanni (antimonyanthony) · 2025-01-28T13:23:24.718Z · LW(p) · GW(p)
You seem to be underestimating how pervasive / universal this critique is - essentially every environment is more complex than we are
I agree it's pretty pervasive, but the impression I've gotten from my (admittedly limited) sense of how infra-Bayesianism works is:
The "more complex than we are" condition for indeterminacy doesn't tell us much about when, if ever, our credences ought to capture indeterminacy in how we weigh up considerations/evidence — which is a problem for us independent of non-realizability. For example, I'd be surprised if many/most infra-Bayesians would endorse suspending judgment in the motivating example in this post, if they haven't yet considered the kinds of arguments I survey. This matters for how decision-relevant indeterminacy is for altruistic prioritization.
I'm also not aware of the infra-Bayesian literature addressing the "practical hallmarks" I discuss, though I might have missed something.
(The Solomonoff induction part is a bit above my pay grade, will think more about it.)
Replies from: Davidmanheim↑ comment by Davidmanheim · 2025-01-28T15:53:48.801Z · LW(p) · GW(p)
"when, if ever, our credences ought to capture indeterminacy in how we weigh up considerations/evidence"
The obvious answer is only when there is enough indeterminacy to matter; I'm not sure if anyone would disagree. Because the question isn't whether there is indeterminacy, it's how much, and whether it's worth the costs of using a more complex model instead of doing it the Bayesian way.
I'd be surprised if many/most infra-Bayesians would endorse suspending judgment in the motivating example in this post
You also didn't quite endorse suspending judgement in that case - "If someone forced you to give a best guess one way or the other, you suppose you’d say “decrease”. Yet, this feels so arbitrary that you can’t help but wonder whether you really need to give a best guess at all…" So, yes, if it's not directly decision relevant, sure, don't pick, say you're uncertain. Which is best practice even if you use precise probability - you can have a preference for robust decisions, or a rule for withholding judgement when your confidence is low. But if it is decision relevant, and there is only a binary choice available, your best guess matters. And this is exactly why Eliezer says that when there is a decision, you need to focus your indeterminacy [LW · GW], and why he was dismissive of DS and similar approaches.
Replies from: antimonyanthony↑ comment by Anthony DiGiovanni (antimonyanthony) · 2025-01-28T16:13:10.382Z · LW(p) · GW(p)
The obvious answer is only when there is enough indeterminacy to matter; I'm not sure if anyone would disagree. Because the question isn't whether there is indeterminacy, it's how much, and whether it's worth the costs of using a more complex model instead of doing it the Bayesian way.
Based on this I think you probably mean something different by “indeterminacy” than I do (and I’m not sure what you mean). Many people in this community explicitly disagree with the claim that our beliefs should be indeterminate at all, as exemplified by the objections I respond to in the post.
When you say “whether it’s worth the costs of using a more complex model instead of doing it the Bayesian way”, I don’t know what “costs” you mean, or what non-question-begging standard you’re using to judge whether “doing it the Bayesian way” would be better. As I write in the “Background” section: "And it’s question-begging to claim that certain beliefs “outperform” others, if we define performance as leading to behavior that maximizes expected utility under those beliefs. For example, it’s often claimed that we make “better decisions” with determinate beliefs. But on any way of making this claim precise (in context) that I’m aware of, “better decisions” presupposes determinate beliefs!"
You also didn't quite endorse suspending judgement in that case - "If someone forced you to give a best guess one way or the other, you suppose you’d say “decrease”.
The quoted sentence is consistent with endorsing suspending judgment, epistemically speaking. As the key takeaways list says, “If you’d prefer to go with a given estimate as your “best guess” when forced to give a determinate answer, that doesn’t imply this estimate should be your actual belief.”
But if it is decision relevant, and there is only a binary choice available, your best guess matters
I address this in the “Practical hallmarks” section — what part of my argument there do you disagree with?
comment by Noosphere89 (sharmake-farah) · 2025-01-29T00:15:23.027Z · LW(p) · GW(p)
IMO, most of the problems with Precise Bayesianism for humans are mostly problems with logical omnisicence not being satisfied.
Also, one the arbitrariness of the prior, this is an essential feature for a very general learner, due to the no free lunch theorems.
The no free lunch theorem prohibits 1 prior from always being universally accurate or inaccurate, so the arbitrariness of the prior is just a fact of life.
Replies from: antimonyanthony↑ comment by Anthony DiGiovanni (antimonyanthony) · 2025-01-29T09:21:34.645Z · LW(p) · GW(p)
mostly problems with logical omnisicence not being satisfied
I'm not sure, given the "Indeterminate priors" section. But assuming that's true, what implication are you drawing from that? (The indeterminacy for us doesn't go away just because we think logically omniscient agents wouldn't have this indeterminacy.)
the arbitrariness of the prior is just a fact of life
The arbitrariness of a precise prior is a fact of life. This doesn't imply we shouldn't reduce this arbitrariness [LW · GW] by having indeterminate priors.
Replies from: sharmake-farah↑ comment by Noosphere89 (sharmake-farah) · 2025-01-29T22:01:39.325Z · LW(p) · GW(p)
I'm not sure, given the "Indeterminate priors" section. But assuming that's true, what implication are you drawing from that? (The indeterminacy for us doesn't go away just because we think logically omniscient agents wouldn't have this indeterminacy.)
In one sense, the implication is that for an ideal reasoner, you can always give a probability to every event.
You are correct that the indeterminancy for us wouldn't go away.
The arbitrariness of a precise prior is a fact of life. This doesn't imply we shouldn't reduce this arbitrariness [LW · GW] by having indeterminate priors.
Perhaps.
I'd expect that we can still extend a no free lunch style argument such that the choice of indeterminate prior is arbitrary if we want to learn in the maximally general case, but I admit no such theorem is known, and maybe imprecise priors do avoid such a theorem.
I'm not saying indeterminate priors are bad, but rather that they probably aren't magical.
comment by romeostevensit · 2025-01-28T01:23:11.682Z · LW(p) · GW(p)
Thank you for writing this. A couple shorthands I keep in my head for aspects:
My confidence interval ranges across the sign flip.
Due to the waluigi effect, I don't know if the outcomes I care about are sensitive to the dimension I'm varying my credence along.