err, disagree? free will is logical uncertainty about your own choice. it's not written until you write it, which you were always going to do. then you find out you wrote it.

I'm curious--is this your attempt to "do your homework", according to Eliezer [LW · GW]? I ask because it doesn't seem like you're addressing the question he actually asked. I would really try to do that before opening this spoiler. The question is: what is the belief in free will? How does it work? What does it actually do?

Meditators who find their feeling of free will and self "dissolved' aren't noted to act any differently by their family/coworkers afterwards, a fact that many find very confusing and upsetting. So if there isn't a difference in behavior (other than talking about meditation a lot), why would there be a selection advantage?

Curiously, people are willing to impute thought, choice, decision-making, understanding, purpose, action to achieve purposes, and so on, to machines like the GPT family, yet are often diffident or hostile to attributing these things to people. I find this very strange.

I left "free will" out of that list, because it is not clear to me what people are asserting, when they say they have "free will", or what they are denying, when they say they do not.

We can all agree that there is no such thing as a unicorn. We can do that because we all know what the word means: a supposed horse-like creature with a single horn in its forehead, and having magical attributes variously told of. Because we know what we mean, we would recognise one if we found one. But we know it is only a legend. Explorers have never found any such creature, not even something close enough to have inspired the legend. There is no such thing as a unicorn.

But when speaking of free will, whether to assert it or deny it, people describe it only in phrases that seem to point at the same thing in different words, whatever that thing is. "Choice", "the possibility of having done differently", and so on. Contrast "unicorn", which is described in terms that make no reference to unicorns.

Thus "free will" is the name given to a certain subjective experience that people are unable to articulate any further, the experience of acting in the world. When they try to articulate it they confabulate non-answers, answers that no more answer the question "what is free will?" than "gravity" answers "why do things fall?", or "energy" answers "how does a clockwork toy work?", or "fluency" answers "how do you speak a foreign language well?"^[1] These non-answers are easily shown to be empty or absurd, but this does not negate the existence of the phenomena that they fail to elucidate. Things fall, clockwork toys move, and mastery of a foreign language is possible.

People clearly do think, choose, make decisions, understand things, have purposes, act to achieve them, and so on. Given that, what is left to talk about on the subject of "free will"? Free will is the quale of doing such things, and can be no more articulated than the sensation of hearing a trumpet.

I don't understand the counterfactual you're using to declare the false belief "useful".  Are you saying a p-zombie would somehow behave differently, or that it's impossible, or something else?

I mostly bite the bullet that it's an illusion, but I suspect it's a side-effect of the efficient behavior encoding in a human brain, not a separable thing that can be evaluated as to utility.  I do not know if other active deterministic structures have or need such an illusion.

lsusr, if it was proven that the human brain actually does work on quantum principles, how would that change your view on free will?

One mind knows that the universe is purely deterministic

Some thoughts:

• The word "knows" does a lot of work here. What does it mean for a brain to know that it is deterministic? I guess you appeal to an intuitive understanding, but that makes the argument weaker.
• I also don't think that the conclusion follows. Many organisms get a lot of things done without knowing it.
• Evolution cannot work on concepts in the neocortex, and thus your argument doesn't work unless you assume some mechanism for how the brain stem brings about the concepts in the neocortex. In this case, the argument is really about that mechanism and not about our high-level summary "knows itself". 
• If your argument works, though, that would indicate a new (?) alignment strategy: Sufficiently strongly ensuring that the mind doesn't produce self-awareness could reduce agency to zero.

The human brain is too big and too warm for quantum indeterminacy to matter

What you read was: the human brain is to big and too warm for large scale superpositions to form. Any single particle experiment amplifies quantum indeterminacy to the macroscopic level.

ETA

Most people feel like they have free will. But nobody actually does

What makes that true? Just the claim about quantum.mechanics?