[link] Scott Aaronson on free will

As far as I can see, the only hope for avoiding these difficulties is if—because of chaos, the limits of quantum measurement, or whatever other obstruction—minds can’t be copied perfectly from one physical substrate to another, as can programs on standard digital computers.

Even if Aaronson's speculation that human minds are not copyable turns out to be correct, that doesn't rule out copyable minds being built in the future, either de novo AIs or what he (on page 58) calls "mockups" of human minds that are functionally close enough to the originals to fool their close friends. The philosophical problems with copyable minds will still be an issue for those minds, and therefore minds not being copyable can't be the only hope of avoiding these difficulties.

To put this another way, suppose Aaronson definitively shows that according to quantum physics, minds of biological humans can't be copied exactly. But how does he know that he is actually one of the original biological humans, and not for example a "mockup" living inside a digital simulation, and hence copyable? I think that is reason enough for him to directly attack the philosophical problems associated with copyable minds instead of trying to dodge them.

I really don't like the term "LW consensus" (isn't there a LW post about how you should separate out bundles of ideas and consider them separately because there's no reason to expect the truth of one idea in a bundle to correlate strongly with the truth of the others? If there isn't, there should be). I've been using "LW memeplex" instead to emphasize that these ideas have been bundled together for not necessarily systematically good reasons.

I think that last paragraph you quote needs the following extra bit of context:

To clarify, we can’t use any philosophical difficulties that would arise if minds were copyable, as evidence for the empirical claim that they’re not copyable. The universe has never shown any particular tendency to cater to human philosophical prejudices! But I’d say the difficulties provide more than enough reason to care about the copyability question.

... because otherwise it looks as if Aaronson is saying something really silly, which he isn't.

If we could fax ourselves to Mars, or undergo uploading, then still wonder whether we're still "us" -- the same as we wonder now when such capabilities are just theoretical/hypotheticals -- that should count as a strong indication that such questions are not very practically relevant, contrary to Aaronson's assertion. Surely we'd need some legal rules, but the basis for those wouldn't be much different than any basis we have now -- we'd still be none the wiser about what identity means, even standing around with our clones.

For example, if we were to wonder about a question of "what effect will a foom-able AI have on our civilization", surely asking after the fact would yield different answers to asking before. With copies / uploads etc., you and your perfect copy could hold a meeting contemplating who stays married with the wife, and still start from the same basis with the same difficulty of finding the "true" answer as if you'd discussed the topic with a pal roleplaying your clone, in the present time.

In particular:

whenever it’s been possible to make definite progress on ancient philosophical problems, such progress has almost always involved a [kind of] “bait-and-switch.” In other words: one replaces an unanswerable philosophical riddle Q by a “merely” scientific or mathematical question Q′, which captures part of what people have wanted to know when they’ve asked Q. Then, with luck, one solves Q′.

Of course, even if Q′ is solved, centuries later philosophers might still be debating the exact relation between Q and Q′! And further exploration might lead to other scientific or mathematical questions — Q′′, Q′′′, and so on — which capture aspects of Q that Q′ left untouched. But from my perspective, this process of “breaking off” answerable parts of unanswerable riddles, then trying to answer those parts, is the closest thing to philosophical progress that there is.

Successful examples of this breaking-off process fill intellectual history. The use of calculus to treat infinite series, the link between mental activity and nerve impulses, natural selection, set theory and first-order logic, special relativity, Gödel’s theorem, game theory, information theory, computability and complexity theory, the Bell inequality, the theory of common knowledge, Bayesian causal networks — each of these advances addressed questions that could rightly have been called “philosophical” before the advance was made. And after each advance, there was still plenty for philosophers to debate about truth and provability and infinity, space and time and causality, probability and information and life and mind. But crucially, it seems to me that the technical advances transformed the philosophical discussion as philosophical discussion itself rarely transforms it! And therefore, if such advances don’t count as “philosophical progress,” then it’s not clear that anything should.

Appropriately for this essay, perhaps the best precedent for my bait-and-switch is the Turing Test... with legendary abruptness, Turing simply replaced the original question by a different one: “Are there imaginable digital computers which would do well in the imitation game?”...

...The claim is not that the new question, about the imitation game, is identical to the original question about machine intelligence. The claim, rather, is that the new question is a worthy candidate for what we should have asked or meant to have asked, if our goal was to learn something new rather than endlessly debating definitions. [Luke adds: I'm reminded of Dennett's quip that "Philosophy... is what you have to do until you figure out what questions you should have been asking in the first place."] In math and science, the process of revising one’s original question is often the core of a research project, with the actual answering of the revised question being the relatively easy part!

A good replacement question Q′ should satisfy two properties:

(a) Q′ should capture some aspect of the original question Q — so that an answer to Q′ would be hard to ignore in any subsequent discussion of Q.

(b) Q′ should be precise enough that one can see what it would mean to make progress on Q′: what experiments one would need to do, what theorems one would need to prove, etc.

The Turing Test, I think, captured people’s imaginations precisely because it succeeded so well at (a) and (b). Let me put it this way: if a digital computer were built that aced the imitation game, then it’s hard to see what more science could possibly say in support of machine intelligence being possible. Conversely, if digital computers were proved unable to win the imitation game, then it’s hard to see what more science could say in support of machine intelligence not being possible. Either way, though, we’re no longer “slashing air,” trying to pin down the true meanings of words like “machine” and “think”: we’ve hit the relatively-solid ground of a science and engineering problem. Now if we want to go further we need to dig (that is, do research in cognitive science, machine learning, etc). This digging might take centuries of backbreaking work; we have no idea if we’ll ever reach the bottom. But at least it’s something humans know how to do and have done before. Just as important, diggers (unlike air-slashers) tend to uncover countless treasures besides the ones they were looking for.

Intertemporal solidarity is just as much a choice today as it will be should teleporters arrive.

That is admirably concise, correct, and totally on target.

"Intertemporal solidarity is just as much a choice today as it will be should teleporters arrive."

I should clarify that I see no special philosophical problem with teleportation that necessarily destroys the original copy, as quantum teleportation would (see the end of Section 3.2). As you suggest, that strikes me as hardly more perplexing than someone's boarding a plane at Newark and getting off at LAX.

For me, all the difficulties arise when we imagine that the teleportation would leave the original copy intact, so that the "new" and "original" copies could then interact with each other, and you'd face conundrums like whether "you" will experience pain if you shoot your teleported doppelganger. This sort of issue simply doesn't arise with the traditional problem of intertemporal identity, unless of course we posit closed timelike curves.

You are not a COPY (perfect or otherwise) of yourself from one moment to the next. Not by any meaningful definition of the word copy.

"But calling this Knightian unpredictability 'free will' just confuses both issues."

torekp, a quick clarification: I never DO identify Knightian unpredictability with "free will" in the essay. On the contrary, precisely because "free will" has too many overloaded meanings, I make a point of separating out what I'm talking about, and of referring to it as "freedom," "Knightian freedom," or "Knightian unpredictability," but never free will.

On the other hand, I also offer arguments for why I think unpredictability IS at least indirectly relevant to what most people want to know about when they discuss "free will" -- in much the same way that intelligent behavior (e.g., passing the Turing Test) is relevant to what people want to know about when they discuss consciousness. It's not that I'm unaware of the arguments that there's no connection whatsoever between the two; it's just that I disagree with them!

Downvoted for extremely uncharitable reading of the paper.

It's worse than I thought. Aaronson really does want to address the free will debate in philosophy - and utterly botches the job.

Aaronson speaks for some of his interlocutors (who are very smart - i.e. they say what I would say ;) ):

Suppose that only “ordinary” quantum randomness and classical chaos turned out to be involved: how on earth could that matter, outside the narrow confines of free-will debates? Is the variety of free will that apparently interests you—one based on the physical unpredictability of our choices—really a variety “worth wanting,” in Daniel Dennett’s famous phrase [27]?

and answers:

As a first remark, if there’s anything in this debate that all sides can agree on, hopefully they can agree that the truth (whatever it is) doesn’t care what we want, consider “worth wanting,” or think is necessary or sufficient to make our lives meaningful!

Uh, so: the truth about the modal number of hairs on a human foot, for example, doesn't care what we want. But, if I were to claim that (a good portion of) a famous philosophical debate amounted to the question of how many hairs are on our feet, you could reject the claim immediately. And you could cite the fact that nobody gives a damn about that as a sufficient reason to reject the substitution of the new question for the old. Sure, in this toy example, you could cite many other reasons too. But with philosophical (using the next word super broadly) definitions, sometimes the most obvious flaw is that nobody gives a damn about the definiens, while lots care passionately about the concept supposedly defined. Dennett rightly nails much philosophizing about free will, on precisely this point.

Interlocutors:

But if the event is undetermined, it isn’t “free” either: it’s merely arbitrary, capricious, and random.

Aaronson:

An event can be “arbitrary,” in the sense of being undetermined by previous events, without being random in the narrower technical sense of being generated by some known or knowable probabilistic process.

But this doesn't address the point, unless Knightian-uncertain "actions" somehow fall more under the control of the agent than do probabilistic processes. The truth is the opposite. I have the most control over actions that flow deterministically from my beliefs and desires. I have almost as much, if it is highly probable that my act will be the one that the beliefs and desires indicate is best. And I have none, if it is completely uncertain. For example, if I am pondering whether to eat a berry and then realize that this type of berry is fatally poisonous, this realization should ideally be decisive, with certainty. But I'll settle for 99.99...% probability that I don't eat it. If it is wide-open uncertain whether I will eat it - arbitrary in a deep way - that does not help my sense of control and agency. To put it mildly!

Finally, Aaronson has a close brush with the truth when he rejects the following premise of the Consequence Argument (section 2.9):

(ii) The state of the universe 100 million years ago is clearly outside our ability to alter.

Aaronson replies:

there might be many possible settings of the past microfacts—the polarizations of individual photons, etc. [...] our choices today might play a role in selecting one past from a giant ensemble of macroscopically-identical but microscopically-different pasts.

Only one microscopic past is consistent with our choice today. But the same can be said of the whole past on a classically-determinist scientific picture. As Carl Hoefer says in the Stanford Encyclopedia

they can be viewed as bi-directionally deterministic. That is, a specification of the state of the world at a time t, along with the laws, determines not only how things go after t, but also how things go before t. Philosophers, while not exactly unaware of this symmetry, tend to ignore it when thinking of the bearing of determinism on the free will issue.

And that's a mistake, as Hoefer indicates in his conclusion. The physics of deterministic theories (and also, I would add, any probabilistic quantum theory that gives sufficient probabilistic weight to our beliefs, desires, and reasoning in producing action) supports strong compatibilism, not Aaronson's weak version.

Missed it by (fingers close together) that much!

Not bad at all :)

Eh, I don't think I count as a luminary, but thanks :-)

Aaronson's crediting me is mostly due to our exchanges on the blog for his paper/class about philosophy and theoretical computer science.

One of them, about Newcomb's problem where my main criticisms were

a) he's overstating the level and kind of precision you would need when measuring a human for prediction; and

b) that the interesting philosophical implications of Newcomb's problem follow from already-achievable predictor accuracies.

The other, about average-human performance on 3SAT, where I was skeptical the average person actually notices global symmetries like the pigeonhole principle. (And, to a lesser extent, whether the other in which you stack objects affects their height...)

Predicting with 99.99% accuracy that any person, put in front of the dilemma of tasting a pleasant cake or receive a kick in the teeth (or, to stay in the Portal metaphor, to be burned alive) will chose the cake, is clearly not relevant to the free will debate.

Is there an existing principled explanation for why this is not relevant to the free will debate, but predicting less obvious behaviors is?

It is completely not about being more or less impressive.

So better-than-chance predictability is just too low a bar for clearing it to have any relevance to the free-will debate.

This is plainly wrong, as any Bayesian-minded person will know: it all depends on the prior information you are using.

If you can throw out the fMRI data and get better predictive power, something is wrong with the fMRI data.

At the same time, predicting what will be the next choice in pressing the button, exclusively from neurological data (and very broadly aggregated, as is the case of fMRI) with 60% accuracy, is in direct contrast with the Knightian unpredictability thesis.

The fMRI results are not relevant, because quantum effects in the brain are noise on an fMRI. Aaronson explicitly locates any Knightian noise left in the system to be at the microscopic level; see the third paragraph under section 3.

TL;DR: 2.12 is about forestalling a bad counterargument (that being the heading of 2.12) and does not give evidence against Knightian upredictability.

What? Just because predicting human behaviour one minute before it's happening with 99% accuracy is more impressive, it doesn't mean that it involves any kind of different process than predicting human behaviour 5 seconds before with 60% accurateness. Admittedly, it might imply different kind, maybe even unachievable or uncomputable kind of process, but it also may be just a matter of better probes/more computational power.

So would you have been wiling to draw the same conclusion from an experiment that predicted the button pushing 1 second before with 99.99999% probability by scanning the neurons in the arm?

Simple but misleading.

This is because Newcomb's problem is not reliant on the predictor being perfectly accurate. All they need to do is predict you so well that people who one-box walk away with more expected utility than people who two-box. This is easy - even humans can predict other humans this well (though we kinda evolved to be good at it).

So if it's still worth it to one-box even if you're not being copied, what good is an argument that relies on you being copied to work?

It seems way simpler to leave out the "freely willed decision" part altogether.

If we posit that the Predictor can reliably predict my future choice based on currently available evidence, it follows that my future choice is constrained by the current state of the world. Given that, what remains to be explained?

I'm not quite sure what you mean here. Do you mean that if he's right, then LW consensus is wrong, and that makes LW consensus a religion?

That seems both wrong and rather mean to both LW consensus and religion.

I would guess that he thinks that the probability of this hypothetical---worlds in which brain scanning isn't possible---is pretty low (based on having discussed it briefly with him). I'm sure everyone around here thinks it is possible as well, it's just a question of how likely it is. It may be worth fleshing out the perspective even if it is relatively improbable.

In particular, the probability that you can't get a functional human out of a brain scan seems extremely low (indeed, basically 0 if you interpret "brain scan" liberally), and this is the part that's relevant to most futurism.

Whether there can be important aspects of your identity or continuity of experience that are locked up in uncopyable quantum state is more up for grabs, and I would be much more hesitant to bet against that at 100:1 odds. Again, I would guess that Scott takes a similar view.

we can easily imagine worlds consistent with quantum mechanics (and all other known physics and biology) where the answer to the question [scanning of minds possible?] is yes, and other such worlds where the answer is no

If it is so easy, could someone please explain me the main idea in less than 85 pages?

(Let's suppose that the scanned mind does not have to be an absolutely perfect copy; that differences as big as the difference between me now and me 1 second later are acceptable.)

Easily imagining worlds doesn't mean they are possible or even consistent, as per the p-zombie world.

This is not argument against Aaronson paper in general, although I think it's far from correct, but against your deduction.

Plus, I think there exist multiple, reasonable and independent arguments that favors LW consensus, and this is evidential weight against Aaronson paper, not the opposite.

LW consensus is religion

What an offensive analogy! Please don't tar a vast, nuanced thing like religion with hasty analogies to something as trifling and insignificant as LW consensus. After all, denotation may win arguments, technically, but connotation changes minds—so I beg thee, be careful.