S.E.A.R.L.E's COBOL room

The Chinese Room argument is actually pretty good if you read it as a criticism of suggestively named LISP tokens, which I think were popular roughly around that time. But of course, it fails completely once you try to make it into a general proof of why computers can't think. Then again, Searle didn't claim it impossible for computers to think, he just said that they'd need similar "causal powers" as the human brain.

Also, the argument that "When they claim that a mind can emerge from "a system" without saying what the system is or how such a thing might give rise to a mind, then they are under the grip of an ideology" is actually pretty reasonable. Steelmanned, the Chinese Room would be an attack on people who were putting together suggestively named tokens and building systems that could perform crude manipulations on their input and then claiming that this was major progress towards building a mind, while having no good theory of why exactly these particular kinds of symbol manipulations should be expected to produce a mind.

Or look at something like SHRDLU: it's superficially very impressive and gives an impression that you're dealing with something intelligent, but IIRC, it was just a huge bunch of hand-coded rules for addressing various kinds of queries, and the approach didn't scale to more complex domains because the amount of rules you'd needed to have program in would have blown up. In the context of programs like those, Searle's complaints about dumb systems that do symbol manipulation without any real understanding of what they're doing make a lot more sense.

It took me years to realize that my agreement with the Chinese Room argument was due almost entirely to how convincing Searle sounded standing in front of a classroom, and to his skill at appealing to both common sense and vanity. The argument was flagged as correct and cached, acquired a support structure of evasions and convenient flinches, and so it remained, assumed and unquestioned.

I wish I could remember an exact moment of realization that destroyed the whole thing, roots and all, but I don't. I suspect it was more of a gradual shift, where the support structure was slowly eroded by new ideas (i.e. I started reading LessWrong), and the contradictions were exposed. Until one day it came up in a discussion and I realized that, far from wanting to defend it, I didn't actually believe it anymore.

I should probably go back through my class notes (if I can find them) and see what else I might have carelessly cached.

This was a fun read. Reminds me of Terry Bisson's "They're made out of meat."

(I like variations on these intuition pumps involving questions like whether a depiction of a person in a video recording has understanding/consciousness, and imagining extending the video recordings in various ways until it would feel unreasonable to assert that something with consciousness/understanding isn't there.)

How is Searle's actual response to the accusation that he has just dressed up the Other Minds Problem at all satisfactory? Does anyone find it convincing?

I used not to take Searle's arguments seriously until I actually understood what they were about.

Before anything, I should say that I disagree with Searle's arguments. However, it is important to understand them if we are to have a rational discussion.

Most importantly, Searle does not claim that machines can never understand, or that there is something inherently special about the human brain that cannot be replicated in a computer. He acknowledges that the human brain is governed by physics and is probably subject to the church-turing thesis.

Searle's main argument is this: sophistication of computation does not by itself lead to understanding. That is, just because a computer is doing something that a human could not do without understanding does not mean the computer must be understanding it as well. It is very hard to argue against this, which is why the Chinese room argument has stuck around for so long.

Searle is of the opinion that if we can find the 'mechanism' of understanding in the brain and replicate it in the computer, the computer can understand as well.

To get down to the nuts and bolts of the argument, he maintains that a precise molecular-level simulation of a human brain would be able to understand, but a computer that just happened to act intelligent might not be able to.

In my opinion, this argument just hides yet another form of vitalism. That there is something above and beyond the mechanical. However, everywhere in the brain we've looked, we've found just neurons doing simple computations on their inputs. I believe that that is all there is to it - that something with the capabilities of the human brain also has the ability to understand.

However, this is just a belief at this point. There is no way to prove it. There probably will be no way until we can figure out what consciousness is.

So there you have it. The chinese room argument is really just another form of the Hard Problem of consciousness. Nothing new to see here.

When they claim that a mind can emerge from "a system" without saying what the system is or how such a thing might give rise to a mind, then they are under the grip of an ideology.

Is this actually what Searl says?

This seems a lot like Haugeland's Demon, as described in Hofstadter and Dennett's The Mind's I.

If I don't understand COBOL, and the inert neurones certainly don't, how can the conjunction of the two understand COBOL?

This one appears to assume reductionism is false: the same reasoning can apply to properly organized heap of neurons. Indeed an intuition pump, for we are dualists by default.

(Come to think of it, doesn't non-reductionism come from treating our ignorance as positive knowledge? In that case, the mind seems irreducible only because we just don't know how to reduce it yet.)

.