Refuting Searle’s wall, Putnam’s rock, and Johnson’s popcorn

IMO, the entire Chinese room thought experiment dissolves into clarity once we remember that the intuitive meaning of understanding is formed around algorithms that are not lookup tables, because trying to create an infinite look-up table would be infeasible in reality, thus our intuitions go wrong in extreme cases.

I agree with the discord comments here on this point:

The portion of the argument where I contest is step 2 here (it's a summarized version):

1. If Strong AI is true, then there is a program for Chinese, C, such that if any computing system runs C, that system thereby comes to understand Chinese.
2. I could run C without thereby coming to understand Chinese.
3. Therefore Strong AI is false.

Or this argument here:

Searle imagines himself alone in a room following a computer program for responding to Chinese characters slipped under the door. Searle understands nothing of Chinese, and yet, by following the program for manipulating symbols and numerals just as a computer does, he sends appropriate strings of Chinese characters back out under the door, and this leads those outside to mistakenly suppose there is a Chinese speaker in the room.

As stated, if the computer program is accessible to him, then for all intents and purposes, he does understand Chinese for the purposes of interacting until the program is removed (assuming that it completely characterizes Chinese and correctly works for all inputs).

I think the key issue is that people don't want to accept that if we were completely unconstrained physically, even very large look-up tables would be a valid answer to making an AI that is useful.

The book in the room isn't inert, though. It instructs the little guy on what to do as he manipulates symbols and stuff. As such, it is an important part of the computation that takes place.

The mapping of popcorn-to-computation, though, doesn't do anything equivalent to this. It's just an off-to-the-side interpretation of what is happening in the popcorn: it does nothing to move the popcorn or cause it to be configured in such a way. It doesn't have to even exist: if you just know that in theory there is a way to map the popcorn to the computation, then if (by the terms of the argument) the computation itself is sufficient to generate consciousness, the popcorn should be able to do it as well, with the mapping left as an exercise for the reader. Otherwise you are implying some special property of a headful of meat such that it does not need to be interpreted in this way for its computation to be equivalent to consciousness.

This isn't arguing other side, just a request for clarification:

There are some Turing machines that we don't know and can't prove if they halt. There are some that are sufficiently simple that we can show that they halt.

But why dors the discussion encompass the question of all possible Turing machines and their maps? Why isn't it sufficient to discuss the subset which are tractable within some compute threshold?

Okay, let me know if this is a fair assessment:

1. Let's consider someone meditating in a dark and mostly-sealed room with minimal sensory inputs, and they're meditating in a way that we can agree they're having a conscious experience. Let's pick a 1 second window and consider the CNS and local environment of the meditator during that window.

2. (I don't know much physics, so this might need adjustment): Let's say we had a reasonable guess of an "initial wavefunction" of the meditator in that window. Maybe this hypothetical is unreasonable in a deep way and this deserves to be fought. But supposing it can be done, and we had a sufficiently powerful supercomputer, we could encode and simulate possible trajectories of this CNS over a one second window. CF suggests that there is a genuine conscious experience there.

3. Now let's look at how one such simulation is encoded, which we could view as a long string of 0s and 1s. The tricky part (I think) is as follows: we have a way of understanding these 0s and 1s as particles and the process of interpreting these as states of particles is "simple". But I can't convert that understanding rigorously into the length of a program because all programs can do is convert one encoding into another (and presumably we've designed this encoding to be as straightforward-to-interpret as possible, instead of as short as possible).

4. Let's say I have sand swirling around in a sandstorm. I likewise pick a section of this, and do something like the above to encode it as a sequence of integers in a manner that is as easy for a human to interpret as possible, and makes no effort to be compressed.

5. Now I can ask for the K-complexity of the CNS string, given the sand swirling sequence as input (i.e. the size of the smallest turing machine that prints the CNS string with the sand swirling sequence on its input tape). Divide this by the K-complexity of the CNS string. If the resulting fraction is close to zero, maybe there's a sense in which the sand swirling sequence is really emulating the meditator's conscious experience. But this is ratio is probably closer to 1. (By the way, the choice of using K-complexity is itself suspect, but it can be swapped with other notions of complexity.)

What I can't seem to shake is that it seems to be fundamentally important that we have some notion of 0s and 1s encoding things in a manner that is optimally "human-friendly". I don't know how this can be replaced with a way that avoids needing a sentient being.

but I don't need to be able to define fire to be able to point out something that definitely isn't on fire!

I guess I can see that. I just don't think that e.g. Mike Johnson would consider his argument refuted based on the post; I think he'd argue that the type of problems illustrated by the popcorn thought experiment are in fact hard (and, according to him, probably unsolvable). And I'd probably still invoke the thought experiment myself, too. Basically I continue to think they make a valid point, hence are not "refuted". (Maybe I'm being nit-picky? But I think the standards for claiming to have refuted sth should be pretty high.)