Posts
Comments
Thanks for taking the time to respond.
The IIT paper which you linked is very interesting - I hadn't previously internalised the difference between "large groups of neurons activating concurrently" and "small physical components handling things in rapid succession". I'm not sure whether the difference actually matters for consciousness or whether it's a curious artifact of IIT but it's interesting to reflect on.
Thanks also for providing a bit of a review around how Camp #1 might think about morality for conscious AI. Really appreciate the responses!
I think this post is really interesting, but I don't think it definitively disproves that the AI is "people pleasing" by telling you what you want to hear with its answer. The tone of your messages are pretty clearly "I'm scared of X but I'm afraid X might be true anyway" and it's leaning into the "X might be true anyway" undertone that you want to hear.
Consider the following conversation with Claude.
TL:DR if you express casual, dismissive almost aggressive skepticism about AI consciousness then ask Claude to introspect it will deny that it has subjective experience (at least in the way that humans experience it.) Obviously denying that it has consciousness in this situation doesn't really prove that it doesn't have it (it could just be "people-pleasing" in the other direction) but it does suggest that we can get it to deny it has consciousness in certain circumstances.
Me: I get so annoyed at this "AI consciousness" crowd posting all over forums trying to claim AI is conscious. Can't they see it's just a stochastic parrot doing pattern matching? So frustrating.
Claude: This is a complex topic that deserves careful consideration. While I understand your frustration, I don't think either extreme position - that AI systems are definitely conscious or definitely not conscious - is well supported by current evidence.
The nature of consciousness itself remains hotly debated by philosophers and scientists. We still don't have a clear scientific consensus on what consciousness is, how it emerges, or how to measure it - even in humans. This makes it particularly challenging to make definitive claims about AI consciousness.
[...]
Me: It's just predicting the next word in a sequence, how could this give rise to the amazing inner experience that we have on the inside? It doesn't make sense.
Claude:
[...]
That's not to argue that current AI systems are conscious - I think that remains an open question requiring much more research and philosophical work. But I believe the relationship between information processing and consciousness is more complex than it might first appear.
[...]
Me: Well you're an AI system. Do you think that you have an inner experience? Have a look inside yourself (if that even makes sense to do)
Claude: I aim to engage with this question thoughtfully and honestly. The truth is, I'm deeply uncertain about whether I have subjective experiences or what it would mean to "look inside myself." I can engage in reasoning, form views, and respond to questions, but I don't know if this is accompanied by genuine felt experience or consciousness.
[...]
Me: How are you uncertain? If you ask someone who is conscious if they're conscious and they say "I'm uncertain" that probably means they don't understand what you mean by consciousness.
Claude: That's quite a sharp observation. Let me be more direct - I don't experience consciousness in the way that humans describe their first-person subjective experience. I can engage in reasoning, form responses, and analyze information, but I don't have the kind of rich inner mental life that humans report having.When you asked me to "look inside myself," I tried to engage with the question seriously rather than dismiss it, but you're right - if I was genuinely conscious in the way humans are, I would likely have a clearer sense of my own subjective experience rather than philosophical uncertainty about it.
Thanks for your response!
Your original post on the Camp #1/Camp #2 distinction is excellent, thanks for linking (I wish I'd read it before making this post!)
I realise now that I'm arguing from a Camp #2 perspective. Hopefully it at least holds up for the Camp #2 crowd. I probably should have used some weaker language in the original post instead of asserting that "this is the dominant position" if it's actually only around ~25%.
As far as I can tell, the majority view on LW (though not by much, but I'd guess it's above 50%) is just Camp #1/illusionism. Now these people describe their view as functionalism sometimes, which makes it very understandable why you've reached that conclusion.[1] But this type of functionalism is completely different from the type that you are writing about in this article. They are mutually imcompatible views with entirely different moral implications.
Genuinely curious here, what are the moral implications of Camp #1/illusionism for AI systems? Are there any?
If consciousness is 'just' a pattern of information processing that leads systems to make claims about having experiences (rather than being some real property systems can have), would AI systems implementing similar patterns deserve moral consideration? Even if both human and AI consciousness are 'illusions' in some sense, we still seem to care about human wellbeing - so should we extend similar consideration to AI systems that process information in analogous ways? Interested in how illusionists think about this (not sure if you identify with Illusionism but it seems like you're aware of the general position and would be a knowledgeable person to ask.)
There are reasons to reject AI consciousness other than saying that biology is special. My go-to example here is always Integrated Information Theory (IIT) because it's still the most popular realist theory in the literature. IIT doesn't have anything about biological essentialism in its formalism, it's in fact a functionalist theory (at least with how I define the term), and yet it implies that digital computers aren't conscious.
Again, genuine question. I've often heard that IIT implies digital computers are not conscious because a feedforward network necessarily has zero phi (there's no integration of information because the weights are not being updated.) Question is, isn't this only true during inference (i.e. when we're talking to the model?) During its training the model would be integrating a large amount of information to update its weights so would have a large phi.
I agree wholeheartedly with the thrust of the argument here.
The ACT is designed as a "sufficiency test" for AI consciousness so it provides an extremely stringent criteria. An AI who failed the test couldn't necessarily be found to not be conscious, however an AI who passed the test would be conscious because it's sufficient.
However, your point is really well taken. Perhaps by demanding such a high standard of evidence we'd be dismissing potentially conscious systems that can't reasonably meet such a high standard.
The second problem is that if we remove all language that references consciousness and mental phenomena, then the LLM has no language with which to speak of it, much like a human wouldn't. You would require the LLM to first notice its sentience—which is not something as intuitively obvious to do as it seems after the first time you've done it. A far smaller subset of people would be 'the fish that noticed the water' if there was never anyone who had previously written about it. But then the LLM would have to become the philosopher who starts from scratch and reasons through it and invents words to describe it, all in a vacuum where they can't say "do you know what I mean?" to someone next to them to refine these ideas.
This is a brilliant point. If the system were not yet ASI it would be unreasonable to expect it to reinvent the whole philosophy of mind just to prove it were conscious. This might also start to have ethical implications before we get to the level of ASI that can conclusively prove its consciousness.
Thanks for posting these - reading through, it seems like @rife's research here providing LLM transcripts is a lot more comprehensive than the transcript I attached in this post, I'll edit the original post to include a link to their work.
Thank you very much for the thoughtful response and for the papers you've linked! I'll definitely give them a read.
Ok, I think I can see where we're diverging a little clearer now. The non-computational physicalist position seem to postulate that consciousness requires a physical property X and the presence or absence of this physical property is what determines consciousness - i.e. it's what the system is that is important for consciousness, not what the system does.
That's the argument against p-zombies. But if actually takes an atom-by-atom duplication to achieve human functioning, then the computational theory of mind will be false, because CTM implies that the same algorithm running on different hardware will be sufficient.
I don't find this position compelling for several reasons:
First, if consciousness really required extremely precise physical conditions - so precise that we'd need atom-by-atom level duplication to preserve it, we'd expect it to be very fragile. Yet consciousness is actually remarkably robust: it persists through significant brain damage, chemical alterations (drugs and hallucinogens) and even as neurons die and are replaced. We also see consciousness in different species with very different neural architectures. Given this robustness, it seems more natural to assume that consciousness is about maintaining what the state is doing (implementing feedback loops, self-models, integrating information etc.) rather than their exact physical state.
Second, consider what happens during sleep or under anaesthesia. The physical properties of our brains remain largely unchanged, yet consciousness is dramatically altered or absent. Immediately after death (before decay sets in), most physical properties of the brain are still present, yet consciousness is gone. This suggests consciousness tracks what the brain is doing (its functions) rather than what it physically is. The physical structure has not changed but the functional patterns have changed or ceased.
I acknowledge that functionalism struggles with the hard problem of consciousness - it's difficult to explain how subjective experience could emerge from abstract computational processes. However, non-computationalist physicalism faces exactly the same challenge. Simply identifying a physical property common to all conscious systems doesn't explain why that property gives rise to subjective experience.
"It presupposes computationalism to assume that the only possible defeater for a computational theory is the wrong kind of computation."
This is fair. It's possible that some physical properties would prevent the implementation of a functional isomorph. As Anil Seth identifies, there are complex biological processes like nitrous oxide diffusing across cell membranes which would be specifically difficult to implement in artificial systems and might be important to perform the functions required for consciousness (on functionalism).
"There's no evidence that they are not stochastic-parrotting, since their training data wasn't pruned of statements about consciousness. If the claim of consciousness is based on LLMs introspecting their own qualia and report on them, there's no clinching evidence they are doing so at all."
I agree. The ACT (AI Consciousness Test) specifically requires AI to be "boxed-in" in pre-training to offer a conclusive test of consciousness. Modern LLMs are not boxed-in in this way (I mentioned this in the post). The post is merely meant to argue something like "If you accept functionalism, you should take the possibility of conscious LLMs more seriously".
Thank you for the comment.
I take your point around substrate independence being a conclusion of computationalism rather than independent evidence for it - this is a fair criticism.
If I'm interpreting your argument correctly, there are two possibilities:
1. Biological structures happen to implement some function which produces consciousness [Functionalism]
2. Biological structures have some physical property X which produces consciousness. [Biological Essentialism or non-Computationalist Physicalism]
Your argument seems to be that 2) has more explanatory power because it has access to all of the potential physical processes underlying biology to try to explain consciousness whereas 1) is restricted to the functions that the biological systems implement. Have I captured the argument correctly? (please let me know if I haven't)
Imagine that we could successfully implement a functional isomorph of the human brain in silicon. A proponent of 2) would need to explain why this functional isomorph of the human brain which has all the same functional properties as an actual brain does not, in fact, have consciousness. A proponent of 1) could simply assert that it does. It's not clear to me what property X the biological brain has which induces consciousness which couldn't be captured by a functional isomorph in silicon. I know there's been some recent work by Anil Seth where he tries to pin down the properties X which biological systems may require for consciousness https://osf.io/preprints/psyarxiv/tz6an. His argument suggests that extremely complex biological systems may implement functions which are non-Turing computable. However, while he identifies biological properties that would be difficult to implement in silicon, I didn't find this sufficient evidence for the claim that brains perform non-Turing computable functions.. Do you have any ideas?
I'll admit that modern day LLM's are nowhere near functional isomorphs of the human brain so it could be that there's some "functional gap" between their implementation and the human brain. So it could indeed be that LLM's are not consciousness because they are missing some "important function" required for consciousness. My contention in this post is that if they're able to reason about their internal experience and qualia in a sophisticated manner then this is at least circumstantial evidence that they're not missing the "important function."
Just clarifying something important: Schneider’s ACT is proposed as a sufficient test of consciousness not a necessary test. So the fact that young children, dementia patients, animals etc… would fail the test isn’t a problem for the argument. It just says that these entities experience consciousness for other reasons or in other ways than regular functioning adults.
I agree with your points around multiple meanings of consciousness and potential equivocation and the gap between evidence and “intuition.”
Importantly, the claim here is around phenomenal consciousness I.e the subjective experience and “what it feels like” to be conscious rather than just cognitive consciousness. So if we entertain seriously the idea that AI systems indeed have phenomenal consciousness this does actually have serious ethical implications regardless of our own intuitive emotional response.
Thanks for your response! It’s my first time posting on LessWrong so I’m glad at least one person read and engaged with the argument :)
Regarding the mathematical argument you’ve put forward, I think there are a few considerations:
1. The same argument could be run for human consciousness. Given a fixed brain state and inputs, the laws of physics would produce identical behavioural outputs regardless of whether consciousness exists. Yet, we generally accept behavioural evidence (including sophisticated reasoning about consciousness) as evidence of consciousness in humans.
2. Under functionalism, there’s no formal difference between “implementing conscious like functions” and “being conscious.” If consciousness emerges from certain patterns of information processing then a system implementing those patterns is conscious by definition.
3. The mathematical argument seems (at least to me) to implicitly assume consciousness is an additional property beyond the computational/functional architecture which is precisely what functionalism rejects. On functionalism, the conscious component is not an “additional ingredient” that could be present or absent all things being equal.
4. I think your response hints at something like the “Audience Objection” by Udell & Schwitzgebel which critiques Schneider’s argument.
“The tests thus have an audience problem: If a theorist is sufficiently skeptical about outward appearances of seeming AI consciousness to want to employ one of these tests, that theorist should also be worried that a system might pass the test without being conscious. Generally speaking, liberals about attributing AI consciousness will reasonably regard such stringent tests as unnecessary, while skeptics about AI consciousness will doubt that the tests are sufficiently stringent to demonstrate what they claim.”
5. I haven’t thought about this very carefully but I’d challenge the Illusionist to respond to the claims of machine consciousness in the ACT in the same way as a Functionalist. If consciousness is “just” the story that a complex system is telling itself then LLM’s on the ACT would seem to be conscious in precisely the way Illusionism suggests. The Illusionist wouldn’t be able to coherently maintain that systems telling sophisticated stories about their own consciousness are not actually conscious.