Posts
Comments
Yeah, e.g., any convergent series.
This is assuming no expression that converges to the constants exists? Which I think is an open question. (Of course, it would only be finite if there are such expressions for all constants. But even so, I think it's an open question.)
As someone who expects LLMs to be a dead end, I nonetheless think this post makes a valid point and does so using reasonable and easy to understand arguments. I voted +1.
As I already commented, I think the numbers here are such that the post should be considered quite important even though I agree that it fails at establishing that fish can suffer (and perhaps lacks comparison to fish in the wild). If there was another post with a more nuanced stance on this point, I'd vote for that one instead, but there isn't. I think fish wellbeing should be part of the conversation more than it is right now.
It's also very unpleasant to think or write about these things, so I'm also more willing to overlook flaws than I'd be by default.
Shape can most certainly be emulated by a digital computer. The theory in the paper you linked would make a brain simulation easier, not harder, and the authors would agree with that
Would you bet on this claim? We could probably email James Pang to resolve a bet. (Edit: I put about 30% on Pang saying that it makes simulation easier, but not necessarily 70% on him saying it makes simulation harder, so I'd primarily be interested in a bet if "no idea" also counts as a win for me.)
It is not proposing that we need to think about something other than neuronal axons and dendrites passing information, but rather about how to think about population dynamics.
Really? Isn't the shape of the brain something other than axons and dendrites?
The model used in the paper doesn't take any information about neurons into account, it's just based on a mesh of the geometry of the particular brain region.
So this is the opposite of proposing a more detailed model of brain function is necessary, but proposing a courser-grained approximation.
And they're not addressing what it would take to perfectly understand or reproduce brain dynamics, just a way to approximately understand them.
The results (at least the flagship result) are about a coarse approximation, but the claim that anatomy restricts function still seems to me like contradicting the neuron doctrine.
Admittedly the neuron doctrine isn't well-defined, and there are interpretations where there's no contradiction. But shape in particular is a property that can't be emulated by digital computers, so it's a contradiction as far as the OP goes (if in fact the paper is onto something).
I mean, we have formalized simplicity metrics (Solomonoff Induction, minimal description length) for a reason, and that reason is that we don't need to rely on vague intuitions to determine whether a given theory (like wave function collapse) is plausible.
No reputable neuroscientist argued against it to any strong degree, just for additional supportive methods of information transmission.
I don't think this is correct. This paper argues explicitly against the neuron doctrine (enough so that they've put it into the first two sentences of the abstract), is published in a prestigious journal, has far above average citation count, and as far as I can see, is written by several authors who are considered perfectly fine/serious academics. Not any huge names, but I think enough to clear the "reputable" bar.
I don't think this is very strong evidence since I think you can find people with real degrees supporting all sorts of contradicting views. So I don't think it really presents an issue for your position, just for how you're phrased it here.
Two thoughts here
-
I feel like the actual crux between you and OP is with the claim in post #2 that the brain operates outside the neuron doctrine to a significant extent. This seems to be what your back and forth is heading toward; OP is fine with pseudo-randomness as long as it doesn't play a nontrivial computational function in the brain, so the actual important question is not anything about pseudo-randomness but just whether such computational functions exist. (But maybe I'm missing something, also I kind of feel like this is what most people's objection to the sequence 'should' be, so I might have tunnel vision here.)
-
(Mostly unrelated to the debate, just trying to improve my theory of mind, sorry in advance if this question is annoying.) I don't get what you mean when you say stuff like "would be conscious (to the extent that I am), and it would be my consciousness (to a similar extent that I am)," since afaik you don't actually believe that there is a fact of the matter as to the answers to these questions. Some possibilities what I think you could mean
- I don't actually think these questions are coherent, but I'm pretending as if I did for the sake of argument
- I'm just using consciousness/identity as fuzzy categories here because I assume that the realist conclusions must align with the intuitive judgments (i.e., if it seems like the fuzzy category 'consciousness' applies similarly to both the brain and the simulation, then probably the realist will be forced to say that their consciousness is also the same)
- Actually there is a question worth debating here even if consciousness is just a fuzzy category because ???
- Actually I'm genuinely entertaining the realist view now
- Actually I reject the strict realist/anti-realist distinction because ???
I think causal closure of the kind that matters here just means that the abstract description (in this case, of the brain as performing an algorithm/computation) captures all relevant features of the the physical description, not that it has no dependence on inputs. Should probably be renamed something like "abstraction adequacy" (making this up right now, I don't have a term on shelf for this property). Abstraction (in)adequacy is relevant for CF I believe (I think it's straight-forward why?). Randomness probably doesn't matter since you can include this in the abstract description.
You're using 'bridging law' differently from I was, so let me rephrase.
To explain subjective experience, you need bridging-laws-as-you-define-them. But it could be that consciousness is functional and the bridging laws are implicit in the description of the universe, rather than explicit. Differently put, the bridging laws follow as a logical consequences of how the remaining universe is defined, rather than being an additional degree of freedom.[1]
In that case, since bridging laws do not add to the length of the program,[2] Solomonoff Induction will favor a universe in which they're the same for everyone, since this is what happens by default (you'd have a hard time imagining that bridging laws follow by logical necessity but are different for different people). In fact, there's a sense in which the program that SI finds is the same as the program SI would find for an illusionist universe; the difference is just about whether you think this program implies the existence of implicit bridging laws. But in neither cases is there an explicit set of bridging laws that add to the length of the program.
Most of Eliezer's anti zombie sequence, especially Zombies Redacted can be viewed as an argument for bridging laws being implicit rather than explicit. He phrases this as "consciousness happens within physics" in that post. ↩︎
Also arguable but something I feel very strongly about; I have an unpublished post where I argue at length that and why logical implications shouldn't increase program length in Solomonoff Induction. ↩︎
I reacted locally invalid (but didn't downvote either comment) because I think "computation" as OP is using it is about the level of granularity/abstraction at which consciousness is located, and I think it's logically coherent to believe both (1) materialism[1] and (2) consciousness is located at a fundamental/non-abstract level.
To make a very unrealistic analogy that I think nonetheless makes the point: suppose you believed that all ball-and-disk integrators were conscious. Do you automatically believe that consciousness can be defined with a computation? Not necessarily -- you could have a theory according to which a digital computer computing the same integrals is not consciousness (since, again, consciousness is about the fine-grained physical steps, rather than the abstracted computational steps, and a digital computer calculating performs very different physical steps than a ball-and-disk integrator doing the same). The only way you now care about "computation" is if you think "computation" does refer to low-level physical steps. In that case, your implication is correct, but this isn't what OP means, and OP did define their terms.
as OP defines the term; in my terminology, materialism means something different ↩︎
If consciousness is not functional, then Solomonoff induction will not predict it for other people even if you assert it for yourself.
Agreed. But who's saying that consciousness isn't functional? "Functionalism" and "functional" as you're using it are similar sounding words, but they mean two different things. "Functionalism" is about locating consciousness on an abstracted vs. fundamental level. "Functional" is about consciousness being causally active vs. passive.[1] You can be a consciousness realist, think consciousness is functional, but not a functionalist.
You can also phrase the "is consciousness functional" issue as the existence or non-existence of bridging laws (if consciousness is functional, then there are no bridging laws). Which actually also means that Solomonoff Induction privileges consciousness being functional, all else equal (which circles back to your original point, though of course you can assert that consciousness being functional is logically incoherent and then it doesn't matter if the description is shorter).
I would frame this as dual-aspect monism [≈ consciousness is functional] vs. epiphenomenalism [≈ consciousness is not functional], to have a different sounding word. Although there are many other labels people use to refer to either of the two positions, especially for the first, these are just what I think are clearest. ↩︎
In Solomonoff induction, the mathematical formalization of Occam's razor, it's perfectly legitimate to start by assuming your own phenomenal experience (and then look for hypotheses that would produce that, such as the external world plus some bridging laws). But there's no a priori reason those bridging laws have to apply to other humans.
You can reason that a universe in which you are conscious and everyone else is not is more complex than a universe in which everyone is equally conscious, therefore Solomonoff Induction privileges consciousness for everyone.
Chalmer’s fading qualia [...]
Worth noting that Eliezer uses this argument as well in The Generalized Anti-Zombie Principle, as the first line of his Socrates Dialogue (I don't know if he has it from Chalmers or thought of it independently):
Albert: "Suppose I replaced all the neurons in your head with tiny robotic artificial neurons that had the same connections, the same local input-output behavior, and analogous internal state and learning rules."
He also acknowledges that this could be impossible, but only considers one reason why (which at least I consider highly implausible):
Sir Roger Penrose: "The thought experiment you propose is impossible. You can't duplicate the behavior of neurons without tapping into quantum gravity. That said, there's not much point in me taking further part in this conversation." (Wanders away.)
Also worth noting that another logical possibility (which you sort of get at in footnote 9) is that the thought experiment does go through, and a human with silicon chips instead of neurons would still be conscious, but CF is still false. Maybe it's not the substrate but the spatial location of neurons that's relevant. ("Substrate-indendence" is not actually a super well-defined concept, either.)
If you do reject CF but do believe in realist consciousness, then it's interesting to consider what other property is the key factor for human consciousness. If you're also a physicalist, then whichever property that is probably has to play a significant computational role in the brain, otherwise you run into contradicts when you compare the brain with a system that doesn't have the property and is otherwise as similar as possible. Spatial location has at least some things going for it here (e.g., ephaptic coupling and neuron synchronization).
For example, consciousness isn’t a function under this view, it probably still plays a function in biology.[12] If that function is useful for future AI, then we can predict that consciousness will eventually appear in AI systems, since whatever property creates consciousness will be engineered into AI to improve its capabilities.
This is a decent argument for why AI consciousness will happen, but actually "AI consciousness is possible" is a weaker claim. And it's pretty hard to see how that weaker claim could ever be false, especially if one is a physicalist (aka what you call "materialist" in your assumptions of this post). It would imply that consciousness in the brain depends on a physical property, but that physical property is impossible to instantiate in an artificial system; that seems highly suspect.
but I don't need to be able to define fire to be able to point out something that definitely isn't on fire!
I guess I can see that. I just don't think that e.g. Mike Johnson would consider his argument refuted based on the post; I think he'd argue that the type of problems illustrated by the popcorn thought experiment are in fact hard (and, according to him, probably unsolvable). And I'd probably still invoke the thought experiment myself, too. Basically I continue to think they make a valid point, hence are not "refuted". (Maybe I'm being nit-picky? But I think the standards for claiming to have refuted sth should be pretty high.)
I think the point is valid, but (as I said here) I don't think this refutes the thought experiments (that purport to show that a wall/bag-of-popcorn/whatever is doing computation). I think the thought experiments show that it's very hard to objectively interpret computation, and your point is that there is a way to make nonzero progress on the problem by positing a criterion that rules out some interpretations
Imo it's kind of like if someone made a thought experiment arguing that AI alignment is hard because there's a large policy space, and you responded by showing that there's an impact measure by which not every policy in the space is penalized equally. It would be a coherent argument, but it would be strange to say that it "refuted" the initial thought experiment, even if the impact measure happened to penalize a particular failure mode that the initial thought experiment used as an example. A refutation would more be a complete solution to the alignment problem, which no one has -- just as no one has a complete solution to the problem of objectively interpreting computation.
(I also wonder if this post is legible to anyone not already familiar with the arguments? It seems very content-dense per number of words/asking a lot of the reader.)
although I’m not quite sure what to make of it
(Kinda figured this, almost decided not to post the comment.)
Also, just for context, do you think you’ve experienced PNSE? Thanks!
Yes with some caveats. I think I've experienced no-self, which is what you describe from 6.2 onward. But if you'd asked me how far I am to enlightenment, I'd have said maybe 15%. Which is to say, I think no-self is a real thing you can achieve (and I definitely think it's a net positive), but I think from there, the ladder goes way way higher. Like the "take one day vs. entire life" comment implies a goodness multiplier of at least 20000x compared to the minds of regular people. Even if we assume this is widely exaggerated (although people keep insisting that it's not) and that the real multiplier is two OOMs smaller, than that' still 200x, whereas I'd put no-self at somewhere between 1x and 2x. If someone did claim that just the no-self part gives you even a 200x multiplier (which I doubt the person in the twitter comment would say), then I'd just be scratching my head at that.
... which could be a sign that I'm delusional and haven't really experienced no-self, but I think my experience fits quite well with your description (less anxiety, less self-reflective thoughts, identification with everything in awareness, etc.). Actually I think no-self + flow state is really very similar to regular flow state, which is again why the multiplier can't be that high. So, yeah, in my model enlightenment and no-self are two radically different things, the first is way harder to achieve and presumably way way better, and I think I've experienced the second but I know I'm nowhere close to the first -- if in fact it exists, which I suspect it does. (Sorry for the rambly answer.)
Also, I also want to distinguish two aspects of an emotion. In one, “duration of an emotion” is kinda like “duration of wearing my green hat”. I don’t have to be thinking about it the whole time, but it’s a thing happening with my body, and if I go to look, I’ll see that it’s there. Another aspect is the involuntary attention. As long as it’s there, I can’t not think about it, unlike my green hat. I expect that even black-belt PNSE meditators are unable to instantly turn off anger / anxiety / etc. in the former sense. I think these things are brainstem reactions that can be gradually unwound but not instantly. I do expect that those meditators would be able to more instantly prevent the anger / anxiety / etc. from controlling their thought process. What do you think?
Agree with all this.
More specifically, I feel like when you say “apply equanimity to X”, you mean something vaguely like “do a specific tricky learned attention-control maneuver that has something to do with the sensory input of X”. That same maneuver could contribute to equanimity, if it’s applied to something like anxiety. But the maneuver itself is not what I would call “equanimity”. I
I don't think it feels that way. What it feels like is that, if you pick any item in awareness, there's by default a tension with that thing, which makes it feel lower valence. If you apply equanimity -- which as I said, I can best describe as 'try not to resist' -- then the apparent tension lessens. With pain, this like experiencing the pain but not suffering. With positive sensations, the best way I can describe it is that if you succeed in applying a decent amount of equanimity, you realize afterward that your enjoyment wasn't "pure" but was plagued by attachment/craving. A decent way to describe it is that "pleasure turns into fulfillment"; I think that's the term associated with good-sensations-that-have-no-craving-aspect. But in both cases they definitely become higher valence. And with neutral sensations it kinda still feels like you've removed craving or resistance, even though this doesn't particularly make sense. Anyway, it really doesn't feel like it's an attention-control maneuver, it feels like it's a property of the sensation.
I sorta have in mind a definition kinda like: neither very happy, nor very sad, nor very excited, nor very tired, etc. Google gives the example: “she accepted both the good and the bad with equanimity”
Imo meditators are often evasive when it comes to this topic and refuse to just say that meditation is supposed to make you feel better, even though it obviously does, and this is probably causally upstream of you writing this sentence. i think it's just because 'feeling better' is generally associated with 'feel more nice things', and trying to chase pleasures is the opposite of meditation; you're supposed to be content with what is (again, equanimity feels like not resisting; it's sometimes analogized to the inverse of friction in a mechanical system). So yeah, I mean, applying tons of equanimity doesn't make you feel more pleasure, but yeah it does feel really good/high-valence, just in a non-pleasur-y but fullfilment-y sense. (The one time I was on a formal retreat, the meditation teacher even complained when I mentioned that I had a goal for meditating, and I had to specify that this doesn't mean I'm thinking about the goal while meditating; tbqh imo many people are just kinda bad at differentiating these things, but it's really not that complicated.)
Hmm. I think that none of this refutes the point I was making, which is that practical CF as defined by OP is a position that many people actually hold,[1] hence OP's argument isn't just a strawman/missing the point. (Whether or not the argument succeeds is a different question.)
Well, I guess this discussion should really be focused more on personal identity than consciousness (OP wrote: “Whether or not a simulation can have consciousness at all is a broader discussion I'm saving for later in the sequence, and is relevant to a weaker version of CF.”).
I don't think you have to bring identity into this. (And if you don't have to, I'd strongly advise leaving it out because identity is another huge rabbit hole.) There's three claims with strictly increasing strength here: digital simulations can be conscious, a digital simulation of a brain exhibits similar consciousness to that brain, and if a simulation of my brain is created, then that simulation is me. I think only is about identity, and OP's post is arguing against . (All three claims are talking about realist consciousness.)
This is also why I don't think noise matters. Granting all of (A)-(D) doesn't really affect ; a practical simulation could work with similar noise and be pseudo-nondeterministic in the same way that the brain is. I think it's pretty coherent to just ask about how similar the consciousness is, under a realist framework (i.e., asking ), without stepping onto the identity hornets nest.
a caveat here is that it's actually quite hard to write down any philosophical position (except illusionism) such that a lot of people give blanket endorsements (again because everyone has slightly different ideas of what different terms mean), but I think OP has done a pretty good job, definitely better than most, in formulating an opinion that at least a good number of people would probably endorse. ↩︎
I think a lot of misunderstandings on this topic are because of a lack of clarity about what exact position is being debated/argued for. I think two relevant positions are
- (1) it is impossible to say anything meaningful whatsoever about what any system is computing. (You could write this semi-formally as physical system computation the claims " performs " and " performs " are equally plausible)
- (2) it is impossible to have a single, formal, universally applicable rule that tells you which computation a physical system is running that does not produce nonsense results. (I call the problem of finding such a rule the "interpretation problem", so (2) is saying that the interpretation problem is impossible)
(1) is a much stronger claim and definitely implies (2), but (2) is already sufficient to rule out the type of realist functionalism that OP is attacking. So OP doesn't have to argue (1). (I'm not sure if they would argue for (1), but they don't have to in order to make their argument.) And Scott Aaronson's essay is (as far as I can see) just arguing against (1) by proposing a criterion according to which a waterfall isn't playing chess (whereas stockfish is). So, you can agree with him and conclude that (1) is false, but this doesn't get you very far.
The waterfall thought experiment also doesn't prove (2). It's very hard to prove (2) because (2) is just saying that a problem cannot be solved, and it's hard to prove that anything can't be done. But the waterfall thought experiment is an argument for (2) by showing that the interpretation problem looks pretty hard. This is how I was using it in my reply to Steven on the first post of this sequence; I didn't say "and therefore, I've proven that no solution to the interpretation problem exists"; I've just pointed out that you start off with infinitely many interpretations and currently no one has figured out how to narrow it done to just one, at least not in a way such that the answers have the properties everyone is looking for.
One thing I worry about is that the same disagreement happens with a lot of other users who, unlike Steven, just downvote the post rather than writing a comment.
In general, when I've read through the entire LW catalogue of posts with the consciousness tag, I've noticed that almost all well received posts with the consciousness tag take what I call the camp #1 perspective (i.e., discuss consciousness from an illusionist lens, even if it's not always stated explicitly). Iirc the only major exceptions are the posts from Eliezer, which, well, are from Eliezer. So it could be that post who discuss consciousness from a realist PoV consistently receive certain amount of downvotes from camp #1 people to whom the post just seems gibberish/a waste of time. I don't have any data to prove that this is the mechanism, it's just a guess, but the pattern is pretty consistent. I also think you generally wouldn't predict this if you just read the comment sections. (And idk if clarifying the perspective would help since no one does it.)
I'm pretty poor right now so didn't donate, but I do generally believe that the Lightcone team has done a good job, overall, and is worth donating to.
I agree that “practical CF” as thus defined is false—indeed I think it’s so obviously false that this post is massive overkill in justifying it.
But I also think that “practical CF” as thus defined is not in fact a claim that computational functionalists tend to make.
The term 'functionalist' is overloaded. A lot of philosophical terms are overloaded, but 'functionalist' is the most egregiously overloaded of all philosophical terms because it refers to two groups of people with two literally incompatible sets of beliefs:
- (1) the people who are consciousness realists and think there's this well-defined consciousness stuff exhibited from human brains, and also that the way this stuff emerges depends on what computational steps/functions/algorithms are executed (whatever that means exactly)
- (2) the people who think consciousness is only an intuitive model, in which case functionalism is kinda trivial and not really a thing that can be proved or disproved, anyway
Unless I'm misinterpreting things here (and OP can correct me if I am), the post is arguing against (1), but you are (2), which is why you're talking past each other here. (I don't think this sequence in general is relevant to your personal views, which is what I also tried to say here.) In the definition you rephrased
would cause the same conscious experience as that brain, in the specific sense of thinking literally the exact same sequence of thoughts in the exact same order, in perpetuity.
... consciousness realists will read the 'thinking' part as referring to thinking in the conscious mind, not to thinking in the physical brain. So to you this reads obviously false to you because you don't think there is a conscious mind separate from the physical brain, and the thoughts in the physical brain aren't 'literally exactly the same' in the biological brain vs. the simulation -- obviously! But the (1) group does, in fact, believe in such a thing, and their position does more or less imply that it would be thinking the same thoughts.
I believe this is what OP is trying to gesture at as well with their reply here.
Fair enough; the more accurate response would have been that evolution might be an example, depending on how the theory was derived (which I don't know). Maybe it's not actually an example.
The crux would be when exactly he got the idea; if the idea came first and the examples later, then it's still largely analogous (imo); if the examples were causally upstream of the core idea, then not so much.
Can you give an example in another domain where philosophical reasoning about a topic led to empirical facts about that topic?
Yes -- I think evolution is a pretty clean example. Darwin didn't have any more facts than other biologists or philosophers, and he didn't derive his theory by collecting facts; he was just doing better philosophy than everyone else. His philosophy led to a large set of empirical predictions, those predictions were validated, and that's how and when the theory was broadly accepted. (Edit: I think that's a pretty accurate description of what happened, maybe you could argue with some parts of it?)
I think we should expect that consciousness works out the same way -- the problem has been solved, the solution comes with a large set of empirical predictions, it will be broadly accepted once the empirical evidence is overwhelming, and not before. (I would count camp #1 broadly as a 'theory' with the empirical prediction that no crisp divide between conscious and unconscious processing exists in the brain, and that consciousness has no elegant mathematical structure in any meaningful sense. I'd consider this empirically validated once all higher cognitive functions have been reverse-engineered as regular algorithms with no crisp/qualitative features separating conscious and unconscious cognition.)
(GPT-4o says that before evolution was proposed, the evolution of humans was considered a question of philosophy, so I think it's quite analogous in that sense.)
I see your point, but I don't think this undermines the example? Like okay, the 'life is not a crisp category' claim has nuance to it, but we could imagine the hypothetical smart philosopher figuring out that as well. I.e., life is not a crisp category in the territory, but it is an abstraction that's well-defined in most cases and actually a useful category because of this <+ any other nuance that's appropriate>.
It's true that the example here (figuring out that life isn't a binary/well-defined thing) is not as practically relevant as figuring out stuff about consciousness. (Nonetheless I think the property of 'being correct doesn't entail being persuasive' still holds.) I'm not sure if there is a good example of an insight that has been derived philosophically, is now widely accepted, and has clear practical benefits. (Free Will and implications for the morality of punishment are pretty useful imo, but they're not universally accepted so not a real example, and also no clear empirical predictions.)
Couldn't you imagine that you use philosophical reasoning to derive accurate facts about consciousness, which will come with insights about the biological/computational structure of consciousness in the human brain, which will then tell you things about which features are critical / how hard human brains are to simulate / etc.? This would be in the realm of "empirical predictions derived from philosophical reasoning that are theoretically testable but not practically testable".
I think most solutions to consciousness should be like that, although I'd grant that it's not strictly necessary. (Integrated Information Theory might be an example of a theory that's difficult to test even in principle if it were true.)
I'd call that an empirical problem that has philosophical consequences :)
That's arguable, but I think the key point is that if the reasoning used to solve the problem is philosophical, then a correct solution is quite unlikely to be recognized as such just because someone posted it somewhere. Even if it's in a peer-reviewed journal somewhere. That's the claim I would make, anyway. (I think when it comes to consciousness, whatever philosophical solution you have will probably have empirical consequences in principle, but they'll often not be practically measurable with current neurotech.)
What about if it's a philosophical problem that has empirical consequences? I.e., suppose answering the philosophical questions tells you enough about the brain that you know how hard it would be to simulate it on a digital computer. In this case, the answer can be tested -- but not yet -- and I still think you wouldn't know if someone had the answer already.
I don't know the answer, and I'm pretty sure nobody else does either.
I see similar statements all the time (no one has solved consciousness yet / no one knows whether LLMs/chickens/insects/fish are conscious / I can only speculate and I'm pretty sure this is true for everyone else / etc.) ... and I don't see how this confidence is justified. The idea seems to be that no one can't have found the correct answers to the philosophical problems yet because if they had, those answers would immediately make waves and soon everyone on LessWrong would know about it. But it's like... do you really think this? People can't even agree on whether consciousness is a well-defined thing or a lossy abstraction; do we really think that if someone had the right answers, they would automatically convince everyone of them?
There are other fields where those kinds of statements make sense, like physics. If someone finds the correct answer to a physics question, they can run an experiment and prove it. But if someone finds the correct answer to a philosophical question, then they can... try to write essays about it explaining the answer? Which maybe will be slightly more effective than essays arguing for any number of different positions because the answer is true?
I can imagine someone several hundred years ago having figured out, purely based on first-principles reasoning, that life is no crisp category at the territory but just a lossy conceptual abstraction. I can imagine them being highly confident in this result because they've derived it for correct reasons and they've verified all the steps that got them there. And I can imagine someone else throwing their hands up and saying "I don't know what mysterious force is behind the phenomenon of life, and I'm pretty sure no one else does, either".
Which is all just to say -- isn't it much more likely that the problem has been solved, and there are people who are highly confident in the solution because they have verified all the steps that led them there, and they know with high confidence which features need to be replicated to preserve consciousness... but you just don't know about it because "find the correct solution" and "convince people of a solution" are mostly independent problems, and there's just no reason why the correct solution would organically spread?
(As I've mentioned, the "we know that no one knows" thing is something I see expressed all the time, usually just stated as a self-evident fact -- so I'm equally arguing against everyone else who's expressed it. This just happens to the be first time that I've decided to formulate my objection.)
Sort of not obvious what exactly "causal closure" means if the error tolerance is not specified. We could differentiate literally 100% perfect causal closure, almost perfect causal closure, and "approximate" causal closure. Literally 100% perfect causal closure is impossible for any abstraction due to every electron exerting nonzero force on any other electron in its future lightcone. Almost perfect causal closure (like 99.9%+) might be given for your laptop if it doesn't have a wiring issue(?), maybe if a few more details are included in the abstraction. And then whether or there exists an abstraction for the brain with approximate causal closure (95% maybe?) is an open question.
I'd argue that almost perfect causal closure is enough for an abstraction to contain relevant information about consciousness, and approximate causal closure probably as well. Of course there's not really a bright line between those two, either. But I think insofar as OP's argument is one against approximate causal closure, those details don't really matter.
No software/hardware separation in the brain: empirical evidence
I feel like the evidence in this section isn't strong enough to support the conclusion. Neuroscience is like nutrition -- no one agrees on anything, and you can find real people with real degrees and reputations supporting just about any view. Especially if it's something as non-committal as "this mechanism could maybe matter". Does that really invalidate the neuron doctrine? Maybe if you don't simulate ATP, the only thing that changes is that you have gotten rid of an error source. Maybe it changes some isolated neuron firings, but the brain has enough redundancy that it basically computes the same functions.
Or even if it does have a desirable computational function, maybe it's easy to substitute with some additional code.
I feel like the required standard of evidence is to demonstrate that there's a mechanism-not-captured-by-the-neuron-doctrine that plays a major computational role, not just any computational role. (Aren't most people talking about neuroscience still basically assuming that this is not the case?)
We can expect natural selection to result in a web of contingencies between different levels of abstraction.[6]
Mhh yeah I think the plausibility argument has some merit.
As simon has already said, your math is wrong because the cases aren't equiprobable. For k=2 you can fix this by doubling the cases of H since they're twice as likely as the others (so the proper state space is with .) For you'd have to quadruple H and double HT, which would give 4x H and 4x HT and 4x HTT and 8x TTT I believe, leading to probability of TTT. (Up from 1x H, 2x HT, 4x HTT, 8x TTT.) In general, I believe the probability of only Ts approaches 0, as does the probability of H.
Regardless, are these the right betting odds? Yup! If we repeat this experiment for any and you are making a bet every time you wake up, then these are the odds according to which you should take or reject bets to maximize profit. You can verify this by writing a simulation, if you want.
If you make the experiment non-repeating, then I think this is just a version of the presumptuous philospoher argument which (imo) shows that you have to treat logical uncertainty differently from randomness (I addressed this case here).
I think there's a practically infinite number of edge cases. For a system to run the algorithm, it would have to perform a sequence of operations on natural numbers. If we simplify this a bit, we could just look at the values of the variables in the program (like a, x, y; I don't actually know the algorithm, I'm just looking at the pseudo-code on Wikipedia). If the algorithm is running, then each variable goes through a particular sequence, so we could just use this as a criterion and say the system runs the algorithm iff one of these particular sequences is instantiated.
Even in this simplified setting, figuring this out requires a mapping of physical states to numbers. If you start agreeing on a fixed mapping (like with a computer, we agree that this set of voltages at this location corresponds to the numbers 0-255), then that's possible to verify. But in general you don't know, which means you have to check whether there exists at least one mapping that does represent these sequences. Considered very literally, this is probably always true since you could have really absurd and discontinuous mappings (if this pebble here has mass between 0.5g and 0.51g it represents the number 723; if it's between 0.51g and 0.52g it represents 911...) -- actually you have infinitely many mappings even after you agree on how the system partitions into objects, which is also debatable.
So without any assumptions, you start with a completely intractable problem and then have to figure out how to deal with this (... allow only reasonable mappings? but what's reasonable? ...), which in practice doesn't seem like something anyone has been able to do. So even if I just show you a bunch of sand trickling through someone's hands, it's already a hard problem to prove that this doesn't represent the Miller-Rabin test algorithm. It probably represents some sequence of numbers in a not too absurd way. There are some philosophers who have just bitten the bullet and concluded that any and all physical systems compute, which is called panpcomputationalism. The only actually formal rule for figuring out a mapping that I know is from IIT, which is famously hated on LW (and also has conclusions that most people find absurd, such that digital computers can't be conscious at all bc the criterion ends up caring more about the hardware than the software). The thing is that most of the particles inside a computer don't actually change all that much depending on the program, it's really only a few specific locations where electrons move around, which is enough if we decide that our mapping only cares about those locations but not so much if you start with a rule applicable to arbitrary physical systems.
That all said, I think illusionists have the pretty easy out of just saying that computation is frame-dependent, i.e., that the answer to "what is this system computing" depends on the frame of reference, specifically the mapping from physical states to mathematical objects. It's really only a problem you must solve if you both think that (a) consciousness is well-defined, frame-invariant, camp #2 style, etc., and also (b) the consciousness of a system depends on what it computes.
I don't know if you are going to address this, but if I were to write a sequence of posts on functionalism, I'd start with the problem that "computation" isn't very well defined, and hence functionalism isn't very well-defined, either. In practice it's often clear enough whether or not a system is computing something, but you're going to have a hard time giving a fully general, rigorous, universally applicable definition of what exactly a physical process has to do to count as computing something (and if so, what precisely it is computing). Similarly, your definition of the Practical CF inherits this problem because it's not at all clear what "capturing the dynamics of the brain on some coarse-grained level of abstraction" means. This problem is usually brushed over but imo that's where all the difficulty lies.
(Of course, many people think consciousness is inherently fuzzy, in which case associating it with similarly fuzzy concepts isn't a problem. But I'm assuming you're taking a realist point of view here and assume consciousness is well-defined, since otherwise there's not much of a question to answer. If consciousness is just an abstraction, functionalism becomes vacuously true as a descriptive statement.)
Fwiw I was too much of a coward/too conflict averse to say anything myself but I agree with this critique. (As I've said in my post I think half of all people are talking about mostly the same thing when they say 'consciousness', which imo is fully compatible with the set of responses Andrew listed in his post given how they were collected.)
Schindler's List: we can talk about specific rationality lessons all day, but we all know the biggest bottleneck is trying in the first place. This movie is the transformation of an ethical egoist into a utilitarian.
It also shows the value of Money: the Unit of Caring.
I have my own benchmark of tasks that I think measure general reasoning to decide when I freak out about LLMs, and they haven't improved on them. I was ready to be cautiously optimistic that LLMs can't scale to AGI (and would have reduced by p(doom) slightly) even if they keep scaling by conventional metrics, so the fact that scaling itself also seems to break down (maybe, possibly, partially, to whatever extent it does in fact break down, I haven't looked into it much) and we're reaching physical limits are all good things.
I'm not particularly more optimistic about alignment working anytime soon, just about very long timelines.
(You did respond to all the important parts, rest of my comment is very much optional.)
I’m sure you’re aware that people feel like they have a broader continuous awareness of their visual field than they actully do. There are lots of demonstrations of this—e.g. change blindness, selective attention test, the fact that peripheral vision has terrible resolution and terrible color perception and makes faces look creepy. There’s a refrigerator light illusion thing—if X is in my peripheral vision, then maybe it’s currently active as just a little pointer in a tiny sub-area of my cortex, but as soon as I turn my attention to X it immediately unfolds in full detail across the global workspace.
Yes -- and my point was that appealing to these phenomena is the kind of thing you will probably have to do to explain the meta problem of seeing. Which raises all kinds of issues -- for example, change blindness by itself doesn't logically prove anything, since it's possible not to notice that something changed even if it was represented. Only the reverse conclusion is valid -- if a subject can tell that X changed, then X was in awareness, but if they can't tell, X may or may not have been in awareness. So teasing out exactly how much information is really present in awareness, given the positive and negative evidence, is a pretty big rabbit hole. (Poor resolution in peripheral vision does prove absence of information, but as with the memory example I've complained about in post #2, this is an example of something people don't endorse under reflection anyway, so it doesn't get you very far. Like, there is a very, very big difference between arguing that peripheral resolution is poor, which people will agree with as soon as they actually pay attention to their peripheral vision for the first time, and arguing that the continuous visual image they think they see is not really there, which most people will stubbornly disagree with regardless of how much attention they pay to it.)
Anyway, that's the only claim I was making -- I was only trying to go as far as "this is why I think the problem is nontrivial and you haven't solved it yet", not "and that's why you can't solve it".
The contents of IT are really truly different from the contents of LIP [I didn’t check where the visual information gets to the cortex in blindsight, I’m just guessing LIP for concreteness]. Querying IT is a different operation than querying LIP. IT holds different types of information than LIP does, and does different things with that information, including leading to different visceral reactions, motivations, semantic knowledge, etc., all of which correspond to neuroscientific differences in how IT versus LIP is wired up.
All these differences between IT vs LIP are in the territory, not the map. So I definitely agree that “the distinction [between seeing and vague-sense-of-presence] isn’t just that we happen to call them by different labels”. They’re different like how the concept “hand” is different from the concept “foot”—a distinction on the map downstream of a distinction in the territory.
Right, and I agree that this makes it apriori plausible that they could account for the differences in how people talk about, e.g., vivid seeing vs. intangible intuitions. But it doesn't prove that they do, it only shows that this is the kind of explanation that, on first glance, looks like it could work. To actually solve the meta problem, you still have to do the work of explaining all the properties of introspective reports, which requires going into a lot of detail.
As of above, this is the only claim I was making -- I'm not saying any of these issues are provably impossible with your approach, I'm only saying that your approach hasn't provided a full solution yet. (And that I genuinely think most of the difficulty happens to be in these still unaddressed details; this was the point of the carrot/plant analogy.)
I think that’s compatible with my models, because those meditators still have a cortex, in which patterns of neurons can be firing or not firing at any particular time. And that’s the core aspect of the “territory” which corresponds to “conscious awareness” in the “map”. No amount of meditation, drugs, etc., can change that.
Fair enough, but I think it does show that free will isn't that central of a piece.
The way intuitive models work (I claim) is that there are concepts, and associations / implications / connotations of those concepts. There’s a core intuitive concept “carrot”, and it has implications about shape, color, taste, botanical origin, etc. And if you specify the shape, color, etc. of a thing, and they’re somewhat different from most normal carrots, then people will feel like there’s a question “but now is it really a carrot?” that goes beyond the complete list of its actual properties. But there isn’t, really. Once you list all the properties, there’s no additional unanswered question. It just feels like there is. This is an aspect of how intuitive models work, but it doesn’t veridically correspond to anything of substance.
Mhhhmhh. Let me see if I can work with the carrot example to where it fits my view of the debate.
A botanist is charged with filling a small field with plants, any plants. A chemist hands him a perfect plastic replica of a carrot, perfect in shape, color, texture, and (miraculously) taste. The botanist says that it's not a plant. The chemist, who has never seen plants other than carrots, points out the matching qualities to the plants he knows. The botanist says okay but those are just properties that a particular kind of plant happens to have, they're not the integral property of what makes something a plant. "The core intuitive concept 'plant' has implications about shape, color, texture, taste, et cetera", says the chemist. "If all those properties are met, people may think there's an additional question about the true plant-ness of the object, but [...]." The botanist points out that he is not talking about an intangible, immeasurable, or non-physical property but rather about the fact that this carrot won't grow and spread seeds when planted into the earth. The chemist, having conversed extensively with people who define plants primarily by their shape, color, texture, and taste (which are all those of carrots because they've also not seen other plants) just sighs, rolling his eyes at the attempt to redefine plant-ness to be entirely about this one obscure feature that also just happens to be the most difficult one to test.
Which is to say that I get -- or at least I think I get -- the sense that we're successfully explaining important features of consciousness and the case for linking it to anything special is clearly diminishing -- but I don't think it's correct. When I say that the hard meta problem of seeing probably contains ~90% of the difficulty of the hard meta problem of consciousness whereas the meta problem of free will contains 0% and the problem of awareness ~2%, then I'm not changing my model in response to new evidence. I've always thought Free Will was nonsense!
(The botanist separately points out that there in fact other plants with different shape, texture, and taste, although they all do have green leaves, to which the chemist replies that ?????. This is just to come back to the point that people report advanced meditative states that lose many of the common properties of consciousness, including Free Will, the feeling of having a self (I've experienced that one!) and even the presence of any information content whatsoever, and afaik they tend to be more "impressed", roughly speaking, with consciousness as a result of those experiences, not less.)
[seeing stuff]
Attempt to rephrase: the brain has several different intuitive models in different places. These models have different causal profiles, which explains how they can correspond to different introspective reports. One model corresponds to the person talking about smelling stuff. Another corresponds to the person talking about seeing stuff. Yet another corresponds to the person talking about obtaining vague intuitions about the presence and location of objects. The latter two are triggered by visual inputs. Blindsight turns off the second but not the third.
If this is roughly correct, my response to it is that proposing different categories isn't enough because the distinction between visually vivid experience and vague intuitions isn't just that we happen to call them by different labels. (And the analogous thing is true for every other sensory modality, although the case is the least confusing with vision.) Claiming to see a visual image is different from claiming to have a vague intuition in all the ways that it's different; people claim to see something made out of pixels, which can look beautiful or ugly, seems to have form, depth, spatial location, etc. They also claim to perceive a full visual image constantly, which presumably isn't possible(?) since it would contain more information than can actually be there, so a solution has to explain how this illusion of having access to so much information is possible. (Is awareness really a serial processor in any meaningful way if it can contain as much information at once as a visual image seems to contain?)
(I didn't actually intend to get into a discussion about any of this though, I was just using it as a demonstration of why I think the hard metaproblem of consciousness has at least one real subset and hence isn't empty.)
Hard Problem
Yeah, I mean, since I'm on board with reducing everything to the meta problem, the hard problem itself can just be sidestepped entirely.
But since you brought it up, I'll just shamelessly use this opportunity to make a philosophical point that I've never seen anyone else make, which is that imo the common belief that no empirical data can help distinguish an illusionist from a realist universe... is actually false! The reason is that consciousness is a high-level phenomenon in the illusionist universe and a low phenomenon in at least some versions of the realist universe, and we have different priors for how high-level vs. low-level phenomena behave.
The analogy I like is, imagine there's a drug that makes people see ghosts, and some think these ghosts tap into the fundamental equations of physics, whereas others think the brain is just making stuff up. One way you can go about this is to have a thousand people describe their ghosts in detail. If you find that the brightness of hallucinated ghosts is consistently proportional to their height, then you've pretty much disproved the "the brain is just making stuff up hypothesis". (Whereas if you find no such relationships, you've strengthened the hypothesis.) This is difficult to operationalize for consciousness, but I think determining the presence of absence of elegant mathematical structure within human consciousness is, at least in principle, an answer to the question of "[w]hat would progress on the 'breathes fire' question even look like".
I think this post fails as an explanation of equanimity. Which, of course, is dependent on my opinion about how equanimity works, so you have a pretty easy response of just disputing that the way I think equanimity works is correct. But idk what to do about this, so I'll just go ahead with a critique based on how I think equanimity works. So I'd say a bunch of things:
-
Your mechanism describes how PNSE or equanimity leads to a decrease in anxiety via breaking the feedback loop. But equanimity doesn't actually decrease the severity of an emotion, it just increases the valence! It's true that you can decrease the emotion (or reduce the time during which you feel it), but imE this is an entirely separate mechanism. So between the two mechanisms of (a) decreasing the duration of an emotion (presumably by breaking the feedback loop) and (b) applying equanimity to make it higher valence, I think you can vary each one freely independent of the other. You could do a ton of (a) with zero (b), a ton of (b) with zero (a), a lot of both, or (which is the default state) neither.
-
Your mechanism mostly applies to mental discomfort, but equanimity is actually much easier to apply to physical pain. You can also apply it to anxiety, but it's very hard. I can reduce suffering from moderately severe physical pain on demand (although there is very much a limit) and ditto with itching sensations, but I'm still struggling a lot with mental discomfort.
-
You can apply equanimity to positive sensations and it makes them better! This is a point I'd emphasize the most because imo it's such a clear and important aspect of how equanimity works. One of the ways to feel really really good is to have a pleasant sensation, like listening to music you love, and then applying maximum equanimity to it. I'm pretty sure you can enter the first jhana this way (although to my continuous disappointment I've never managed to reach the first jhana with music, so I can't guarantee it.)
... actually, you can apply equanimity to literally any conscious percept. Like literally anything; you can apply equanimity to the sense of space around you, or to the blackness in your visual field, or to white noise (or any other sounds), or to the sensation of breathing. The way to do this is hard to put into words (similar to how an elementary motor command like lifting a finger is hard to put into words); the way it's usually described is by trying to accept/not fight a sensation. (Which imo is problematic because it sounds like equanimity means stopping to do something, when I'm pretty sure it's actively doing something. Afaik there are ~zero examples of animals who learn to no longer care about pain, so it very much seems like the default is that pain is negative valence, and applying equanimity is an active process that increases valence.)
I mean again, you can just say you've talked about something else using the same term, but imo all of the above are actually not that difficult to verify. At least for me, it didn't take me that long to figure out how to apply equanimity to minor physical pain, and from there, everything is just a matter of skill to do it more -- it's very much a continuous scale of being able to apply more and more equanimity, and I think the limit is very high -- and of realizing that you can just do same thing wrt sensations that don't have negative valence in the first place.
After finishing the sequence, I'm in the odd position where most of my thoughts aren't about the sequence itself, but rather about why I think you didn't actually explain why people claim to be conscious. So it's strange because it means I'm gonna talk a whole bunch about what you didn't write about, rather than what you did write about. I do think it's still worth writing this comment, but with the major disclaimer/apology that I realize most of this isn't actually a response to the substance of your arguments.
First to clarify, the way I think about this is that there's two relevant axes along which to decompose the problem of consciousness:
- the easy vs. hard axis, which is essentially about the describing the coarse functional behavior vs. why it exists at all; and
- the [no-prefix] vs. meta axis, which is about explaining the thing itself vs. why people talk about the thing. So for every , the meta problem of is "explain why people talk about "
(So this gives four problems: the easy problem, the hard problem, the easy meta problem, and the hard meta problem.)
I've said in this comment that I'm convinced the meta problem is sufficient to solve the entire problem. And I very much stand by that, so I don't think you have to solve the hard problem -- but you do have to solve the hard meta problem! Like, you actually have to explain why people claim to be conscious, not just why they report the coarse profile of functional properties! And (I'm sure you see where this is going), I think you've only addressed the easy meta problem throughout this sequence.
Part of the reason why this is relevant is because you've said in your introductory post that you want to address this (which I translate to the meta problem in my terminology):
STEP 1: Explain the chain-of-causation in the physical universe that leads to self-reports about consciousness, free will, etc.—and not just people’s declarations that those things exist at all, but also all the specific properties that people ascribe to those things.
Imo you actually did explain why people talk about free will,[1] so you've already delivered on at least half of this. Which is just to say that, again, this is not really a critique, but I do think it's worth explaining why I don't think you've delivered on the other half.
Alright, so why do I think that you didn't address the hard meta problem? Well, post #2 is about conscious awareness so it gets the closest, but you only really talk about how there is a serial processing stream in the brain whose contents roughly correspond to what we claim is in awareness -- which I'd argue is just the coarse functional behavior, i.e., the macro problem. This doesn't seem very related to the hard meta problem because I can imagine either one of the problems not existing without the other. I.e., I can imagine that (a) people do claim to be conscious but in a very different way, and (b) people don't claim to be conscious, but their high-level functional recollection does match the model you describe in the post. And if that's the case, then by definition they're independent.
A possible objection to the above would be that the hard and easy meta problem aren't really distinct -- like, perhaps people do just claim to be conscious because they have this serial processing stream, and attempts to separate the two are conceptually confused...
... but I'm convinced that this isn't true. One reason is just that, if you actually ask camp #2 people, I think they'll tell you that the problem isn't really about the macro functional behavior of awareness. But the more important reason is the hard meta problem can be considered in just a single sensory modality! So for example, with vision, there's the fact that people don't just obtain intangible information about their surroundings but claim to see continuous images.
Copying the above terminology, we could phrase the hard problem of seeing as explaining why people see images, and the hard meta problem of seeing as explaining why people claim to see images.[2] (And once again, I'd argue it's fine/sufficient to only answer the meta problem -- but only if you do, in fact, answer the meta problem!) Then since the hard meta problem of seeing is a subset of the hard meta problem of consciousness, and since the contents of your post very much don't say anything about this, it seems like they can't really have conclusively addressed the hard meta problem in general.
Again, not really a critique of the actual posts; the annoying thing for me is just that I think the hard meta problem is where all the juicy insights about the brain are hidden, so I'm continuously disappointed that no one talks about it. ImE this is a very consistent pattern where whenever someone says they'll talk about it, they then end up not actually talking it, usually missing it even more than you did here (cough Dennett cough). Actually there is at least one phenomenon you do talk about that I think is very interesting (namely equanimity), but I'll make a separate comment for that.
Alas I don't view Free Will as related to consciousness. I understand putting them into the same bucket of "intuitive self-models with questionable veridicality". But the problem is that people who meditate -- which arguably is like paying more attention -- tend to be less likely to think Free Will is real, but I'd strongly expect that they're more likely to say that consciousness is real, rather than less. (GPT-4 says there's no data on this; would be very interesting to make a survey correlating camp#1 vs. camp#2 views by how much someone has meditated, though proving causation will be tricky.) If this is true, imo they don't seem to belong into the same category. ↩︎
Also, I think the hard meta problem of seeing has the major advantage that people tend to agree it's real -- many people claim not to experience any qualia, but everyone seems to agree that they seem to see images. Basically I think talking about seeing is just a really neat way to reduce conceptual confusion while retaining the hard part of the problem. And then there's also blindsight where people claim not to see and retain visual processing capabilities -- but much very much reduced capabilities! -- so there's some preliminary evidence that it's possible to tease out the empirical/causal effects of the hard meta problem. ↩︎
Feeling better about this prediction now fwiw. (But I still don't want to justify this any further since I think progress toward AGI bad and LLMs little progress toward AGI, and hence more investment into LLMs probably good.)
You should be good (though I have only bet once; haven't withdrawn yet, so can't guarantee it). I think the gist of it is that Polymarket uses layer 2 and so is cheaper.
Feels empirically true. I remember cases where I thought about a memory and was initially uncertain about some aspect of it, but then when I think about it later it feels either true or false in my memory, so I have to be like, "well no, I know that I was 50/50 on this when the memory was more recent, so that's what my probability should be now, even though it doesn't feel like it anymore".
Seems like the fact that I put about a 50% probability on the thing survived (easy/clear enough to remember), but the reasons did not, so the probability no longer feels accurate.
That's what I mean (I'm talking about the input/output behavior of individual neurons).
Ah, I see. Nvm then. (I misunderstood the previous comment to apply to the entire brain -- idk why, it was pretty clear that you were talking about a single neuron. My bad.)
Nice; I think we're on the same page now. And fwiw, I agree (except that I think you need just a little more than just "fire at the same time"). But yes, if the artificial neurons affect the electromagnetic field in the same way -- so not only fire at the same time, but with precisely the same strength, and also have the same level of charge when they're not firing -- then this should preserve both communication via synaptic connections and gap junctions, as well as any potential non-local ephaptic coupling or brain wave shenanigans, and therefore, the change to the overall behavior of the brain will be so minimal that it shouldn't affect its consciousness. (And note that concerns the brain's entire behavior, i.e., the algorithm it's running, not just its input/output map.)
If you want to work more on this topic, I would highly recommend trying to write a proof for why simulations of humans on digital computers must also be conscious -- which, as I said in the other thread, I think is harder than the proof you've given here. Like, try to figure out exactly what assumptions you do and do not require -- both assumptions about how consciousness works and how the brain works -- and try to be as formal/exact as possible. I predict that actually trying to do this will lead to genuine insights at unexpected places. No one has ever attempted this on LW (or at least there were no attempts that are any good),[1] so this would be a genuinely novel post.
I'm claiming this based on having read every post with the consciousness tag -- so I guess it's possible that someone has written something like this and didn't tag it, and I've just never seen it. ↩︎
The fact that it's so formalized is part of the absurdity of IIT. There are a bunch of equations that are completely meaningless and not based in anything empirical whatsoever. The goal of my effort with this proof, regardless of whether there is a flaw in the logic somewhere, is that I think if we can take a single inch forward based on logical or axiomatic proofs, this can begin to narrow down our sea of endless speculative hypotheses, then those inches matter.
I'm totally on board with everything you said here. But I didn't bring up IIT as a rebuttal to anything you said in your post. In fact, your argument about swapping out neurons specifically avoids the problem I'm talking about in this above comment. The formalism of IIT actually agrees with you that swapping out neurons in a brain doesn't change consciousness (given the assumptions I've mentioned in the other comment)!
I've brought up IIT as a response to a specific claim -- which I'm just going state again since I feel like I keep getting misunderstood as making more vague/general claims than I'm in fact making. The claim (which I've seen made on LW before) is that we know for a fact that a simulation of a human brain on a digital computer is conscious because of the Turing thesis. Or at least, that we know this for a fact if we assume some very basic things about the universe like laws of physics are complete and functionalism is true. So like, the claim is that every theory of consciousness that agrees with these two premises also states that a simulation of a human brain has the same consciousness as that human brain.
Well, IIT is a theory that agrees with both of these premises -- it's a functionalist proposal that doesn't postulate any violation to the laws of physics -- and it says that simulations of human brains have completely different consciousness than human brains themselves. Therefore, the above claim doesn't seem true. This is my point; no more, no less. If there is a counter-example to an implication , then the implication isn't true; it doesn't matter if the counter-example is stupid.
Again, does not apply to your post because you talked about swapping neurons in a brain, which is different -- IIT agrees with your argument but disagrees with green_leaf's argument.
I don't see how it's an assumption. Are we considering that the brain might not obey the laws of physics?
If you consider the full set of causal effects of a physical object, then the only way to replicate those exactly is with the same object. This is just generally true; if you change anything about an object, this has changes to the particle structure, and that comes with measurable changes. An artificial neuron is not going to have exactly 100% the same behavior as a biological neuron.
This is why I made the comment about the plank of wood -- it's just to make the point that, in general, across all physical processes, substrate is causally relevant. This is a direct implication of the laws of physics; every particle has a continuous effect that depends on its precise location, any two objects have particles in different places, so there is no such thing as having a different object that does exactly the same thing.
So any step like "we're going to take out this thing and then replace it with a different thing that has the same behavior" makes assumptions about the structure of the process. Since the behavior isn't literally the same, you're assuming that the system as a whole is such that the differences that do exist "fizzle out". E.g., you might assume that it's enough to replicate the changes to the flow of current, whereas the fact the new neurons have a different mass will fizzle out immediately and not meaningfully affect the process. (If you read my initial post, this is what I was getting at with the abstraction description thing; I was not just making a vague appeal to complexity.)
it seemed that part of your argument is that the neuron black box is unimplementable
Absolutely not; I'm not saying that any of these assumptions are wrong or even hard to justify. I'm just pointing out that this is, in fact, an assumption. Maybe this is so pedantic that it's not worth mentioning? But I think if you're going to use the word proof, you should get even minor assumptions right. And I do think you can genuinely prove things; I'm not in the "proof is too strong a word for anything like this" camp. So by analogy, if you miss a step in a mathematical proof, you'd get points deducted even if the thing you're proving is still true, and even if the step isn't difficult go get right. I really just want people to be more precise when they discuss this topic.
Also, here's a sufficient reason why this isn't true. As far as I know, Integrated Information Theory is currently the only highly formalized theory of consciousness in the literature. It's also a functionalist theory (at least according to my operationalization of the term.) If you apply the formalism of IIT, it says that simulations on classical computers are minimally conscious at best, regardless of what software is run.
Now I'm not saying IIT is correct; in fact, my actual opinion on IIT is "100% wrong, no relation how consciousness actually works". But nonetheless, if the only formalized proposal for consciousness doesn't have the property that simulations preserve consciousness, then clearly the property is not guaranteed.
So why does IIT not have this property? Well because IIT analyzes the information flow/computational steps of a system -- abstracting away the physical details, which is why I'm calling it functionalist -- and a simulation of a system performs completely different computational steps than the original system. I mean it's the same thing I said in my other reply; a simulation does not do the same thing as the thing it's simulating, it only arrives at the same outputs, so any theory looking at computational steps will evaluate them differently. They're two different algorithms/computations/programs, which is the level of abstraction that is generally believed to matter on LW. Idk how else to put this.