Posts
Comments
I don't think so. According to Many Worlds, all weights exist, so there's no uncertainty in the territory -- and I don't think there's a good reason to doubt Many Worlds.
I dispute the premise. Weights of quantum configurations are not probabilities, they just share some superficial similarities. (They're modeled with complex numbers!) Iirc Eliezer was very clear about this point in the quantum sequence.
(Self-Review.)
I still endorse every claim in this post. The one thing I keep wondering is whether I should have used real examples from discussion threads on LessWrong to illustrate the application of the two camp model, rather than making up a fictional discussion as I did in the post. I think that would probably help, but it would require singling out someone and using them as a negative example, which I don't want to do. I'm still reading every new post and comment section about consciousness and often link to this post when I see something that looks like miscommunication to me; I think that works reasonably well.
However, I did streamline the second half of the post (took out the part about modeling the brain as a graph, I don't think that was necessary to make the point about research) and added a new section about terminology. I think that should make it a little easier to diagnose when the model is relevant in real discussions.
Not that one; I would not be shocked if this market resolves Yes. I don't have an alternative operationalization on hand; would have to be about AI doing serious intellectual work on real problems without any human input. (My model permits AI to be very useful in assisting humans.)
Gotcha. I'm happy to offer 600 of my reputation points vs. 200 of yours on your description of 2026-2028 not panning out. (In general if it becomes obvious[1] that we're racing toward ASI in the next few years, then people should probably not take me seriously anymore.)
well, so obvious that I agree, anyway; apparently it's already obvious to some people. ↩︎
I feel like a bet is fundamentally unfair here because in the cases where I'm wrong, there's a high chance that I'll be dead anyway and don't have to pay. The combination of long timelines but high P(doom|AGI soon) means I'm not really risking my reputation/money in the way I'm supposed to with a bet. Are you optimistic about alignment, or does this asymmetry not bother you for other reasons? (And I don't have the money to make a big bet regardless.)
Just regular o1, I have the 20$/month subscription not the 200$/month
You could call them logic puzzles. I do think most smart people on LW would get 10/10 without too many problems, if they had enough time, although I've never tested this.
About two years ago I made a set of 10 problems that imo measure progress toward AGI and decided I'd freak out if/when LLMs solve them. They're still 1/10 and nothing has changed in the past year, and I doubt o3 will do better. (But I'm not making them public.)
Will write a reply to this comment when I can test it.
- Because if you don't like it you can always kill yourself and be in the same spot as the non-survival case anyway.
Not to get too morbid here but I don't think this is a good argument. People tend not to commit suicide even if they have strongly net negative lives
My probably contrarian take is that I don't think improvement on a benchmark of math problems is particularly scary or relevant. It's not nothing -- I'd prefer if it didn't improve at all -- but it only makes me slightly more worried.
The Stanford Enyclopedia thing is a language game. Trying to make deductions in natural language about unrelated statements is not the kind of thing that can tell you what time is, one way or another. It can only tell you something about how we use language.
But also, why do we need an argument against presentism? Presentism seems a priori quite implausible; seems a lot simpler for the universe to be an unchanging 4d block than a 3d block that "changes over time", which introduces a new ontological primitive that can't be formalized. I've never seen a mathematical object that changes over time, I've only seen mathematical objects that have internal axes.
This all seems correct. The one thing I might add is that imE the usual effect of stating, however politely, that someone may not be 100% acting in good faith is to turn the conversation into much more of a conflict than it already was, which is why pretending as if it's an object level disagreement is almost always the correct strategy. But I agree that actually believing the other person is acting in good faith is usually quite silly.
(I also think the term is horrendous; irrc I've never used either "good faith" or "bad faith" in conversation.)
((This post also contributes to this nagging sense that I sometimes have that Zack is the ~only person on this platform who is actually doing rationality in a completely straight-forward way as intended, and everyone else is playing some kind of social game in which other considerations restrict the move set and rationality is only used to navigate within the subset of still permissible moves. I'm not in the business of fighting this battle, but in another timeline maybe I would be.))
Yeah, e.g., any convergent series.
This is assuming no expression that converges to the constants exists? Which I think is an open question. (Of course, it would only be finite if there are such expressions for all constants. But even so, I think it's an open question.)
As someone who expects LLMs to be a dead end, I nonetheless think this post makes a valid point and does so using reasonable and easy to understand arguments. I voted +1.
As I already commented, I think the numbers here are such that the post should be considered quite important even though I agree that it fails at establishing that fish can suffer (and perhaps lacks comparison to fish in the wild). If there was another post with a more nuanced stance on this point, I'd vote for that one instead, but there isn't. I think fish wellbeing should be part of the conversation more than it is right now.
It's also very unpleasant to think or write about these things, so I'm also more willing to overlook flaws than I'd be by default.
Shape can most certainly be emulated by a digital computer. The theory in the paper you linked would make a brain simulation easier, not harder, and the authors would agree with that
Would you bet on this claim? We could probably email James Pang to resolve a bet. (Edit: I put about 30% on Pang saying that it makes simulation easier, but not necessarily 70% on him saying it makes simulation harder, so I'd primarily be interested in a bet if "no idea" also counts as a win for me.)
It is not proposing that we need to think about something other than neuronal axons and dendrites passing information, but rather about how to think about population dynamics.
Really? Isn't the shape of the brain something other than axons and dendrites?
The model used in the paper doesn't take any information about neurons into account, it's just based on a mesh of the geometry of the particular brain region.
So this is the opposite of proposing a more detailed model of brain function is necessary, but proposing a courser-grained approximation.
And they're not addressing what it would take to perfectly understand or reproduce brain dynamics, just a way to approximately understand them.
The results (at least the flagship result) are about a coarse approximation, but the claim that anatomy restricts function still seems to me like contradicting the neuron doctrine.
Admittedly the neuron doctrine isn't well-defined, and there are interpretations where there's no contradiction. But shape in particular is a property that can't be emulated by digital computers, so it's a contradiction as far as the OP goes (if in fact the paper is onto something).
I mean, we have formalized simplicity metrics (Solomonoff Induction, minimal description length) for a reason, and that reason is that we don't need to rely on vague intuitions to determine whether a given theory (like wave function collapse) is plausible.
No reputable neuroscientist argued against it to any strong degree, just for additional supportive methods of information transmission.
I don't think this is correct. This paper argues explicitly against the neuron doctrine (enough so that they've put it into the first two sentences of the abstract), is published in a prestigious journal, has far above average citation count, and as far as I can see, is written by several authors who are considered perfectly fine/serious academics. Not any huge names, but I think enough to clear the "reputable" bar.
I don't think this is very strong evidence since I think you can find people with real degrees supporting all sorts of contradicting views. So I don't think it really presents an issue for your position, just for how you're phrased it here.
Two thoughts here
-
I feel like the actual crux between you and OP is with the claim in post #2 that the brain operates outside the neuron doctrine to a significant extent. This seems to be what your back and forth is heading toward; OP is fine with pseudo-randomness as long as it doesn't play a nontrivial computational function in the brain, so the actual important question is not anything about pseudo-randomness but just whether such computational functions exist. (But maybe I'm missing something, also I kind of feel like this is what most people's objection to the sequence 'should' be, so I might have tunnel vision here.)
-
(Mostly unrelated to the debate, just trying to improve my theory of mind, sorry in advance if this question is annoying.) I don't get what you mean when you say stuff like "would be conscious (to the extent that I am), and it would be my consciousness (to a similar extent that I am)," since afaik you don't actually believe that there is a fact of the matter as to the answers to these questions. Some possibilities what I think you could mean
- I don't actually think these questions are coherent, but I'm pretending as if I did for the sake of argument
- I'm just using consciousness/identity as fuzzy categories here because I assume that the realist conclusions must align with the intuitive judgments (i.e., if it seems like the fuzzy category 'consciousness' applies similarly to both the brain and the simulation, then probably the realist will be forced to say that their consciousness is also the same)
- Actually there is a question worth debating here even if consciousness is just a fuzzy category because ???
- Actually I'm genuinely entertaining the realist view now
- Actually I reject the strict realist/anti-realist distinction because ???
I think causal closure of the kind that matters here just means that the abstract description (in this case, of the brain as performing an algorithm/computation) captures all relevant features of the the physical description, not that it has no dependence on inputs. Should probably be renamed something like "abstraction adequacy" (making this up right now, I don't have a term on shelf for this property). Abstraction (in)adequacy is relevant for CF I believe (I think it's straight-forward why?). Randomness probably doesn't matter since you can include this in the abstract description.
You're using 'bridging law' differently from I was, so let me rephrase.
To explain subjective experience, you need bridging-laws-as-you-define-them. But it could be that consciousness is functional and the bridging laws are implicit in the description of the universe, rather than explicit. Differently put, the bridging laws follow as a logical consequences of how the remaining universe is defined, rather than being an additional degree of freedom.[1]
In that case, since bridging laws do not add to the length of the program,[2] Solomonoff Induction will favor a universe in which they're the same for everyone, since this is what happens by default (you'd have a hard time imagining that bridging laws follow by logical necessity but are different for different people). In fact, there's a sense in which the program that SI finds is the same as the program SI would find for an illusionist universe; the difference is just about whether you think this program implies the existence of implicit bridging laws. But in neither cases is there an explicit set of bridging laws that add to the length of the program.
Most of Eliezer's anti zombie sequence, especially Zombies Redacted can be viewed as an argument for bridging laws being implicit rather than explicit. He phrases this as "consciousness happens within physics" in that post. ↩︎
Also arguable but something I feel very strongly about; I have an unpublished post where I argue at length that and why logical implications shouldn't increase program length in Solomonoff Induction. ↩︎
I reacted locally invalid (but didn't downvote either comment) because I think "computation" as OP is using it is about the level of granularity/abstraction at which consciousness is located, and I think it's logically coherent to believe both (1) materialism[1] and (2) consciousness is located at a fundamental/non-abstract level.
To make a very unrealistic analogy that I think nonetheless makes the point: suppose you believed that all ball-and-disk integrators were conscious. Do you automatically believe that consciousness can be defined with a computation? Not necessarily -- you could have a theory according to which a digital computer computing the same integrals is not consciousness (since, again, consciousness is about the fine-grained physical steps, rather than the abstracted computational steps, and a digital computer calculating performs very different physical steps than a ball-and-disk integrator doing the same). The only way you now care about "computation" is if you think "computation" does refer to low-level physical steps. In that case, your implication is correct, but this isn't what OP means, and OP did define their terms.
as OP defines the term; in my terminology, materialism means something different ↩︎
If consciousness is not functional, then Solomonoff induction will not predict it for other people even if you assert it for yourself.
Agreed. But who's saying that consciousness isn't functional? "Functionalism" and "functional" as you're using it are similar sounding words, but they mean two different things. "Functionalism" is about locating consciousness on an abstracted vs. fundamental level. "Functional" is about consciousness being causally active vs. passive.[1] You can be a consciousness realist, think consciousness is functional, but not a functionalist.
You can also phrase the "is consciousness functional" issue as the existence or non-existence of bridging laws (if consciousness is functional, then there are no bridging laws). Which actually also means that Solomonoff Induction privileges consciousness being functional, all else equal (which circles back to your original point, though of course you can assert that consciousness being functional is logically incoherent and then it doesn't matter if the description is shorter).
I would frame this as dual-aspect monism [≈ consciousness is functional] vs. epiphenomenalism [≈ consciousness is not functional], to have a different sounding word. Although there are many other labels people use to refer to either of the two positions, especially for the first, these are just what I think are clearest. ↩︎
In Solomonoff induction, the mathematical formalization of Occam's razor, it's perfectly legitimate to start by assuming your own phenomenal experience (and then look for hypotheses that would produce that, such as the external world plus some bridging laws). But there's no a priori reason those bridging laws have to apply to other humans.
You can reason that a universe in which you are conscious and everyone else is not is more complex than a universe in which everyone is equally conscious, therefore Solomonoff Induction privileges consciousness for everyone.
Chalmer’s fading qualia [...]
Worth noting that Eliezer uses this argument as well in The Generalized Anti-Zombie Principle, as the first line of his Socrates Dialogue (I don't know if he has it from Chalmers or thought of it independently):
Albert: "Suppose I replaced all the neurons in your head with tiny robotic artificial neurons that had the same connections, the same local input-output behavior, and analogous internal state and learning rules."
He also acknowledges that this could be impossible, but only considers one reason why (which at least I consider highly implausible):
Sir Roger Penrose: "The thought experiment you propose is impossible. You can't duplicate the behavior of neurons without tapping into quantum gravity. That said, there's not much point in me taking further part in this conversation." (Wanders away.)
Also worth noting that another logical possibility (which you sort of get at in footnote 9) is that the thought experiment does go through, and a human with silicon chips instead of neurons would still be conscious, but CF is still false. Maybe it's not the substrate but the spatial location of neurons that's relevant. ("Substrate-indendence" is not actually a super well-defined concept, either.)
If you do reject CF but do believe in realist consciousness, then it's interesting to consider what other property is the key factor for human consciousness. If you're also a physicalist, then whichever property that is probably has to play a significant computational role in the brain, otherwise you run into contradicts when you compare the brain with a system that doesn't have the property and is otherwise as similar as possible. Spatial location has at least some things going for it here (e.g., ephaptic coupling and neuron synchronization).
For example, consciousness isn’t a function under this view, it probably still plays a function in biology.[12] If that function is useful for future AI, then we can predict that consciousness will eventually appear in AI systems, since whatever property creates consciousness will be engineered into AI to improve its capabilities.
This is a decent argument for why AI consciousness will happen, but actually "AI consciousness is possible" is a weaker claim. And it's pretty hard to see how that weaker claim could ever be false, especially if one is a physicalist (aka what you call "materialist" in your assumptions of this post). It would imply that consciousness in the brain depends on a physical property, but that physical property is impossible to instantiate in an artificial system; that seems highly suspect.
but I don't need to be able to define fire to be able to point out something that definitely isn't on fire!
I guess I can see that. I just don't think that e.g. Mike Johnson would consider his argument refuted based on the post; I think he'd argue that the type of problems illustrated by the popcorn thought experiment are in fact hard (and, according to him, probably unsolvable). And I'd probably still invoke the thought experiment myself, too. Basically I continue to think they make a valid point, hence are not "refuted". (Maybe I'm being nit-picky? But I think the standards for claiming to have refuted sth should be pretty high.)
I think the point is valid, but (as I said here) I don't think this refutes the thought experiments (that purport to show that a wall/bag-of-popcorn/whatever is doing computation). I think the thought experiments show that it's very hard to objectively interpret computation, and your point is that there is a way to make nonzero progress on the problem by positing a criterion that rules out some interpretations
Imo it's kind of like if someone made a thought experiment arguing that AI alignment is hard because there's a large policy space, and you responded by showing that there's an impact measure by which not every policy in the space is penalized equally. It would be a coherent argument, but it would be strange to say that it "refuted" the initial thought experiment, even if the impact measure happened to penalize a particular failure mode that the initial thought experiment used as an example. A refutation would more be a complete solution to the alignment problem, which no one has -- just as no one has a complete solution to the problem of objectively interpreting computation.
(I also wonder if this post is legible to anyone not already familiar with the arguments? It seems very content-dense per number of words/asking a lot of the reader.)
although I’m not quite sure what to make of it
(Kinda figured this, almost decided not to post the comment.)
Also, just for context, do you think you’ve experienced PNSE? Thanks!
Yes with some caveats. I think I've experienced no-self, which is what you describe from 6.2 onward. But if you'd asked me how far I am to enlightenment, I'd have said maybe 15%. Which is to say, I think no-self is a real thing you can achieve (and I definitely think it's a net positive), but I think from there, the ladder goes way way higher. Like the "take one day vs. entire life" comment implies a goodness multiplier of at least 20000x compared to the minds of regular people. Even if we assume this is widely exaggerated (although people keep insisting that it's not) and that the real multiplier is two OOMs smaller, than that' still 200x, whereas I'd put no-self at somewhere between 1x and 2x. If someone did claim that just the no-self part gives you even a 200x multiplier (which I doubt the person in the twitter comment would say), then I'd just be scratching my head at that.
... which could be a sign that I'm delusional and haven't really experienced no-self, but I think my experience fits quite well with your description (less anxiety, less self-reflective thoughts, identification with everything in awareness, etc.). Actually I think no-self + flow state is really very similar to regular flow state, which is again why the multiplier can't be that high. So, yeah, in my model enlightenment and no-self are two radically different things, the first is way harder to achieve and presumably way way better, and I think I've experienced the second but I know I'm nowhere close to the first -- if in fact it exists, which I suspect it does. (Sorry for the rambly answer.)
Also, I also want to distinguish two aspects of an emotion. In one, “duration of an emotion” is kinda like “duration of wearing my green hat”. I don’t have to be thinking about it the whole time, but it’s a thing happening with my body, and if I go to look, I’ll see that it’s there. Another aspect is the involuntary attention. As long as it’s there, I can’t not think about it, unlike my green hat. I expect that even black-belt PNSE meditators are unable to instantly turn off anger / anxiety / etc. in the former sense. I think these things are brainstem reactions that can be gradually unwound but not instantly. I do expect that those meditators would be able to more instantly prevent the anger / anxiety / etc. from controlling their thought process. What do you think?
Agree with all this.
More specifically, I feel like when you say “apply equanimity to X”, you mean something vaguely like “do a specific tricky learned attention-control maneuver that has something to do with the sensory input of X”. That same maneuver could contribute to equanimity, if it’s applied to something like anxiety. But the maneuver itself is not what I would call “equanimity”. I
I don't think it feels that way. What it feels like is that, if you pick any item in awareness, there's by default a tension with that thing, which makes it feel lower valence. If you apply equanimity -- which as I said, I can best describe as 'try not to resist' -- then the apparent tension lessens. With pain, this like experiencing the pain but not suffering. With positive sensations, the best way I can describe it is that if you succeed in applying a decent amount of equanimity, you realize afterward that your enjoyment wasn't "pure" but was plagued by attachment/craving. A decent way to describe it is that "pleasure turns into fulfillment"; I think that's the term associated with good-sensations-that-have-no-craving-aspect. But in both cases they definitely become higher valence. And with neutral sensations it kinda still feels like you've removed craving or resistance, even though this doesn't particularly make sense. Anyway, it really doesn't feel like it's an attention-control maneuver, it feels like it's a property of the sensation.
I sorta have in mind a definition kinda like: neither very happy, nor very sad, nor very excited, nor very tired, etc. Google gives the example: “she accepted both the good and the bad with equanimity”
Imo meditators are often evasive when it comes to this topic and refuse to just say that meditation is supposed to make you feel better, even though it obviously does, and this is probably causally upstream of you writing this sentence. i think it's just because 'feeling better' is generally associated with 'feel more nice things', and trying to chase pleasures is the opposite of meditation; you're supposed to be content with what is (again, equanimity feels like not resisting; it's sometimes analogized to the inverse of friction in a mechanical system). So yeah, I mean, applying tons of equanimity doesn't make you feel more pleasure, but yeah it does feel really good/high-valence, just in a non-pleasur-y but fullfilment-y sense. (The one time I was on a formal retreat, the meditation teacher even complained when I mentioned that I had a goal for meditating, and I had to specify that this doesn't mean I'm thinking about the goal while meditating; tbqh imo many people are just kinda bad at differentiating these things, but it's really not that complicated.)
Hmm. I think that none of this refutes the point I was making, which is that practical CF as defined by OP is a position that many people actually hold,[1] hence OP's argument isn't just a strawman/missing the point. (Whether or not the argument succeeds is a different question.)
Well, I guess this discussion should really be focused more on personal identity than consciousness (OP wrote: “Whether or not a simulation can have consciousness at all is a broader discussion I'm saving for later in the sequence, and is relevant to a weaker version of CF.”).
I don't think you have to bring identity into this. (And if you don't have to, I'd strongly advise leaving it out because identity is another huge rabbit hole.) There's three claims with strictly increasing strength here: digital simulations can be conscious, a digital simulation of a brain exhibits similar consciousness to that brain, and if a simulation of my brain is created, then that simulation is me. I think only is about identity, and OP's post is arguing against . (All three claims are talking about realist consciousness.)
This is also why I don't think noise matters. Granting all of (A)-(D) doesn't really affect ; a practical simulation could work with similar noise and be pseudo-nondeterministic in the same way that the brain is. I think it's pretty coherent to just ask about how similar the consciousness is, under a realist framework (i.e., asking ), without stepping onto the identity hornets nest.
a caveat here is that it's actually quite hard to write down any philosophical position (except illusionism) such that a lot of people give blanket endorsements (again because everyone has slightly different ideas of what different terms mean), but I think OP has done a pretty good job, definitely better than most, in formulating an opinion that at least a good number of people would probably endorse. ↩︎
I think a lot of misunderstandings on this topic are because of a lack of clarity about what exact position is being debated/argued for. I think two relevant positions are
- (1) it is impossible to say anything meaningful whatsoever about what any system is computing. (You could write this semi-formally as physical system computation the claims " performs " and " performs " are equally plausible)
- (2) it is impossible to have a single, formal, universally applicable rule that tells you which computation a physical system is running that does not produce nonsense results. (I call the problem of finding such a rule the "interpretation problem", so (2) is saying that the interpretation problem is impossible)
(1) is a much stronger claim and definitely implies (2), but (2) is already sufficient to rule out the type of realist functionalism that OP is attacking. So OP doesn't have to argue (1). (I'm not sure if they would argue for (1), but they don't have to in order to make their argument.) And Scott Aaronson's essay is (as far as I can see) just arguing against (1) by proposing a criterion according to which a waterfall isn't playing chess (whereas stockfish is). So, you can agree with him and conclude that (1) is false, but this doesn't get you very far.
The waterfall thought experiment also doesn't prove (2). It's very hard to prove (2) because (2) is just saying that a problem cannot be solved, and it's hard to prove that anything can't be done. But the waterfall thought experiment is an argument for (2) by showing that the interpretation problem looks pretty hard. This is how I was using it in my reply to Steven on the first post of this sequence; I didn't say "and therefore, I've proven that no solution to the interpretation problem exists"; I've just pointed out that you start off with infinitely many interpretations and currently no one has figured out how to narrow it done to just one, at least not in a way such that the answers have the properties everyone is looking for.
One thing I worry about is that the same disagreement happens with a lot of other users who, unlike Steven, just downvote the post rather than writing a comment.
In general, when I've read through the entire LW catalogue of posts with the consciousness tag, I've noticed that almost all well received posts with the consciousness tag take what I call the camp #1 perspective (i.e., discuss consciousness from an illusionist lens, even if it's not always stated explicitly). Iirc the only major exceptions are the posts from Eliezer, which, well, are from Eliezer. So it could be that post who discuss consciousness from a realist PoV consistently receive certain amount of downvotes from camp #1 people to whom the post just seems gibberish/a waste of time. I don't have any data to prove that this is the mechanism, it's just a guess, but the pattern is pretty consistent. I also think you generally wouldn't predict this if you just read the comment sections. (And idk if clarifying the perspective would help since no one does it.)
I'm pretty poor right now so didn't donate, but I do generally believe that the Lightcone team has done a good job, overall, and is worth donating to.
I agree that “practical CF” as thus defined is false—indeed I think it’s so obviously false that this post is massive overkill in justifying it.
But I also think that “practical CF” as thus defined is not in fact a claim that computational functionalists tend to make.
The term 'functionalist' is overloaded. A lot of philosophical terms are overloaded, but 'functionalist' is the most egregiously overloaded of all philosophical terms because it refers to two groups of people with two literally incompatible sets of beliefs:
- (1) the people who are consciousness realists and think there's this well-defined consciousness stuff exhibited from human brains, and also that the way this stuff emerges depends on what computational steps/functions/algorithms are executed (whatever that means exactly)
- (2) the people who think consciousness is only an intuitive model, in which case functionalism is kinda trivial and not really a thing that can be proved or disproved, anyway
Unless I'm misinterpreting things here (and OP can correct me if I am), the post is arguing against (1), but you are (2), which is why you're talking past each other here. (I don't think this sequence in general is relevant to your personal views, which is what I also tried to say here.) In the definition you rephrased
would cause the same conscious experience as that brain, in the specific sense of thinking literally the exact same sequence of thoughts in the exact same order, in perpetuity.
... consciousness realists will read the 'thinking' part as referring to thinking in the conscious mind, not to thinking in the physical brain. So to you this reads obviously false to you because you don't think there is a conscious mind separate from the physical brain, and the thoughts in the physical brain aren't 'literally exactly the same' in the biological brain vs. the simulation -- obviously! But the (1) group does, in fact, believe in such a thing, and their position does more or less imply that it would be thinking the same thoughts.
I believe this is what OP is trying to gesture at as well with their reply here.
Fair enough; the more accurate response would have been that evolution might be an example, depending on how the theory was derived (which I don't know). Maybe it's not actually an example.
The crux would be when exactly he got the idea; if the idea came first and the examples later, then it's still largely analogous (imo); if the examples were causally upstream of the core idea, then not so much.
Can you give an example in another domain where philosophical reasoning about a topic led to empirical facts about that topic?
Yes -- I think evolution is a pretty clean example. Darwin didn't have any more facts than other biologists or philosophers, and he didn't derive his theory by collecting facts; he was just doing better philosophy than everyone else. His philosophy led to a large set of empirical predictions, those predictions were validated, and that's how and when the theory was broadly accepted. (Edit: I think that's a pretty accurate description of what happened, maybe you could argue with some parts of it?)
I think we should expect that consciousness works out the same way -- the problem has been solved, the solution comes with a large set of empirical predictions, it will be broadly accepted once the empirical evidence is overwhelming, and not before. (I would count camp #1 broadly as a 'theory' with the empirical prediction that no crisp divide between conscious and unconscious processing exists in the brain, and that consciousness has no elegant mathematical structure in any meaningful sense. I'd consider this empirically validated once all higher cognitive functions have been reverse-engineered as regular algorithms with no crisp/qualitative features separating conscious and unconscious cognition.)
(GPT-4o says that before evolution was proposed, the evolution of humans was considered a question of philosophy, so I think it's quite analogous in that sense.)
I see your point, but I don't think this undermines the example? Like okay, the 'life is not a crisp category' claim has nuance to it, but we could imagine the hypothetical smart philosopher figuring out that as well. I.e., life is not a crisp category in the territory, but it is an abstraction that's well-defined in most cases and actually a useful category because of this <+ any other nuance that's appropriate>.
It's true that the example here (figuring out that life isn't a binary/well-defined thing) is not as practically relevant as figuring out stuff about consciousness. (Nonetheless I think the property of 'being correct doesn't entail being persuasive' still holds.) I'm not sure if there is a good example of an insight that has been derived philosophically, is now widely accepted, and has clear practical benefits. (Free Will and implications for the morality of punishment are pretty useful imo, but they're not universally accepted so not a real example, and also no clear empirical predictions.)
Couldn't you imagine that you use philosophical reasoning to derive accurate facts about consciousness, which will come with insights about the biological/computational structure of consciousness in the human brain, which will then tell you things about which features are critical / how hard human brains are to simulate / etc.? This would be in the realm of "empirical predictions derived from philosophical reasoning that are theoretically testable but not practically testable".
I think most solutions to consciousness should be like that, although I'd grant that it's not strictly necessary. (Integrated Information Theory might be an example of a theory that's difficult to test even in principle if it were true.)
I'd call that an empirical problem that has philosophical consequences :)
That's arguable, but I think the key point is that if the reasoning used to solve the problem is philosophical, then a correct solution is quite unlikely to be recognized as such just because someone posted it somewhere. Even if it's in a peer-reviewed journal somewhere. That's the claim I would make, anyway. (I think when it comes to consciousness, whatever philosophical solution you have will probably have empirical consequences in principle, but they'll often not be practically measurable with current neurotech.)
What about if it's a philosophical problem that has empirical consequences? I.e., suppose answering the philosophical questions tells you enough about the brain that you know how hard it would be to simulate it on a digital computer. In this case, the answer can be tested -- but not yet -- and I still think you wouldn't know if someone had the answer already.
I don't know the answer, and I'm pretty sure nobody else does either.
I see similar statements all the time (no one has solved consciousness yet / no one knows whether LLMs/chickens/insects/fish are conscious / I can only speculate and I'm pretty sure this is true for everyone else / etc.) ... and I don't see how this confidence is justified. The idea seems to be that no one can't have found the correct answers to the philosophical problems yet because if they had, those answers would immediately make waves and soon everyone on LessWrong would know about it. But it's like... do you really think this? People can't even agree on whether consciousness is a well-defined thing or a lossy abstraction; do we really think that if someone had the right answers, they would automatically convince everyone of them?
There are other fields where those kinds of statements make sense, like physics. If someone finds the correct answer to a physics question, they can run an experiment and prove it. But if someone finds the correct answer to a philosophical question, then they can... try to write essays about it explaining the answer? Which maybe will be slightly more effective than essays arguing for any number of different positions because the answer is true?
I can imagine someone several hundred years ago having figured out, purely based on first-principles reasoning, that life is no crisp category at the territory but just a lossy conceptual abstraction. I can imagine them being highly confident in this result because they've derived it for correct reasons and they've verified all the steps that got them there. And I can imagine someone else throwing their hands up and saying "I don't know what mysterious force is behind the phenomenon of life, and I'm pretty sure no one else does, either".
Which is all just to say -- isn't it much more likely that the problem has been solved, and there are people who are highly confident in the solution because they have verified all the steps that led them there, and they know with high confidence which features need to be replicated to preserve consciousness... but you just don't know about it because "find the correct solution" and "convince people of a solution" are mostly independent problems, and there's just no reason why the correct solution would organically spread?
(As I've mentioned, the "we know that no one knows" thing is something I see expressed all the time, usually just stated as a self-evident fact -- so I'm equally arguing against everyone else who's expressed it. This just happens to the be first time that I've decided to formulate my objection.)
Sort of not obvious what exactly "causal closure" means if the error tolerance is not specified. We could differentiate literally 100% perfect causal closure, almost perfect causal closure, and "approximate" causal closure. Literally 100% perfect causal closure is impossible for any abstraction due to every electron exerting nonzero force on any other electron in its future lightcone. Almost perfect causal closure (like 99.9%+) might be given for your laptop if it doesn't have a wiring issue(?), maybe if a few more details are included in the abstraction. And then whether or there exists an abstraction for the brain with approximate causal closure (95% maybe?) is an open question.
I'd argue that almost perfect causal closure is enough for an abstraction to contain relevant information about consciousness, and approximate causal closure probably as well. Of course there's not really a bright line between those two, either. But I think insofar as OP's argument is one against approximate causal closure, those details don't really matter.
No software/hardware separation in the brain: empirical evidence
I feel like the evidence in this section isn't strong enough to support the conclusion. Neuroscience is like nutrition -- no one agrees on anything, and you can find real people with real degrees and reputations supporting just about any view. Especially if it's something as non-committal as "this mechanism could maybe matter". Does that really invalidate the neuron doctrine? Maybe if you don't simulate ATP, the only thing that changes is that you have gotten rid of an error source. Maybe it changes some isolated neuron firings, but the brain has enough redundancy that it basically computes the same functions.
Or even if it does have a desirable computational function, maybe it's easy to substitute with some additional code.
I feel like the required standard of evidence is to demonstrate that there's a mechanism-not-captured-by-the-neuron-doctrine that plays a major computational role, not just any computational role. (Aren't most people talking about neuroscience still basically assuming that this is not the case?)
We can expect natural selection to result in a web of contingencies between different levels of abstraction.[6]
Mhh yeah I think the plausibility argument has some merit.
As simon has already said, your math is wrong because the cases aren't equiprobable. For k=2 you can fix this by doubling the cases of H since they're twice as likely as the others (so the proper state space is with .) For you'd have to quadruple H and double HT, which would give 4x H and 4x HT and 4x HTT and 8x TTT I believe, leading to probability of TTT. (Up from 1x H, 2x HT, 4x HTT, 8x TTT.) In general, I believe the probability of only Ts approaches 0, as does the probability of H.
Regardless, are these the right betting odds? Yup! If we repeat this experiment for any and you are making a bet every time you wake up, then these are the odds according to which you should take or reject bets to maximize profit. You can verify this by writing a simulation, if you want.
If you make the experiment non-repeating, then I think this is just a version of the presumptuous philospoher argument which (imo) shows that you have to treat logical uncertainty differently from randomness (I addressed this case here).
I think there's a practically infinite number of edge cases. For a system to run the algorithm, it would have to perform a sequence of operations on natural numbers. If we simplify this a bit, we could just look at the values of the variables in the program (like a, x, y; I don't actually know the algorithm, I'm just looking at the pseudo-code on Wikipedia). If the algorithm is running, then each variable goes through a particular sequence, so we could just use this as a criterion and say the system runs the algorithm iff one of these particular sequences is instantiated.
Even in this simplified setting, figuring this out requires a mapping of physical states to numbers. If you start agreeing on a fixed mapping (like with a computer, we agree that this set of voltages at this location corresponds to the numbers 0-255), then that's possible to verify. But in general you don't know, which means you have to check whether there exists at least one mapping that does represent these sequences. Considered very literally, this is probably always true since you could have really absurd and discontinuous mappings (if this pebble here has mass between 0.5g and 0.51g it represents the number 723; if it's between 0.51g and 0.52g it represents 911...) -- actually you have infinitely many mappings even after you agree on how the system partitions into objects, which is also debatable.
So without any assumptions, you start with a completely intractable problem and then have to figure out how to deal with this (... allow only reasonable mappings? but what's reasonable? ...), which in practice doesn't seem like something anyone has been able to do. So even if I just show you a bunch of sand trickling through someone's hands, it's already a hard problem to prove that this doesn't represent the Miller-Rabin test algorithm. It probably represents some sequence of numbers in a not too absurd way. There are some philosophers who have just bitten the bullet and concluded that any and all physical systems compute, which is called panpcomputationalism. The only actually formal rule for figuring out a mapping that I know is from IIT, which is famously hated on LW (and also has conclusions that most people find absurd, such that digital computers can't be conscious at all bc the criterion ends up caring more about the hardware than the software). The thing is that most of the particles inside a computer don't actually change all that much depending on the program, it's really only a few specific locations where electrons move around, which is enough if we decide that our mapping only cares about those locations but not so much if you start with a rule applicable to arbitrary physical systems.
That all said, I think illusionists have the pretty easy out of just saying that computation is frame-dependent, i.e., that the answer to "what is this system computing" depends on the frame of reference, specifically the mapping from physical states to mathematical objects. It's really only a problem you must solve if you both think that (a) consciousness is well-defined, frame-invariant, camp #2 style, etc., and also (b) the consciousness of a system depends on what it computes.
I don't know if you are going to address this, but if I were to write a sequence of posts on functionalism, I'd start with the problem that "computation" isn't very well defined, and hence functionalism isn't very well-defined, either. In practice it's often clear enough whether or not a system is computing something, but you're going to have a hard time giving a fully general, rigorous, universally applicable definition of what exactly a physical process has to do to count as computing something (and if so, what precisely it is computing). Similarly, your definition of the Practical CF inherits this problem because it's not at all clear what "capturing the dynamics of the brain on some coarse-grained level of abstraction" means. This problem is usually brushed over but imo that's where all the difficulty lies.
(Of course, many people think consciousness is inherently fuzzy, in which case associating it with similarly fuzzy concepts isn't a problem. But I'm assuming you're taking a realist point of view here and assume consciousness is well-defined, since otherwise there's not much of a question to answer. If consciousness is just an abstraction, functionalism becomes vacuously true as a descriptive statement.)
Fwiw I was too much of a coward/too conflict averse to say anything myself but I agree with this critique. (As I've said in my post I think half of all people are talking about mostly the same thing when they say 'consciousness', which imo is fully compatible with the set of responses Andrew listed in his post given how they were collected.)
Schindler's List: we can talk about specific rationality lessons all day, but we all know the biggest bottleneck is trying in the first place. This movie is the transformation of an ethical egoist into a utilitarian.
It also shows the value of Money: the Unit of Caring.