TAG's Shortform 2020-08-13T09:30:22.058Z


Comment by TAG on niplav's Shortform · 2024-04-16T15:15:36.114Z · LW · GW

The claim that humans are at least TM's is quite different to the claim that humans are at most TM's. Only the second is computationalism.

Comment by TAG on Ackshually, many worlds is wrong · 2024-04-16T14:05:33.917Z · LW · GW

Meanwhile the many-worlds interpretation suffers from the problem that it is hard to bridge to experience,

Operationally, it's straightforward: you keep "erasing the part of the (alleged) wavefunction that is inconsistent with my indexical observations, and then re-normalizing the wavefunction"...all the time murmering under your breath "this is not collapse..this is not collapse".

(Lubos Motl is quoted making a similar comment here

Comment by TAG on Against John Searle, Gary Marcus, the Chinese Room thought experiment and its world · 2024-04-15T14:39:31.929Z · LW · GW

That claim is unjustified and unjustifiable

Nothing complex is a black box , because it has components, which can potentially be understood.

Nothing artificial is a black box to the person who built it.

An LLM is , of course, complex and artificial.

Everything is fundamentally a black box until proven otherwise.

What justifies that claim?

Our ability to imagine systems behaving in ways that are 100% predictable and our ability to test systems so as to ensure that they behave predictably

I wasn't arguing on that basis.

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-14T18:21:11.102Z · LW · GW

every particle interaction creates n parallel universes which never physically interfere with each other”

Although a fairly standard way of explaining MWI, this is an example of conflating coherence and decoherence. To get branches that never interact with each other again, you need decoherence, but decoherence is a complex dynamical takes some it is not going to occur once per elementary interaction. It's reasonable to suppose that elementary interactions produce coherent superpositions, on the other hand, but these are not mutually isolated "worlds". And we have fairly strong evidence for them.. quantum computing relies on complex coherent any idea that all superpositions just automatically and instantly decohere must be rejected.

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-14T17:00:17.324Z · LW · GW

People keep coming up with derivations, and other people keep coming up with criticisms if then, which is why people keep coming up with new ones.

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-14T16:20:36.215Z · LW · GW

I don’t think this is correct, either (although it’s closer). You can’t build a ball-and-disk integrator out of pebbles, hence computation is not necessarily substrate neutral.

Meaning that a strong version of computational substrate independence , where any substrate will do, is false? Maybe, but I was arguing against hypothetical, that "the substrate independence of computation implies the substrate independence of consciousness", not *for* the antecedent, the substrate independence of computation.

What the Turing Thesis says is that a Turing machine, and also any system capable of emulating a Turing machine, is computationally general (i.e., can solve any problem that can be solved at all). You can build a Turing machine out of lots of substrates (including pebbles), hence lots of substrates are computationally general. So it’s possible to integrate a function using pebbles, but it’s not possible to do it using the same computation as the ball-and-disk integrator uses—the pebbles system will perform a very different computation to obtain the same result.

I don't see the relevance.

So even if you do hold that certain computations/algorithms are sufficient for consciousness, it still doesn’t follow that a simulated brain has identical consciousness to an original brain. You need an additional argument that says that the algorithms run by both systems are sufficiently similar.

OK. A crappy computational emulation might not be conscious, because it's crappy. It still doesn't follow that a good emulation is necessarily conscious. You're just pointing out another possible defeater.

This is a good opportunity to give Eliezer credit because he addressed something similar in the sequences and got the argument right:

Which argument? Are you saying that a good enough emulation is necessarily conscious?

Albert: “Suppose I replaced all the neurons in your head with tiny robotic artificial neurons that had the same connections, the same local input-output behavior, and analogous internal state and learning rules.” Note that this isn’t “I upload a brain” (which doesn’t guarantee that the same algorithm is run)

If it's detailed enough, it's guaranteed to. That's what "enough" means

but rather “here is a specific way in which I can change the substrate such that the algorithm run by the system remains unaffected”.

Ok...that might prove the substrate independence of computation, which I wasn't arguing against. Past that, I don't see your point

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-13T18:11:50.578Z · LW · GW

The result (at least partially) of a particular physical substrate. Physicalism and computationalism are both not-dualism , but they are not the same as each other.

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-13T16:27:18.880Z · LW · GW

The Church-Turing thesis gives us the “substrate independence principle”. In principle, AI could be conscious.

The C-T thesis gives you the substrate independence of computation. To get to the substrate independence of consciousness, you need the further premise that the performance of certain computations is sufficient for consciousness, including qualia. This is, of course, not known.

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-13T16:11:50.801Z · LW · GW

MW has to show that decoherence is a natural consequence, which is the same thing. It can't be taken on faith, any more than entropy should be. Proofs of entropy were supplied a long time ago, proofs of decoherence of a suitable kind, are a work in progress.

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-12T17:55:32.934Z · LW · GW

What does highly sensitive mean? In classical physics, an observer can produce an energy output much greater than the energy input of the observation. ,but no splitting is implied. In bare Everettian theory, an observer becomes entangled with the coherent superposition they are observing, and goes into a coherent superposition themself no decoherentsplitting is implied. You still haven't said where and the initial decoherent splitting occurs.

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-12T08:33:34.968Z · LW · GW

Bohmian mechanics adds hidden variables. why would it be simpler?

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-12T05:50:49.325Z · LW · GW

Amount of calculation isn’t so much the concern here as the amount of bits used to implement that calculation. And there’s no law that forces the amount of bits encoding the computation to be equal. Copenhagen can just waste bits on computations that MWI doesn’t have to do

And vice versa. You can do unnecessary calculation under any interpretation, so that's an uninteresting observation.

The importantly is that the minimum amount of calculation you have to do get an empirically adequate theory is the same under any interpretation, because interpretations don't change the maths, they just ... interpret it.... differently. In particular, a.follower many worlder has to discard unobserved results in the same way as a Copenhagenist -- it's just that they interpret doing so as the unobserved results existing in another branch, rather than being snipped off by collapse. The maths is the same, the interpretation is different. You can also do the maths without interpreting it, as in Shut Up And Calculate.

Copenhagen has to have rules for when measurements occur and what basis they occur in

This gets back to a long-standing confusion between Copenhagen and objective collapse theories (here, I mean, not in the actual physics community). Copenhagen ,properly speaking, only claims that collapse occurs on or before measurement. It also claims that nothing is known about the ontology of.the system before collapse -- it's not the case that anything "is" a wave function. An interpretation of QM doesn't have to have an ontology, and many dont. Which, of course, is another factor that renders the whole Kolmogorov. Complexity approach inoperable.

Objective collapse theories like GRW do have to specify when and collapse occurs...but MW theories have to specify when and how decoherence occurs. Decoherence isn't simple.

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-11T17:28:07.550Z · LW · GW
Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-11T12:20:32.174Z · LW · GW

You're saying that if you have decoherent splitting of an observer, that leads to more decoherent splitting. But where does the initial decoherent splitting come from?

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-11T03:10:34.304Z · LW · GW

We don't have to regard basis as objective, ITFP.

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-11T03:04:21.466Z · LW · GW

Why? If you could prove that large environments must cause decoherence into n>1 branches you would have solved the measurement problem as it is currently understood.

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-11T02:59:21.247Z · LW · GW

but rather that Copenhagen used up some extra bits in the machine that generates the output tape in order to implement the wavefunction collapse procedure. (

Again: that's some less calculation that the reader of the tape has to do.

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-10T18:50:35.798Z · LW · GW

I'm not talking about the code complexity of interleaving the SI's output.

I am talking about interpreting the serial output of the SI , as it were. If you account for that , then the total complexity is exactly the same as Copenhagen and that's the point. I'm not a dogmatic Copenhagenist, so that's not a gotcha.

Basically , the amount of calculation you have to do to get an empirically adequate theory is the same under any interpretation, because interpretations don't change the maths, they just ... interpret it .....differently. The SI argument for MWI only seems to work because it encourages the reader to neglect the complexity implicit in interpreting the output tape.

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-10T17:18:19.689Z · LW · GW

"it" isn't a single theory.

The argument that Everettian MW is favoured by Solomonoff induction, is flawed.

If the program running the SWE outputs information about all worlds on a single output tape, they are going to have to be concatenated or interleaved somehow. Which means that to make use of the information, you gave to identify the subset of bits relating to your world. That's extra complexity which isn't accounted for because it's being done by hand, as it were..

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-10T14:34:05.346Z · LW · GW

I'm not trying to say all forms of MW are hopeless. I am saying

  • there is more than one form
  • there are trade offs between simplicity and correctness -- there's no simple and adequate MWI.

Decoherence isn't simple -- you can't find it by naively looking at the SWE, and it took three or four decades for physicists to notice.

It also doesnt't unequivocally support MW -- when we observe decoherence, we observe it one universe at a time, and maybe in the one and only universe.

"Decoherence does half the job of solving the measurement problem. In short, it tells you that you will not in practice be able to observe that Schroodinger's cat is in a superposition, because the phase between the two parts of the superposition would not be sufficiently stable. But the concept of decoherence does not, on its own, yield an answer to the question "how come the experimental outcome turns out to be one of A or B, not both A and B carried forward together into the future?"

The half-job that decoherence succeeds in doing is to elucidate the physical process whereby a preferred basis or pointer basis is established. As you say in the question, any given quantum state can be expressed as a superposition in some basis, but this ignores the dynamical situation that physical systems are in. In practice, when interactions with large systems are involved, states in one basis will stay still, states in another basis will evolve VERY rapidly, especially in the phase factors that appear as off-diagonal elements of density matrices. The pointer basis is the one where, if the system is in a state in that basis, then it does not have this very fast evolution.

But as I say, this observation does not in and of itself solve the measurement problem in full; it merely adds some relevant information. It is the next stage where the measurement problem really lies, and where people disagree. Some people think the pointer basis is telling us about different parts of a 'multiverse' which all should be regarded as 'real'. Other people think the pointer basis is telling us when and where it is legitimate to assert 'one thing and not both things happen'.

That's it. That's my answer to your question.

But I can't resist the lure, the sweet call of the siren, "so tell us: what is really going on in quantum measurement?" So (briefly!) here goes.

I think one cannot get a good insight into the interpretation of QM until one has got as far as the fully relativistic treatment and therefore field theory. Until you get that far you find yourself trying to interpret the 'state' of a system; but you need to get into another mindset, in which you take an interest in events, and how one event influences another. Field theory naturally invites one to a kind of 'input-output' way of thinking, where the mathematical apparatus is not trying to say everything at once, but is a way of allowing one to ask and find answers to well-posed questions. There is a distinction between maths and physical stuff. The physical things evolve from one state to another; the mathematical apparatus tells us the probabilities of the outcomes we put to it once we have specified what is the system and what is its environment. Every system has an environment and quantum physics is a language which only makes sense in the context of an environment.

In the latter approach (which I think is on the right track) the concept of 'wavefunction of the whole universe' is as empty of meaning as the concept of 'the velocity of the whole universe'. The effort to describe the parts of such a 'universal wavefunction' is a bit like describing the components of the velocity of the whole universe. In saying this I have gone beyond your question, but I hope in a useful way."


“Despite how tidy the decoherence story seems, there are some peo-ple for whom it remains unsatisfying. One reason is that the deco-herence story had to bring in a lot of assumptions seemingly extra-neous to quantum mechanics itself: about the behavior of typicalphysical systems, the classicality of the brain, and even the nature ofsubjective experience. A second reason is that the decoherence storynever did answer our question about the probability you see the dotchange color – instead the story simply tried to convince us the ques-tion was meaningless.” Quantum Computing since Democritus, 2nd Ed, P. 169.

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-10T13:55:06.160Z · LW · GW

Regarding basis as an observers own choice of "co-ordinate grid", and regarding observing (or instrument) as having a natural basis , is a simple and powerful theory of basis. Since an observer's natural basis is the one that minimises superpositions, the fact that observers make quasi-classical observations drops out naturally, without any cosmological assumptions. But, since there is no longer a need for a global and objective basis, a basis that is a feature of the universe, there is no longer a possibility of many worlds as an objective feature of the universe: since an objective basis is needed to objectively define a division into worlds, there such a division is no longer possible, and splitting is an observer-dependent phenomenon.

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-10T12:20:02.040Z · LW · GW

What would you expect to see if you were a superposition?

If I were in a coherent superposition, I would expect to see non classical stuff. Entanglement alone is not enough to explain my sharp-valued, quasi classical observations.

It isn't just a matter of definition, because I don't perceive non classical stuff, so I lack motivation to define "I" in way that mispredicts that I do. You don't get to arbitrarily relabel things if you are in the truth seeking business.

The objection isn't to using "I" or "the observer" to label a superposed bundle of sub-persons , each of which individually is unaware of the others and has normal, classical style experience, because that doesn't mispredict my experience. There problem is that "super posed bundle of persons , each of which is unaware of the others and has normal, classical style experience" is what you get from a decoherent superposition, and I am specifically talking about coherent superposition. ("In the Everett theory, everything that starts in a coherent superposition, stays in one.") Decoherence was introduced precisely to solve the problem with Everett's RSI.

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-09T19:29:02.647Z · LW · GW

Surely the math itself says that when you open the box, you end up in a superposition of |see the cat alive> + |see the cat dead>.

In a classical basis. But you could rewrite the superposition in other bases that we dont observe. That's one problem.

As Penrose writes (Road to Reality 29.8) "Why do we not permit these superposed perception states? Until we know exactly what it is about a quantum state that allows it to be considered as a ‘perception’, and consequently see that such superpositions are ‘not allowed’, we have really got nowhere in explaining why the real world of our experiences cannot involve superpositions of live and dead cats." Penrose gives the example of 2|psi>1⁄4 {jperceiving live cati þ jperceiving dead cati} {jlive cati þ j dead cati} þ {jperceiving live cati jperceiving dead cati} {jlive cati jdead cati} as an example of a surreal, non-classical superposition.

And/or, we have no reason to believe, given only the formalism itself , that the two versions of the observer will be unaware of each other, and able to report unambiguously on their individual observations. That's the point of the de/coherence distinction. In the Everett theory, everything that starts in a coherent superposition, stays in one.

"According to Everett’s pure wave mechanics, when our observer makes a measurement of the electron he does not cause a collapse, but instead becomes correlated with the electron. What this means is that where once we had a system that consisted of just an electron, there is now a system that consists of the electron and the observer. The mathematical equation that describes the state of the new system has one summand in which the electron is z-spin up and the observer measured “z-spin up” and another in which the electron is z-spin down and the observer measured “z-spin down.” In both summands our observer got a determinate measurement record, so in both, if we ask him whether he got a determinate record, he will say “yes.” If, as in this case, all summands share a property (in this case the property of our observer saying “yes” when asked if he got a determinate measurement record), then that property is determinate.

This is strange because he did not in fact get a determinate measurement record; he instead recorded a superposition of two outcomes. After our observer measures an x-spin up electron’s z-spin, he will not have determinately gotten either “*z-*spin up” or “*z-*spin down” as his record. Rather he will have determinately gotten “*z-*spin up or *z-*spin down,” since his state will have become correlated with the state of the electron due to his interaction with it through measurement. Everett believed he had explained determinate experience through the use of relative states (Everett 1957b: 146; Everett 1973: 63, 68–70, 98–9). That he did not succeed is largely agreed upon in the community of Everettians."


Many attempts have been made to fix the problem, notably decoherence based approaches.

The expected mechanism of decoherence is interaction with a larger environment... which is assumed to already be in a set of decoherent branches on a classical basis. But why? At this point , it becomes a cosmological problem. You can't write a WF of the universe without some cosmological assumptions about the initial state, and so on, so whether it looks many-worldish or not depends on the assumptions.

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-09T18:52:41.549Z · LW · GW

The math doesn't describe a multiverse, in the sense that if you solve the Schrödinger equation for the universe, you get some structure with clearly separated decoherent branches, every time. You need additional assumptions, which have their own complexity cost.

In fact, MWI is usually argued from models of a few particles. These can show coherent superposition, but to spin an MW theory worthy of the name out of that, you need superpositions that can be maintained at large scale, and also decohere into non interacting branches , preferably by a mechanism that can be found entirely in the standard formalism.

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-09T16:59:11.115Z · LW · GW

Everett's thesis doesn't give an answer to how an observer makes sharp-valued classiscal observations, and doesn't flag the issue either, although much of the subsequent literature does.

Eg. for an overview (also why it's more than one theory, and a work-in-progress).

Comment by TAG on Any evidence or reason to expect a multiverse / Everett branches? · 2024-04-09T12:17:28.339Z · LW · GW

The main reason for not favouring the Everett interpretation is that it doesn't predict classical observations , unless you make further assumptions about basis, the "preferred basis problem". There is therefore room for an even simpler interpretation.

There is an approach to MWI based on coherent superpositions, and a version based on decoherence. These are (for all practical purposes) incompatible opposites, but are treated as interchangeable in Yudkowsky's writings.

The original, Everettian, or coherence based approach , is minimal, but fails to predict classical observations. (At all. It fails to predict the appearance of a broadly classical universe). The later, decoherence based approach, is more emprically adequate, but seems to require additional structure, placing its simplicity in doubt

Coherent superpositions probably exist, but their components aren't worlds in any intuitive sense. Decoherent branches would be worlds in the intuitive sense, and while there is evidence of decoherence, there is no evidence of decoherent branching.There could be a theoretical justification for decoherent branching , but that is what much of the ongoing research is about -- it isn't a done deal, and therefore not a "slam dunk". And, inasmuch as there is no agreed mechanism for decoherent branching, there is no definite fact about the simplicity of decoherent MWI.

Comment by TAG on My intellectual journey to (dis)solve the hard problem of consciousness · 2024-04-08T04:54:37.517Z · LW · GW

Explaining everything involves explaining phenomenal consciousness, so it's literally solving the Hard Problem, as opposed to dissolving it.

Comment by TAG on My intellectual journey to (dis)solve the hard problem of consciousness · 2024-04-07T12:01:11.966Z · LW · GW

I don't think it would tell you much, because it only excludes the case where the HP is a meme, not the case where it's a repeatable error.

Boxed AI tells you there is an HP.

  • Could be because it is phenomenally conscious, and it has noticed there is a real HP.
  • Could be because it is repeating a conceptual conclusion.

Boxed AI tells you there is not an HP.

  • Could be because it is a zombie, so it can't understand what PC is.
  • Could be because it has PC, but isn't subject to the erroneous thinking that causes the HP.

Note that you can't specify whether an AI is or isn't conscious, or that it's a perfect reasoner.

Note that philosophers don't agree on what constitutes a conceptual confusion.

Note that being able to trace back the causal history of an output doesn't tell you it wasn't caused by PC: one of the possible solutions to the HP is that certain kinds of physical activity, or information processing are identical to PC, so there is.not necessarily an xor between PC and physical causation. Of course, there is also a fact that human pronouncements have alone sort of causal history, and it doesn't settle much.

Note that, as things stand, the thought experiment is an intuition pump like Mary's Room, etc.

Comment by TAG on My intellectual journey to (dis)solve the hard problem of consciousness · 2024-04-07T02:59:11.052Z · LW · GW

So maybe consciousness has always been a linguistic debate?

It has always been at least a linguistic debate , but that does not show that it as at most a linguistic debate. The thing is that the Hard Problem is already only about one specific sub-meaning of "consciousness", namely phenomenal you can't dissolve it just by saying "consciousness" means more than one thing, or "let's focus on the components of the problem". A problem is at least as hard as its hardest sub-problem.

And "let's focus on the components of the problem" isn't eliminativism... eliminativism the claim that there is nothing for the problem to be about.

And breaking the problem into a set of sub problems that dont contain the Hard Problem is not dissolving it. (C.f. Kaj's comments)

Don’t start by asking ‘what is consciousness’ or ‘what are qualia’; start by asking ‘what are the cognitive causes of people talking about consciousness and qualia

Which might be real things that are adequately designated by the words "consciousness"and "qualia" or real things that are not adequately designated by the words "consciousness"and "qualia", or by nothing, or...

The dictum tells you nothing.

Comment by TAG on My intellectual journey to (dis)solve the hard problem of consciousness · 2024-04-07T02:37:08.121Z · LW · GW
Comment by TAG on Back to Basics: Truth is Unitary · 2024-03-31T15:30:58.458Z · LW · GW

The trick is to validate your beliefs against competent people who have objectives orthogonal to your personal narrative.

For instance, don't just read the sequences, read what the mainstream has to say as well.

And, while you should do this, it doesn't lead to unitary truth without the further assumption that epistemology is universally agreed and fixed. The assumption is unrealistic, because epistemologies are often embedded in comprehensive belief systems. Imagine a group of Marxists and a group of Christians trying to settle their differences. Comparing different theories is necessary but insufficient to arrive at universal truth.

Comment by TAG on Many arguments for AI x-risk are wrong · 2024-03-28T22:26:37.825Z · LW · GW

The issue is what is likeliest, not what is possible.

Comment by TAG on The Simple Truth · 2024-03-26T05:59:16.594Z · LW · GW

People who disbelieve in naive correspondence theories of truth can do.for sophisticated reasons,.ie. there are various objections to the correspondence theory, and the OP does not address them.

Comment by TAG on Are AIs conscious? It might depend · 2024-03-20T18:37:00.390Z · LW · GW

One class of theories about consciousness holds that beings are conscious by virtue of possessing a set of properties. Folk consciousness, for example, holds that humans are conscious by virtue of possessing souls. Pan-psychism holds that everything is conscious by virtue of existing. IIT holds that beings are conscious by virtue of being able to hold large sums of information in their head.

As a mathematical realist, however, I find these theories difficult to accept. Within the digits of pi, there exist infinite copies of the information that describes me. Yet I do not consider my consciousness to exist in these copies, but rather in the real world. If I were to die tomorrow, I would find very little comfort in knowing that I would continue to live on in the digits of pi.

I don't see how mathematical realism affects any of the theories you mention. Souls don't have to be mathematical. Panpsyschism applies to everything , not just entities with a certain mathematical representation. IIT (and computationalism) require information to be processed, not just exist eternally.

On top of that, you have offered no evidence for MR. We have direct evidence that consciousness exists, but no evidence that numbers exist outside of the mind. If MR does conflict with the existence of consciousness, we should therefore drop MR.

Similarly, the many-worlds interpretation states that there are an endless number of versions of me created whenever a quantum moment takes place. And yet I likewise give these copies very little regard.

Maybe you are wrong.

Comment by TAG on Celiefs · 2024-03-17T17:02:03.598Z · LW · GW

If you have a meta belief that none of your beliefs are certain, does that make all your beliefs celiefs?

Comment by TAG on Many arguments for AI x-risk are wrong · 2024-03-15T15:06:25.132Z · LW · GW

That argument doesn't work its own terms: we have extinguished far fewer species than we have not.

Comment by TAG on Evolution did a surprising good job at aligning social status · 2024-03-13T14:26:38.358Z · LW · GW

So humans are "aligned" if humans have any kind of values? That's not how alignment is usually used.

Comment by TAG on Deconstructing Bostrom's Classic Argument for AI Doom · 2024-03-12T15:02:27.132Z · LW · GW

The Orthogonality Thesis asserts that there can exist arbitrarily intelligent agents pursuing any kind of goal.”

The Ortogonality Thesis is often used in a way that "smuggles in" the idea that an AI will necessarily have a stable goal, even though goals can be very variewd. But similar reasoning shows that any combination of goal (in)stability and goallessness is possible, as well. mindspace contains agents with fixed goals, randomnly drifting goals, corrigble (externally controlable goals) , as well as non-agentive minds with no goals.

Comment by TAG on Scientific Method · 2024-03-08T02:08:38.157Z · LW · GW

We must always start with the simplest possible explanations for the phenomena that surround us.


The fewer components, abstractions, or entities required for a hypothesis, the better the hypothesis.


(Not doubting Occam's razor, pointing out that it needs an explanation).

There is more than one way to correctly describe reality.

That goes against he law of non-contradiction: if the two ways are different, they cannot both be correct.

Newton’s theory was nominally refuted by Einstein’s relativism, but this did not stop it from working

"Working" means making correct predictions, not describing reality.

However, Stephen Hawking suggests instead that we consider them all true: that a theory accurately describes the fundamental nature of things is of less importance to us than that it gives us reliable mechanisms for interacting with reality.

How important something is depends on ones values.

“All models are wrong, but some of them are useful.” the opposite of "There is more than one way to correctly describe reality.". Unless you start changing the meanings of "works"/"useful" versus "true"/"describes reality".

PS. Nothing to say about induction?

Comment by TAG on Many arguments for AI x-risk are wrong · 2024-03-07T16:54:00.464Z · LW · GW

It's not two things, risk versus safety, it's three things: existential risk versus sub-existential risk versus no risk. Sub existential risk is the most likely on the priors.

Comment by TAG on Even if we lose, we win · 2024-03-05T23:34:05.834Z · LW · GW

That would be a philosophical problem...

Comment by TAG on Agreeing With Stalin in Ways That Exhibit Generally Rationalist Principles · 2024-03-04T15:07:53.727Z · LW · GW

Truth and Feelings can be reconciled, so long as you are not extreme about either.: if your true beliefs are hurtful you can keep them to yourself. Your worldview can be kept separate from your persona. The problem is when you bring a third thing -- the thing known as sincerity or tactlessness, depending on whether or not you believe in it -- into the picture. If you feel obliged to tell the truth, you are going to hurt feelings.

This used to be well known, but is becoming unknown because of an increasing tendency to use words like "truth and "honesty" in a way that encompasses offering unsolicited opinions in addition to avoid lying. If you can't make a verbal distinction, its hard to make a conceptual one.

He visibly cared about other people being in touch with reality. “I’ve informed a number of male college students that they have large, clearly detectable body odors. In every single case so far, they say nobody has ever told them that before,” he wrote. (I can testify that this is true: while sharing a car ride with Anna Salamon in 2011, he told me I had B.O.)[21]

Well, that goes beyond having true beliefs and only making true statements.

Comment by TAG on Wei Dai's Shortform · 2024-03-02T23:40:29.623Z · LW · GW

"Read the sequences....just the sequences"

Something a better , future version of rationalism could do is build bridges and facilitate communication between these little bubbles. The answet-to-everything approach has been tried too many times.

Comment by TAG on On the abolition of man · 2024-03-01T22:47:42.283Z · LW · GW
Comment by TAG on On the abolition of man · 2024-03-01T22:31:14.621Z · LW · GW

Either the Tao can influence the world in the present, in which case the conditioners can never *really *prevent it from reasserting itself; or it can’t, in which case how did we first find it anyway; or it controlled the beginning as first cause in which case whatever happens anywhere ever is what it intended; or it intended something different but it’s not very good at it’s job.

Or it influences the world in proportion to how much it is recognised, and how much you influence the world is proportional to how much you recognise it. The Tao that controls you is not the Tao: the Tao you control is not the Tao. The Tao that does everything is not the Tao; the Tao that does nothing is not the Tao.

Comment by TAG on How is Chat-GPT4 Not Conscious? · 2024-03-01T13:58:14.029Z · LW · GW

Footnotes three and four are the sources behind today today’s understanding of consciousness as including “any kind of cognition....” as well as “awareness”.

The wikipedia quote doesn't to show that independence is necessary for consciousness, and your arguments from the behaviour of the LLM don't to show that there is any awareness, or anything beyond forms of cognition.

I think of cognition as involving the process of reasoning.

The question is the relationship between cognition and consciousness, not reasoning. Your quotes show that, at best, cognition is necessary but insufficient for consciousness.

If I google “define Independent,” the first definition that comes up is “free from outside control; not depending on another’s authority.”

Independence in an absolute sense might be impossible: any deterministic system can be controlled if you know how it works , and you can set the initial conditions.

Right now, my computer is running programs, but that is based on programming from someone else’s cognition. The key here is that, if we dissect Chat-GPT4, I don’t believe you would find Python/Java/C++ or any known programming language that a programmer used in order to tell GPT4 how to solve the particular problems I gave it in the four sessions (from my original post and my own reply/addendum to my original post).

That seems to be the heart of the issue. No, its responses are not explictly programmed in. Yes, its reponses show the ability to learn and synthesise. Which means...minimally...that's it actually is an AI .... not a glorified search engine. That's what AI is supposed to do.

The question is whether there is a slope from

*Shows learning and synthesis in cognition *Has independent cognition *Is conscious. *(Has personhood?....should be a citizen...?)

If your think that learning and synthesis in cognition are sufficient for consciousness conscious, you are effectively assuming that all AIs are conscious. But, historically, Artificial Consciousness has been regarded as a much higher bar than artificial intelligence.

Comment by TAG on How is Chat-GPT4 Not Conscious? · 2024-02-29T16:32:10.826Z · LW · GW

most definitions of consciousness indicate that—if a being has independent cognition (i.e. a stream of consciousness)--then the being is conscious.

I don't think that's true. For instance, none of the definitions given in the LW wiki give that definition. And the whole argument rests on that claim-- which rests on the meaning of "independent". What is "independent", anyway?

Comment by TAG on Why you, personally, should want a larger human population · 2024-02-28T04:42:07.851Z · LW · GW

I assume it isn't always like a bell curve, because smaller and poorer societies can't afford the deadweight of useless knowledge.

Comment by TAG on The Pointer Resolution Problem · 2024-02-28T03:04:24.777Z · LW · GW

How do we use Bayes to find kinds of truth other than predictiveness?

Comment by TAG on Rawls's Veil of Ignorance Doesn't Make Any Sense · 2024-02-27T17:57:34.846Z · LW · GW

And really the conditions of the OP are actively contrary to good decision-making, e.g. that you don’t know your particular conception of the good (??) or that you’re essentially self-interested. . .

Well, they're inimical to good personal self-interested decision making, but why would that matter? Do you think justice and self interested rationality are the same? If they are differerent, what's the problem? Rawl's theory would not necessarily predict the behaviour of a self interested agent , but it's not supposed to. It's a normative theory: justly is how people should behave, not how they invariably do. If they have their own theories of ethics, well they are theories and not necessarily correct. Mere disagreement between the front-of-the-veil and behind-the-veil versions of a person doesn't tell you much.

There’s no reason to think, generally, that people disagree with John Rawls only because of their social position or psychological quirks

They might have a well constructed case against him, he might have a well constructed case against them.