LessWrong 2.0 Reader

The fact that it's so formalized is part of the absurdity of IIT. There are a bunch of equations that are completely meaningless and not based in anything empirical whatsoever. The goal of my effort with this proof, regardless of whether there is a flaw in the logic somewhere, is that I think if we can take a single inch forward based on logical or axiomatic proofs, this can begin to narrow down our sea of endless speculative hypotheses, then those inches matter.

I'm totally on board with everything you said here. But I didn't bring up IIT as a rebuttal to anything you said in your post. In fact, your argument about swapping out neurons specifically avoids the problem I'm talking about in this above comment. The formalism of IIT actually agrees with you that swapping out neurons in a brain doesn't change consciousness (given the assumptions I've mentioned in the other comment)!

I've brought up IIT as a response to a specific claim -- which I'm just going state again since I feel like I keep getting misunderstood as making more vague/general claims than I'm in fact making. The claim (which I've seen made on LW before) is that we know for a fact that a simulation of a human brain on a digital computer is conscious because of the Turing thesis. Or at least, that we know this for a fact if we assume some very basic things about the universe like laws of physics are complete and functionalism is true. So like, the claim is that every theory of consciousness that agrees with these two premises also states that a simulation of a human brain has the same consciousness as that human brain.

Well, IIT is a theory that agrees with both of these premises -- it's a functionalist proposal that doesn't postulate any violation to the laws of physics -- and it says that simulations of human brains have completely different consciousness than human brains themselves. Therefore, the above claim doesn't seem true. This is my point; no more, no less. If there is a counter-example to an implication $A⟹B$, then the implication isn't true; it doesn't matter if the counter-example is stupid.

Again, does not apply to your post because you talked about swapping neurons in a brain, which is different -- IIT agrees with your argument but disagree with green_leaf's argument.

"100% wrong, no relation how consciousness actually works"

indeed.  I think we should stop there though.  The fact that it's so formalized is part of the absurdity of IIT.  There are a bunch of equations that are completely meaningless and not based in anything empirical whatsoever.  

The goal of my effort with this proof, regardless of whether there is a flaw in the logic somewhere, is that I think if we can take a single inch forward based on logical or axiomatic proofs, this can begin to narrow down our sea of endless speculative hypotheses, then those inches matter.

I don't think just because we have no way of solving the hard problem yet, or formulating a complete theory of consciousness, that this doesn't mean we can't make at least a couple of tiny inferences we can know with a high degree of certainty.  I think it's a disservice to this field that most high profile efforts have a complete framework of the entirety of consciousness stated as theory, when it's completely possible to start moving forward one tiny step at a time without relying on speculation.  

I don't see how it's an assumption.  Are we considering that the brain might not obey the laws of physics?

I mentioned complexity because you brought up a specific aspect of what determines the firing patterns, and my response is just to say 'sure, our replacement neurons will take in additional factors as part of their input and output'

basically, it seemed that part of your argument is that the neuron black box is unimplementable.  I just don't buy into the idea that neurons operate so vastly differently than the rest of reality to the point their behaviour can't be replicated

Most people need to eat something [citation needed] and it's hard to eat if you don't work.

This is honestly some of the best feedback that I've received on this site, so thank you for your comment. I edited the introduction and I clarified what I meant by "redundant" research.

I once tried to quantify the validity of academic research, but I gave up on trying that. I talk more about this in my reply to Seth Herd [LW(p) · GW(p)].

I see, thank you for the clarification. I should have been more careful with mischaracterizing your views.

I do have a question or two about your views if you would entertain me. You say humans wikl be economically obsolete and will 'retire' but there will still be trade between humans and AI. Does trade here just means humans consuming, I.e. trading money for AI goods and services? That doesn't sound like trading in the usual sense where it is a reciprocal exchange of goods and services.

How many 'different' AI individuals do you expect there to be ?

Not a Pascal's mugging to the best of my knowledge [LW · GW].

Well, this isn't the assumption, it's the conclusion (right or wrong). It appears from what I can tell is that the substrate is the firing patterns themselves.

You say "Now, replace one neuron with a functionally identical unit, one that takes the same inputs and fires the same way" and then go from there. This step is where you make the third assumption, which you don't justify.

I think focusing on the complexity required by the replacement neurons is missing the bigger picture.

Agreed -- I didn't say anything that complexity itself is a problem, though, I said something much more specific.

(that means a classical computer can run software that acts the same way).

No. Computability shows that you can have a classical computer that has the same input/output behavior, not that you can have a classical computer that acts the same way. Input/Output behavior is generally not considered to be enough to guarantee same consciousness, so this doesn't give you what you need. Without arguing about the internal workings of the brain, a simulation of a brain is just a different physical process doing different computational steps that arrives at the same result. A GLUT (giant look-up table) is also a different physical process doing different computational steps that arrives at the same result, and Eliezer himself argued that GLUT isn't conscious.

The "let's swap neurons in the brain with artificial neurons" is actually a much better argument than "let's build a simulation of the human brain on a different physical system" for this exact reason, and I don't think it's a coincidence that Eliezer used the former argument in his post.

A recently commonly heard viewpoint on the development of AI states that AI will be economically impactful but will not upend the dominancy of humans. Instead AI and humans will flourish together, trading and cooperating with one another. This view is particularly popular with a certain kind of libertarian economist: Tyler Cowen, Matthew Barnett, Robin Hanson.

They share the curious conviction that the probablity of AI-caused extinction p(Doom) is neglible. They base this with analogizing AI with previous technological transition of humanity, like the industrial revolution or the development of new communication mediums. A core assumption/argument is that AI will not disempower humanity because they will respect the existing legal system, apparently because they can gain from trades with humans.

I think this summarizes my view quite poorly on a number of points. For example, I think that:

1. AI is likely to be much more impactful than the development of new communication mediums. My default prediction is that AI will fundamentally increase the economic growth rate, rather than merely continuing the trend of the last few centuries.

2. Biological humans are very unlikely to remain dominant in the future, pretty much no matter how this is measured. Instead, I predict that artificial minds and humans who upgrade their cognition will likely capture the majority of future wealth, political influence, and social power, with non-upgraded biological humans becoming an increasingly small force in the world over time.

3. The legal system will likely evolve to cope with the challenges of incorporating and integrating non-human minds. This will likely involve a series of revolutionary reforms, and will look very different from the idea of "AIs will fit neatly into human social roles and obey human-controlled institutions indefinitely".

A more accurate description of my view is that humans will become economically obsolete after AGI, but this obsolescence will happen peacefully, without a massive genocide of biological humans. In the scenario I find most likely, humans will have time to prepare and adapt to the changing world, allowing us to secure a comfortable retirement, and/or join the AIs via mind uploading. Trade between AIs and humans will likely persist even into our retirement, but this doesn't mean that humans will own everything or control the whole legal system forever.