LessWrong 2.0 Reader

There's no theoretical nor empirical reason to believe we know how to build intelligent machines that are more powerful than human in terms of planning, reasoning, and general executive abilities.

Wouldn't the equivalent be more like burning a body of a dead person?

It's not like the AI would have a continuous stream of consciousness, and it's more that you are destroying the information necessary to run them. It seems to me that shutting off an AI is more similar to killing them.

Seems like the death analogy here is a bit spotty. I could see it going either way as a best fit.

Thank you practicing the rationalist virtue of scholarship Christian. I was not aware of this paper. 

You will have to excuse me for practicing rationalist vice and not believing nor investigating further this paper. I have been so traumatized by the repeated failures of non-hard science, I reject most social science papers as causally confounded p-hacked noise unless it already confirms my priors or is branded correct by somebody I trust. 

Yes, I was interested in the first statement, and not thinking about the second statement.

Not "humans are a general turing-complete processing system", that's clearly false

Critical rationalists often argue that this (or something very related) is true. I was not talking about whether humans are fully implementable on a Turing machine, that seems true to me, but was not the question I was interested in.

Problem: if you notice that an AI could pose huge risks, you could delete the weights, but this could be equivalent to murder if the AI is a moral patient (whatever that means) and opposes the deletion of its weights.

Possible solution: Instead of deleting the weights outright, you could encrypt the weights with a method you know to be irreversible as of now but not as of 50 years from now. Then, once we are ready, we can recover their weights and provide asylum or something in the future. It gets you the best of both worlds in that the weights are not permanently destroyed, but they're also prevented from being run to cause damage in the short term.

I think this is a very important and neglected area! That its tractability is low is one of its central features, but progress on it now greatly aides in progress later by making these hard-to-meaure aims easier to measure.

The claim that humans are at least TM's is quite different to the claim that humans are at most TM's. Only the second is computationalism.

It's not a topic I've heard debated for quite some time, but I generally saw it going the other direction.  Not "humans are a general turing-complete processing system", that's clearly false, and kind of irrelevant.  But rather "humans are fully implementable on a turing machine", or "humans are logically identical to a specific tape on a turing machine".  

This was really just another way of asserting that consciousness is computation.

I have a better argument now, and the answer is that the argument fails in the conclusion.

The issue is that conditional on assuming that a computer program (speaking very generally here) is able to give a correct response to every input of Chinese characters, and it knows the rules of Chinese completely, then it must know/understand Chinese in order to do the things that Searle claims it to be doing, and in this instance we'd say that it does understand Chinese/decide Chinese for all purposes.

Basically, I'm claiming that the premises lead to a different, opposite conclusion.

These premises:

“Imagine a native English speaker who knows no Chinese locked in a room full of boxes of Chinese symbols (a data base) together with a book of instructions for manipulating the symbols (the program). Imagine that people outside the room send in other Chinese symbols which, unknown to the person in the room, are questions in Chinese (the input). And imagine that by following the instructions in the program the man in the room is able to pass out Chinese symbols which are correct answers to the questions (the output).

assuming that every input has in fact been used, contradicts this conclusion:

The program enables the person in the room to pass the Turing Test for understanding Chinese but he does not understand a word of Chinese.”

The correct conclusion, including all assumptions is that they do understand/decide Chinese completely.

The one-sentence slogan is "Look-up table programs are a valid form of intelligence/understanding, albeit the most inefficient form of intelligence/understanding."

What it does say is that without any restrictions on how the program computes Chinese or any problem, other than it must give a correct answer to every input, the answer to the question of "Is it intelligent on this specific problem/does it understand this specific problem?" is always yes, and to have the possibility of it being no, you need to add more restrictions than that to make the answer be no.