LessWrong 2.0 Reader

Thank you, this information was useful for a project I'm working on.

Your reasoning makes sense with regards to how a more authoritarian government would make it more likely that we can avoid x-risk, but how do you weigh that against the possibility that an AGI that is intent-aligned (but willing to accept harmful commands) would be more likely to create s-risks in the hands of an authoritarian state, as the post author has alluded to?

Also, what do you make of the author's comment below [LW(p) · GW(p)]?

• In general, the public seems pretty bought-in on AI risk being a real issue and is interested in regulation. Having democratic instincts would perhaps push in the direction of good regulation (though the relationship here seems a little less clear).

The thing I would bet is "your 'build a lifeboat for some people-like-you to move to somewhere other than EA' plan will work at least a bit, and, one of the important mechanisms for it working will be those effortful posts you wrote."

A thing I wanted to check: were you grokking the general premise that calculus and much of physics haven't been invented yet, and the metaphor here is more about an early stage physicist who has gotten a sense of how "I feel confused here, and I might need to invent [something that will turn out to be calculus]", but, it's at an early enough stage that crisp physics to easily explain it doesn't exist yet?

(If you did get that part, I'm interested in hearing a little bit more about what felt annoying, and if you didn't get that, I'm interested in what sort of things might have helped make the pre-physics/calculus part more clear)

I definitely wouldn't bet money that EA will have evolved into something you can live with (Neither EA nor the threads of rationality that he affeted evolved into things Ben Hoffman could live with)

But, I do think there is something important about the fact that, despite that, it is inaccurate to say "the critiques dropped like a stone through water" (or, what I interpret that poetry to mean, which is something like "basically nobody listened at all". I don't think I misunderstood that part but if I did then I do retract my claim)

How much are you arguing about wording, vs genuinely believe and would bet money that in 3-5 years my work will have moved EA to something I can live with?

Your brain has a giant space of possible generative models^[2] that map from underlying states of the world (e.g. “there’s a silhouette dancer with thus-and-such 3D shape spinning clockwise against a white background etc.”) to how the photoreceptor cells would send signals into the brain (“this part of my visual field is bright, that part is dark, etc.”)

How do you argue that the models are really implemented backwards like this in the brain?

Yes. That is still planned!!! I'm just very bad at writing. 
Primer: I've been collecting more data since and something super weird happened. I tried to gain more weight again to redo experiments, it was suprisingly harder than expected to gain more weight, but I managed. But super weird. After gaining more weight, going back on the half-assed potato diet didn't work as well anymore. I still didn't manage to loose the weight I intentionally gained! If I went on a total potato I would loose weight. But the semi-potato diet is not enough to compensate the days where I go to a party (or there is some sort of event with lots of food)! Super interesting because previously the semi-potato diet could easy compensate for those big meals days.  

Even though I can't critique the details, I do think it is important to note that I often find claims of similarity like this in areas I understand better to not be very illuminating because people want to find similarities/analogies to understand it more easily.

Agreed, that's definitely a general failure mode.

As other people have said, this is a known argument; specifically, it’s in The Generalized Anti-Zombie Principle in the Physicalism 201 series. From the very early days of LessWrong

Albert: “Suppose I replaced all the neurons in your head with tiny robotic artificial neurons that had the same connections, the same local input-output behavior, and analogous internal state and learning rules.”

I think this proof relies on three assumptions. The first (which you address in the post) is that consciousness must happen within physics. (The opposing view would be substance dualism where consciousness causally acts on physics from the outside.) The second (which you also address in the post) is that consciousness and reports about consciousness aren’t aligned by chance. (The opposing view would be epiphenomenalism, which is also what Eliezer trashes extensively in this sequence.) physical duplicate might do the same, although. that would imply the original's consciousness is epiphenomenal. Which is itself a reason to disbelieve in p-zombies , although not an impossibility proof.

This of course contradicts the Generalised Anti Zombie Principle announced by Eliezer Yudowsky. The original idea was that in a zombie world, it would be incredibly unlikely for an entity's claims of consciousness to be caused by something other than consciousness. "

Excluding coincidence doesn't proved that an entity's reports of consciousness are directly caused by its own consciousness. Robo-Chalmers will claim to be conscious because Chalmers does. It might actually be conscious it it might not. The fact that claim is made does not distinguish the two cases. Yudkowsky makes much of the fact that Robo-Chalmers claim.would be caused *indirectly* by consciousness -- Chalmers has to be conscious in order to make a computational duplicate of his consciousness -- but at best that refutes the possibility of a zombie world, where entities claim to be conscious, although consciousness has never existed. Robo-Chalmers would still be possible in this world for reasons Yudkowsky accepts. So there is one possible kind of zombie, so the *Generalised* Anti Zombie Principle is false

(Note that I am talking about computational zombies, or c-zombies, not p-zombies

Computationalism isn't a direct consequence of physicalism. Physicalism has it that an exact atom-by-atom duplicate of a person will be a person and not a zombie, because there is no nonphysical element to go missing. That's the argument against p-zombies. But if actually takes an atom-by-atom duplication to achieve human functioning, then the computational theory of mind will be false, because there CTM implies that the same algorithm running on different hardware will be sufficient. Physicalism doesn't imply computationalism, and arguments against p-zombies don't imply the non existence of c-zombies-duplicates that are identical computationally, but not physically).

@Richard_Kennaway

That sounds like a Chalmers paper. https://consc.net/papers/qualia.html