LessWrong 2.0 Reader

Alas, thank you for looking into it.

Two physics riddles, since my last riddle has positive karma:

1. Why do we use the right-hand rule to calculate the Lorentz force, rather than using the left-hand rule?

2. Why do planetary orbits stabilize in two dimensions, rather than three dimensions [i.e. a shell] or zero [i.e. relative fixity]? [ It's clear why they don't stabilize in one dimension, at least: they would have to pass through the center of mass of the system, which the EMF usually prevents. ]

My understanding of something here is probably very off, but I'll try stating what my intuition tells me anyway:
I feel like assuming solipsism+idealism patches the issue here. Like the issue here is caused by the fact that the prior the oracle uses to explain its experiences put more weight into being in a universe where there are a lot of simulations of oracles. If it were instead just looking at what program might have generated its past experiences as output, it wouldn't run into the same issue (This is the solipsist-idealist patch I was talking about).

I think a lot of misunderstandings on this topic are because of a lack of clarity about what exact position is being debated/argued for. I think two relevant positions are

• (1) it is impossible to say anything meaningful whatsoever about what any system is computing. (You could write this semi-formally as $\forall$ physical system $x\forall$ computation $c_1,c_2:$ the claims "$x$ performs $c_1$" and "$x$ performs $c_2$" are equally plausible)
• (2) it is impossible to have a single, formal, universally applicable rule that tells you which computation a physical system is running that does not produce nonsense results. (I call the problem of finding such a rule the "interpretation problem", so (2) is saying that the interpretation problem is impossible)

(1) is a much stronger claim and definitely implies (2), but (2) is already sufficient to rule out the type of realist functionalism that OP is attacking. So OP doesn't have to argue (1). (I'm not sure if they would argue for (1), but they don't have to in order to make their argument.) And Scott Aaronson's essay is (as far as I can see) just arguing against (1) by proposing a criterion according to which a waterfall isn't playing chess (whereas stockfish is). So, you can agree with him and conclude that (1) is false, but this doesn't get you very far.

The waterfall thought experiment also doesn't prove (2). It's very hard to prove (2) because (2) is just saying that a problem cannot be solved, and it's hard to prove that anything can't be done. But the waterfall thought experiment is an argument for (2) by showing that the interpretation problem looks pretty hard. This is how I was using it in my reply to Steven on the first post of this sequence [LW(p) · GW(p)]; I didn't say "and therefore, I've proven that no solution to the interpretation problem exists"; I've just pointed out that you start off with infinitely many interpretations and currently no one has figured out how to narrow it done to just one, at least not in a way such that the answers have the properties everyone is looking for.

I think the question isn't whether but when. AGI most obviously is a huge national security opportunity and risk. The closer we get to it, the more evidence there will be.and the more we talk about it, the more attention will be devoted to it by the national security apparatus.

The likely path to takeoff is relatively slow and continuous. People will get to talk to fully human-level entities before they're smart enough to take over. Those people will recognize the potential of a new intelligent species in a visceral way that abstract arguments don't provide.

Can you explain more about why you think [AGI requires] a shared feature of mammals and not, say, humans or other particular species?

The example in the post below is not about an Artificial Intelligence literally at all! If the post were about what AIs supposedly can't do, the central example would have used an AI!

Contra this assertion, Yudkowksy-2007 was very capable of using parables. The "genie" in this article is easily recognized as metaphorically referring to an imagined AI. For example, here is Yudkowsky-2007 in Lost Purposes [? · GW], linking here:

I have seen many people go astray when they wish to the genie [? · GW] of an imagined AI, dreaming up wish after wish that seems good to them, sometimes with many patches and sometimes without even that pretense of caution.

Similarly, portions of [Project Lawful](https://www.lesswrong.com/posts/SA9hDewwsYgnuscae/projectlawful-com-eliezer-s-latest-story-past-1m-words) are about AI, [That Alien Message](https://www.lesswrong.com/posts/5wMcKNAwB6X4mp9og/that-alien-message) is about AI, and so forth.

I'm very sympathetic to the claim that this parable has been misinterpreted. This is a common problem with parables! They are good at provoking thought, they are good at strategic ambiguity, and they are bad at clear communication.

I'm not sympathetic to the claim that this post is not about AI literally at all.

The interpreter, if it would exist, would have complexity. The useless unconnected calculation in the waterfall/rock, which could be but isn't usually interpreted, also has complexity. 

Your/Aaronson's claim is that only the fully connected, sensibly interacting calculation matters.  I agree that this calculation is important - it's the only type we should probably consider from a moral standpoint, for example. And the complexity of that calculation certainly seems to be located in the interpreter, not in the rock/waterfall.

But in order to claim that only the externally connected calculation has conscious experience, we would need to have it be the case that these connections are essential to the internal conscious experience even in the "normal" case - and that to me is a strange claim! I find it more natural to assume that there are many internal experiences, but only some interact with the world in a sensible way.

Regulation to reduce racing. Government regulation could temper racing between multiple western projects. So there are ways to reduce racing between western projects, besides centralising.

Can you say more about the kinds of regulations you're envisioning? What are your favorite ideas for regulations for (a) the current Overton Window and (b) a wider Overton Window but one that still has some constraints?

I appreciate your response, and I understand that you are not arguing in favor of this perspective. Nevertheless, since you have posited it, I have decided to respond to it myself and expand upon why I ultimately disagree with it (or at the very least, why I remain uncomfortable with it because it doesn't seem to resolve my confusions).

I think revealed preferences show I am a huge fan [LW(p) · GW(p)] of explanations of confusing questions that ultimately claim the concepts we are reifying are ultimately inconsistent/incoherent, and that instead of hitting our heads against the wall over and over, we should take a step back and ponder the topic at a more fundamental level first [LW · GW]. So I am certainly open to the idea that “do I nonetheless continue living (in the sense of, say, anticipating the same kind of experiences)?” is a confused question.

But, as I see it, there are a ton of problems with applying this general approach in this particular case. First of all, if anticipated experiences [LW · GW] are an ultimately incoherent concept that we cannot analyze without first (unjustifiably) reifying [LW · GW] a theory-ladden framework, how precisely are we to proceed from an epistemological perspective? When the foundation of 'truth' [LW · GW] (or at least, what I conceive of it to be) is based around comparing and contrasting what we expect to see with what we actually observe experimentally, doesn't the entire edifice collapse once the essential constituent piece of 'experiences' breaks down? Recall the classic (and eternally underappreciated) paragraph from Eliezer:

I pause. “Well . . .” I say slowly. “Frankly, I’m not entirely sure myself where this ‘reality’ business comes from. I can’t create my own reality in the lab, so I must not understand it yet. But occasionally I believe strongly that something is going to happen, and then something else happens instead. I need a name for whatever-it-is that determines my experimental results, so I call it ‘reality’. This ‘reality’ is somehow separate from even my very best hypotheses. Even when I have a simple hypothesis, strongly supported by all the evidence I know, sometimes I’m still surprised. So I need different names for the thingies that determine my predictions and the thingy that determines my experimental results. I call the former thingies ‘belief,’ and the latter thingy ‘reality.’ ”

What exactly do we do once we give up on precisely pinpointing the phrases "I believe", "my [...] hypotheses", "surprised", "my predictions", etc.? Nihilism, attractive as it may be to some from a philosophical or 'contrarian coolness' perspective, is not decision-theoretically useful when you have problems to deal with and tasks to accomplish. Note that while Eliezer himself is not [LW · GW] what he considers a logical positivist, I think I... might be [LW(p) · GW(p)]?

I really don't understand what "best explanation", "true", or "exist" mean, as stand-alone words divorced from predictions about observations we might ultimately make about them.

This isn't just a semantic point, I think. If there are no observations we can make that ultimately reflect whether something exists in this (seems to me to be) free-floating sense, I don't understand what it can mean to have evidence for or against such a proposition. So I don't understand how I am even supposed to ever justifiably change my mind on this topic, even if I were to accept it as something worth discussing on the object-level.

Everything I believe, my whole theory of epistemology and everything else logically downstream of it (aka, virtually everything I believe), relies on the thesis (axiom, if you will) that there is a 'me' out there doing some sort of 'prediction + observation + updating' in response to stimuli from the outside world. I get that this might be [LW · GW] like reifying ghosts [LW · GW] in a Wentworthian sense when you drill down on it, but I still have desires about the world, dammit, even if they don't make coherent sense as concepts! And I want them to be fulfilled [LW(p) · GW(p)] regardless.

And, moreover, one of those preferences is maintaining a coherent flow of existence, avoiding changes that would be tantamount to death [LW(p) · GW(p)] (even if they are not as literal as 'someone blows my brains out'). As a human being, I have preferences over what I experience too, not just over what state the random excitations of quantum fields in the Universe are [LW(p) · GW(p)] at some point past my expiration date. As far as I see, the hard problem of consciousness (i.e., the nature of qualia) has not been close to solved [LW(p) · GW(p)]; any answer to it would have to give me a practical handbook for answering the initial questions I posed to jbash.