LessWrong 2.0 Reader

My aim here wasn't to give a complete account of pandemic weirdness, just to show how alternative language can help us dissect the situation in a more detailed way than seems possible with "social reality vs. physical reality."

But it's very different when "there are some people that don't trust the true information we're getting" and "none of the institutions that were supposed to be giving accurate information were." 

One of these points at an idiosyncratic problem with a specific group getting a bit confused, and one points at a systemic problem with our institutions' relationship with truth.  Which thing you focus on has direct implications for how useful this post is.

I also tried to give specific examples, see the "friends" example in my other comment

Ah, I hadn't seen that. I don't feel convinced, because it assumes that the AI system has a "goal" that isn't "be corrigible". Or perhaps the argument is that the goal moves from "be corrigible" to "care for the operator's friends"? Or maybe that the goal stays as "be corrigible / help the user" but the AI system has a firm unshakeable belief that the user wants her friends to be cared for?

we'll make powerful systems

But... why can't I apply the argument to "powerful", and say that it is extremely unlikely for an AI system to be powerful? Predictive, sure, but powerful?

My model of you responds "powerful is upstream of goal-accomplishing" or "powerful is downstream of goal-directedness which is upstream of goal-accomplishing", but it seems like you could say that for corrigibility too: "corrigibility is upstream of effectively helping the user".

As for "why can we hope to solve it", I can imagine lots of possible solution directions

Thanks, that was convincing (that even under radical uncertainty there are still avenues to pursue).

None of this is to say that religions don't sometimes do bad things

Furthermore, self-improvement has a lot of potential for harm too.

Out of curiosity I looked up what you were teaching in the spring - the problem of induction, right? (I'll be surprised and impressed if you managed to foist a reading from Li and Vitanyi on your students :P ) I'm definitely curious about what you think of the progress in probability, and what morals one could draw from it.

I'd actually checked because I thought it would be the philosophy of psychology. That seems like one of those areas where there were, in hindsight, obvious past mistakes, and it's not clear how much of the progress has been emprirical versus things that could have been figured out using the empirical knowledge of the time.

I spent time working in fatal car crash investigation (reading crash reports and doing engineering analysis, nothing as gory as you're probably picturing), and car crashes often involved massive head trauma or would, at a minimum, require *hours* of lag time before the cryonics team could make it there. I'd say at a complete guess that only about 10% involved people dying in hospital later on (i.e. under circumstances that a cryo team could get to them in time to prepare the body).

My impression of the technology is that it's too much in its infancy to be able to say with any sort of confidence that a body that had been left with minimal treatment for a good 8-10 hours would be in a good state for preservation. And my understanding is that after only a few minutes/hours the brain starts to really degrade.

This is a major reason I'm not considering yet. I also live in a country without a good cryo organisation, and the exchange rates make the fees for Alcor quite a lot when I am not convinced I'd get the value. I also think the 5% figure is way too high.

I mean admittedly, pascal's wager comes into play a bit here, but I'm not convinced that my current jurisdiction is a good place to die and be cryopreserved, and I have no plans to move.

There's a huge structural similarity between the proof that '1 + 1 != 3' and '1+1 != 4'; like, both are generic instances of the class '1 + 1 != n \forall n != 2'. We can increase the number of numbers without decreasing the plausibility of this claim (like, consider it in Z/4, then Z/8, then Z/16, then...).

I feel like that's exactly my point? Showing that something is a conjunction of a bunch of claims should not always make you think that claim is low probability, because there could be structural similarity between those claims such that a single argument is enough to argue for all of them.

(The claims "If X drifts away from corrigibility along dimension {N}, it will get pulled back" are clearly structurally similar, and the broad basin of corrigibility argument is meant to be an argument that argues for all of them.)

Similarly, if we make an argument that something is an attractor in N-dimensional space, that does actually grow less plausible the more dimensions there are, since there are more ways for the thing to have a derivative that points away from the 'attractor,' if we think the dimensions aren't all symmetric.

1. Why aren't the dimensions symmetric?

2. I somewhat buy the differential argument (more dimensions => less plausible) but not the absolute argument (therefore not plausible); this post is arguing for the absolute version:

it starts to feel awfully unlikely that corrigibility is really a broad basin of attraction after all

3. I'm not sure where the idea of a "derivative" is coming from -- I thought we were talking about small random edits to the weights of a neural network. If we're training the network on some objective that doesn't incentivize corrigibility then certainly it won't stay corrigible.

Rather than having to petition the commune for a day’s use of a red shirt, I can simply decide to wear my red shirt today. I can make solid plans around decisions that I’m the only major input to. This will sometimes lead to socially suboptimal decisions if coordination were free--maybe I look really bad in red, and a wise commune would give me blue instead--but given that coordination is not free, this is often our best available option.

It's also worth pointing out also that coordination isn't always desirable. If the average member of the commune thinks the best way to deal with a disease is to direct resources at empty passenger airlines and military helicopters, but you think spending time and resources developing vaccines is more important, you can use your own property to develop them instead of trying to get permission.

I think the general case of this is that private property lets you do things that the government* doesn't think are worth it, at the expense of not being able to direct as many resources as the things the government thinks are most important.

* I'm not sure of the right term to use here but government seems like the best word for it

Oh I completely agree. My aim here wasn't to give a complete account of pandemic weirdness, just to show how alternative language can help us dissect the situation in a more detailed way than seems possible with "social reality vs. physical reality."

In addition to the coordination problem, there is often an incentive problem. Too many decision-makers can be a problem, but even a small number of decision-makers who don't get the benefit/harm of their decision can be problematic.

Private property is one way to ensure that the cost or reward of a decision is felt by the decision-maker.

Here you go (sorry, I actually forgot the title)

https://www.lesswrong.com/posts/qDmnyEMtJkE9Wrpau/simulacra-levels-and-their-interactions [LW · GW]