The Gears of Impact

post by TurnTrout · 2019-10-07T14:44:51.212Z · score: 43 (15 votes) · LW · GW · 5 comments

Scheduling: The remainder of the sequence will be released after some delay.

Exercise: Why does instrumental convergence happen? Would it be coherent to imagine a reality without it?

Notes

5 comments

Comments sorted by top scores.

comment by adamShimi · 2020-03-06T13:11:32.033Z · score: 4 (2 votes) · LW(p) · GW(p)

I don't get why the client AU from the perspective of the robber doesn't drop when the robber enters, or just before? Because even if I'm the robber and I know they won't like it and won't be able to do things after I'm in, they can still do things in the bank before I'm in. And if they're out before I come in, their AU will be the same than if I was never there.

comment by TurnTrout · 2020-03-06T14:44:57.078Z · score: 2 (1 votes) · LW(p) · GW(p)

I guess I implicitly imagined the robber could predict the clients with fairly high accuracy. What you described is also plausible.

comment by G Gordon Worley III (gworley) · 2019-10-16T02:12:20.570Z · score: 4 (2 votes) · LW(p) · GW(p)

The way you presented AU here makes me think of it in terms of "attachment", as in the way we tend to get attached to outcomes that haven't happened yet but that we expect to and then can be surprised in good and bad ways when the outcomes are better or worse than we expected. In this way impact seems tied in with our capacity to expect to see what we expect to see (meta-expectations?), e.g. I 100% expect a 40% chance of X and a 60% chance of Y happening. That 100% meta-expectation creates a kind of attachment that doesn't leave any room for being wrong, and so just seeing something happen in a way that makes you want to update your object level expectations of X and Y after the fact seems to create a sense of impact.

comment by JohnBuridan · 2020-04-30T15:13:50.510Z · score: 1 (1 votes) · LW(p) · GW(p)

You said, "Once promoted to your attention, you notice that the new plan isn't so much worse after all. The impact vanishes." Just to clarify, you mean that the negative impact of the original plan falling through vanishes, right?

When I think about the difference between value impact and objective impact, I keep getting confused.

Is money a type of AU? Money both functions as a resource for trading up (personal realization of goals) AND as a value itself (for example when it is an asset).

If this is the case, then any form of value based upon optionality violates the "No 'coulds' rule," doesn't it?

For example, imagine I have a choice between hosting a rationalist meetup and going on a long bike ride. There's a 50/50 chance of me doing either of those. Then something happens which removes one of those options (say a pandemic sweeps the country or something like that). If I'm interpreting this right, then the loss of the option has some personal impact, but zero objective impact.

Is that right?

Let's say an agent works in a low paying job that has a lot of positive impact for her clients - both by helping them attain their values and helping them increase resources for the world. Does the job have high objective impact and low personal impact? Is the agent in bad equilibrium when achievable objective impact mugs her of personal value realization?

Let's take your examples of sad person with (P, EU):

Mope and watch Netflix (.90, 1). Text ex (.06, -500). Work (.04, 10). If suddenly one of these options disappeared is that a big deal? Behind my question is the worry that we are missing something about impact being exploited by one of the two terms which compose it and about whether agents in this framework get stuck in strange equilibria because of the way probabilities change based on time.

Help would be appreciated.

comment by TurnTrout · 2020-04-30T20:39:19.311Z · score: 2 (1 votes) · LW(p) · GW(p)

Just to clarify, you mean that the negative impact of the original plan falling through vanishes, right?

Yes.

When I think about the difference between value impact and objective impact, I keep getting confused.

  • Value impact affects agents with goals very similar to yours (e.g. torture of humans on the faraway planet). Think of this as "narrow" impact.
  • Objective impact matters ~no matter what your goal. Therefore, getting rich is usually objectively impactful. Being unable to do a bunch of stuff because of a pandemic – objectively impactful.

There's a 50/50 chance of me doing either of those. Then something happens which removes one of those options (say a pandemic sweeps the country or something like that). If I'm interpreting this right, then the loss of the option has some personal impact, but zero objective impact.

Is that right?

Not quite – being confined is objectively impactful, and has some narrow value impact (not being able to see your family, perhaps).

Let's say an agent works in a low paying job that has a lot of positive impact for her clients - both by helping them attain their values and helping them increase resources for the world. Does the job have high objective impact and low personal impact? Is the agent in bad equilibrium when achievable objective impact mugs her of personal value realization?

Just because something has a positive objective impact on me doesn't mean I haven't been positively impacted. Value/objective is just a way of decomposing the total impact on an agent – they don't trade off against each other. For example, if something really good happens to Mary, she might think: "I got a raise (objective impact!) and Bob told me he likes me (value impact!). Both of these are great", and they are both great (for her)!