Incoherence of unbounded selfishness

post by emmab · 2022-07-26T22:27:21.757Z · LW · GW · 2 comments

Contents

2 comments

Unbounded agentic selfishness isn't possible unless you can unboundedly index on and retrocausally compress "self". Otherwise future "you" is just this animal walking around, and how do you specify them in particular for good things to happen to? In practice, evil people do this by cancer like "make good things happen to people with my skin color".

If you believe humans are "not coherent agents" i.e. what values someone ends up with is underdetermined, then why would you believe that acquisition of ability to index on values as self is coacquired with the underdetermined acquisition of those values? Like, that would be inconsistently postulating both having and lacking "inner alignment".

(Note that Lesswrongers don't think about cybernetics much like "but how does the thing reference itself", if this style of argument seems unusual to you)

2 comments

Comments sorted by top scores.

comment by wolajacy · 2022-07-26T23:19:03.557Z · LW(p) · GW(p)

The argument is very unclear clear to me. What does "unbounded" mean? What does it mean to "retrocausally compress 'self'"?

 Are you postulating that:
 - the notion of "an individual" does not make sense even in principle
 - there exists something like "self"/"individual" in general, but we don't know how to define rigorously
 - there exists something like "self"/"individual", but specific individuals (people, in this case) are not able to precisely define 'themselves'
 - some fourth option?

(The second and third paragraph are even less clear to me, so if they present separate lines of thought, maybe let's start with the first one)

Replies from: emmab
↑ comment by emmab · 2022-07-26T23:49:17.479Z · LW(p) · GW(p)

What does "unbounded" mean?

Without bound, as in, without there existing some specific bound you will never surpass.

What does it mean to "retrocausally compress 'self'"

Make yourself easier for your past self to index on. e.g. for an evil version, if Horde Prime wants Horde clones to work to benefit Horde Prime, Horde Prime can work to place himself in the center of the universe, which he previously programmed other Hordes to care about.

 Are you postulating that: [...]

I'm saying something closest to #3. In order to specify an individual, you have to be able to point at them in some way.