LessWrong 2.0 Reader

I can imagine this being the solution, but

• this would require a pretty small universe
• if this is not the solution, my understanding is that IBP agents wouldn’t know or care, as regardless of how likely it is that we live in naive MWI or Tegmark IV, they focus on the minimal worlds required. Sure, in these worlds, not all Everett branches coexist, and it is coherent for an agent to focus only on these worlds; but it doesn’t tell us much about how likely we’re in a small world. (I.e., if we thought atoms are ontologically basic, we could build a coherent ASI that only cared about worlds with ontologically basic atoms and only cared about things made of ontologically basic atoms. After observing the world, it would assume it’s running in a simulation of a quantum world on a computer build of ontologically basic atoms, and it would try to influence the atoms outside the simulation and wouldn’t care about our universe. Some coherent ASIs being able to think atoms are ontologically basic shouldn’t tell us anything about whether atoms are indeed ontologically basic.)

Conditional on a small universe, I would prefer the IBP explanation (or other versions of not running all of the branches and producing the Born rule). Without it, there’s clearly some sort of sampling going on.

Is there some level of discomfort short of extreme torture for a billion to suffer where the balance shifts?

For clarity – I do agree that a simple stacking model like this has its flaws as each individual ‘unit’ of discomfort / pain will not never be exactly equal in practice.

If you have the time, I’d like you to read my response to CstineSublime’s comment.

Regardless of whether or not some version of this thought experiment ever becomes a serious practical problem, if there was some non-zero chance of having a situation where a decision like this had to be made, wouldn’t it make sense to have some method of comparison?

In reality, every ‘stacking theory’ requires a bunch of assumptions, but if the decision had to be made for some reason, do you think a logarithmic scale is a more appropriate one to use?

There isn't a housing shortage. There are more houses than there are households. This from Ian Mulheirn:

No. Back in 1991 there were just over 3.0% more houses than there were households in the UK according to government data. Today, using the ONS’s latest household estimates, there appear to be 5.2% more places to live than there are households that want to live in them. In fact growth in the stock of dwellings appears to have outstripped that of households over the past 50 years or so. This is a strange sort of ‘endemic shortage’.

COMMENT THREAD

If you comment anywhere other than here, Nathan will delete your comment.

I think you're imagining that we modify the shrink-and-reposition functions each iteration, lowering their scope? I. e., that if we picked the topmost triangle for the first iteration, then in iteration two we pick one of the three sub-triangles making up the topmost triangle, rather than choosing one of the "highest-level" sub-triangles?

Something like this:

If we did it this way, then yes, we'd eventually end up jumping around an infinitesimally small area. But that's not how it works, we always pick one of the highest-level sub-triangles:

Note also that we take in the "global" coordinates of the point we shrink-and-reposition (i. e., its position within the whole triangle), rather than its "local" coordinates (i. e., position within the sub-triangle to which it was copied).

Here's a (slightly botched?) video explanation.

I suppose you’re right.

Maybe ‘death’ was a poor example as it inherently leads us to a state of relief from discomfort. If we instead take the example of ‘extreme torture’, then it makes more sense to compare the two.

The ‘discomfort’ I was referring to was more from a ‘physical sensation’ perspective rather than any second-order effects.

Imagine these experiences occur in a closed system with no influence on the outside world. Each person has been brought into existence by some higher power specifically for the purposes of this experiment. They have no family, no friends, and are genetically identical.

Imagine 1,000,000,000 participants with a single rational observer. The observer is forced by the higher power to make a choice – so some method of comparison is required.

Would it make more sense for the observer to choose for every single one of the participants to be burdened with a speck of dust in their eye, or for one single participant to be subjected to ‘extreme torture’?

Is there any point where increasing the number changes your mind?

For me it doesn’t matter how many participants there are – the option of torture should never be taken.

The ‘logarithmic stacking theory’ allows this to work mathematically, while a linear model does not.

Two notes: 

1. I think the title is a somewhat obscure pun referencing the old saying that Stanford was the "Hardvard of the West". If one is not familiar with that saying, I guess some of the nuance is lost in the choice of term. (I personally had never heard that saying before recently, and I'm not even quite sure I'm referencing the right "X of the West" pun)
2. habryka did have a call with Nick Bostrom a few weeks back, to discuss his idea for an "FHI of the West", and I'm quite confident he referred to it with that phrase on the call, too. Far as I'm aware Nick didn't particularly react to it with more than a bit humor. 

Thanks for pointing this out, I missed a word. I have added it now.

I don't think this phenomenon is just related to the training data alone because in RLLMv3, the " Leilan" glitch mode persisted while " petertodd" became entirely unrelated to bitcoin. It's like some glitch tokens can be affected by the amount of re-training and some aren't. I believe that there is something much deeper is happening here, an architectural flaw that might be related to the token selection/construction process.