LessWrong 2.0 Reader

Yeah, I’ve written about that in §2.7.3 here [LW · GW].

I kinda want to say that there are many possible future outcomes that we should feel happy about. It’s true that many of those possible outcomes would judge others of those possible outcomes to be a huge missed opportunity, and that we’ll be picking from this set somewhat arbitrarily (if all goes well), but oh well, there’s just some irreducible arbitrariness is the nature of goodness itself.

What is often left out in papers is all of these experiments and the though chains people had about them.

I keep hearing about dual-use risk concerns when I mention automated AI safety research. Here’s a simple solution that could even work in a startup setting:

Keep all of the infrastructure internally and only share with vetted partners/researchers.

You can hit two birds with one stone:

• Does not turn into a mass-market product that leads to dual-use risks.
• Builds a moat where you have complex internal infrastructure which is not shared, only the product of that system is shared. Investors love moats, you just got to convince them that this is the way to go for a product like this these days.

You don’t market the product to mass-market, you just find partners and use the system to spin out products and businesses that have nothing to do with frontier models. So, you can repurpose the system for specific application areas without releasing the platform and process, which would be copied in a day in the age of AI anyways.

Isn't this what the "coherent" part is about? (I forget.)

I think there's a strong motivation to believe in hell for other people. The wicked flourish like the green bay tree, and where is justice?

Alternatively, belief in hell for other people is mere spitefulness.

Also, I believe inventing the tortures of hell is very like the same drive that causes people to write horror fiction, though I have no idea of why they do it, or why I like horror fiction.

It's evidence that God loves complexity even more than He loves beetles.

Is there really some particular human whose volition you'd like to coherently extrapolate over eternity but where you refrain because you're worried it will generate infighting? Or is it more like, you can't think of anybody you'd pick, so you want a decision procedure to pick for you?

If there is some particular human, who is it?

Thanks for raising this important point. When modeling these situations carefully, we need to give terms like "today" a precise semantics that's well-defined for the agent. With proper semantics established, we can examine what credences make sense under different ways of handling indexicals. Matthias Hild's paper "Auto-epistemology and updating" demonstrates how to carefully construct time-indexed probability updates. We could then add centered worlds or other approaches for self-locating probabilities.

Some cases might lead to puzzles, particularly where epistemic fixed points don't exist. This might push us toward modeling credences differently or finding other solutions. But once we properly formalize "today" as an event, we can work on satisfying richness conditions. Whether this leads to inconsistent attitudes depends on what constraints we place on those attitudes - something that reasonable people might disagree about, as debates over sleeping beauty suggest.
 

Pasek did couchsurf at my place in the days after a LessWrong Community Weekend in Berlin. That was before he went to the Bay Area, so probably 8 or 9 years ago and before he seemed to make contact with Ziz which was after Pazek left the Bay Area and moved to live with other rationalists in a group house in Gran Canaria. Pazek's contact with Ziz seemed to be mostly online while living in Gran Canaria.

If you read Pasek's post where he thinks about committing suicide, there's plenty of TDT-thinking in it. I matched my idea of how Pasek thinks even before engaging with Ziz.

Pasek was TDT-ish vegan. 

Pasek had some QS tracking for how he spent every waking hour of the day that he did on paper and seemed to not suffer from akrasia while guiding his actions. 

If I remember right, that he said that stealing is okay in cases where the TDT calculation would be in favor of stealing where traditional morality would say stealing is bad. I don't think that resulted in Pasek actually stealing things but I think we talked about some case where he thought it was justified to steal which surprised me at the time. My memory is here very fussy.

I dislike raw oysters quite a bit, but they're okay cooked.

Speaking of logical fallacies, the fact that one person loves a thing means that other people will even tolerate it is not strongly likely. I don't know that people even have an obligation to try things other people love.

And yet, the temptation to think that other people do or should love what one loves it very strong. "I think this is great!" just doesn't feel as true as "This is great!".