LessWrong 2.0 Reader

I wanted to reply to this because it's easy for me to do and the subject sounds important to you ("a crux I have with the entire alignment community").

In particular, I'd expect a surprising amount of disagreement on whether making a hell is good, if you managed to sell it as eternally punishing a favored enemy.

Agreed for current humans (it's not surprising for me having witnessed it).

I'm less confident about augmented humans, although I do believe at least some will still want hells. I would not go so far as to say "idealizations are so similar that no conflicts are expected", but I think there will be some convergence when humans (and others) are augmented, at least in cases where their previous desire to torture someone was based in false beliefs, which may encompass more cases than expected when we consider metaphysical beliefs^[1].

though I think that people like Eliezer don't realize that human values conflicts sort of break collective CEV type solutions

What is a "collective CEV type solution"? I see two possible meanings:

1. "The CEV of {everyone combined}"
2. The CEVs of each individual, all combined (simplistically, e.g. averaged^[2]) into one utility function.

The first one seems impractical to me because "everyone combined" is much less coherent or unitary of an entity than a single human. The second one seems as possible as the CEV of one individual^[3], and of course it contains some conflicts, like over whether hells should exist (anywhere), or whether it's okay for anyone anywhere^[4] to (for example) have grey eyes.

I would be surprised if Eliezer does not realize this (high confidence).

And yes, my preferred solution does involve declaring "no hells anywhere" as a hard line. Though, even as someone with an analogy to grey eyes, I wouldn't mind leaving the question of grey eyes one and almost all other imaginable universe-wide-values disagreements to be resolved by some sort of everyone-wide process (which does of course still mean they have to be resolved, just by a process most would prefer instead of 'the most powerful being makes all the rules').

Agree with this, which handles some cases, but my worry is that there are still likely to be big values conflicts where one value set must ultimately win out over another.

Hm, I wonder what cases you think it doesn't handle? Like, yeah there's the case of the hypothetical person who only values hell, but doesn't the second half of what you agreed with handle most realistic cases? (The below quote)

also on the "lose out" phrasing: even if someone "wants at least some people to have tormentful lives", they don't "lose out" overall if they also positively value other things / still negatively value any of the vast majority of beings having tormentful lives.

But of course, such terminal conflicts have to be resolved in some way, even if they don't sway the entire scale for nearly all beings. Is there more to your position, or just that? I wonder if this is really something you and the alignment community disagree on, or maybe if what I understood as your claims were actually weaker than your true meaning or beliefs.

1. ^{^}

   (As a trivial, non-metaphysical case: "Humans with [for example] grey eyes intrinsically seek to do bad."

   Examples of metaphysical beliefs which can be required parts of many's preference for certain others being harmed:

   1. "libertarian free will exists and people have souls, so [implicitly] people who do bad things have evil souls"
   2. "suffering is something which can be inflicted on individual personalities, rather than just on the selfless neural architectures which host those personalities"

   Of course not everyone needs to learn the correct metaphysics

2. ^{^}

   Actually, something like: {Scale the output of each function so that the average of its absolute value assigned to each possible world equals 1}, then maximize {the average all the functions ('s non-absolute but still scaled values)}

3. ^{^}

   (although certainly some individuals are better modeled as having CEVs (being idealizeable without fundamental change the original them would regard as ~a different person) than others)

4. ^{^}

   (in contrast, a preference to exist in a localized context, of very many contexts, where everyone naturally has grey eyes (or some ~spirituality/etc) and okay with this, and others are not allowed to enter, would not itself cause as much conflict in the end, as it is not a preference which seeks a universe-wide restriction, as "no hells" and "no grey eyes" do.)

I don’t believe burnout is real. I have theories on why people think it’s real

More interesting would be to hear why you don’t think it’s real. (“Why do people think it’s real” is the easiest thing in the world to answer: “Because they have experienced it”, of course. Additional theorizing is then needed to explain why the obvious conclusion should not be drawn from those experiences.)

Thank you again. 

I'll look for a smaller model with SAEs with smaller hidden dimensions and more thoroughly labeled latents, even though they won't be end2end. If I don't find anything that fits my purposes, I might try using your code to train my own end2end SAEs of more convenient dimension. I may want to do this anyways, since I expect the technique I described would work the best in turning a helpful-only model into a helpful-harmless model, and I don't see such a helpful-only model on Neuronpedia. 

If the FFNN has a hidden dimension of 16, then it would have around 1.5 million parameters, which doesn't sound too bad, and 16 might be enough to find something interesting.

Low-rank factorization might help with the parameter counts. 

Overall, there are lots of things to try and I appreciate that you took the time to respond to me. Keep up the great work!

Wednesday yes, sorry.

I procrastinated on this, but donated $200 today (the day after the fundraiser ended). I have been reading the site for 4 years, and it has been worth significantly more than $50 per year.

According to the site metrics there are around 15,000 logged-in monthly active users, so if everyone donated $200 that would exactly equal the $3,000,000 target. This made $200 seem like a reasonable number. 

Yes, and (for certain mainstream interpretations) nothing in quantum mechanics is probabilistic at all: the only uncertainty is indexical.

on extensions: note you can use ublock origin^[1] to stop sites from loading by setting a filter against their main elements^[2]. this is preferable to minimize the amount of extensions which can read site data^[3], at least if you already use ublock.

1. ^{^}

   https://github.com/gorhill/uBlock/wiki/uBlock-Origin-is-completely-unrelated-to-the-web-site-ublock.org

2. ^{^}

   (for example, now if i go to reddit, only the header bar loads and the rest is white; if i go to youtube, no video panes load, so i can only open videos found elsewhere, like by searching for a song and clicking the link)

3. ^{^}

   i remember finding an extension which claimed to block some individual site, but tried to request read access on all sites

I had a vaguely favorable reaction to this post when it was first posted.

When I wrote my recent post on corrigibility [LW · GW], I grew increasingly concerned about the possible conflicts between goals learned during pretraining and goals that are introduced later. That caused me to remember this post, and decide it felt more important now than it did before.

I'll estimate a 1 in 5000 chance that the general ideas in this post turn out to be necessary for humans to flourish.

Thanks very much! Yeah, I agree political will seems like a big issue. But I also hear people saying that they don't know what to push for, so I wanted to try to offer a concrete example of a system that wasn't as destructive to any constituency's interests as e.g. a total pause.

Stable is not a virtue, nor is our equilibrium well-tolerated. The problems it causes in terms of health, cost and homelessness are central political issues and have been for a long time.

I also have no idea why you assume I’m “ignoring” these “lessons” you’re handwaving at. It’s a pretty annoying rhetorical move.