LessWrong 2.0 Reader

I have grown pessimistic about our ability to solve the open technical problems even given 100 years of work on them. 

Why? 

Just to confirm, this means that the thing I put in quotes would probably end up being dynamically inconsistent? In order to avoid that, I need to put in an additional step of also ruling out plans that would be dominated from some constant prior perspective? (It’s a good point that these won’t be dominated from my current perspective.)

I'm not sure how I feel about seed oils [LW · GW] generally, but I know they're higher in Omega-6 fatty acids. From the NIH: 

Omega-3s are utilized by the body to resolve and lower inflammation, whereas omega-6 polyunsaturated fatty acids are primarily used for increasing inflammation. Thus, the rise in the omega-6/3 ratio over the past 100 years may be driving chronic low-grade inflammatory conditions including autoimmune diseases, allergies and asthma.

I'm not entirely sure what you've looked at in the literature; have you seen "Direct Validation of the Information Bottleneck Principle for Deep Nets" (Elad et al.)? They use the Fenchel conjugate

This turns finding the KL-divergence into an optimisation problem for \(f^*(x) = \log \frac{p(x)}{q(x)}\). Since

you can train a neural network to predict the mutual information. For the information bottleneck, you would train two additional networks. Ideal lossy compression maximises the information bottleneck, so this can be used as regularization for autoencoders. I did this for a twenty-bit autoencoder of MNIST (code). Here are the encodings without mutual information regularization, and then with it:

Notice how the digits are more "spread out" with regularization. This is exactly the benefit variational autoencoders give, but actually based in theory! In this case, my randomness comes from the softmax choice for each bit, but you could also use continuous latents like Gaussians. In fact, you could even have a deterministic encoder, since the mutual information predictor network is adversarial, not omniscient. The encoder can fool it during training by having small updates in its weights make large updates in the latent space, which means (1) it's essentially random in the same way lava lamps are random, (2) the decoder will learn to ignore noise, and (3) the initial distribution of encoder weights will concentrate around "spines" in the loss landscape, which have lots of symmetries in all dimensions except the important features you're encoding.

The mutual information is cooperative for the decoder network, which means the decoder should be deterministic. Since mutual information is convex, i.e. if we had two decoders \(\phi_1, \phi_2: Z\to Y\), then

Every stochastic decoder could be written as a sum of deterministic ones, so you might as well use the best deterministic decoder. Then

so you're really just adding in the entropy of the decoded distribution. The paper "The Deterministic Information Bottleneck" (Strauss & Schwab) argues that the encoder \(\psi: X\to Z\) should also be deterministic, since maximising

seems to reward adding noise irrelevant to \(X\). But that misses the point; you want to be overwriting unimportant features of the image with noise. It's more clear this is happening if you use the identity 

(the second term is constant). This is the same idea behind variational autoencoders, but they use \(\mathrm{KL}(\psi(X)||\mathcal{N}(0, 1))\) as a cheap proxy to \(H(X|\psi(X))\).

The AI-generate prose is annoying to read. I haven’t read this closely, but my guess is these arguments also imply that CNNs can’t classify hand-drawn digits.

One benefit of his 'no-nut January' is that by cutting out peanuts entirely, he's also avoiding problems from oxalates. I would expect powdered peanut butter to be as dangerous in that regard.

due to Taking Seriously things like radical veganism

I take seriously radical animal-suffering-is-bad-ism^[1], but we would only save a small portion of animals by trading ourselves off 1-for-1 against animal eaters, and just convincing one of them to go vegan would prevent at least as many torturous animal lives in expectation, while being legal. I think there must be additional causes, like the weird decision theory people have mentioned, although I think even that is insufficiently explanatory, as I explain near the end.

That said, taking animal suffering seriously does change the moral status of killing an average knowing animal-eater to something which is deontologically understandable, even if it's still strategically very bad.

So while I don't endorse the actions, I mostly feel empathy for Ophelia and the others and hope that they'll be okay. Maybe it's like how I'd feel empathy for an altruist who couldn't handle living in this world and committed suicide, cause that's also strategically bad and reckless.. but very understandable to me, as one who knows how alienating it can be.

I haven't seen others on LW with this sentiment, maybe they've felt afraid to express it (as I do). In which they were alienated altruists who couldn't handle this world and seemingly went a little insane (given the incorrect beliefs about decision theory). Most people struggle to stay dispassionately rational when faced with something which they regard as very morally bad. It is hard to live in a world one believes to contain atrocities.

It was once harder for me to live in this world too, but I adapted myself into a better consequentialist. That is a grueling and non-default thing to do; "There will soon be horror in front of you, young altruist, but you are not allowed to directly intervene, because if you do you will be arrested, and you won't be able to stop others from doing the same horror. That's right, there are many, many others doing the same horror, and you will often have to not voice objection to it while you plan how to make it stop in a lasting way." That is not the kind of situation a standard human is capable of handling well. 

Now combine this with the default human bias of ignoring things outside one's own story (a standard example being suffering in another country); a decision theory that always says 'escalate conflict' cannot itself support escalating conflicts only with the ones around them in particular (a friend's abusive parents, a cop stopping your car, a landlord), instead of e.g. animal farming CEOs. Indeed, this kind of scope-sensitivity, when taken sufficiently seriously, generalizes to "do the altruistically best action, whatever it is, whether or not it looks like fighting back."

I doubt that this is the full explanation. For example, I imagine they were aware of the concept of scope sensitivity and agreed with it. Maybe it plays a part, though, since being aware of biases doesn't make you fully immune to them. I see no other explanation for this.

Given the purpose of this thread as sense-making, and that courage is listed as a virtue which it did take to post this, I hope this will be welcomed and help with sense-making.

1. ^{^}

   For a valid analogue between how bad this is in my morality and something that would be equally bad in a human-focused morality, you can imagine being born into a world with widespread human factory farms. Or the slaughter and slavery of human-like orcs, in case of this EY fiction.

Can I double-click on what "does not understand politics at [a] very deep level" means? Can someone explain what they have in mind? I think Eliezer has probably better models than most of what our political institutions are capable of, and probably isn't very skilled at personally politicking. I'm not sure what other people have in mind.

The former, but the latter is a valid response too.

Someone doing a good job of painting an overall picture is a good opportunity to reflect on the overall picture and what changes to make, or what counter-arguments to present to this account.

For what it's worth, I have grown pessimistic about our ability to solve the open technical problems even given 100 years of work on them. I think it possible but not probable in most plausible scenarios.

Correspondingly the importance I assign to increasing the intelligence of humans has drastically increased.