LessWrong 2.0 Reader

I think that's a great idea. I don't think the duplication of polls would be a problem, since it would be easy to filter aggregate results for user-created vs official polls.

A few questions:

• The literature review is very strange to me. Where is the section on certified robustness against epsilon-ball adversarial examples? The techniques used in that literature (e.g. interval propagation) are nearly identical to what you discuss here.
• Relatedly, what's the source of hope for these kinds of methods outperforming adversarial training? My sense from the certified defenses literature is that the estimates they produce are very weak, because of the problems with failing to model all the information in activations. (Note I'm not sure how weak the estimates actually are, since they usually report fraction of inputs which could be certified robust, rather than an estimate of the probability that a sampled input will cause a misclassification, which would be more analogous to your setting.)
• If your catastrophe detector involves a weak model running many many inferences, then it seems like the total number of layers is vastly larger than the number of layers in M, which seems like it will exacerbate the problems above by a lot. Any ideas for dealing with this? 
• What's your proposal for the distribution $P_0$ for Method 2 (independent linear features)?

This suggests that we must model the entire distribution of activations simultaneously, instead of modeling each individual layer.

• Why think this is a cost you can pay? Even if we ignore the existence of C and just focus on M, and we just require modeling the correlations between any pair of layers (which of course can be broken by higher-order correlations), that is still quadratic in the number of parameters of M and so has a cost similar to training M in the first place. In practice I would assume it is a much higher cost (not least because C is so much larger than M).

I partly disagree; steganography is only useful when it's possible for the outside / receiving system to detect and interpret the hidden messages, so if the messages are of a type that outside systems would identify, they can and should be detectable by the gating system as well. 

That said, I'd be very interested in looking at formal guarantees that the outputs are minimally complex in some computationally tractable sense, or something similar - it definitely seems like something that @davidad [LW · GW] would want to consider.

I really like that idea, and the clarity it provides, and have renamed the post to reflect it! (Sorryr this was so slow- I'm travelling.)

That seems fair!

@Ericf [LW · GW] you are right, I didn't calculate it like this. I´m going to adjust the formula to see if it changes the overall outcome of the essay or not.

If the idea is "you should use AI to work on your personal projects", the problem is I'm already doing that (notice the $50/month).  I'm looking for ways to spend 20x more on AI without spending 20x more of my time (which is physically impossible).

What's the use in saying that values change, rather than just saying that you aren't interested in concepts involving values, then?

I don't have one. What would I use it for? I don't think anyone else yet has one, at least not something mathematically founded, with the simplicity and inevitability of VNM. People put forward various ideas and discuss the various "monsters" I listed, but I see no sign of a consensus.

A generic example is kind of an anti-example though.

If you reject utility theory, what approach do you use for modelling values instead, and what makes you feel that approach is helpful?