LessWrong 2.0 Reader

guesses: 1. in most cases, children on net detract from other major projects for common-sense time/attention/optionality management reasons (as well as because they sometimes commit people to a world view of relatively slow change), 2. whether to have children isn't each other's business and pressure against doing normal human things like this is net socially harmful (conservatives in particular are alienated by a culture of childlessness, though maybe that's net strategically useful), 3. people conflate 2 with not-1 on an emotional level and feel 1 is false because 2 is true

I am pretty confused on this, and as I said above, don't put much weight on this study because I also have some sense that the author isn't super trustworthy (though I haven't found any critique of this specific paper). 

Overall, my current sense is that the effect on women in particular is quite strong, and women who choose to have children will reduce their chance of major achievement by at least 40% or so. For men it's probably weaker, and I am a lot less sure what the data says. 

Could this be solved just by posting your work and then immediately sharing the link with people you specifically want feedback from? That way there's no expectation that they would have already seen it. (Granted, this is slightly different from a gdoc in that you can share a gdoc with one person, get their feedback, then share with another person, while what I suggested requires asking everyone you want feedback from all at once.)

I'm from Ukraine, and it's similar here (also, parsnip, radish, beetroot and pumpkin are among the most common things). A curious variation common in the villages but not in the cities is that if you grow it, you don't buy it. This often means that people in the city get theirs starting earlier in the season.

The main point, as I see it, is essentially that functions with good generalisation correspond to large volumes in parameter-space, and that SGD finds functions with a probability roughly proportional to their volume.

What I'm suggesting is that volume in high-dimensions can concentrate on the boundary. To be clear, when I say SGD only typically reaches the boundary, I'm talking about early stopping and the main experimental setup in your paper where training is stopped upon reaching zero train error.

We have done overtraining, which should allow SGD to penetrate into the region. This doesn’t seem to make much difference for the probabilities we get.

This does seem to invalidate the model. However, something tells me that the difference here is more about degree. Since you use the word 'should' I'll use the wiggle room to propose an argument for what 'should' happen.

If SGD is run with early stopping, as described above, then my argument is that this is roughly equivalent to random sampling via an appeal to concentration of measure in high-dimensions.

If SGD is not run with early stopping, it's enclosed by the boundary of zero train error functions. Because these are most likely in the interior these functions are unlikely to be produced by random sampling. Thus, on a log-log plot I'd expect overtraining to 'tilt' the correspondence between SGD and random sampling likelihoods downward.

Falsifiable Hypothesis: Compare SGD with overtaining to the random sampling algorithm. You will see that functions that are unlikely to be generated by random sampling will be more likely under SGD with overtraining. Moreover, functions that are more likely with random sampling will be become less likely under SGD with overtraining.

Planned summary for the Alignment Newsletter:

This post distinguishes between three kinds of “alignment”:
1. Not building an AI system at all,
2. Building Friendly AI that will remain perfectly aligned for all time and capability levels,
3. _Bootstrapped alignment_, in which we build AI systems that may not be perfectly aligned but are at least aligned enough that we can use them to build perfectly aligned systems.

The post argues that optimization-based approaches can’t lead to perfect alignment, because there will always eventually be Goodhart effects.

I would worry about the counter-party risk with annuities; if a single company goes out of business, you might be bust.  Even if you distribute across many companies, I'd think it's more likely that the whole sector goes bust than that your portfolio devalues to 0 in some other way.

For that reason I'd lean toward not putting too much of my assets in annuities -- but maybe it works out so that the counter-party risk is smaller than the risk of running out of money otherwise.

I agree it's not vacuous. It sounds like you're mostly stating the same argument I gave but in different words. Can you tell me what's wrong or missing from my summary of the argument?

Since it is possible to compress high-probability events using an optimal code for the probability distribution, you might expect that functions with high probability in the neural network prior can be compressed more than functions with low probability. Since high probability functions are more likely, this means that the more likely functions correspond to shorter programs. Since shorter programs are necessarily more likely in the prior that simulates all possible programs, they should be expected to be better programs, and so generalize well.

(Even if you are talking about the overparameterized case, where this argument is not vacuous and also applies primarily to neural nets and not other ML models, I don't find this argument very compelling a priori, though I agree that based on empirical evidence it is probably mostly correct.)

I am from Russia and I am surprised by what you list as typical examples of vegetables. I wonder what vegetables are popular in different parts of the world. Can you all, people from different countries, write the most popular veggies where you're at? I'll start.

I am from Moscow, Russia. I can't say for the entirety of Moscow, only for my family and friends. Here the most popular vegetables are BY FAR tomatos, cucumbers, potatoes, onion, and sweet peppers. People eat tomatoes, cucumbers, and sweet peppers raw either on their own with pepper and salt, or in all kinds of salads. Less popular veggies are eggplant, tsukini, cauliflower, carrots, garlic, green peas, sweet corn. Celery, spinach and broccoli are a little rare. Asparagus is very rare (and expensive).

The main point, as I see it, is essentially that functions with good generalisation correspond to large volumes in parameter-space, and that SGD finds functions with a probability roughly proportional to their volume.

This seems right, but I'm not sure how that's different from Zach's phrasing of the main point? Zach's phrasing was "SGD approximately equals random sampling", and random sampling finds functions with probability exactly proportional to their volume. Combine that with the fact that empirically we get good generalization and we get the thing you said.

(Maybe you weren't disagreeing with Zach and were just saying the same thing a different way?)

Saying that SGD is “Bayesian” is one way of saying the latter

This feels similar [LW · GW] to:

Saying that MLK was a "criminal" is one way of saying that MLK thought and acted as though he had a moral responsibility to break unjust laws and to take direct action.

(This is an exaggeration but I think it is directionally correct. Certainly when I read the title "neural networks are fundamentally Bayesian" I was thinking of something very different.)

the Kolmogorov complexity stuff is a way to formalise some intuitions around the former.

I've discussed this above [LW(p) · GW(p)], I'll continue the discussion there.

The rest of the comment is about stuff that I didn't have a strong opinion on, so I'll leave it for Zach to answer if he wants.