Buck's Shortform 2019-08-18T07:22:26.247Z · score: 11 (2 votes)
"Other people are wrong" vs "I am right" 2019-02-22T20:01:16.012Z · score: 195 (73 votes)


Comment by buck on Six AI Risk/Strategy Ideas · 2019-08-29T05:02:46.939Z · score: 3 (2 votes) · LW · GW

Minor point: I think asteroid strikes are probably very highly correlated between Everett branches (though maybe the timing of spotting an asteroid on a collision course is variable).

Comment by buck on Buck's Shortform · 2019-08-21T01:20:18.379Z · score: 32 (7 votes) · LW · GW

A couple weeks ago I spent an hour talking over video chat with Daniel Cantu, a UCLA neuroscience postdoc who I hired on to spend an hour answering a variety of questions about neuroscience I had. (Thanks Daniel for reviewing this blog post for me!)

The most interesting thing I learned is that I had quite substantially misunderstood the connection between convolutional neural nets and the human visual system. People claim that these are somewhat bio-inspired, and that if you look at early layers of the visual cortex you'll find that it operates kind of like the early layers of a CNN, and so on.

The claim that the visual system works like a CNN didn’t quite make sense to me though. According to my extremely rough understanding, biological neurons operate kind of like the artificial neurons in a fully connected neural net layer--they have some input connections and a nonlinearity and some output connections, and they have some kind of mechanism for Hebbian learning or backpropagation or something. But that story doesn't seem to have a mechanism for how neurons do weight tying, which to me is the key feature of CNNs.

Daniel claimed that indeed human brains don't have weight tying, and we achieve the efficiency gains over dense neural nets by two other mechanisms instead:

Firstly, the early layers of the visual cortex are set up to recognize particular low-level visual features like edges and motion, but this is largely genetically encoded rather than learned with weight-sharing. One way that we know this is that mice develop a lot of these features before their eyes open. These low-level features can be reinforced by positive signals from later layers, like other neurons, but these updates aren't done with weight-tying. So the weight-sharing and learning here is done at the genetic level.

Secondly, he thinks that we get around the need for weight-sharing at later levels by not trying to be able to recognize complicated details with different neurons. Our vision is way more detailed in the center of our field of view than around the edges, and if we need to look at something closely we move our eyes over it. He claims that this gets around the need to have weight tying, because we only need to be able to recognize images centered in one place.

I was pretty skeptical of this claim at first. I pointed out that I can in fact read letters that are a variety of distances from the center of my visual field; his guess is that I learned to read all of these separately. I'm also kind of confused by how this story fits in with the fact that humans seem to relatively quickly learn to adapt to inversion goggled. I would love to check what some other people who know neuroscience think about this.

I found this pretty mindblowing. I've heard people use CNNs as an example of how understanding brains helped us figure out how to do ML stuff better; people use this as an argument for why future AI advances will need to be based on improved neuroscience. This argument seems basically completely wrong if the story I presented here is correct.

Comment by buck on Buck's Shortform · 2019-08-20T21:04:12.980Z · score: 1 (1 votes) · LW · GW

I recommend looking on Wyzant.

Comment by buck on Buck's Shortform · 2019-08-18T07:22:26.379Z · score: 53 (24 votes) · LW · GW

I think that an extremely effective way to get a better feel for a new subject is to pay an online tutor to answer your questions about it for an hour.

It turns that there are a bunch of grad students on Wyzant who mostly work tutoring high school math or whatever but who are very happy to spend an hour answering your weird questions.

For example, a few weeks ago I had a session with a first-year Harvard synthetic biology PhD. Before the session, I spent a ten-minute timer writing down things that I currently didn't get about biology. (This is an exercise worth doing even if you're not going to have a tutor, IMO.) We spent the time talking about some mix of the questions I'd prepared, various tangents that came up during those explanations, and his sense of the field overall.

I came away with a whole bunch of my minor misconceptions fixed, a few pointers to topics I wanted to learn more about, and a way better sense of what the field feels like and what the important problems and recent developments are.

There are a few reasons that having a paid tutor is a way better way of learning about a field than trying to meet people who happen to be in that field. I really like it that I'm paying them, and so I can aggressively direct the conversation to wherever my curiosity is, whether it's about their work or some minor point or whatever. I don't need to worry about them getting bored with me, so I can just keep asking questions until I get something.

Conversational moves I particularly like:

  • "I'm going to try to give the thirty second explanation of how gene expression is controlled in animals; you should tell me the most important things I'm wrong about."
  • "Why don't people talk about X?"
  • "What should I read to learn more about X, based on what you know about me from this conversation?"

All of the above are way faster with a live human than with the internet.

I think that doing this for an hour or two weekly will make me substantially more knowledgeable over the next year.

Various other notes on online tutors:

  • Online language tutors are super cheap--I had some Japanese tutor who was like $10 an hour. They're a great way to practice conversation. They're also super fun IMO.
  • Sadly, tutors from well paid fields like programming or ML are way more expensive.
  • If you wanted to save money, you could gamble more on less credentialed tutors, who are often $20-$40 an hour.

If you end up doing this, I'd love to hear your experience.

Comment by buck on "Other people are wrong" vs "I am right" · 2019-02-25T01:12:01.891Z · score: 5 (3 votes) · LW · GW

I'm confused about what point you're making with the bike thief example. I'm reading through that post and its comments to see if I can understand your post better with that as background context, but you might want to clarify that part of the post (with a reader who doesn't have that context in mind).

Can you clarify what is unclear about it?

Comment by buck on Current AI Safety Roles for Software Engineers · 2018-11-10T04:40:13.666Z · score: 25 (9 votes) · LW · GW
I believe they would like to hire several engineers in the next few years.

We would like to hire many more than several engineers--we want to hire as many people as engineers as possible; this would be dozens if we could, but it's hard to hire, so we'll more likely end up hiring more like ten over the next year.

I think that MIRI engineering is a really high impact opportunity, and I think it's definitely worth the time for EA computer science people to apply or email me (

Comment by buck on Weird question: could we see distant aliens? · 2018-04-21T01:04:28.924Z · score: 2 (1 votes) · LW · GW

My main concern with this is the same as the problem listed on Wei Dai's answer: whether a star near us is likely to block out this light. The sun is about 10^9m across. A star that's 10 thousand light years away (this is 10% of the diameter of the Milky Way) occupies about (1e9m / (10000 lightyears * 2 * pi))**2 = 10^-24 of the night sky. A galaxy that's 20 billion light years away occupies something like (100000 lightyears / 20 billion lightyears) ** 2 ~= 2.5e-11. So galaxies occupy more space than stars. So it would be weird if individual stars blocked out a whole galaxy.

Comment by buck on Weird question: could we see distant aliens? · 2018-04-21T01:04:24.485Z · score: 2 (1 votes) · LW · GW

Another piece of idea: If you're extremely techno-optimistic, then I think it would be better to emit light at weird wavelengths than to just emit a lot of light. Eg emitting light at two wavelengths with ratio pi or something. This seems much more unmistakably intelligence-caused than an extremely bright light.

Comment by buck on Weird question: could we see distant aliens? · 2018-04-20T18:50:10.385Z · score: 15 (4 votes) · LW · GW

My first idea is to make two really big black holes and then make them merge. We observed gravitational waves from two black holes with solar masses of around 25 solar masses each located 1.8 billion light years away. Presumably this force decreases as an inverse square times exponential decay; ignoring the exponential decay this suggests to me that we need 100 times as much mass to be as prominent from 18 billion light years. A galaxy mass is around 10^12 solar masses. So if we spent 2500 solar masses on this each year, it would be at least as prominent as the gravitational wave that we detected, and we could do this a billion times with a galaxy. To be safe, I'd 10x the strength of the waves, so that we could do it 100 million times with a galaxy.

Currently our instruments aren't sensitive enough to detect which galaxy was emitting these bizarrely strong gravitational waves. So I'd combine this with Wei Dai's suggestion of making an extremely bright beacon using the accretion disks resulting from the creation of these black holes.