Posts

Model Mis-specification and Inverse Reinforcement Learning 2018-11-09T15:33:02.630Z · score: 34 (12 votes)
Latent Variables and Model Mis-Specification 2018-11-07T14:48:40.434Z · score: 22 (9 votes)
[link] Essay on AI Safety 2015-06-26T07:42:11.581Z · score: 12 (13 votes)
The Power of Noise 2014-06-16T17:26:30.329Z · score: 28 (31 votes)
A Fervent Defense of Frequentist Statistics 2014-02-18T20:08:48.833Z · score: 54 (58 votes)
Another Critique of Effective Altruism 2014-01-05T09:51:12.231Z · score: 20 (23 votes)
Macro, not Micro 2013-01-06T05:29:38.689Z · score: 19 (22 votes)
Beyond Bayesians and Frequentists 2012-10-31T07:03:00.818Z · score: 36 (41 votes)
Recommendations for good audio books? 2012-09-16T23:43:31.596Z · score: 6 (10 votes)
What is the evidence in favor of paleo? 2012-08-27T07:07:07.105Z · score: 13 (18 votes)
PM system is not working 2012-08-02T16:09:06.846Z · score: 11 (12 votes)
Looking for a roommate in Mountain View 2012-08-01T19:04:59.872Z · score: 11 (18 votes)
Philosophy and Machine Learning Panel on Ethics 2011-12-17T23:32:20.026Z · score: 8 (11 votes)
Help me fix a cognitive bug 2011-06-25T22:22:31.484Z · score: 4 (7 votes)
Utility is unintuitive 2010-12-09T05:39:34.176Z · score: -5 (16 votes)
Interesting talk on Bayesians and frequentists 2010-10-23T04:10:27.684Z · score: 7 (12 votes)

Comments

Comment by jsteinhardt on Why Boston? · 2020-10-13T06:50:04.150Z · score: 6 (4 votes) · LW · GW

I noticed the prudishness, but "rudeness" to me parses as people actually telling you what's on their mind, rather than the passive-aggressive fake niceness that seems to dominate in the Bay Area. I'll personally take the rudeness :).

Comment by jsteinhardt on Why Boston? · 2020-10-13T06:46:42.306Z · score: 8 (3 votes) · LW · GW

On the other hand, the second-best place selects for people who don't care strongly about optimizing for legible signals, which is probably a plus. (An instance of this: In undergrad the dorm that, in my opinion, had the best culture was the run-down dorm that was far from campus.)

Comment by jsteinhardt on Why Boston? · 2020-10-11T05:29:57.228Z · score: 6 (5 votes) · LW · GW

Many of the factors affecting number of deaths are beyond a place's control, such as how early on the pandemic spread to that place, and how densely populated the city is. I don't have a strong opinion about MA but measuring by deaths per capita isn't a good way of judging the response.

Comment by jsteinhardt on What's Wrong with Social Science and How to Fix It: Reflections After Reading 2578 Papers · 2020-09-17T02:02:03.053Z · score: 7 (4 votes) · LW · GW

That's not really what a p-value means though, right? The actual replication rate should depend on the prior and the power of the studies.

Comment by jsteinhardt on What's Wrong with Social Science and How to Fix It: Reflections After Reading 2578 Papers · 2020-09-12T18:03:46.264Z · score: 1 (3 votes) · LW · GW

What are some of the recommendations that seem most off base to you?

Comment by jsteinhardt on Covid-19 6/11: Bracing For a Second Wave · 2020-06-13T19:58:18.847Z · score: 4 (2 votes) · LW · GW

My prediction: infections will either go down or only slowly rise in most places, with the exception of one or two metropolitan areas. If I had to pick one it would be LA, not sure what the second one will be. The places where people are currently talking about spikes won't have much correlation with the places that look bad two weeks from now (i.e. people are mostly chasing noise).

I'm not highly confident in this, but it's been a pretty reliable prediction for the past month at least...

Comment by jsteinhardt on Estimating COVID-19 Mortality Rates · 2020-06-13T08:00:32.737Z · score: 4 (2 votes) · LW · GW

Here is a study that a colleague recommends: https://www.medrxiv.org/content/10.1101/2020.05.03.20089854v3. Tweet version: https://mobile.twitter.com/gidmk/status/1270171589170966529?s=21

Their point estimate is 0.64% but with likely heterogeneity across settings.

Comment by jsteinhardt on Quarantine Bubbles Require Directness, and Tolerance of Rudeness · 2020-06-10T02:33:59.701Z · score: 4 (2 votes) · LW · GW

I don't think bubble size is the right thing to measure; instead you should measure the amount of contract you have with people, weighted by time, distance, indoor/outdoor, mask-wearing, and how likely the other person is to be infected (I.e. how careful they are).

An important part of my mental model is that infection risk is roughly linear in contact time.

Comment by jsteinhardt on Quarantine Bubbles Require Directness, and Tolerance of Rudeness · 2020-06-08T07:50:07.485Z · score: 2 (1 votes) · LW · GW

As a background assumption, I'm focused on the societal costs of getting infected, rather than the personal costs, since in most places the latter seem negligible unless you have pre-existing health conditions. I think this is also the right lens through which to evaluate Alameda's policy, although I'll discuss the personal calculation at the end.

From a social perspective, I think it's quite clear that the average person is far from being effectively isolated, since R is around 0.9 and you can only get to around half of that via only household infection. So a 12 person bubble isn't really a bubble... It's 12 people who each bring in non trivial risk from the outside world. On the other hand they're also not that likely to infect each other.

From a personal perspective, I think the real thing to care about is whether the other people are about as careful as you. By symmetry there's no reason to think that another house that practices a similar level of precaution is more likely to get an outside infection than your house is. But by the same logic there's nothing special about a 12 person bubble: you should be trying to interact with people with the same or better risk profile as you (from a personal perspective; from a societal perspective you should interact with riskier people, at least if you're low risk, because bubbles full of risky people are the worst possible configuration and you want to help break those up).

Comment by jsteinhardt on Quarantine Bubbles Require Directness, and Tolerance of Rudeness · 2020-06-08T04:52:54.259Z · score: 2 (1 votes) · LW · GW

I think the biggest issue with the bubble rule is that the math doesn't work out. The secondary attack rate between house members is ~30% and probably much lower between other contacts. At that low of a rate, these games with the graph structure buy very little and may be harmful because they increase the fraction of contact occurring between similar people (which is bad because the social cost of a pair of people interacting is roughly the product of their infection risks).

Comment by jsteinhardt on Estimating COVID-19 Mortality Rates · 2020-06-07T20:14:30.925Z · score: 14 (4 votes) · LW · GW

I'm not trying to intimidate; I'm trying to point out that I think you're making errors that could be corrected by more research, which I hoped would be helpful. I've provided one link (which took me some time to dig up). If you don't find this useful that's fine, you're not obligated to believe me and I'm not obligated to turn a LW comment into a lit review.

Comment by jsteinhardt on Estimating COVID-19 Mortality Rates · 2020-06-07T18:58:05.422Z · score: 7 (4 votes) · LW · GW

The CFR will shift substantially over time and location as testing changes. I'm not sure how you would reliably use this information. IFR should not change much and tells you how bad it is for you personally to get sick.

I wouldn't call the model Zvi links expert-promoted. Every expert I talked to thought it had problems, and the people behind it are economists not epidemiologists or statisticians.

For IFR you can start with seroprevalence data here and then work back from death rates: https://twitter.com/ScottGottliebMD/status/1268191059009581056

Regarding back-of-the-envelope calculations, I think we have different approaches to evidence/data. I started with back-of-the-envelope calculations 3 months ago. But I would have based things on a variety of BOTECs and not a single one. Now I've found other sources that are taking the BOTEC and doing smarter stuff on top of it, so I mostly defer to those sources, or to experts with a good track record. This is easier for me because I've worked full-time on COVID for the past 3 months; if I weren't in that position I'd probably combine some of my own BOTECs with opinions of people I trusted. In your case, I predict Zvi if you asked him would also say the IFR was in the range I gave.

Comment by jsteinhardt on Estimating COVID-19 Mortality Rates · 2020-06-07T17:04:38.882Z · score: 19 (8 votes) · LW · GW

Ben, I think you're failing to account for under-testing. You're computing the case fatality rate when you want the infection fatality rate. Most experts, as well as the well-done meta analyses, place the IFR in the 0.5%-1% range. I'm a little bit confused why you're relying on this back of the envelope rather than the pretty extensive body of work on this question.

Comment by jsteinhardt on Ben Hoffman's donor recommendations · 2018-07-30T17:59:04.732Z · score: 2 (1 votes) · LW · GW

I don't understand why this is evidence that "EA Funds (other than the global health and development one) currently funges heavily with GiveWell recommended charities", which was Howie's original question. It seems like evidence that donations to OpenPhil (which afaik cannot be made by individual donors) funge against donations to the long-term future EA fund.

Comment by jsteinhardt on RFC: Philosophical Conservatism in AI Alignment Research · 2018-05-15T04:24:03.648Z · score: 13 (3 votes) · LW · GW

I like the general thrust here, although I have a different version of this idea, which I would call "minimizing philosophical pre-commitments". For instance, there is a great deal of debate about whether Bayesian probability is a reasonable philosophical foundation for statistical reasoning. It seems that it would be better, all else equal, for approaches to AI alignment to not hinge on being on the right side of this debate.

I think there are some places where it is hard to avoid pre-commitments. For instance, while this isn't quite a philosophical pre-commitment, it is probably hard to develop approaches that are simultaneously optimized for short and long timelines. In this case it is probably better to explicitly do case splitting on the two worlds and have some subset of people pursuing approaches that are good in each individual world.

Comment by jsteinhardt on [deleted post] 2018-03-19T19:43:11.984Z

FWIW I understood Zvi's comment, but feel like I might not have understood it if I hadn't played Magic: The Gathering in the past.

EDIT: Although I don't understand the link to Sir Arthur's green knight, unless it was a reference to the fact that M:tG doesn't actually have a green knight card.

Comment by jsteinhardt on Takeoff Speed: Simple Asymptotics in a Toy Model. · 2018-03-06T13:54:41.342Z · score: 28 (8 votes) · LW · GW

Thanks for writing this Aaron! (And for engaging with some of the common arguments for/against AI safety work.)

I personally am very uncertain about whether to expect a singularity/fast take-off (I think it is plausible but far from certain). Some reasons that I am still very interested in AI safety are the following:

  • I think AI safety likely involves solving a number of difficult conceptual problems, such that it would take >5 years (I would guess something like 10-30 years, with very wide error bars) of research to have solutions that we are happy with. Moreover, many of the relevant problems have short-term analogues that can be worked on today. (Indeed, some of these align with your own research interests, e.g. imputing value functions of agents from actions/decisions; although I am particularly interested in the agnostic case where the value function might lie outside of the given model family, which I think makes things much harder.)
  • I suppose the summary point of the above is that even if you think AI is a ways off (my median estimate is ~50 years, again with high error bars) research is not something that can happen instantaneously, and conceptual research in particular can move slowly due to being harder to work on / parallelize.
  • While I have uncertainty about fast take-off, that still leaves some probability that fast take-off will happen, and in that world it is an important enough problem that it is worth thinking about. (It is also very worthwhile to think about the probability of fast take-off, as better estimates would help to better direct resources even within the AI safety space.)
  • Finally, I think there are a number of important safety problems even from sub-human AI systems. Tech-driven unemployment is I guess the standard one here, although I spend more time thinking about cyber-warfare/autonomous weapons, as well as changes in the balance of power between nation-states and corporations. These are not as clearly an existential risk as unfriendly AI, but I think in some forms would qualify as a global catastrophic risk; on the other hand I would guess that most people who care about AI safety (at least on this website) do not care about it for this reason, so this is more idiosyncratic to me.

Happy to expand on/discuss any of the above points if you are interested.

Best,

Jacob

Comment by jsteinhardt on Takeoff Speed: Simple Asymptotics in a Toy Model. · 2018-03-06T13:32:48.176Z · score: 13 (3 votes) · LW · GW

Very minor nitpick, but just to add, FLI is as far as I know not formally affiliated with MIT. (FHI is in fact a formal institute at Oxford.)

Comment by jsteinhardt on Zeroing Out · 2017-11-05T22:19:45.863Z · score: 28 (12 votes) · LW · GW

Hi Zvi,

I enjoy reading your posts because they often consist of clear explanations of concepts I wish more people appreciated. But I think this is the first instance where I feel I got something that I actually hadn't thought about before at all, so I wanted to convey extra appreciation for writing it up.

Best,

Jacob

Comment by jsteinhardt on Seek Fair Expectations of Others’ Models · 2017-10-20T03:53:12.702Z · score: 8 (4 votes) · LW · GW

I think the conflation is "decades out" and "far away".

Comment by jsteinhardt on [deleted post] 2017-10-17T03:04:59.264Z

Galfour was specifically asked to write his thought up in this thread: https://www.lesserwrong.com/posts/BEtzRE2M5m9YEAQpX/there-s-no-fire-alarm-for-artificial-general-intelligence/kAywLDdLrNsCvXztL

It seems either this was posted to the wrong place, or there is some disagreement within the community (e.g. between Ben in that thread and the people downvoting).

Comment by jsteinhardt on Oxford Prioritisation Project Review · 2017-10-14T18:08:10.872Z · score: 3 (2 votes) · LW · GW

Points 1-5 at the beginning of the post are all primarily about community-building and personal development externalities of the project, and not about the donation itself.

Comment by jsteinhardt on Oxford Prioritisation Project Review · 2017-10-14T03:58:56.583Z · score: 13 (3 votes) · LW · GW

?? If you literally mean minimum-wage, I think that is less than 10,000 pounds... although agree with the general thrust of your point about the money being more valuanle than the time (but think you are missing the spirit of the exercise as outlined in the post).

Comment by jsteinhardt on Robustness as a Path to AI Alignment · 2017-10-11T05:46:54.275Z · score: 11 (3 votes) · LW · GW

You might be interested in my work on learning from untrusted data (see also earlier work on aggregating unreliable human input). I think it is pretty relevant to what you discussed, although if you do not think it is, then I would also be pretty interested in understanding that.

Unrelated, but for quantilizers, isn't the biggest issue going to be that if you need to make a sequence of decisions, the probabilities are going to accumulate and give exponential decay? I don't see how to make a sequence of 100 decisions in a quantilizing way unless the base distribution of policies is very close to the target policy.

Comment by jsteinhardt on [deleted post] 2017-05-27T20:51:14.016Z

Parts of the house setup pattern-match to a cult, cult members aren't good at realizing when they need to leave, but their friends can probably tell much more easily.

(I don't mean the above as negatively as it sounds connotatively, but it's the most straightforward way to say what I think is the reason to want external people. I also think this reasoning degrades gracefully with the amount of cultishness.)

Comment by jsteinhardt on [deleted post] 2017-05-27T17:29:34.887Z

I think there's a difference between a friend that one could talk to (if they decide to), and a friend tasked with the specific responsibility of checking in and intervening if things seem to be going badly.

Comment by jsteinhardt on Scenario analysis: a parody · 2017-04-28T04:52:09.218Z · score: 1 (1 votes) · LW · GW

I feel like you're straw-manning scenario analysis. Here's an actual example of a document produced via scenario analysis: Global Trends 2035.

Comment by jsteinhardt on Effective altruism is self-recommending · 2017-04-21T23:35:42.407Z · score: 5 (5 votes) · LW · GW

When you downvote something on the EA forum, it becomes hidden. Have you tried viewing it while not logged in to your account? It's still visible to me.

Comment by jsteinhardt on An OpenAI board seat is surprisingly expensive · 2017-04-20T21:03:58.348Z · score: 1 (2 votes) · LW · GW

then the technical advisors at OPP must have a very specific approach to AI safety they are pushing very hard to get support for, but are unwilling or unable to articulate why they prefer theirs so strongly.

I don't think there is consensus among technical advisors on what directions are most promising. Also, Paul has written substantially about his preferred approach (see here for instance), and I've started to do the same, although so far I've been mostly talking about obstacles rather than positive approaches. But you can see some of my writing here and here. Also my thoughts in slide form here, although those slides are aimed at ML experts.

Comment by jsteinhardt on I Want To Live In A Baugruppe · 2017-03-20T02:31:47.253Z · score: 3 (3 votes) · LW · GW

Any attempt to enforce rationalists moving in is illegal.

Is this really true? Based on my experience (not any legal experience, just seeing what people generally do that is considered fine) I think in the Bay Area the following are all okay:

  • Only listing a house to your friends / social circle.
  • Interviewing people who want to live with you and deciding based on how much you like them.

The following are not okay:

  • Having a rule against pets that doesn't have an exception for seeing-eye dogs.
  • Explicitly deciding not to take someone as a house-mate only on the basis of some protected trait like race, etc. (but gender seems to be fine?).
Comment by jsteinhardt on [deleted post] 2016-12-13T21:03:05.679Z

Hi Eugene!

Comment by jsteinhardt on CFAR's new mission statement (on our website) · 2016-12-10T10:10:48.630Z · score: 8 (8 votes) · LW · GW

Thanks for posting this, I think it's good to make these things explicit even if it requires effort. One piece of feedback: I think someone who reads this who doesn't already know what "existential risk" and "AI safety" are will be confused (they suddenly show up in the second bullet point without being defined, though it's possible I'm missing some context here).

Comment by jsteinhardt on [deleted post] 2016-12-02T05:55:36.341Z

70% this account is Eugene.

Comment by jsteinhardt on Why GiveWell can't recommend MIRI or anything like it · 2016-12-01T02:59:20.093Z · score: 4 (4 votes) · LW · GW

I don't think you are actually making this argument, but this comes close to an uncharitable view of GiveWell that I strongly disagree with, which goes something like "GiveWell can't recommend MIRI because it would look weird and be bad for their brand, even if they think that MIRI is actually the best place to donate to." I think GiveWell / OpenPhil are fairly insensitive to considerations like this and really just want to recommend the things they actually think are best independent of public opinion. The separate branding decision seems like a clearly good idea to me, but I think that if for some reason OpenPhil were forced to have inseparable branding from GiveWell, they would be making the same recommendations.

Comment by jsteinhardt on Why GiveWell can't recommend MIRI or anything like it · 2016-11-30T04:34:17.989Z · score: 7 (7 votes) · LW · GW

Also like: here is a 4000-word evaluation of MIRI by OpenPhil. ???

Comment by jsteinhardt on Open Thread, Aug. 15. - Aug 21. 2016 · 2016-08-16T08:29:28.145Z · score: 12 (12 votes) · LW · GW

(FYI, this was almost certainly downvoted by Eugene_Nier's sockpuppets rather than actual people. I upvoted, hopefully others will as well to counteract the trollling.)

Comment by jsteinhardt on Open thread, Jul. 18 - Jul. 24, 2016 · 2016-07-18T17:31:28.007Z · score: 1 (1 votes) · LW · GW

Wait what? How are you supposed to meet your co-founder / early employees without connections? College is like the ideal place to meet people to start start-ups with.

Comment by jsteinhardt on Open thread, Jul. 11 - Jul. 17, 2016 · 2016-07-14T01:17:27.506Z · score: 1 (1 votes) · LW · GW

I don't think I need that for my argument to work. My claim is that if people get, say, less than 70% of a meal's worth of food, an appreciable fraction (say at least 30%) will get cranky.

Comment by jsteinhardt on Open thread, Jul. 11 - Jul. 17, 2016 · 2016-07-13T21:01:50.643Z · score: 1 (1 votes) · LW · GW

But like, there's variation in how much food people will end up eating, and at least some of that is not variation that you can predict in advance. So unless you have enough food that you routinely end up with more than can be eaten, you are going to end up with a lot of cranky people a non-trivial fraction of the time. You're not trying to peg production to the mean consumption, but (e.g.) to the 99th percentile of consumption.

Comment by jsteinhardt on Open thread, Jul. 11 - Jul. 17, 2016 · 2016-07-13T15:45:52.406Z · score: 2 (2 votes) · LW · GW

I don't think this is really a status thing, more a "don't be a dick to your guests" thing. Many people get cranky if they are hungry, and putting 30+ cranky people together in a room is going to be a recipe for unpleasantness.

Comment by jsteinhardt on Open thread, Jul. 04 - Jul. 10, 2016 · 2016-07-06T06:58:50.094Z · score: 2 (2 votes) · LW · GW

I don't really think this is spending idiosyncrasy credits... but maybe we hang out in different social circles.

Comment by jsteinhardt on Open Thread April 25 - May 1, 2016 · 2016-04-29T06:26:53.461Z · score: 1 (1 votes) · LW · GW

I assume at least some of the downvotes are from Eugene sockpuppets (he tends to downvote any suggestions that would make it harder to do his trolling).

Comment by jsteinhardt on Yoshua Bengio on AI progress, hype and risks · 2016-01-30T08:01:43.829Z · score: 13 (15 votes) · LW · GW

+1 To go even further, I would add that it's unproductive to think of these researchers as being on anyone's "side". These are smart, nuanced people and rounding their comments down to a specific agenda is a recipe for misunderstanding.

Comment by jsteinhardt on Voiceofra is banned · 2015-12-25T22:11:05.511Z · score: 4 (6 votes) · LW · GW

I'm well aware. It is therefore even more problematic if this account is abused --- note that there have been multiple confirmations that username2 has been used to downvote the same people that VoiceOfRa was downvoting before; in addition, VoiceOfRa has used the username2 account to start arguments with NancyLebovitz in a way that makes it look like a 3rd party is disagreeing with the decision, rather than VoiceOfRa himself. At the very least, it is better if everyone is aware of this situation, and ideally we would come up with a way to prevent such abuse.

Comment by jsteinhardt on Voiceofra is banned · 2015-12-24T22:09:40.905Z · score: 4 (4 votes) · LW · GW

I was 85% sure at the time that username2's comment was posted. I'm now 98% sure for a variety of reasons.

I'm only 75% sure that the upvotes on "username2"/VoiceOfRa's comments above are from sockpuppets.

Comment by jsteinhardt on Voiceofra is banned · 2015-12-24T07:43:01.802Z · score: 9 (9 votes) · LW · GW

Requesting a transparency report.

I think it's bad form to make costly (in terms of time) requests to moderators unless you're willing to be part of the solution. In this case, it would be good at minimum to outline exactly what you mean by a "transparency report" --- concretely, what sort of information would you like to see, and why would it be helpful? It would be even better if you were willing to volunteer to help in creating the report to the extent that the help can be utilized.

Comment by jsteinhardt on Voiceofra is banned · 2015-12-24T06:20:20.024Z · score: 7 (15 votes) · LW · GW

I'm 85% sure that you're VoiceOfRa.

Comment by jsteinhardt on Voiceofra is banned · 2015-12-24T06:16:47.920Z · score: 11 (13 votes) · LW · GW

I'm dubious that that constitutes abusing her power; AdvancedAtheist was highly and consistently downvoted for a long period of time before being banned.

Comment by jsteinhardt on Voiceofra is banned · 2015-12-24T02:14:55.722Z · score: 5 (5 votes) · LW · GW

As Romeo noted, Nancy was appointed roughly by popular acclaim (more like, a small number of highly dedicated and respected users appointing her, and no one objecting). I think it's reasonable in general to give mods a lot of discretionary power, and trust other veteran users to step in if things take a turn for the worse.

Comment by jsteinhardt on Marketing Rationality · 2015-11-20T17:15:28.992Z · score: 12 (16 votes) · LW · GW

My main update from this discussion has been a strong positive update about Gleb Tsipursky's character. I've been generally impressed by his ability to stay positive even in the face of criticism, and to continue seeking feedback for improving his approaches.