Delegate a Forecast

post by Amandango · 2020-07-28T17:43:58.461Z · score: 41 (12 votes) · LW · GW · 20 comments

This is a link post for


  Some examples of questions you could ask:

EDIT: We've stopped answering questions for now, sorry if we didn't get to your question! We're still really interested in what kinds of questions people want forecasted, feedback on how useful it is to delegate forecasts, and Elicit as a tool, so feel free to keep commenting these thoughts. We also forecasted questions on the EA Forum version of this post [EA · GW].

Hi everyone! We, Ought, have been working on Elicit, a tool to express beliefs in probability distributions. This is an extension of our previous work on delegating reasoning. We’re experimenting with breaking down the reasoning process in forecasting into smaller steps and building tools that support and automate these steps.

In this specific post, we’re exploring the dynamics of Q&A with distributions by offering to make a forecast for a question you want answered. Our goal is to learn:

  1. Whether people would appreciate delegating predictions to a third party, and what types of predictions they want to delegate
  2. Whether a distribution can more efficiently convey information (or convey different types of information) than text-based interactions
  3. Whether conversing in distributions isolates disagreements or assumptions that may be obscured in text
  4. How to translate the questions people care about or think about naturally into more precise distributions (and what gets lost in that translation)

We also think that making forecasts is quite fun. In that spirit, you can ask us (mainly Amanda Ngo and Eli Lifland) to forecast any continuous question that you want answered. Just make a comment on this post with a question, and we’ll make a distribution to answer it.

Some examples of questions you could ask:

We’ll spend <=1 hour on each one, so you should expect about that much rigor and information density. If there’s context on you or the question that we won’t be able to find online, you can include it in the comment to help us out.

We’ll answer as many questions as we can from now until Monday 8/3. We expect to spend about 10-15 hours on this, so we may not get to all the questions. We’ll post our distributions in the comments below. If you disagree or think we missed something, you can respond with your own distribution for the question.

We’d love to hear people’s thoughts and feedback on outsourcing forecasts, providing beliefs in probability distribution, or Elicit generally as a tool. If you’re interested in more of what we’re working on, you can also check out the competition we’re currently running on LessWrong to amplify Rohin Shah’s forecast [LW · GW] on when the majority of AGI researchers will agree with safety concerns.


Comments sorted by top scores.

comment by Raemon · 2020-07-29T01:18:08.634Z · score: 7 (4 votes) · LW(p) · GW(p)

If I held an outdoor event near Berkeley CA, in December, where ~250 people sat outdoors in social-bubble groups 30' from each other, would at least one person get Covid-19?

[edit: event is 3 hours, with lots of shouting and singing]

comment by Owain_Evans · 2020-07-29T10:43:05.443Z · score: 3 (2 votes) · LW(p) · GW(p)

How long is the event? Is there perfect compliance with groups staying apart? Is everyone wearing masks? Are people singing/shouting?

comment by Raemon · 2020-07-29T19:48:18.577Z · score: 3 (2 votes) · LW(p) · GW(p)

Event is 3 hours with lots of shouting/singing.

comment by Richard_Ngo (ricraz) · 2020-07-29T09:52:22.805Z · score: 3 (2 votes) · LW(p) · GW(p)

I think the intention is that the forecasts are of continuous variables. Are you interested in the expected number of people who get covid?

comment by Raemon · 2020-07-29T19:07:58.943Z · score: 3 (2 votes) · LW(p) · GW(p)

Oh, yes that’s even better. I had assumed I was supposed to phrase it as a discrete question. 
(also interested in fractional people if it’s less than one)

I guess ‘number of microcovids generated by the event’ is what I actually want. Not sure what the requirements of the question are. 

comment by Amandango · 2020-07-30T17:05:53.347Z · score: 3 (2 votes) · LW(p) · GW(p)

Either expected number of people who get covid or number of microcovids generated by the event works as a question! My instinctive sense is that # of people who get covid will be easier to quickly reason about, but I'll see as I'm forecasting it.

comment by Raemon · 2020-07-30T17:19:24.909Z · score: 4 (2 votes) · LW(p) · GW(p)

Nod. My guess is that that number of people in expectation will likely be 0 (if it turns out to be 1 or more than that's a dealbreaker for the event), and "fractional chance at least one person gets it" would be decision relevant for me.

*I guess further consideration that's harder to reason about is "I'm expecting most of the attendees to be the sort of person who takes caution seriously most of the time, which lowers the rate of people who are likely to have in the first place to transmit it." But, not sure how to quantify that and don't really want to rely on it.

comment by Amandango · 2020-08-01T23:55:26.850Z · score: 1 (1 votes) · LW(p) · GW(p)

Here's my prediction, and here's a spreadsheet with more details (I predicted expected # of people who would get COVID). Some caveats/assumptions:

  • There's a lot of uncertainty in each of the variables that I didn't have time to research in-depth
  • I didn't adjust for this being outdoors, you can add a row and adjust for that if you have a good sense of how it would affect it.
  • I wasn't sure how to account for the time being 3 hours. My sense is that if you're singing loudly at people < 1m for 3 hours, this is going to be a pretty high infection rate. Also, I assumed they weren't wearing masks because of the singing. I'm most uncertain about this though
  • You didn't mention how big the pods are. I assumed 10 people in a pod, but it would change it if this were much smaller.
comment by Raemon · 2020-08-02T01:19:45.409Z · score: 2 (1 votes) · LW(p) · GW(p)


(fwiw: I was assuming the people in pods would already have infected each other or would be about to infect each other anyways (due to sharing a house and lots of airspace), so the intent was something more about how much additional spread would be between pods, outdoors)

comment by philh · 2020-07-31T23:22:48.755Z · score: 6 (3 votes) · LW(p) · GW(p)

This is super self absorbed, and maybe not reasonably doable with the info you have, but... no harm in asking I guess.

I'd like to know my chances of winning the SSC book review contest (assuming Scott starts asking for entries again following this).

(In my favor: I think I'm a pretty good writer, reviewing a book that's totally on-brand. Against me: he had ~20 entries more than a month before the original deadline (and by Sturgeon's law, two of those might have been decent), and I haven't finished yet.)

comment by whyzenna · 2020-07-29T03:08:11.679Z · score: 4 (3 votes) · LW(p) · GW(p)

By 2030 how many logical qubits will the leading commercially available quantum device be able to compute with?

comment by elifland · 2020-08-03T01:48:52.475Z · score: 2 (2 votes) · LW(p) · GW(p)

My forecast is based on:

I don't have a background in quantum computing, so there's a chance I'm misinterpreting the question in some way, but I learned a lot doing the research for the forecast (like that there's a lot of controversy regarding whether quantum supremacy has been achieved yet).

Amusingly, during my research I stumbled upon this Metaculus question about when a >49 qubit quantum computer would be created which resolved ambiguously due to the issue of how well-controlled the qubits are. For the purposes of this forecast I assumed it would resolve based on the raw number of qubits, without adjusting for control.

comment by rossry · 2020-07-28T22:59:31.284Z · score: 4 (3 votes) · LW(p) · GW(p)

When will air travel from New York to Hong Kong no longer require arriving passengers to self-quarantine?

comment by SoerenMind · 2020-07-30T19:49:55.138Z · score: 2 (2 votes) · LW(p) · GW(p)

I'm not sure if a probability counts as continuous?

If so, what's the probability that this paper would get into Nature (main journal) if submitted? Or even better, how much more likely is it to get into The Lancet Public Health vs Nature? I can give context by PM.

comment by lambdaloop · 2020-07-29T21:55:18.082Z · score: 2 (2 votes) · LW(p) · GW(p)

In what year would more than 30% of US adults own a device whose primary interaction mode is through augmented reality (such as google glass or the rumored apple AR glasses)?

comment by Amandango · 2020-08-02T00:11:28.110Z · score: 3 (2 votes) · LW(p) · GW(p)

Here’s my prediction for this! I predicted a median of March 1, 2029. Below are some of the data sources that informed my thinking.

Related Metaculus question: When will sales of a non-screen technology be greater than sales of a screen technology?

comment by GuySrinivasan · 2020-07-29T17:34:06.252Z · score: 2 (1 votes) · LW(p) · GW(p)

A house in the Seattle area sold for $1,000,000 in April 2019.

How much would the same house sell for in April 2021? Assume away "someone wanted to sell after only 2 years" considerations; maybe instead a better phrasing is "a house in the Seattle area had an extremely credible offer for $1,000,000 in April 2019, which was not acted upon and records of which are not publicly available".

comment by elifland · 2020-08-02T22:48:07.940Z · score: 1 (1 votes) · LW(p) · GW(p)

My forecast is based on historical data from Zillow. I explained my reasoning in the notes. The summary is that housing prices haven't changed very much in Seattle since April 2019 (on the whole it's risen 1%). On the other hand, prices in more expensive areas have stayed the same or declined slightly. I settled on a boring median of the price staying the same. Due to how stable the prices have been recently, I think most of the variation will come from the individual house and which neighborhood it's in, with an outside chance of large Seattle home value fluctuations.

comment by chris_painter · 2020-07-30T09:16:57.657Z · score: 1 (1 votes) · LW(p) · GW(p)

How much money will be raised by special purpose acquisition companies (SPACs) in initial public offerings in the year 2021?

To clarify, I'm looking for the number of new SPACs that issue equity times the average amount that they raised. I am NOT looking for the amount of money raised by the companies that were eventually the acquisition targets of these SPACs.







comment by chris_painter · 2020-07-30T08:11:31.292Z · score: 1 (1 votes) · LW(p) · GW(p)

How many days after August 1st, 2020, will the FDA issue an Emergency Use Authorization for a product that performs some form of COVID testing directly in the home, meaning sample collection, molecular biology, and diagnostic result delivery can all be done without a user leaving their home?

Access to cloud computing is fine. Tele-guidance from a trained healthcare practitioner or anyone else is fine.