[LINK] Sign up for DAGGRE to improve science and technology forecasting
post by Qiaochu_Yuan · 2013-05-26T00:08:55.793Z · LW · GW · Legacy · 9 commentsContents
9 comments
Link:
In When Will AI Be Created?, I named four methods that might improve our forecasts of AI and other important technologies. Two of these methods were explicit quantification and leveraging aggregation, as exemplified by IARPA's ACE program, which aims to “dramatically enhance the accuracy, precision, and timeliness of… forecasts for a broad range of event types, through the development of advanced techniques that elicit, weight, and combine the judgments of many analysts.
GMU's DAGGRE program, one of five teams participating in ACE, recently announced a transition from geopolitical forecasting to science & technology forecasting:
DAGGRE will continue, but it will transition from geo-political forecasting to science and technology (S&T) forecasting to better use its combinatorial capabilities. We will have a brand new shiny, friendly and informative interface co-designed by Inkling Markets, opportunities for you to provide your own forecasting questions and more!
Another exciting development is that our S&T forecasting prediction market will be open to everyone in the world who is at least eighteen years of age. We’re going global!
If you want help improve humanity’s ability to forecast important technological developments like AI, please register for DAGGRE’s new S&T prediction website here.
Experienced PredictionBook veterans should do well.
9 comments
Comments sorted by top scores.
comment by ozziegooen · 2013-05-26T01:17:54.975Z · LW(p) · GW(p)
I couldn't believe the registration "questions".
- "A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?"
- "If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets?"
- "In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it takes 48 days for the patch to cover the entire lake, how long would it take for the patch to cover half the lake?"
These are the exact questions from this blog post, to the letter (and are also all very well known outside that, but sometimes in different orders). What?
Then there were a few that state "Discontinued", including one with the answers (this was multiple choice), "Completely a Fox", "In between a Fox and a Hedgehog", and "Completely a Hedgehog" (with a few others in between). Why these questions were not just removed from the form is beyond me. I'm also really interested in how, at least for a while, they may have assumed readers knew about this dichotomy, especially readers who they assumed did not know the 3 most popular analysis/rationality questions on the internet.
The main site seems to be written in Ember or Backbone.js or something, and yet it's really tough to use and is very buggy in chrome. (Closer inspection shows they may have done it without a MVC js framework. Also, they haven't minified their main JS files for some reason.)
The existing platform has all the markings of a big bureaucratic project combined with recent web and prediction market trends. Bizarre.
Replies from: Qiaochu_Yuan, ctwardy, ozziegooen↑ comment by Qiaochu_Yuan · 2013-05-26T01:50:15.311Z · LW(p) · GW(p)
Having a very low bar to cross might still be helpful. Answering those questions correctly might not tell you much, but answering them incorrectly is a bad sign. I agree the "Discontinued" questions were weird.
Replies from: ozziegooen↑ comment by ozziegooen · 2013-05-26T01:55:56.753Z · LW(p) · GW(p)
A low bar should still be helpful, but this is more of a google search than a rationality check. They could have at least changed the numbers slightly or the nouns by a bit. If it were to check if you were a computer, it wouldn't work at all because the answers are the same each time.
Of course, answering them incorrectly is still a bad sign. I just imagine that with a 3 minutes of more effort (especially compared with the multi-year long complex web application) they could have had something substantially more useful and interesting.
Replies from: Viliam_Bur↑ comment by Viliam_Bur · 2013-05-26T12:56:33.339Z · LW(p) · GW(p)
this is more of a google search than a rationality check
Knowing when to use google is a part of instrumental rationality. :D
↑ comment by ctwardy · 2013-05-30T12:15:15.421Z · LW(p) · GW(p)
Whoa whoa whoa! That's the old registration site. Go to signup.daggre.org to register for the S&T market (like the original post said).
And, er, sorry about the disrepair of the old intake questions. I thought that had been taken care of ages ago. (But the Cognitive Reflectance Test questions are actually a pretty good screen. Fewer than half the people answer them correctly.) Anyway, the main site is going down shortly while we retool. Use signup.daggre.org to get in the queue for the Science & Technology forecasting site. We're going to take advantage of downtime this summer to make some fundamental changes we simply couldn't do while running a production site in a tournament. (We did do substantially better than a baseline opinion pool and a baseline prediction market on the geopolitical questions.)
I'll also ask our participant coordinator to check everyone who has signed up to the main site in the last week. Thanks for posting your problems.
Charles Twardy PI for the (soon-to-be-renamed) DAGGRE project
↑ comment by ozziegooen · 2013-05-26T01:46:54.702Z · LW(p) · GW(p)
On that note, seriously, check out IARPA if you get a chance. It seems to be a recent (2008) offshoot of DARPA, that's like "Darpa, but for Spies". Their mission is to "invest in high-risk/high-payoff research programs that have the potential to provide the United States with an overwhelming intelligence advantage over future adversaries." link
Basically, it seems like they are the government's bleeding edge of spying and internet security. Some of their researchers seem to be doing projects focussed on quantum computing and computer security. Others on using public big data to predict possible threats. Others to train people to understand their cognitive biases by playing games. It's almost as if someone cloned LessWrong and had them work for the government but closed sourced and significantly less idealistic.
It's really a pity that it's all closed, with the mission itself implying a zero-sum game. However, there is a surprising amount of public information on their website. For example, check out the complete sirius page. Not only can you find a pretty precise documentation of how the IARPA is trying to make this game to reduce cognitive biases, but you can also find all of the proposals by different agencies to get this contract. Really interesting stuff.
comment by tgb · 2013-05-26T23:30:11.496Z · LW(p) · GW(p)
You also get entered into a drawing for $50 Amazon gift certificates based on activity (not based on performance - they're don't want to touch gambling regulations). I have gotten 2 this twice now with pretty low activity, which is nice.
The current and previous questions were focused on geo-political predictions which proved to be a surprisingly interesting to someone who doesn't really follow the news to closely. It provides a nice alternative way of learning about the world other than headlines - each question is a mini research project to quickly learn about the developments of North Korea's multistage rocket program or the likelihood of US increasing certain sanctions.
Replies from: ctwardycomment by ctwardy · 2013-05-30T12:27:24.570Z · LW(p) · GW(p)
Thanks for the shout! We're excited to move into Science & Tech which lets us be much more open about recruiting and sourcing. I really hope we get lots of people from LW.
As noted in my reply below, please use the new registration site (it has an atom logo, not a dagger). That will put you in the queue to be notified when we bring up the new site. The old site will go dark this Saturday, though we will still have occasional updates on the blog. The new site will be in testing over the summer (there is an email link if you're interested) and go live in the autumn.