An overview of forecasting for politics, conflict, and political violence
post by VipulNaik · 2014-06-24T22:10:39.093Z · LW · GW · Legacy · 0 commentsContents
Actors interested in making predictions about global politics Databases used in predictive analytics for global politics Global opinion polling and influence/sentiment measurement Some predictive algorithms Websites with a good overview of different datasets and predictive algorithms Blogs to follow Important people Journals Popular magazines Additional reading None No comments
Global politics is a high-stakes affair, and being able to predict it, prepare for it, and in some cases manipulate it, could be a game-changer. Political forecasting, construed broadly as forecasting of events that affect the structure of political systems and the configurations of political power, is therefore quite an important activity.
In relatively stable, prosperous, democratic, and developed countries, political forecasting largely involves two related forecasting activities:
- Predicting what individual or political parties will win elections
- Predicting what sort of policies will be implemented by the ruling parties
The tools used here are relatively simple: public opinion (both on who people will vote for and what policies they want to see implemented) is tracked through opinion polls. For forecasts made far out in the future, the opinion poll results are combined with some models about how changing economic or political conditions are likely to affect people's voting choices. (In fact, as of the time of writing this post, the Wikipedia page on political forecasting somewhat narrowly defines it as election forecasting, thereby taking the perspective of a relatively stable democratic country).
A blog post on the SAS Business Forecasting blog reviews the main methods used in election forecasting, and compares their performance on US presidential elections. It identifies three broad categories of models that seem to make somewhat reliable and high-quality predictions:
- Prediction markets such as Intrade (website, Wikipedia) and the Iowa Electonic Markets (website, Wikipedia).
- Combination models such as PollyVote (website, Wikipedia), created by J. Scott Armstrong. A paper by Armstrong with Andreas Graefe claims that despite being quite stable and accurate, PollyVote has not received much media coverage, and speculates as to the reasons.
- Polling aggregators such as FiveThirtyEight (website, Wikipedia) (run by Nate Silver) and Votamatic (website, Wikipedia). See also the blog post Was Nate Silver the Most Accurate 2012 Election Pundit? by Luke Muehlhauser and Gwern Banwen, November 9, 2012.
When considering global politics, however, the narrow focus on public opinion and election performance is misguided, for two reasons. First, the interactions between nation-states are governed by rules somewhat different from those that govern local politics, and they often do not map easily to public opinion. Second, and more importantly, in many countries around the world, what transpires politically is far from an accurate reflection of majority public opinion. Of course, public opinion does affect political outcomes, even in dictatorships, but the nature of the relationship is more complex: the opinions of subsets of the population that have more of an ability to stage a coup matter more. Also, in some countries, there are centers of political power or activity that are not (officially) linked with the state. These include rebel groups, separatists, terrorists, and local militia. Even countries that do have elections may not necessarily have free and fair elections, so the outcome of the election may be governed more by who controls the polling apparatus than by what the people want.
This is not to suggest that most parts of the world are mired in continuous, frequent conflict. Many parts of the world, even poor parts and undemocratic parts, are generally peaceful most of the time. But threats of intergroup or interfactional violence play an important role in governing the trajectory of events, even if violence itself is rare.
The problem of predicting global politics is therefore tricky because it's not even clear what questions we should be asking. The simple question "who will win the election in 2014?" isn't good enough. Lots of other questions, such as "will the army stage a coup?" or "will the president be willing to have a free and fair election, or conduct a sham referendum to consolidate his power?" or "will the government purge <insert unpopular minority group here> from the big cities?" could be worth asking. And it's sometimes not even clear whether a question is worth asking until it has been answered in the affirmative.
For the purpose of this post, then, we will discuss together the domains of political forecasting (that largely involves election forecasting and public opinion forecasting) and the forecasting of conflicts, terrorism and international crises.
Actors interested in making predictions about global politics
So who gets in the business of trying to figure out how global politics will unfold? One obvious answer is: governments of other countries, partly with the goal of protecting the country's own economic interest in those other countries, and partly with humanitarian goals of avoiding the other country getting into violence. In addition, agencies whose goal is to combat terrorism are also interested in political developments that might create breeding grounds or support infrastructure for terrorism. Industries that depend on resources available only in a few countries are interested in making sure that those countries remain sufficiently stable that they can continue extracting the resources (and/or that they are on sufficiently good terms with the rulers that the resource extraction can continue despite the presence of conflict).
Some specific actors are listed below:
- Intelligence Advanced Research Projects Activity (IARPA) (website, Wikipedia), a research agency under the Office of National Intelligence in the United States. IARPA sponsors the Aggregative Contingent Estimation (ACE) program (website, Wikipedia) that funds The Good Judgment Project, one of the best political forecasting tools today.
- Political Instability Task Force (PITF) (website, Wikipedia), funded by the Central Intelligence Agency (CIA) in the United States.
- Various governmental and nongovernmental initiatives aimed at genocide prevention, such as the Center for the Prevention of Genocide (website, Wikipedia) at the United States Holocaust Memorial Museum, and the Sentinel Project for Genocide Prevention (website, Wikipedia)
- Terrorism-related initiatives, such as the National Consotrium for the Study of Terrorism and Responses to Terrorism (START) (website, Wikipedia) and the Chicago Project on Security and Terrorism (CPOST) (website, Wikipedia).
Databases used in predictive analytics for global politics
Database name | Areas covered | Year range | Availability and update frequency |
---|---|---|---|
Global Database of Events, Language, and Tone (GDELT) (GDELT project, Wikipedia) | All political events, using CAMEO codes | 1979-present | Publicly available, updated daily |
Integrated Conflict Early Warning System (ICEWS) (website, Wikipedia) | All political events using CAMEO framework. However, unlike GDELT, follows a more traditional approach to event data in seeking to encode a chronology of events that reflects in some sense the putative ground truth of what occurred. | ? | Not publicly available, but was made available to forecasters for The Good Judgment Project |
Armed Conflict Location and Event Data Project (ACLED) (website, Wikipedia) | Armed conflicts | 1997-present (for Africa), fewer years for South Asia | Publicly available, updated daily |
Uppsala Conflict Data Program/Peace Research Institute of Oslo (UCDP/PRIO) Armed Conflict Dataset (website, Wikipedia) | Armed conflicts | 1946-present (data collected live only since the 1970s) | Publicly available, updated annually |
Worldwide Atrocities Dataset (website, Wikipedia) | Atrocities, i.e., the deliberate use of lethal violence against non-combatant civilians by actors engaged in a wider political or military conflict | 1995-present, with a four-month embargo period | Monthly (with a four-month embargo period) |
Correlates of War (website, Wikipedia) | Data sets on Militarized Interstate Disputes (MID), National Material Capabilities, World Religion, Formal Alliances, Territorial Change, Direct Contiguity, and many more (see here for a full list) | Some of the data sets are available from about 1815 or 1816, the data sets proceed till about 2005 or 2006, but are likely to be updated to include later years. | Unclear |
International Crisis Behavior Project (website, Wikipedia) | International crises, loosely defined as something that had the potential to lead to a conflict or war | 1918-2007 | A new version of the database is released every few years, adding more years at the end. The database remains a few years behind the times. For instance, as of June 2014, the most recent release is from July 2010 and covers data till 2007. All data is freely available online. |
Global Terrorism Database (website, Wikipedia) | All terrorist attacks | 1970-2012, excluding 1993 (more years to be added) | Approximately annually |
Suicide Attack Database (website, Wikipedia) | All suicide attacks | 1982-2013 | Approximately annually |
Manifesto Project Database (earlier known as the Comparative Manifestos Project) (website, Wikipedia) | Political manifestors and election performance of over 50 free democratic countries | 1945 onward | ? |
Hatebase (website, Wikipedia) | Instances of hate speech that might be predictors of potential violence or persecution | Presumably 2013 onward, when it launched |
Continuous, relies on user submissions, publicly available |
Global opinion polling and influence/sentiment measurement
In addition to databases of political events, worldwide opinion polls are also useful in political forecasting. Examples of agencies that conduct worldwide opinion polls are:
In addition to opinion polling, other tools used to measure global public sentiment include analytics for web service usage. There has been considerable research on using information from sources such as Google Trends, Twitter, and Wikipedia. The United Nations Global Pulse (website, Wikipedia) is an example of an effort to use the digital trails of people to extract meaningful information.
Some predictive algorithms
How do we use the mass of structured and unstructured data to make smart political predictions? There are a number of people that claim to have good prediction strategies, but some of them have been debunked, while the jury on others is still out. Some examples are listed below.
- The most widely respected source for political predictions is The Good Judgment Project (website, Wikipedia). This is a forecasting competition where anybody can participate. Participants are given a set of questions and can basically collect freely available online information (in some rounds, participants were given additional access to some proprietary data). They then use that to make predictions. The aggregate predictions are quite good. For more information, visit the website or see the references in the Wikipedia article. In particular, this Economist article and this Business Insider article are worth reading.
- The Integrated Conflict Early Warning System (ICEWS) (website, Wikipedia) run by Lockheed Martin combines data collection on global political events with a predictive algorithm to provide early warning of conflicts. it is in use by the United States Pacific Command and United States Southern Command. However, ICEWS data and predictions are not publicly available, so it is hard to independently gauge their accuracy. For more, see this post by Patrick Meier and other posts in the references of the Wikipedia page on the ICEWS.
- Steve Levine at Quartz claims to have a geopolitical prediction algorithm with high accuracy, and that successfully predicted a number of events in 2013. The main critical analysis (that makes mostly commonsensical points) I could find was this by Jay Ulfelder.
- Bruce Buenos de Mesquita claims to have a good forecasting track record (see for instance this Cato Unbound July 2011 discusssion and this news article). I found a rather lengthy online critique of his work online but I haven't vetted the substance of the critique. In general, people in the blogs I list below are skeptical of his methods.
- Professor Lincoln P. Bloomfield developed the MIT Cascon System for Analyzing International Conflict (website, Wikipedia). I'm not really sure what this is about.
Websites with a good overview of different datasets and predictive algorithms
The Forecasting Principles website has some interesting overviews related to the forecasting realm. The two relevant Special Interest Groups are:
Blogs to follow
- Predictive Heuristics, a group blog
- Dart-Throwing Chimp, the blog of Jay Ulfelder
- Bad Hessian, a group blog
- Political Violence at a Glance
- iRevolution by Patrick Meier. The author earlier blogged at the Early Warning Project.
- For more blogs, see the blogroll of Predictive Heuristics
- The Monkey Cage, formerly an independent blog, now with The Washington Post
Important people
- Jay Ulfelder (about page on his own blog, Wikipedia): An expert on political forecasting, Ulfelder was at the helm of the CIA's Political Instability Task Force and is now advising the Early Warning Project of the Center for the Prevention of Genocide.
- Kalev Leetaru (website, Wikipedia): He is the co-creator of GDELT and has previously worked on other big data projects.
- Philip A. Schrodt (about page on his own blog, Wikipedia): He is a legend in the world of automated event database creation. He is a co-creator of the CAMEO framework and also a co-creator of GDELT.
- Michael D. Ward (website, Ward Lab, Wikipedia)
- Jay Yonamine
- Philip E. Tetlock (website, Wikipedia): Although not a political forecasters himself, his study of the track record of expert predictions has influenced thinking in the subject. He currently co-runs The Good Judgment Project, a political forecasting competition, along with Barbara Mellers and Don Moore
Journals
To my knowledge, there are no journals exclusively devoted to forecasting, but the work of the people listed above has generally appeared in a relatively small set of journals, listed below.
- Journal of Conflict Resolution (website, Wikipedia)
- Journal of Peace Research (website, Wikipedia)
- International Studies Review (website, Wikipedia)
There are also some journals devoted to forecasting in general (list to appear in a future post of mine) but as far as I can make out, very little of the conflict forecasting literature is published in such journals.
Popular magazines
Two magazine worth checking out for discussions of political science, that occasionally discuss issues related to forecasting, are:
- Foreign Policy (website, Wikipedia): This is by far the most likely source of information. Kalev Leetaru was an occasional contributor. It has included pieces from Jay Ulfelder and Michael D. Ward. Joshua Keating has also discussed many themes related to predictive analytics and forecasting while at FP.
- Foreign Affairs (website, Wikipedia)
Additional reading
On election forecasting (these links are supplementary to the links in the main text):
- Princeton Election Consortium
- Poll aggregation and election forecasting by Andrew Gelman at The Monkey Cage
- Some thoughts on election forecasting by Andrew Gelman on his own blog
On global political forecasting and general considerations relevant to it:
- Why the World Can't Have a Nate Silver. The quants are riding high after Team Data crushed Team Gut in the U.S. election forecasts. But predicting the Electoral College vote is child's play next to some of these hard targets. by Jay Ulfelder.
- Predicting the Future is Easier Than it Looks by Michael D. Ward and Nils Metternich (response to Ulfelder).
- Cato Unbound July 2011 discusssion featuring Philip Tetlock, Dan Gardner, Robin Hanson, John Cochrane, and Bruce Bueno des Mesquita.
- In Defense of Political Science and Forecasting by Jay Ulfelder, responding to an op-ed by Jacqueline Stevens.
UPDATE: I found what seems to be a fairly interesting and thorough paper titled Stepping into the future: the next generation of crisis forecasting models by some of the people listed here (specifically Michael D. Ward, Nils Metternich, and co-authors at the Ward Lab). I haven't had time to examine it closely. It includes a discussion of ICEWS.
0 comments
Comments sorted by top scores.