Good Judgment in Forecasting International Affairs

The Economist’s The World in 2014 issue focuses international attention on the geopolitical outcomes we can expect to see over the next 12-14 months hits the newsstand.  It features an article by University of Pennsylvania psychologist Phil Tetlock and journalist Dan Gardner on the Good Judgment Project.  That said article isa research study funded by the Intelligence Advanced Research Projects Activity (IARPA, the U.S. government’s analog to DARPA), as a result, makes such geopolitical predictions each day.

IARPA has posed approximately 100-150 questions every year to research teams partaking in its ACE forecasting tournament on topics like the Syrian civil war, the constancy of the Eurozone and Sino-Japanese relations since 2011.  Every research team was obliged to collect individual forecasts coming from many forecasters online and to produce daily collective forecasts that allocate sensible probabilities to potential outcomes.

The Good Judgment Project came out as the evident winner and the Good Judgment Project forecasters have established the capability to produce more right forecasts that have surpassed even a few of the most positive approximation at the start of the tournament.  The supplementary graphic shows the calculation from three GJP forecasting techniques on a up to date question about whether the first round of chemical weapons inspections in Syria would be completed before Dec. 1.

From the said condition, the question resolved as a “yes” since from the one who was conducting the inspections which is the Organization for the Prohibition of Chemical Weapons (OPCW) confirmed that they had completed the first round of inspection before Dec. 1.  Because the question resolved as a “yes”, the closer each forecasting method’s predictions were to 1, the better they did.  According to the results of the graph, after some first hesitations about if inspections would occur or not, our forecasters in general met on the correct answer well previous to the outcome.

The Project utilizes new social-science methods ranging from controlling the understanding of mass to prediction markets to situating the teams of forecasters together.  The GJP research team feature its success to a mixture of getting the accurate people on the bus, tendering basic tutorials on inferential traps to keep away from and best practices to put in the practice everyday, engaging the most brilliant forecasters into great teams, and continually fine-tuning the aggregation algorithms it uses to unite individual forecasts into a collective prediction on every forecasting question.  The Project’s best forecasters are normally talented and very motivated amateurs, instead of the subject matter experts.