How Hypermind predicts the future

The Hypermind prediction market is a forecasting contest on geopolitical, business and economic issues. It is designed to consolidate the informed guesses of thousands of brains worldwide into accurate probabilities. For instance, on the eve of 2020, the market judges that Donald Trump is 51% likely to be reelected president of the USA next November, compared to 40% for another man to replace him, or 9% for a woman. (All our predictions are published in real time here.)

USPREZ2020WHO-EN

As this example shows, the prediction market is not a binary oracle that states with certainty what will or won’t happen. Its pronouncements are more subtle. It doesn’t say that the next president of the USA won’t be a woman, it says that there is currently 91% probability that it will be a man.

One naturally wonders whether these probabilities are reliable or pulled out of thin air. If Trump or another man is elected in november, will it mean that the market was overestimating womankind’s chances to seize the White House? Or if a woman does win, will it mean that the market was severely underestimating her chances? To answer these concerns, one would have to rerun the election a hundred times: if a woman won in about 9 runs out of a hundred, then the 9% probability was correctly estimated. But if a woman won much less often, say only 3 times, or too often, say 18 times, then the 9% estimate was, respectively, 3 times too large, or too small by half. Wrong in any case.

Unfortunately, elections or other historical events cannot be rerun, not even once. Does that mean it is impossible to assess the accuracy of probability forecasts? No, there is another way, less ideal but satisfactory nonetheless: consider allthe events that the market has forecasted with the same probability, say 10%, and count how many have actually happened. If about 10% for them did occur, then the 10% probability forecast was correct. Otherwise it wasn’t. Repeating this procedure over all probability levels, from 1% to 99%, gives a complete picture of the forecasting accuracy of the prediction market.

To perform such analysis, lots of data are required. Hypermind has them. Over 5 years, from the market’s launch in the spring of 2014 until the spring of 2019 (when this analysis was performed), it has generated 537,640 probability forecasts about 1,185 possible answers to 400 forecasting questions in the geopolitical, business, and economic domains. In the graph below, each data point answers the question: « What is the proportion of events forecasted with probability pthat actually occurred? »Amazingly, reality seems to literally align itself to the market’s predictions.

calibration-en

This alignment shows the amazing ability of the market’s collective intelligence to discern, not the future itself, but its underlying probabilities. From this we may draw some conclusions:

  1. The future is not determined by the world’s present state. It is fundamentally probabilistic. It is always wrong to think, after the fact, that whatever happened hadto happen. There always was just a probability that it could happen, and another that it might not. It always could have turned out otherwise if the great dice roll in the sky had landed differently.
  2. Collective intelligence, as expressed in a prediction market, is able to accurately discern the probability field that underlies reality.
  3. The forecasts of the Hypermind prediction market are particularly reliable.

To explore these ideas further, in practice:

2 Comments

  1. How was this chart made? Can you share details? For instance: when were market predictions sampled, relative to the outcomes… a day prior, a week prior…?

    Like

    Reply

    1. For each probability p between 1 and 99:
      1) Compile all outcomes that have ever been traded at least once at price p, at any point during their lifetime;
      2) Plot the proportion of these outcomes that actually occurred.

      Like

      Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s