Polls are dead, long live markets

NFCROWD

The polling fiasco in the 2015 UK general election is just the latest in a string of high-profile failures over the last few months. This contrasts with the good performance of prediction markets, and Hypermind in particular.

Let’s start with the referendum on Scottish independence in september 2014. In the final weeks before the referendum, the polls consistently announced a cliffhanger with Yes and No tied within the margin of error. Yet the actual results gave “No” a large majority of 55%, 10 points ahead of “Yes” (45%).

The betting markets on the other hand clearly favored the “No” vote throughout. Witness for instance how the “Yes” vote on Hypermind always stayed below the 50% likelihood threshold, and was given a low probability just before the referendum took place on September 18th.

SCOTLAND

Then came the midterm congressional elections in the U.S., in november 2014. The big question then was whether the Republicans would recapture control of the Senate, which they did. The polls mostly saw this coming, but were much more timid in their forecasts than the betting markets.

In fact, as discussed earlier in this blog, Hypermind out-predicted all the poll-aggregation models operated by the biggest U.S. media, as well as Nate Silver’s FiveThirtyEight. (Only the Washington Post model ended-up out-predicting Hypermind at the very end, but its prediction was all over the place beforehand, as can be seen in the chart below.)

midterms2014Senate

The Israeli elections in March 2015 again stumped the polls and the pundits. The closer we got to election day, the more Benyamin Netanyahu was given up for dead, politically. The latest polls even predicted his Likud party would be 4 seats behind his leftist rival, and considered how difficult it would be for him to assemble a 61 seat majority coalition in the Knesset. Instead, Likud scored 6 more seats than its closest rival, and Bibi was able to remain prime minister for a 4th term.

What about the betting markets ? On the day before the election, while noting that the election was a rare instance of an “actual tossup“, the New York Times also noted that Hypermind was giving Netanyahu 55% chances of staying prime minister. In fact, Hypermind had clearly kept Netanyahu in the favorite seat all through the campaign.

BIBISTAY

Which now brings us to the UK general election 2015. It concluded yesterday with a big win for David Cameron’s Conservative Party, a hair-breadth away from an absolute majority in parliament. This was in contrast to all the polling data which had Labour tied with the Conservatives, both very far from a majority. Based on the poll projections of a hung parliament, the pundits could not see how Cameron could gather a governing coalition, even when adding up Ukip and the LibDems. Everyone gave Labour’s Miliband a much better chance of forming a government, with tacit support from the Scottish Nationalist Party. In fact, the polls gave the Labour+SNP a clear majority in the House…

The story was different in the betting markets. At worst, Cameron’s chances of forming the next government remained close to 50%, tied with Labour’s Miliband’s, a far cry from the large Labour advantage everyone assumed from the parliamentary arithmetic based on poll projections. On Hypermind, a Cameron rebound even occurred just before election day.

UKPRIME-full-en

It will take some time to understand why election polls, which had served the media so well for so long, seem to be suddenly experiencing a global meltdown. Perhaps the simple, powerful idea of the “representative panel” just no longer works well when individualism is pushed to the extreme in modern societies…

What is encouraging, though, is that betting markets – an approach that preexisted polls by decades – are proving more reliable, especially when the going gets tough. This is probably related to the idea, explored earlier in this blog, that predicting human affairs is in general best left to human brains than to algorithms and statistics.

5 Comments

  1. This is a bit of an unfair comparison. You are comparing single unprocessed polls to betting markets that incorporate that information and more. Moreover, the betting markets really haven’t done all that well (but yes, they’ve done better than polls).

    For example, in the UK General Election, you’ve chosen a very specific bet – ‘Will the next PM belong to the Conservative party’, and yeah betting markets did sort of okay on that one, but still the odds on the election day were hovering around 50% for him and 50% for Miliband, with here and there jumping to a few percentage points higher, while the actual results made it look silly to even have Miliband in there. And if you look at the other bets like what government would have formed, things get even less impressive.

    There is also another important thing, which makes the bookies look better – the odds we see are higher because of the bookies’ premium (which makes it look like they were closer to the truth), but if we look at odds for the losing positions they also would be higher.

    TLDR: Yes, bookies outperform simple polls, but it is mostly exaggerated by how much.

    Like

    Reply

    1. Note: The last point might not apply (as much) to Hypermind in particular.
      Also fun fact: Fivethirtyeight’s model had put 0.00 probability on a Conservative Majority in the UK General Election.

      Like

      Reply

  2. You are missing the most important point: All these election results were conservative, right-wing victories, including the Scottish referendum. Opinion polls underestimate right wing numbers because of the “shy-tory” effect – A significant number of voters are secret right-wingers. In Australia its impossible to find anyone who says they voted for Tony Abbott’s Liberal/National coalition government.

    Like

    Reply

  3. Marché prédictif : Qui remportera la primaire à droite… Ou l’art et la manière d’influencer l’électorat porté à sa quintessence … On sait que les sondages à répétition ont quelque retentissement dans le résultat des urnes, mais pour le coup la technique du marché prédictif, bébé d’hypermind, dépasse tout ce qui existait en laissant apparaître le résultat de la libre opinion comme science prédictive ! Joli coup de lobbying au profit d’un seul candidat , auquel Le Point s’associe en marquant insidieusement la mémoire inconsciente des lecteurs longtemps avant l’évènement.

    Like

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s