Nassim Taleb on Election Forecasting

The math isn’t super important to see why it’s so cool. His model seems to be that we should try to forecast the election outcome, including uncertainty between now and the end date, rather than build a forecast that takes current poll numbers and implicitly assumes nothing changes.

I don't think the Markov Models that Silver uses assume that nothing changes. 2 of his 3 models assumed change.

When you read Silver (or your preferred reputable election forecaster, I like Andrew Gelman) post their forecasts prior to the election, do you accept them as equal or better than any estimate you could come up with?

(out of memory) A year ago I think I gave Trump 40% conditional on him winning the Republican nomination on GJOpen. I think two months ago I moved from numbers were over the GJOpen average to

Scott Adams claimed to be able to predict that Trump would persuade everyone to vote for him. While it's tempting to write him off as a pundit charlatan, or claim he doesn't have sufficient proof, we also can't prove his model was wrong either

Scott Adams predicted a Trump landslide. That scenario didn't happen, Trump lost the popular vote. He didn't get elected because he convinced the majority of the population but because of Electorate College math.

Thanks for posting this! I have a longer reply to Taleb's post that I'll post soon. But first:

When you read Silver (or your preferred reputable election forecaster, I like Andrew Gelman) post their forecasts prior to the election, do you accept them as equal or better than any estimate you could come up with? Or do you do a mental adjustment or discounting based on some factor you think they've left out?

I think it depends on the model. First, note that all forecasting models only take into account a specific set of signals. If there are factors influencing the vote that I'm both aware of and don't think are reflected in the signals, then you should update their forecast to reflect this. For example, I think that because Nate Silver's model was based on polls that lag behind current events, if you had some evidence that a given event was really bad or really good for one of the two candidates, such as the Comey letter or the Trump video, you should update in favor of/against a Trump Presidency before it becomes reflected in the polls.

The math is based on assumptions though that with high uncertainty, far out from the election, the best forecast is 50-50.

Not really. The key assumption is that your forecasts are a Wiener process - a continuous time martingale with normally-distributed increments. (I find this funny because Taleb spends multiple books railing against normality assumptions.) This is kind of a troubling assumption, as Lumifer points out below. If your forecast is continuous (though it need not be), then it can be thought of as a time-transformed Wiener process, but as far as I can tell he doesn't account for the time-transformation.

Everyone agrees that as uncertainty becomes really high, the best forecast is 50-50. Conversely, if you make a confident forecast (say 90-10) and you're properly calibrated, you're also implying that you're unlikely to change your forecast by very much in the future (with high probability, you won't forecast 1-99).

I think the question to ask is - how much volatility should make you doubt a forecast? If someone's forecast varied daily between 1-99 and 99-1, you might learn to just ignore them, for example. Taleb tries to offer one answer to this, but makes some questionable assumptions along the way and I don't really agree with his result.

It's just masturbation with math notation.

We have the election estimate F a function of a state variable W, a Wiener process WLOG

That doesn't look like a reasonable starting point to me.

Going back to the OP...

the process by which two candidates interact is highly dynamic and strategic with respect to the election date

Sure, but it's very difficult to model.

it’s actually remarkable that elections are so incredibly close to 50-50

No, it's not. In a two-party system each party adjusts until it can capture close to 50% of the votes. There is a feedback loop.

When you read Silver (or your preferred reputable election forecaster, I like Andrew Gelman) post their forecasts prior to the election, do you accept them as equal or better than any estimate you could come up with?

I'm an arrogant git, so I accept them as bit worse :-P To quote an old expression, (historical-) data driven models are like driving while looking tinto a rearview mirror. Things will change. In this particular case, the Brexit vote showed that under right conditions people who do not normally vote (and so are ignored by historical-data models) will come out of the woodwork.

to know the true answer

Eh, the existence of a "true answer" is doubtful. If you have a random variable, is each instantiation of it a "true answer"? You end up with a lot of true answers...

Not for the first time, a draft paper by Nassim Taleb makes me think, "mathematically, the paper looks fine, and congratulations on knowing calculus, but isn't this waffling past the issue people are actually interested in?"

Many people who trusted poll-based forecasts were indeed shocked on election day, but not so much because the months-long sequence of forecasts wobbled back & forth too much. They were shocked more because the final forecasts failed to predict the final outcome. And the problem there was the difficulty of polling representative samples of people who'd actually vote, a very different problem to the one Taleb identifies.

I also have a specific question to tie this back to a rationality based framework: When you read Silver (or your preferred reputable election forecaster, I like Andrew Gelman) post their forecasts prior to the election, do you accept them as equal or better than any estimate you could come up with? Or do you do a mental adjustment or discounting based on some factor you think they've left out?

In the specific case of the US election, I did use Silver's forecasts as a guideline, but I also considered a model based on fundamentals (income growth and dead American soldiers), not polls: Doug Hibbs's Bread & Peace model. Hibbs predicted a 53%-54% split in Clinton's favour of the Dem/Rep vote. (That 53%-54% prediction might sound pretty poor with hindsight, but the expected error attached to the prediction was 2%, and Clinton has 51%, so it's within statistical bounds.)

Whether it's prediction market variations

I didn't really pay attention to prediction markets. For elections & referendums I don't believe prediction markets add anything significant beyond polls & fundamentals.

or adjustments based on perceiving changes in nationalism or politician specific skills

I didn't consciously adjust on that basis. As far as I know, ordinary, retrospective economic voting played a big role in explaining even "the Extraordinary Election of Adolf Hitler", so I figured I wouldn't bother putting my thumb on the scale because of vigorous nationalism or charisma.

Upvoted your post as encouragement to post more stuff like this. I shrugged my shoulders at Taleb's paper, but the topic's fascinating and I'd like to see more stuff like Taleb's paper, even if the paper itself is unimpressive.

This comment brought to you by sarahconstantin's and AnnaSalamon's Your Wannabe Rationalist Community Blog Needs You! posts. Without them I'd likely have carried on my 11-month streak of not posting.

It's relatively easy to come up with models that seems exciting because they have or they lack certain features. But the ultimate test of any predictive model is in the result: has anyone 'paper played' Taleb's model against past elections?