How Good Are FiveThirtyEight Forecasts?

All probabilities were published by FiveThirtyEight before the corresponding events occurred. We have removed all 100 percent and 0 percent forecasts for events that were guaranteed or impossible from this analysis; for example, any forecasts made after a team was eliminated from a postseason race. When projects are aggregated across multiple years and multiple model types, only FiveThirtyEight's default model type from each year is evaluated. The unskilled estimates for sports games incorporate home-field advantage by using each sport's historical home team winning percentage in its forecasts, rather than assuming that each team has an equal chance of winning. The “biggest surprises” for sports forecasts exclude the chance of a team finishing the regular season in a specific position or with a specific playoff seed.

When calculating the calibration and skill scores for forecasts that we updated over time, such as election forecasts that we updated every day, we weighted each update by the inverse of the number of updates issued. That way, we counted each forecasted event equally, regardless of how many updates we issued to the forecast. We applied the same weights when calculating the confidence intervals.

Download this data.