UPDATED Nov. 5, 2019 at 10:00 AM

FiveThirtyEight’s Pollster Ratings

Based on the historical accuracy and methodology of each firm’s polls.

Read moreDownload the dataSee the latest polls

PollsterMethodLive Caller With CellphonesNCPP/​AAPOR/​RoperPolls AnalyzedSimple Average ErrorRaces Called CorrectlyAdvanced +/-Predictive +/-538 GradeBanned by 538Mean-Reverted Bias
Pollster
Name of the polling organization. The organization that contributed the most intellectual property to the methodology and execution of the poll (rather than the organization that paid for or sponsored the poll). FiveThirtyEight may group organizations together if they routinely collaborate on polling or share a common methodology.
Method

The methodology or methodologies a pollster routinely uses in its election polls as of the 2016 and 2018 campaign cycles. The following categories are listed:

  • Live — Live telephone interviews, including cellphones.
  • Landline — Live telephone interviews, not including cellphones.
  • Live* — Live telephone interviews, but FiveThirtyEight cannot confirm whether cellphones are included.
  • IVR — Interactive voice response, otherwise known as automated polls or “robopolls.”
  • Online — Poll conducted by internet; generally, this means by web browser, but it’s inclusive of text message or application-based polling of mobile phones.
  • Mail — By U.S. mail or other “snail mail” service.
Note that many pollsters are listed as having multiple methodologies, either because they switch back and forth from poll to poll or (more commonly) because they routinely use more than one mode of data collection within the same survey.

Live caller with cellphones

indicates that the polling firm usually or always conducts polls via live interviewers who place calls to cellphones in addition to landlines. Firms that place live-interview calls to cellphones but blend them with another technique (e.g., automated calls to cellphones) are not included in this category. indicates that a firm conducts its polls at least in part via live interviewers who place calls to cellphones in addition to landlines, but may blend them with another technique or conduct other types of polls that are not live-caller polls. As a default, we assume that a polling firm has not begun to include cellphones in its samples until we have evidence to the contrary. Firms that conduct surveys via text messages to mobile devices are not included. (Only phone calls qualify.) Pollsters that have routinely begun to include cellphones in their samples but are not listed as such should contact FiveThirtyEight. Information is as of Nov. 5, 2019.

NCPP/AAPOR/Roper
indicates the polling firm was a member of the National Council on Public Polls, a signatory to the American Association for Public Opinion Research’s transparency initiative or a contributor to the Roper Center for Public Opinion Research’s data archive as of Nov. 5, 2019. This is a proxy for methodological quality.
Polls analyzed
Number of polls from the firm in FiveThirtyEight’s pollster ratings database, which covers polls conducted in the final three weeks of House, Senate, gubernatorial and presidential general election campaigns since 1998 and the last three weeks of presidential primaries and caucuses since 2000.
Simple average error
The firm's average error, calculated as the difference between the polled result and the actual result for the margin separating the top two finishers in the race.
Races called correctly
The percentage of polls in which the polling firm correctly identified the winner of the race. If the poll indicated a tie for the lead and one of the tied candidates won, the pollster is given credit for half a win.
Advanced +/-
A score that compares a pollster’s result against other polling firms surveying the same races and that weights recent results more heavily. Negative scores are favorable and indicate above-average quality.
Predictive +/-
A projection of how accurate the pollster will be in future elections. It is calculated by reverting a pollster's Advanced Plus-Minus score to a mean based on our proxies for methodological quality. Pollsters with low methodological quality also receive a penalty for “herding.” This rating forms the basis for the weights that FiveThirtyEight assigns to polls in its election models. Negative scores are favorable and indicate above-average quality.
538 grade
A letter grade from A+ to F that reflects a pollster's Predictive Plus-Minus score. Pollsters with a relatively small sample of polling now receive a provisional rating rather than a precise letter grade. (An “A/B” provisional rating means that the pollster has shown strong initial results, a “B/C” rating means it has average initial results, and a “C/D” rating means below-average initial results.) It now takes roughly 20 recent polls (or a larger number of older polls) for a pollster to get a precise pollster rating. Firms banned by FiveThirtyEight are automatically given a grade of F.
Banned by 538
indicates that the polling firm is not used in FiveThirtyEight's election forecasting models because we know or strongly suspect that it has faked polling results.
Mean-reverted bias
A pollster's historical average statistical bias toward Democratic or Republican candidates, reverted to a mean of zero based on the number of polls in the database. A score of “R +1.5,” for example, indicates that the pollster’s polls have historically overrated the performance of Republican candidates.

Comments