UPDATED May 30, 2018 at 11:00 AM

FiveThirtyEight’s Pollster Ratings

Based on the historical accuracy and methodology of each firm’s polls.

Read moreDownload the dataSee the latest polls

PollsterMethodLive Caller With CellphonesNCPP/​AAPOR/​RoperPolls AnalyzedSimple Average ErrorRaces Called CorrectlyAdvanced +/-Predictive +/-538 GradeBanned by 538Mean-Reverted Bias
Pollster
Name of the polling organization. The organization that contributed the most intellectual property to the methodology and execution of the poll (rather than the organization that paid for or sponsored the poll). FiveThirtyEight may group organizations together if they routinely collaborate on polling or share a common methodology.
Method

The methodology or methodologies a pollster routinely uses in its election polls as of the 2016 and 2018 campaign cycles. The following categories are listed:

  • Live — Live telephone interviews, including cellphones.
  • Live* — Live telephone interviews, but FiveThirtyEight cannot confirm whether cellphones are included.
  • IVR — Interactive voice response, otherwise known as automated polls or “robopolls.”
  • Online — Poll conducted by internet; generally, this means by web browser, but it’s inclusive of text message or application-based polling of mobile phones.
  • Mail — By U.S. mail or other “snail mail” service.
Note that many pollsters are listed as having multiple methodologies, either because they switch back and forth from poll to poll or (more commonly) because they routinely use more than one mode of data collection within the same survey.

Live caller with cellphones

indicates that the polling firm usually or always conducts polls via live interviewers who place calls to cellphones in addition to landlines. Firms that place live-interview calls to cellphones but blend them with another technique (e.g., automated calls to cellphones) are not included in this category. Neither are firms that conduct surveys via text messages to mobile devices. (Only phone calls qualify.) As a default, we assume that a polling firm has not begun to include cellphones in its samples until we have evidence to the contrary. Pollsters that have routinely begun to include cellphones in their samples but are not listed as such should contact FiveThirtyEight. Information is as of May 22, 2018.

NCPP/AAPOR/Roper
indicates the polling firm was a member of the National Council on Public Polls, a signatory to the American Association for Public Opinion Research’s transparency initiative or a contributor to the Roper Center for Public Opinion Research’s data archive as of May 22, 2018. This is a proxy for methodological quality.
Polls analyzed
Number of polls from the firm in FiveThirtyEight’s polls database, which covers polls conducted in the final three weeks of House, Senate, gubernatorial and presidential general election campaigns since 1998 and the last three weeks of presidential primaries and caucuses since 2000.
Simple average error
The firm's average error, calculated as the difference between the polled result and the actual result for the margin separating the top two finishers in the race.
Races called correctly
The percentage of polls in which the polling firm correctly identified the winner of the race. If the poll indicated a tie for the lead and one of the tied candidates won, the pollster is given credit for half a win.
Advanced +/-
A score that compares a pollster’s result against other polling firms surveying the same races and that weights recent results more heavily. Negative scores are favorable and indicate above-average quality.
Predictive +/-
A projection of how accurate the pollster will be in future elections. It is calculated by reverting a pollster's Advanced Plus-Minus score to a mean based on our proxies for methodological quality. Pollsters with low methodological quality also receive a penalty for “herding.” This rating forms the basis for the weights that FiveThirtyEight assigns to polls in its election models. Negative scores are favorable and indicate above-average quality.
538 grade
A letter grade from A+ to F that reflects a pollster's Predictive Plus-Minus score. Firms banned by FiveThirtyEight are automatically given a grade of F.
Banned by 538
indicates that the polling firm is not used in FiveThirtyEight's election forecasting models because we know or strongly suspect that it has faked polling results.
Mean-reverted bias
A pollster's historical average statistical bias toward Democratic or Republican candidates, reverted to a mean of zero based on the number of polls in the database. A score of “R +1.5,” for example, indicates that the pollster’s polls have historically overrated the performance of Republican candidates.

Comments