AAPOR
The leading association
of public opinion and
survey research professionals
American Association for Public Opinion Research

Judging the Polls Requires Patience on Election Night, and After

AAPOR Task Force to Issue Report on General Election Poll Performance

CHICAGO, IL, October 26, 2020 - The 2020 general election is shaping up as one of the most complex and drawn-out in the nation’s history, with unprecedented numbers of voters casting ballots early. The final results will be in flux for weeks. The American Association for Public Opinion Research (AAPOR), the nation’s largest organization of survey researchers, is calling on journalists and analysts to wait for final vote results before evaluating the performance of the pre-election polls.

“Patience will be the key for dealing with this year’s general election,” said Dan Merkle, AAPOR President. “It will take weeks for election officials to carefully count all early, absentee and provisional ballots. Any judgment on the polls’ performance before those ballots are counted would be premature.”

Rushing to judge how the polls have performed before the votes have been counted has been a problem in past elections. This year, the increasing use of vote-by-mail and early voting adds a new twist to the analytical challenge. Relying on historical patterns of turnout could easily lead to incorrect conclusions. Some states with millions of early votes, such as Texas, have a history of reporting these votes as soon as the polls close. Other states take much longer. These differences arise because states vary significantly in election laws, voting technology and where and when the votes are actually tallied.

“Americans reacting to the coronavirus pandemic are casting absentee ballots and mail ballots in historically high numbers. In some states these will be counted quickly, but not in others. We must wait until all the votes are counted before analyzing the polls’ performance,” said Merkle.

Voters’ views and preferences may well shift as a result of events in the final days before the election. The 2020 campaign season has been a whirlwind of events, with the coronavirus pandemic and economic woes across the country. The timing of a pre-election poll must be considered when judging how well or poorly that poll performed. Because millions of voters are balloting before Nov. 3, judging the impact of events becomes even more difficult.

This year and every year, analyses of poll performance must consider the fundamental uncertainty in all sample surveys, usually stated as the margin of sampling error. Failure to do so could result in a flawed analysis. Pre-election polls in many swing states may show the race within the margin of error as Nov. 3 approaches.

In addition, many other possible sources of error can affect the accuracy of polls, including the type of sample that is used, how likely voters are recruited and screened, how respondents are contacted and interviewed, whether the questions generate unbiased and reliable answers, how the data are weighted (statistically adjusted), and how the weighted data are analyzed.

In January, AAPOR announced the formation of a special task force to examine the performance of public opinion polls during the 2020 presidential election. This task force – including 19 academic experts, pollsters, and statisticians – builds on the work of the 2016 AAPOR task force that showed the national polls performed well in that election.

In 2016, some journalists and analysts rushed to render judgments on the polls’ performance, haste that led to incorrect conclusions. The 2016 AAPOR Task Force Report showed that the national presidential polls in the general election were quite accurate by historical standards, despite frequent mistaken claims to the contrary. State-level polling was not as accurate in the 2016 general election (notably in Pennsylvania, Michigan and Wisconsin), but state polling was reasonably accurate in the 2016 primary elections. 

The 2020 AAPOR task force issued a report on Oct. 12 on the accuracy of pre-election polling for the 2020 primaries.  The report found that the pre-election polls on the Democratic presidential primaries accurately tracked the tumultuous nomination contest in most states. The pre-primary polls also correctly identified the candidate who won each primary contest four times out of five (81%), roughly equal to the accuracy of pre-primary polls in the presidential primary races since 2008.

AAPOR’s 2020 task force is also analyzing pre-election polls conducted for the general election. It will issue a report after a careful analysis and comparison of poll results with the final certified vote totals in each state.