2008 Presidential Primary Polling
FOR IMMEDIATE RELEASE
Investigating the 2008 Presidential Primary Polls
LENEXA, Kansas ------ After an extensive investigation, a committee of the American Association for Public Opinion Research (AAPOR) has identified several factors that contributed to inaccuracies in 2008 Presidential Primary polls, most notably the miscalling of the New Hampshire Democratic primary.
The special AAPOR committee composed of leading academic and private sector experts in public opinion and survey research conducted the analysis. The results show that several methodological factors combined to undermine the accuracy of predictions in New Hampshire, South Carolina, Wisconsin and California. Although the limited data available made it impossible to conduct definitive tests of all likely sources of different poll performance, the following factors were identified as likely reasons for the polling errors in New Hampshire, where many polls mistakenly predicted an Obama victory:
œThis analysis suggests some important explanations for the errors in the 2008 New Hampshire Presidential Primary and raises significant questions for research on pre-election polling methods, said AAPOR President Richard Kulka.
The committee also concluded that several factors were unlikely to have contributed to estimation errors in the New Hampshire pre-primary polls. Among these factors was the so-called œBradley effect in which it is hypothesized that poll respondents say they support a Black candidate in order to appear unbiased, but then cast their ballots for a white candidate in the privacy of the voting booth.
œMany New Hampshire polls predicted Barack Obama would beat Hillary Clinton in that state, said Michael Traugott, committee chair. œSo when Clinton won, some people pointed to latent racism as the reason. But in the data we have from a wide variety of New Hampshire pre-election and exit polls, we found no evidence that white respondents over-represented their support for Obama.
Other factors that the committee discounted:
A number of other possible explanations for the error in the New Hampshire primary polls were considered, but insufficient information was available to evaluate them. These explanations include the order of names on the ballot and the short time frame for polling in New Hampshire following the Iowa caucus.
Traugott noted that the analysis also revealed wide variation in the œtrial heat question, arguably the most important question that respondents are asked. In New Hampshire, there were 11 different question wordings used polls for the Democratic primary, and 10 different wordings used in polls concerning the Republican primary. In some versions, the candidates™ names were not mentioned at all. In others, only the œmajor candidates were named. Some polls randomized the candidates™ names, while others did not.
œWe also learned that some polling firms are buying lists of registered voters with phone numbers, and then they are contacting people with interactive voice response technology --basically computerized calls -- and that they™re taking information from the person who answers the phone, who may or may not be the person identified in the voter list. said Traugott. œAnother approach was to interview any registered voter in the household, he said. œThese practices should be the focus of further research.
For the report, supported in part by a grant from the University of Michigan Institute for Social Research (ISR), Traugott and colleagues analyzed individual, household-level response data provided by seven polling organizations. They also compared information on question wording, weighting, interviewer characteristics, sampling frames, and other methodological factors from up to 19 other firms, in many cases relying on publicly available information gleaned from the Internet.
# # # # #
The American Association for Public Opinion Research (AAPOR) is the leading professional organization of public opinion and survey research professionals in the U.S., with members from academia, media, government, the non-profit sector and private industry. AAPOR members embrace the principle that public opinion research is essential to a healthy democracy, providing information crucial to informed policymaking and giving voice to the nation™s beliefs, attitudes and desires. It promotes a better public understanding of this role, as well as the sound and ethical conduct and use of public opinion research.