Understanding a “credibility interval” and how it differs from the “margin of sampling error” in a public opinion poll

DEERFIELD, Ill. — October 8, 2012 — The most commonly understood measure of the accuracy of a public opinion poll is the margin of sampling error, commonly referred to as the "plus-or-minus."  A new term is making its way into the polling lexicon -- namely the credibility interval.  AAPOR would like to clarify for journalists and consumers of polls that the margin of sampling error and a credibility interval are not the same thing, even though both assign a “plus-or-minus” to the accuracy of a poll.

The statistical basis for a poll to have a margin of sampling error is that it must be based on a probability sample, where everyone in the population to be surveyed has a chance of being selected and the respondents are selected randomly.  Surveys based on self-selected volunteers such as opt-in online polls do not have such a ‘grounded statistical tie’ to the population.  As a result, estimates from self-selected volunteers are subject to unknown error that cannot be measured.

A credibility interval can be developed to measure the theoretical accuracy of nonprobability surveys.  The credibility interval relies on assumptions that may be difficult to validate, and the results may be sensitive to these assumptions.  So while the adoption of the credibility interval may be appropriate for nonprobability samples such as opt-in online polls, the underlying biases associated with such polls remain a concern.

Consequently, AAPOR urges caution when using credibility intervals or otherwise interpreting results from electoral polls using non-probability online panels. The Association continues to recommend the use of probability based polling to measure the opinions of the general public.

A detailed explanation of a credibility interval and how it differs from the margin of sampling error is available by clicking here.