The reporting of a margin of sampling error associated with an opt-in or self-identified sample (that is, in a survey or poll where respondents are self-selecting) is misleading.

When we draw a sample at random—that is, when every member of the target population
has a known probability of being selected—we can use the sample to make projective,
quantitative estimates about the population. A sample selected at random has known
mathematical properties that allow for the computation of sampling error.

Surveys based on self-selected volunteers do not have that sort of known relationship to the target population and are subject to unknown, non-measurable biases. Even if opt-in surveys
are based on probability samples drawn from very large pools of volunteers, their results
still suffer from unknown biases stemming from the fact that the pool has no knowable
relationships with the full target population.

AAPOR considers it harmful to include statements about the theoretical calculation of
sampling error in descriptions of such studies, especially when those statements mislead
the reader into thinking that the survey is based on a probability sample of the full target
population. The harm comes from the inferences that the margin of sampling error
estimates can be interpreted like those of probability sample surveys.

All sample surveys and polls are subject to multiple sources of error. These include, but
are not limited to, sampling error, coverage error, nonresponse error, measurement error,
and post-survey processing error. AAPOR suggests that descriptions of published
surveys and polls include notation of all possible sources of error.

For opt-in surveys and polls, therefore, responsible researchers and authors of research reports are obligated to disclose that respondents were not randomly selected from among
the total population, but rather from among those who took the initiative or agreed to
volunteer to be a respondent.

AAPOR recommends the following wording for use in online and other surveys conducted among self-selected individuals:  Respondents for this survey were selected from among those who have [volunteered to participate/registered to participate in (company name) online surveys and polls]. The data (have been/have not been) weighted to reflect the demographic composition of (target population). Because the sample is based on those who initially self-selected for participation [in the panel] rather than a probability sample, no estimates of sampling error can be calculated. All sample surveys and polls may be subject to multiple sources of error, including, but not limited to sampling error, coverage error, and measurement error.