AAPOR
The leading association
of public opinion and
survey research professionals
American Association for Public Opinion Research

Coverage and Sampling

Previous: Background

Table of Contents 

Next: Nonresponse in
RDD Cell Phone Surveys

Declining Coverage of U.S. Landline Telephone Frames

The prevalence and use of cell phones in the U.S., also often referred to as "wireless" telephones, has been steadily increasing since 2000. As of 2009, more than 80 percent of adults had at least one wireless phone and a growing number of adults and households are replacing their landline telephone service with cell phone service. According to the National Center for Health Statistics (NCHS) the number of "wireless-only households" has grown from less than 2 percent in 2001 to 24.5 percent of all households by the end of 2009, a percentage that now exceeds that of "landline-only households," which is now only 14.9 percent of households (Blumberg and Luke, 2010). These trends will certainly continue as time passes.

This means that a national RDD sample of landline telephone numbers generated from a frame that covers all U.S. landline households will represent less than 80 percent of telephone households and less than 75 percent of all households. 

Moreover, modeled estimates for 2007 produced by NCHS and State Health Access Data Assistance Center (SHADAC) suggest that the prevalence of wireless-only (also called cell-only) households varied significantly by state; e.g., from a low of approximately 5 percent in Vermont in 2007 to a high of approximately 26 percent in Oklahoma in 2007 (Blumberg et al., 2009).1 Research by Arbitron Inc. suggests that the prevalence of wireless-only also varies by market level within each state. Specifically, market level penetrations of wireless-only households vary by region and demographics such as age, race, and ethnicity, with higher penetrations in markets with college campuses and/or military bases and in high density Hispanic areas (Fleeman et al., 2010). For example, Fleeman and her colleagues cite the state of Illinois where the penetration of cell-only is 20 percent, whereas the various metropolitan area rates within Illinois range from 13 percent to 30 percent. As another example, the New York City metro area shows "significant differences among Long Island, the Boroughs and the New Jersey suburbs."

Equally important to researchers are the demographic attributes of these cell phone only households. As of 2009, the U.S. cell phone only population is more likely to be younger: 38 percent of 18- to 24-year-olds, 49 percent of 25- to 29-year-olds, and 37 percent of 30- to 35-year-olds live in wireless-only households (Blumberg and Luke, 2010). The cell phone only population also includes more renters, a higher proportion of non-whites (e.g., 30 percent of Hispanics), and has a lower income as compared to the entire U.S. landline population. Although cell phone only adults have tended to be unmarried cohabitants, the number of children living in cell phone only households is now growing. At the end of 2009, 40 percent of cell phone only adults were living with children (Blumberg and Luke, 2010). In addition to demographic differences, recent research by the Pew Research Center found differences between the cell-only and landline respondents in terms of political attitudes and behavior, media use, internet use and activity, social views and lifestyle behaviors (Christian et al., 2010). Similarly the NCHS (Blumberg and Luke, 2010) continues to find differences in health and health related behaviors.

Given the growing geographic and demographic biases that might result from excluding the cell phone only population, researchers may want to consider the option of a dual frame telephone sample design where a landline frame is augmented with that of a cell phone frame.

Using dual frame (landline plus wireless) telephone samples is not the only approach to compensating for the increasing number of wireless only households absent from the landline RDD frame. For example, Address Based Sampling (ABS) can provide an alternative single frame approach as well. (See Appendix A for an overview of Address Based Sampling.)

Another intriguing alternative that is being explored by some researchers is the possibility of substituting directory-listed landline phone numbers (also called Electronic White Pages or EWP sample) for some of the entire landline RDD sample. Although it is well known that EWP samples underrepresent certain groups, these are the same groups that are fairly easily reached via cell phone, so they are covered to some extent in a dual frame design that combines cell phone and EWP samples. A dual frame study that uses only EWP sample and cell phone RDD sample (without screening) would cover all telephone households except those that have an unlisted landline and no cell phone available. An analysis of NHIS data by Guterbock and colleagues (2009) suggests that this segment is quite small and is shrinking, and it is not highly different from the rest of the telephone universe, so the coverage error from its exclusion is likely to be very small for most survey results (Guterbock et al., 2010). 2

 

U.S. RDD Cell Phone Frames and Types of Telephone Service
Frames for generating random digit dial (RDD) samples for conducting surveys of cell phones in the United States are available from most sample suppliers. These frames are lists of all possible wireless telephone numbers and are generally built using industry databases that identify the types of service provided by individual prefixes and 1000-blocks.3 

Three important features of the U.S. RDD cell phone frames are:

  • The available data in the frames are administrative and are subject to errors (e.g., a particular number may not be for a cell phone or a carrier has incorrectly classified or not updated some of their telephone information).
  • There are no indicators in the frame, or in any other reliable source, that can accurately identify whether the number is currently working; where the subscriber of the number currently resides; or if the number is subscribed by a person who lives in a household with a landline telephone.
  • There are no commercially available sources with subscriber information such as name, address, or any demographics that can be linked reliably to cell phone numbers on the frame.


Because the wireless frame is administrative in nature, it only provides information about prefixes and 1000-blocks, such as service provider, rate center location, some rate information and the type of service provided. For example, below is a list of the types of service (NXX Type) that might contain cell phone numbers. (Note that the various NXX Types are explained in more detail in the glossary.)

04: Dedicated to Cellular
50: Shared Between Three or More Types of Service -- (Plain Old Telephone Service (POTS), Cellular, Paging, Mobile or Miscellaneous)
40: Shared Between POTS and Cellular
55: Special Billing Option Cellular
58: Special Billing Option Shared Between two or More -- (Cellular, Paging, Mobile)
60: Service Provider Requests SELECTIVE Local Exchange -- (IntraLATA Special Billing)
65: Personal Communications Services (PCS) -- Also Wireless/Cell
66: Shared Between POTS and Personal Communications Services
67: Special Billing Option -- PCS / Personal Communications Services
68: Service Provider Requests SELECTIVE Local Exchange -- (IntraLATA Special Billing -- PCS)

However, not all telephone numbers generated in these NXX types will be for cell phones. According to the U.S. Federal Communications Commission (FCC), as of September 30, 2009, approximately 181,000 wireless numbers had been ported (i.e. converted) to landline service (FCC, 2010). Although these numbers will be found in a cell phone frame, they will connect with a landline telephone. Shared service prefixes and 1000-blocks can contain numbers associated with both wireless and landline services.

Considerations When Purchasing a U.S. RDD Cell Phone Sample

The Sample Provider's Frame. When purchasing a sample of U.S. RDD cell phone numbers, researchers are encouraged to inquire about how the sample provider's frame has been constructed and approximately what percentage of cell phone numbers in the geographical area to be surveyed are excluded from the provider's frame. Individual vendors may construct their frames differently.

The following is a list of issues researchers are encouraged to consider when purchasing RDD cell phone samples:

  • Is the frame based on prefixes, 1000-blocks or 100-blocks?
  • How often is the frame updated?4
  • What types of wireless services are included: Dedicated?  Shared?  Cell?  PCS?  Special Billing?5
  • What is the extent of noncoverage and overlap between the provider's landline frames and cell frames?  What prefixes, 1000-blocks, or 100-blocks are excluded, and why?  What prefixes, 1000-blocks, or 100-blocks are duplicated, and why?
  • How are shared service numbers handled? Shared service prefixes and shared service 1000-blocks are those in which different types of service may be mixed at a lower level, such as within 100-blocks. This means that wireless numbers can exist together with landline numbers within a single prefix, 1000-block or 100-block and depending on the sample supplier's frame construction the same number might exist on both the landline frame and the cell frame (resulting in overlap) or neither frame (resulting in non-coverage).
  • What levels of geography are available for sample selection and how have they been determined?  County-level assignments are generally based on rate center location information for prefixes or 1000-blocks, as provided by the service providers on administrative databases. Therefore, most county-based geographies such as Census Region, Census Division, State, MSA, and DMA will be available, but sub-county geographies such as ZIP code will not be available.


Ported Numbers. Researchers also need to consider how they want to handle ported numbers. Number portability is the ability of users of U.S. telecommunications services to keep their existing telephone number when changing from one local service provider to another within their local exchange or rate center. The FCC has reported that as of September 30, 2009, approximately 2.5 million landline numbers in the U.S. have been ported to wireless service (FCC, 2010), which represents approximately 1 percent of all U.S. working cell phone numbers and 0.5 percent of working landline numbers (including business numbers). Landline numbers ported to wireless service will not be included in a cell phone frame. Sample suppliers normally remove landline numbers ported to wireless service from landline telephone samples since to dial such numbers, or any cell phone number, using automated telephone equipment violates FCC regulations stemming from U.S. Telephone Consumer Protection Act (FCC, 2003). This creates a gap in coverage, which may add coverage bias, since these ported numbers are not in the landline or the cell phone frame.

Sample suppliers use the Neustar Intermodal Ported TN Identification Service to identify and/or remove these ported numbers. Some researchers have obtained their own Neustar Intermodal Ported TN Identification Service license.6 This allows them to receive and identify numbers ported to wireless service in their landline samples so that these numbers can be hand-dialed to avoid violating Federal Trade Commission (FTC) regulations. Having a Neustar license also allows firms to perform their own treatment of ported numbers in a timely (daily) manner. Once a licensee has identified ported numbers, those numbers may be removed from the sample or dialed using hand-dialing. (See section on Legal and Ethical Issues for further details.)

It is important to note that Neustar's Intermodal Ported TN Identification Service licenses limit the use of their data to TCPA compliance activities. In other words, the database may only be used by a licensee in their efforts to comply with TCPA regulations prohibiting calls to cell phones using automated telephone equipment and may not, for example, be used to construct or enhance a cell phone frame. 

 

Evaluating the Adequacy of the Coverage Provided by U.S. RDD Cell Phone Samples

Currently, available frames of cell phone numbers in the U.S. can provide excellent national coverage of the U.S. cell phone population. Nevertheless, many coverage issues need to be considered when designing a cell phone sample, particularly one that is not national in scope. Because of these coverage issues, it is even more important for cell phone general population surveys than for landline surveys to determine the geopolitical residential location of each respondent during the interview for establishing eligibility and for weighting purposes. 

Recent research (Christian, 2009) suggests that in an RDD national sample of cell and landline telephone numbers, 40 percent of cell phone subscribers (and 43 percent of cell phone only subscribers) do not live in the county associated with their rate center, as compared to only 8 percent of landline telephone subscribers. At the state level, such differences are less pronounced but still meaningful (i.e., of non-negligible size). Specifically, 10 percent of cell phone subscribers (and 12percent of cell phone only subscribers) do not live in the state associated with their rate center as compared to only 3 percent of landline subscribers. 

There are two main reasons for the difference:

  • Most wireless service areas or exchange boundaries are significantly larger than their landline equivalents, and frequently cross county borders. Also, different cell phone providers in the same rate center may have different coverage areas for their service. This means that the geography covered by cell phone Provider A in a given rate center may be smaller or larger than the geography covered by cell phone Provider B in the same rate center.
  • The exchange (prefix) associated with a cell phone number represents the original point-of-purchase where the subscriber lived or worked when service was originally acquired. This may not represent where the subscriber currently lives or works. In recent years it has become more common for wireless carriers to allow subscribers to select a prefix that is not as closely tied to their residence as in the past, although it is usually within their metropolitan area.


As a result, defining the sample geography below the level of the entire nation, such as a single county or group of counties, can result in an unknown coverage error due to some or all of the following:

  • It is not uncommon to live in a different county than the county in which the cell phone exchange rate center is located.
  • There may be no rate centers located within one or more of the counties to be sampled.
  • An unknown number of subscribers may live in a neighboring county that is not included in the sample geography.
  • An unknown number of subscribers may live in a county to be sampled, but have cell phone telephone numbers in a rate center located in a county that is not being sampled.
  • A number of subscribers may have a cell phone number provided by an employer that is associated with the location of that business and not with the location of the subscriber's s residence.
  • There are no addresses associated with cell phone numbers that can be used to accurately define exchange coverage areas.
  • Subscribers can move to a different city or state and keep their cell phone telephone number. This cell phone transportability can lead to frame undercoverage due to in-migration (subscribers who have moved into a sampled area but with a cell phone number not associated with that area) and to frame overcoverage due to out-migration (subscribers who have a cell phone number associated with the sampled area but who currently live outside that area). Although overcoverage can be addressed by including proper screening questions in the survey instrument, more comprehensive solutions must be devised for cell phone surveying that effectively solve these coverage problems.


Integrating RDD Samples by Combining Samples from Cell and Landline Frames

Integration of cell phone and landline samples may be accomplished in several ways, and researchers should fully disclose the methods used. 

To produce representative samples, telephone surveys of the general population should collect sufficient information from respondents and their sample provider to be able to construct appropriate weights that reflect the probability of selection of each household and/or respondent. (Discussion of these issues appears in more detail in the Weighting section of this report. Examples of questionnaire items for these purposes appear in Appendix B.)

Cell phone frames and landline frames are overlapping frames in that individuals and households with both landline phones and cell phones will be represented on both frames. This duplication results in multiple probabilities of selection for the affected individuals/households. Since the NHIS (Blumberg and Luke, 2010) estimates that 80 percent of U.S. landline households and 83 percent of U.S. landline adults have one or more cell phones, this significant overlap must be considered at every step of the survey design process

Two different sampling approaches have been used to handle situations in which a household in the U.S. can be reached by both a landline and a cell phone. However, there is as yet no consensus on whether one of the approaches is always preferable.This is further complicated because of the difference between whether someone can in theory be reached via a landline or cell phone (i.e., do they have one or both services?), which is a coverage issue, versus whether they can be reached in practice via either or both types of telephone services (i.e., how often, if at all, will they answer a call on a particular service they have?), which is a noncontact-related nonresponse issue.7

Research by Pew (2009) and Brick (2009) suggests that individuals and households with dual service often have different response propensities depending on whether they are contacted on their cell or landline phones.

To the extent that those with dual service who are more likely to agree to be surveyed when contacted on their cell phone than their landline differ in non-negligible ways on the variables of interest from those with dual service who are more likely to agree when contacted on their landline than their cell phone, error in the form of nonresponse bias will result. In general, people with dual service are thought to be more likely to cooperate with a telephone survey when sampled on the service they use most often. 

The Pew research also suggests that there are demographic differences between dual users and the other groups. These demographic and response propensity differences may lead to differential nonresponse, which if not accounted for can lead to biased estimates. (See further discussion in the sections on Nonresponse and on Weighting.)

The NHIS releases national estimates for the "wireless-mostly" population. Wireless-mostly (also called "cell phone mostly") respondents are defined by NHIS as respondents who receive "all or almost all calls" on their cell phone. Landline-mostly (sometimes called "wired-mostly"), as defined by NCHS, are those that receive "all or almost all" of their calls on their landline phone.8 The two "mostly" groups are individuals or households that are covered by both frames but may potentially be unreachable in one or the other of those frames. During the period of July through December 2009, NHIS estimates that 26 percent of households with both landline and wireless service received "all or almost all"of their calls on their cell phone (Blumberg and Luke, 2010). These wireless-mostly adults, when compared to wireless-only adults, are more likely to be White, over 45 years of age, have a college degree or higher, have a higher income, own their residence and be related adults with no children or adults with children. Since wireless-mostly households also have landlines, if they are excluded from a screened sample because they have a landline phone it can create a possible nonresponse bias. 

Thus, new research needs to be conducted to better understand the realities of response propensities associated with the "wireless-mostly" cohorts in dual frame designs and in estimating the potential response bias of landline frames. This new research includes determining the most reliable wording to use in survey questions to measure these constructs, which includes determining whether there is a non-ignorable difference between what people say they do and what they actually do vis-a-vis their phone service usage. The ultimate question is the extent to which "wireless-mostly" individuals are unreachable on their landline telephone. 

Research by Boyle, Lewis, and Tefft (2009) investigated some of these issues. This research looked at response propensities from a different perspective. The authors suggest that a more accurate gauge of the likelihood of reaching a dual service individual or household on their landline could be operationalized by asking: "Thinking just about the landline home phone, not your cell phone, if that phone rang, and someone was home, under normal circumstances how likely would it be answered?  Would you say it is: extremely likely, very likely, somewhat likely, somewhat unlikely, or not at all likely?" They suggest that this is a more accurate estimate of the percent of the dual user population that is actually unreachable in a landline RDD sample. However, as noted above, there may be a discrepancy between reported intended behavior and actual behavior as well as the role Caller ID, showing the name of the incoming caller (e.g., ZYX  Research), might play in any estimate.

A national dual frame survey conducted by Abt SRBI for the 2009 Traffic Safety Culture Index Survey used both sets of questions in an attempt to differentiate between "wireless-mostly" and the proportion of dual users actually not reachable on their landline. Their findings were that 16 percent of the adults were landline only, 19 percent were cell phone only, and another 16 percent would be classified as "wireless-mostly" based on the NCHS questions related to usage. The results also indicated that only 4 percent of adults reported that it was "somewhat unlikely" or "not at all likely" that their landline phone would be answered. However, the study found that three out of five (61 percent) of these potentially unreachable adults actually completed the survey on their landline phone.

Keeter, Dimock, and Christian (2008) took a similar tack by asking, "If I had called you just now on your landline phone, would I have been able to reach you?" They found that 45 percent of the cell phone respondents reported that they could not have been reached on their landline. Among those receiving the majority of their calls on a cell phone, this figure was 52 percent. So roughly speaking, they estimated that about half of the wireless-mostly will not be reached through the landline frame.

The question of whether heavy cell users will also respond to a survey on their landline phone was studied directly by Kennedy and Everett (2009). They conducted a repeated-measures dual frame RDD experiment in which dual users who responded to an initial survey were later randomly assigned to be interviewed for a subsequent, ostensibly independent, survey on either their landline or cell phone. Dual users originally reached through the landline frame were significantly less likely to respond to the subsequent survey if they were called on their cell phone (AAPOR RR1=39 percent) versus being called on their landline (AAPOR RR1=59 percent). By comparison, dual users originally reached through the cell frame were equally accessible for the subsequent survey on their landline (AAPOR RR1=55 percent) and on their cell (AAPOR RR1=54 percent). These results suggest that the accessibility of the "wireless-mostly" via the landline frame is greater than the accessibility of the "landline-mostly" via the cell frame.

Although there are no national data on the "wireless-mainly" (Boyle et al., 2009) population, a number of researchers has used the estimates for the "wireless-mostly" population from the NHIS to post-stratify the overlap group of dual users into two sub-groups: "Wireless-mostly" and all others (Pew, 2009). This adjustment requires that ownership and usage information be gathered from all respondents during the interview process. Based on NHIS data for the second half of 2009 for adults (adjusted to remove adults with no phone and those with unknown cell phone status) the ownership and usage adjustment parameter would be:

  • 24.5 percent Cell Only
  • 58.2 percent Dual (Cell and Landline)
    • 25.7 percent All/Almost All on Cell
    • 74.3 percent Some/None on Cell
  • 14.9 percent Landline Only


Further discussion of these issues is presented in the sections of this report on Nonresponse, Measurement and Weighting.

Screened Approach. This sampling approach involves conducting the interview only with respondents sampled via the cell phone frame who do not have a landline. This requires the exclusion of numbers from the cell phone sample that are in the overlap by screening out those persons with both a cell phone and a landline. With this approach, persons who have at least one household landline telephone and use at least one cell phone would be eligible for inclusion only when sampled from the landline frame. Only those interviewed via cell phone and without a residential landline would be counted and eligible for inclusion as cell phone only persons/households.

Although this approach removes the overlap of the dual frame and thereby makes weighting simpler, it is much more expensive to field because of the added costs associated with hand dialing the cell phone sample and the relatively low incidence associated with wireless-only status; (see further discussion in the section on Costs).

Overlap Approach. This sampling approach involves conducting the interview regardless of the frame from which the household or individual was sampled. That is, no households are excluded (screened out) based on their type(s) of telephone service. For respondents who can be interviewed by either landline or cell phone, an adjustment should be applied to compensate for the additional chance of selection from more than one telephone number; (see further discussion in the section on Weighting). Consequently, it will be necessary to obtain telephone status for all respondents from both frames during the interview process; (see further discussion in the section on Measurement). 

It also is important to consider that research by Brick (2009) has found that cell phone surveying using the overlap approach will tend to overrepresent wireless-only (and wireless mostly/mainly) respondents in the cell phone sample. This follows from the hypothesis that wireless-only individuals are more likely to answer their cell phone and to agree to be interviewed than are wireless subscribers who also have a landline. (This bias may be reduced by adjusting for telephone ownership and/or usage.)

Both the Screened and Overlap approaches have been applied in practice. Given that the NHIS collects data about the wireless-only population along with selected demographics, the resulting estimates may be used to develop post-stratification adjustments for data obtained from a single landline frame or from a dual frame under the Screened approach. Cell-only estimates for households and adults are also available for individual states. Because the cell-only population varies by state, NCHS worked with researchers from the SHADAC to develop model-based state level estimates. However these estimates should be used with considerable caution given their (a) modeled status, (b) recency (2007 estimates) and (c) resulting large margins of error.

Sample Allocation in a Dual Frame Telephone Survey. The decision on how best to allocate sample between the cell phone and landline frames should take into account cost parameters, such as the higher cost associated with calling cell phones compared to landline phones (see further discussion in section on Costs), an estimate of eligible respondents in the respective frames, and differential response dispositions and rates for each group.

Currently, researchers in the U.S. conducting general population dual frame RDD surveys use different criteria for determining sample allocation based on these considerations and their specific research objectives. However, most researchers have attempted to obtain approximately 20 percent to 30 percent of completed interviews from the wireless frame, and thus have calculated their initially designated sample sizes accordingly. Census region, census division, or state level estimates might be used to determine the number of completes for a given state, set of states or a county or counties within a state (cf. Fleeman et al., 2010). Regardless of the sampling approach used, it is recommended that researchers should obtain enough cell phone completions, including cell-only cases, to avoid large weights for these groups.9

Eligibility. Sample dispositions from 12 dual frame telephone surveys conducted by Pew between June and October 2008 suggest that contact, cooperation, and completion rates were relatively similar across the cell phone and landline frames, although these rates tended to be lower in the cell phone part of the surveys. There also were differences in the Pew surveys by individual dispositions for cell phone surveys as compared to the landline surveys, such as fewer non-residential numbers, fewer no-answer/busy outcomes, but more voice mail outcomes.

Despite the similarities in response rates in the Pew studies, they consistently found that it takes approximately 60 percent more working numbers to gain a completed interview in a cell phone sample (average of 9.5 numbers per completion) than in a landline sample (average of 6.0 numbers per completion), primarily due to so many more cell numbers being ineligible for most general population surveys, compared the rate of ineligibility within a landline sample for the same geography. This difference will add to the overall cost of conducting cell phone surveys and in determining an appropriate initially designated sample allocation of cell phone numbers. 

The major contributors to the difference in ineligibility rates are as follows:

  • Respondent age -- In the 12 Pew surveys referenced above, an average of 33 percent of the cell phone respondents who were screened reported that they were less than 18 years of age. The Pew Internet & American Life Project reported that in 2008, 71 percent of teenagers had a cell phone compared to only 45 percent in 2004 and 63 percent in 2006. Given this trend, it is expected that age ineligibility will continue to be a significant factor when conducting surveys on cell phones.
  • Within sample geography -- As mentioned previously, for cell phone surveying that is less than national in scope, it will be necessary to identify and screen out ineligible respondents based on their geographic area of residence relative to the geography associated with their telephone number on the frame. This incidence (geographic eligibility rate) can vary from 94 percent for census region of residence, to 90 percent for state of residence, to only 60 percent for county of residence (Christian and Dimock, 2009). These eligibility/incidence rates may be even lower for otherwise eligible wireless-only adults, but who tend to be younger and more likely to move or have a cell phone number associated with the residence of their parents.
  • Telephone ownership -- When screening for wireless-only adults, experience continues to suggest that due to their greater proclivity to cooperate with a telephone interview on their cell phone, their incidence will be higher than would normally be expected based on their overall prevalence within the cell phone population. Although NCHS estimates that in the second half of 2009 approximately 23 percent of the U.S. adult population was wireless-only, 34 percent of cooperating cell phone respondents in the Pew surveys reported being wireless-only. In the 2007-2008 CHIS survey of Californian adults, 50 percent of cell phone respondents reported being wireless-only vs. an NHIS estimate of only 18 percent for the West Census Region (cf. Brick, 2009). Additionally, there is the possibility that an individual's personal telephone ownership status might be different depending on which adult in the household is responding for the household. Adults, for example, living with parents who have a landline phone might consider themselves as wireless-only if they do not answer the parents' landline phone. However, they technically are not wireless-only since they would be both eligible and potentially reachable in an RDD landline survey that selects a respondent from among all the adults living in the household. Issues associated with cell phone sharing discussed below may also need to be considered when determining telephone status.
  • Language spoken -- When conducting surveys only in English, a proportion of contacts may be deemed out of scope (ineligible) due to a language barrier. In 11 of the national Pew surveys, an average of 6 percent of their cell phone contacts did not speak English versus 4 percent for the landline sample. In the one survey that was conducted in both English and Spanish (the 2009 Religion & Public Life Survey) only 0.5 percent of cell phone contacts and 2.3 percent of cooperating cell phone numbers were dispositioned as Language Barrier, about the same proportions as for the landline sample. This difference is primarily a function of the disproportionate number of minorities among the U.S. wireless-only and wireless-mostly populations--a difference that is also a factor of survey geography. The NCHS report for the second half of 2009, estimated that 30.4 percent of Hispanic adults and 20.6 percent of Asian adults are wireless-only and that 16.9 percent of Hispanic adults and 18.5 percent of Asian adults are wireless-mostly. In the 2007 California Health Interview Survey (CHIS), which was conducted only in English in California (which has a high incidence of Hispanics), 8 percent of the respondents contacted by cell phone did not speak English. Consequently, language barrier rates can be reduced if a cell phone survey (as well as a landline survey) is conducted in Spanish as well as English.


See sections of this report on Nonresponse, Measurement and Weighting for further discussion of these issues.

 

Within-Household Coverage Issues in U.S. RDD Cell Phone Samples
Cell phone samples also create sampling issues related to within-household coverage that have not been adequately addressed at the time of this report. (This is a topic area that warrants future research.) 

Cell Phone Sharing. The relationship between the number of adults in a household and the number of cell phones in that same household is not always one-to-one. An individual may have more than one cell phone or may share a cell phone or cell phones with other adults or children. Currently, there is little reliable literature regarding the sharing of cell phones. National Health Interview Survey (NHIS) data suggest that one in seven wireless-only households had fewer cell phones than adults (Blumberg, 2009), which implies that at least some sharing is occurring.

In a 2007 cell phone survey conducted in three states for the Behavioral Risk Factor Surveillance System (BRFSS), data were gathered about cell phone sharing. Sharing was defined as "sharing a cell phone one-third of the time or more with another adult in the household." This survey found 11 percent, 15 percent, and 17 percent of respondents in the three respective states shared their phone. The authors also found that the rate of sharing was higher for adults living in wireless-only households. In the landline survey, 11 percent of respondents in households with at least one cell phone reported sharing cell phones (Link, Battaglia, Frankel, Osborne and Mokdad, 2007b). A number of researchers believe that sharing may occur more often between parents and children or between siblings within a household.

In contrast, using Gallup national surveys, Buskirk, Rao and Kaminski (2008) reported that upwards of one third of recent landline "cord cutters" (people who had become cell phone only) shared a cell phone with someone else in their household.

A recent study of mobile phone sharing in Germany (Fuchs and Busse, 2010) found that 36 percent of adults shared a mobile phone: 9 percent were "active sharers only" -- respondents who answer calls on another person's phone; 8 percent were "passive sharers only" -- persons who answer calls on the respondent's phone; and 19 percent were both "active" and "passive" sharers. "Active" sharers were more likely to be young, unemployed, single and living in multi-person high income households. People that allowed "passive" sharing tended to be single, female, lower education and unemployed. The authors also speculate that sharing may occur even with persons outside the respondent's household.

Data from the NHIS for the second half of 2009 (Blumberg and Luke, 2010) confirm that sharing is a reality. All these data suggest that sharing, as it relates to probabilities of selection, is an activity that should be considered when designing a cell phone survey.

Business Cell Phones. Another factor that may impact within-household coverage is the presence of business phones and/or more than one personal cell phone. Do researchers need to know, and thus to ask, if the cell phone is a business phone or used as a combination business and personal phone? Under what circumstances should these conditions result in the sampled number being considered out-of-scope? The answers to these important questions are currently unknown.

Multiple Cell Phones. Many people have more than one cell phone, and thus in theory they are an increased probability of being sampled in a cell phone survey. But this, too, is a percentage for which reliable data have not yet been reported. However, it is known from the NHIS that "wireless-only" households and "wireless-mostly" households are more likely than "landline-mostly" households to have more cell phones than individuals, which is an indication that these types of households contain at least some residents with multiple cell phones (Blumberg and Luke, 2010).

Respondent Selection: Although there is still little empirical evidence regarding the ability to interview a different adult from the one originally reached when a cell phone is shared among adults, there are at least five different methods for selecting a respondent within a household in cell phone surveys. These methods are:

  1. Select the person who answers the phone, with no screening for others who possibly share the phone.
  2. Select the person who is the "primary user" of the phone, with screening for the primary user.
  3. Randomly select a respondent from among all the "eligible" users of the cell phone, after screening for how many eligible persons use the phone and making sure that all such persons do in fact qualify as being "eligible" to be surveyed. However, research (e.g., Brick, 2009; Lavrakas, Tompson, Benford and Fleury, 2009) suggests that response rates are lower when trying to have a "handoff" to another eligible respondent. Furthermore, since other research (e.g., Lavrakas, Tompson and Benford, 2009) suggests that approximately one-third of cell phone respondents are reached while they are away from their home, this further complicates the challenge of getting a randomly selected "someone else" who shares the cell phone to be present and actually speak with an interviewer.
  4. Randomly select from among all eligible persons in a household, if this is a household study, regardless of whether all these people use the cell phone on other occasions, with screening for the number of eligible household members (similar to many landline within-unit selection procedures). However, this would be expected to lead to even more nonresponse by requiring the cell phone to be handed off to someone else who does not use the phone under other circumstances.
  5. If this is a wireless-only household, randomly select from among all eligible persons in a household and other members of the household who share the respondent's phone or do not have their own cell phone. This too would be expected to lead to even more nonresponse by requiring the cell phone to be handed off to someone else who does not use the phone under other circumstances.


Future research will be needed to determine which option above is superior among Methods 1, 2, and 3. Methods 4 and 5 would be chosen only when respondents are proxies for their households rather than responding on their own behalf. Methods 4 or 5 might also be appropriate when both cellular and landline numbers are included in the same survey. Also, regardless of which within-unit selection method a researcher may chose, there is no widely accepted way that can currently be recommended for actually wording the selection request for any of the methods.

Within-unit Selection and Nonresponse. As noted above, there is an additional consideration related to within-household selection. In many cell phone surveys, the person who answers the phone is chosen to be the respondent or at least is chosen from screening to determine if s/he is eligible to be interviewed. If the introduction of a cell phone survey is of shorter duration than is common in landline surveys, the interviewer will be able to start the main part of the questionnaire more quickly. This may help avoid refusals from someone reached via a cell phone that might otherwise occur because of the time it takes after initial contact to start the interview. As a consequence, procedures that strive to improve within-household coverage by not always interviewing the person who initially answers the cell phone are very likely to increase nonresponse and thereby decrease completion rates in cell phone surveys (cf. Brick, Edwards, Cervantes and Lee, 2008).

This is one of many trade-offs that researchers need to consider explicitly when deciding whether to deploy a within-unit selection when calling cell phone numbers. Of course, the same trade-off of better within-unit coverage versus greater unit nonresponse exists for landline RDD studies that select one eligible designated respondent within the household, but in most households the "handoff" to another resident of the household will be easier (i.e., more of an "everyday experience") when the call is made to the household's shared landline phone.

 

Previous: Background

Table of Contents

Next: Nonresponse in
RDD Cell Phone Surveys



1 The Task Force membership was not unanimous in its thinking about how accurate some of the NCHS/SHADAC state point estimates might be.  There is always uncertainty about the utility of the model when a model is used to predict an outcome, and in this case the key determinants or predictors for estimating cell-only status are limited. This suggests that one should be cautious in using these estimates. Similar caution should be used with Arbitron's estimates as they were based on a model that included the NCHS/SHADAC state point estimates.

2 Since interviewing from a EWP sample is far more efficient than interviewing from a landline RDD sample, the savings from substituting EWP sample for landline RDD sample can substantially offset the incremental cost of including the cell phone RDD sample.  A series of recent surveys by the University of Virginia Center for Survey Research has tested this approach in "triple-frame" studies that combine landline RDD, EWP, and cell phone RDD samples. This three-sample approach affords some cost savings over the more usual dual frame design, while not fully abandoning the more expensive, traditional landline RDD sampling frame.  This design has allowed direct comparison of survey estimates drawn from combining the EWP and cell phone samples with those obtained from combining landline RDD and cell phone samples, with promising results (Guterbock et al., 2010).  (In the final results, unlisted phones in the two landline samples should be weighted upward so that their proportion among landlines is equal to that found in the landline RDD sample.)   Although  the University of Virginia has made regular use of the triple-frame design, it remains an experimental design that has not to date been adopted elsewhere, and many issues, such as proper weighting and optimal sample allocation, remain to be explored before its suitability for broader application can be fully assessed. 

3 Telephone numbers in the United States are comprised of 10 digits (123-456-7890). The first three numbers, 123, are the area code. The next three numbers, 456, are the prefix or exchange. The last four numbers, 7890, are the local number which can be divided into segments. A thousand block is comprised of 1,000 consecutive numbers for an area code and prefix in which the local "suffix" starts with the same digit, e.g., starting with 7 (7000-7999). A hundred block is the 100 consecutive numbers in which the local suffix starts with the same two digits, e.g., starting with 78 (7800-7899).

4 For example, one major sample vendor updates its frame monthly based on the latest monthly Telcordia file.

5 One major sample vendor recommends that all these wireless services (Dedicated, Cellular, PCS and Special Billing), except Shared, be included. Shared exchanges/blocks by definition may contain landline and wireless numbers. This creates the possibility of overlap between a vendor's cell phone and landline frames. Inclusion of Shared should depend on whether or not the vendor has removed the overlap of 100-blocks or 1000-blocks between the vendor's landline RDD frame and cell phone frame. If overlap has been removed, then inclusion of Shared also is recommended. If overlap has not been dealt with, exclusion or inclusion could lead to reduction of coverage or duplication respectively.

6 This database is licensed for the "sole purposes of: (1) avoid engaging in TCPA Prohibited Conduct by verifying whether TNs [telephone numbers] are assigned to a paging service,  wireless telephone service, specialized mobile radio service, or other radio common carrier service, or any service for which the called party is charged for the call; (2) disclosing, selling, assigning, leasing or otherwise providing the TN Ports to a third party that itself qualifies as a "Customer" under an Intermodal Ported TN Identification Services Agreement for the sole purpose of avoiding TCPA Prohibited Conduct by verifying whether TNs are assigned to a paging service,  wireless telephone service, specialized mobile radio service, or other radio common carrier service, or any service for which the called party is charged for the call."  (http://www.tcpacompliance.us/content/IntermodalUserAgreement.pdf)

7 A person with dual service that predominantly uses a cell phone has come to be termed "wireless-mostly" or "cell phone mostly," whereas a dual service person that predominantly uses a landline has come to be termed "landline-mostly."

8 The following question wording is used by NCHS to ascertain these statuses: "Of all the telephone calls that you or your family receives, are: All or almost all calls received on cell phones; Some received on cell phones and some on regular phones; or Very few or none on cell phones?".  However, Villar, Krosnick, and DeBelle (2010) call into question the validity of this wording and provide evidence that the word "personal" (as in "personal calls") is a more valid way to word the question.

9 In deciding whether specific weights are too large, some researchers look at the ratio of smallest to largest and try to constrain them based on some cut off. In this decision there are four considerations: (a) cost, (b) variance, (c) analytic domains and (d) bias. Optimal allocation based on cost and variance may suggest a very low sampling rate for one group which in turn leads to "large weights." Furthermore, if large weights are appropriate, reducing them (e.g., by trimming) runs the risk of increasing bias. For more on this topic see Potter (1990).