ePrivacy and GPDR Cookie Consent by Cookie Consent
The leading association
of public opinion and
survey research professionals
American Association for Public Opinion Research

General Questionnaire Design

Click here to purchase this webinar kit.

Member Price: $175.00
Nonmember Price: $235.00

Student pricing available

The Role of Question Characteristics in Designing and Evaluating Survey Questions
About This Course:

Over time researchers have developed recommendations for writing questions for standardized measurement that are based on understanding the effects question characteristics like length and difficulty have on survey data. This course reviews the key and latest insights from this area of research to present a framework for how to think about question characteristics and the impact they have on data quality; uses case studies to illustrate that impact; and offers “best practice” advice to help practitioners design questions that enhance data quality. We compare important approaches for identifying and evaluating question characteristics, such as the Question Appraisal System (Willis 2005) and the Question Understand Aid (QUAID) (Grasser et al. 2006). We discuss challenges in this research including how question characteristics can interact in affecting interviewer and respondent processing, and conclude with an agenda for future study.

Learning Objectives: 

  • Present a framework for thinking about question characteristics and their effect on the quality of respondents’ answers.
  • Synthesize, describe, and analyze key approaches to identifying question characteristics.
  • Summarize findings of their impact on data quality.
  • Illustrate the impact of question characteristics in a case study.
  • Offer a summary, limitations, and future directions for research on question characteristics.

Design Principles for the Use of Filter Questions
About This Course:
To avoid asking respondents questions that do not apply to them, surveys very often use filter questions to determine routing into follow up items. Filter questions can be asked in an interleafed format, in which follow up questions are asked immediately after each relevant filter, or a grouped format, in which follow-up questions are asked only after multiple filters have been administered. Most previous investigations of such questions have found that the grouped format collects more affirmative answers to the filter questions than the interleafed format. There is evidence that respondents in the interleafed format learn to shorten the questionnaire by answering the filter questions negatively. However, there are several other factors which can also affect the responses to the filter questions, such as the repetitiveness of the follow up items, the number of follow up items, and how the questions are displayed in web surveys. The course will also cover a related form of questions, which we call looping questions -- looping questions are commonly used in panel surveys to collect information about the life course, for example to ask about all jobs a respondent has held, or all degrees earned.

Learning Objectives:
  • Participants will receive practical evidence-based guidance on how to ask filter and follow-up questions in surveys.

The Questionnaire Design Pitfalls of Multiple Modes
The increasing cost of fieldwork and declining survey budgets is pushing survey practitioners to look for cheaper ways of collecting survey data. For example, this could be through encouraging a worthwhile portion of respondents to complete questionnaires by web rather than by the more traditional modes such as postal questionnaires, face-to-face and telephone interviews. However, mixing modes of data collection can reduce data comparability because people may answer questions differently depending on the mode. In this webinar, we will provide a conceptual framework for understanding the causes of these measurement differences and present a typology of questions based on the factors that cause measurement differences by mode. From our own studies in the UK, and from what is known in the literature, we will highlight some of the questionnaire design pitfalls of using multiple modes and make recommendations for improving the portability of these question types across modes. We will close the webinar with a discussion of the methods that can be applied to assess any remaining measurement differences.

Learning Objectives
  • Understand the theoretical and practical difference in how respondents react to different modes of data collection.
  • Have greater awareness of specific question attributes that make certain questions less portable across modes.
  • Have greater knowledge and confidence in executing their own mixed modes questionnaires.

Questionnaire Design
This webinar addresses a specific questionnaire format that asks respondents to report whether (or the extent to which) they agree or disagree with a statement. Specifically, the course will review the major problems with using agree-disagree questions and scales composed of agree-disagree questions, discuss the ways that these questions may introduce error into survey measures, consider when it may be appropriate to use agree-disagree questions, and discuss ways to revise agree-disagree questions to use other formats. Examples using existing scales and survey questions will be given.

Learning Objectives
  • Understand the major problems with agree-disagree questions.
  • Understand when it is appropriate to use agree-disagree questions and when it is more appropriate to consider using a different format.
  • Be able to revise agree-disagree questions into a different format to avoid the problems with these types of questions.