The leading association
of public opinion and
survey research professionals
American Association for Public Opinion Research

Short Courses

Cognitive Interviewing:  A Hands-On Approach

Co-sponsored by AAPOR and DC-AAPOR

Presented by Gordon Willis, PhD
July 10, 2017

Bureau of Labor Statistics Conference Center

Bureau of Labor Statistics
Postal Square Building
2 Massachusetts Ave., N.E.
Washington, DC 20212-0001



Price (Includes a copy of Dr. Willis' book "Cognitive Interviewing: A Tool for Improving Questionnaire Design"):
Student Members: $90
Student Nonmembers: $100 (includes AAPOR and DC-AAPOR membership)
Members: $125
Non-Members: $225 (includes AAPOR and DC-AAPOR membership)

Course Objectives:

Cognitive interviewing has become a very popular method for pretesting and evaluating survey questionnaires. The current approach favored by Federal laboratories and private research institutions mainly emphasizes the use of intensive verbal probes that are administered by specially trained interviewers to volunteer respondents, to delve into the cognitive and socio-cultural processes associated with answering survey questions.  Based on this information, the evaluator makes judgments about where questions may produce difficulties in a number of subtle ways, due to cognitive demands they impose, cultural mismatches, or other shortcomings.  The short-course will focus on the specifics of how to conduct cognitive testing, and how to process and communicate the results obtained.  Although an introduction to theory and background perspective is included, the course will focus on the application and practice of cognitive interviewing techniques, as these are targeted toward both interviewer-administered (face-to-face or telephone) and self-administered (paper and computer) surveys.  Participants will practice the conduct of cognitive interviews across administration modes, and will evaluate their results by judging where questions have failed, and what one might do to revise them.  The course aims to provide a working familiarity with cognitive techniques, so that students will be able to begin conducting cognitive interviews on their own.

Who Should Attend?

The course is geared to individuals in government, universities, business, and nonprofit organizations interested in understanding and practicing cognitive interviewing techniques.  

About the Instructor:

Gordon Willis
has practiced and conducted research in a wide range of cognitive interviewing techniques for over thirty years, at Northwestern University, The National Center for Health Statistics, Research Triangle Institute, and currently at the National Cancer Institute, NIH.  He has written “Cognitive Interviewing, A Tool for Improving Questionnaire Design” (2005), and “Analysis of the Cognitive Interview in Questionnaire Design.”  He has also taught short courses on cognitive and other questionnaire evaluation techniques for the Joint Program in Survey Methodology, AAPOR, DC-AAPOR; at conferences of the American Association for Public Opinion Research and the American Statistical Association, and for government and professional organizations such as Statistics Sweden, Statistics Netherlands, The Nielsen Company, and The Odum Institute.  He has co-taught a course in questionnaire design at the University of Maryland/University of North Carolina, and has been adjunct faculty member at the Johns Hopkins School of Public Health, and at the Uniformed Services University of the Health Sciences.  His research interests focus on the evaluation of pretesting techniques, and on their extension to multi-lingual and cross-cultural domains.

Course Agenda:

9:00 –  9:30 Part 1:  Intro to Cognitive Interviewing
9:30 – 10:00 Video:  Mock “Good” Cognitive Interview
10:00 - 10:45 Exercise A:  Interviewer-Administered Cognitive Interview
10:45 – 11:00 BREAK
11:00 – 11:45 Part 2:  Developing Verbal Probes
11:45 --12:45  LUNCH
12:45 – 1:15 Part 2b:  Probes (cont), and Avoiding Probing Pitfalls
1:15  –  1:45 Part 3:  Adapting to Survey Administration Mode
1:45  –  2:30 Exercise B:  Self-Administered Cognitive Interview
2:30  –  2:45 BREAK
2:45  –  3:15 Discuss Exercise B
3:15  -- 3:45 Part 4:  Analysis procedures
3:45  –  4:15 Part 5:  Logistical Considerations
4:15 –  4:30 Course Evaluation

For more information contact: Adam Thocher at athocher@aapor.org or call 847-686-2230