Short Courses

Six short courses are offered to enhance your learning experience. These in-depth courses are taught by well-known experts in the survey research field and cover topics that affect our ever-changing industry. Make the most of your time at the conference and plan to participate in the following courses:

Course 1:   Weighting Sample Survey Data, 201
Course 2:   Twitter and Public Opinion Research: Who, What, When, Where, Why and How
Course 3:   Mail and Telephone Data Collection With Address-Based Samples
Course 4:   Advanced Focus Group Moderator Techniques
Course 5:   How to Publish in Survey Research: Strategies, Venues, Opportunities and Errors to Avoid
Course 6:   Cellular Telephone Methodology: Sampling, Targeting and Dispositioning

NEW THIS YEAR! AAPOR is offering $15 off short course registration to 30 interested students on a first-come, first-serve basis. Please contact Heidi Diederich for more information at hdiederich@aapor.org or 847-205-2651, ext. 295.

Course 1

Weighting Sample Survey Data, 201
Wednesday, May 15, 2:30 p.m. – 6:00 p.m.

Course Overview:
Weighting data collected from a probability sample survey is a standard part of data processing. Most often sample weights are constructed using the initial probabilities of selection that are determined by the sampling design and can be further adjusted for nonresponse and noncoverage. This short course will demonstrate the methodological foundations of weighting procedures, provide examples of different types of weighting adjustment methods and explain the benefits, drawbacks and issues related to the weighted data. We will also provide an explicit example of a typical weighting procedure encountered in public opinion surveys. The material will necessitate light-to-medium use of formulas to explain the specific procedures and differences between alternative methods.

We plan to cover: theoretical foundations of weights in survey statistics; weighting adjustments for non-response, poststratification and other forms of calibration; dual and multiple frames; replicate weights; limits and drawbacks of weighting procedures; available software; and workflow of weighting.

Instructors:
Stas Kolenikov is a senior survey statistician at Abt SRBI. He also worked as a statistical consultant for the World Bank and as assistant professor of statistics at University of Missouri, and taught continuing education courses on survey statistics for the American Statistical Association, Statistical Society of Canada and Stata Corp. His recent projects include resampling variance estimators for weight calibration, multiple imputation for mode adjustment and small area estimation of the prevalence of a rare ethnic population.

Trent Buskirk is research director of advanced methodologies and standards at The Nielsen Company. Prior to joining Nielsen, he was an associate professor of biostatistics at Saint Louis University. His research interests include address-based sampling for panel recruitment, mobile phone surveys and use of non-parametric statistical methods with complex survey data. He has consulted with various companies and universities on survey design, sampling and weighting.

Course Objectives:
The students will leave this course with an understanding of the steps needed to take base weights to final survey weights and how to apply various weighting procedures used in survey weight construction. Students will also understand how sampling weights can and should be incorporated into analyses of complex sample survey data, including the use of replicate weights for variance estimation.

Who Should Attend:
This intermediate level course is aimed at public opinion researchers and specialists who frequently work with sample survey data and need to construct weights or use weights as part of data collection, delivery or analysis. The students in this course will have had some prior exposure to survey weighted data and will be driven to learn how specific methods and procedures work to fulfill weighting and analysis needs relative to sample survey data.

Course 2

Twitter and Public Opinion Research: Who, What, When, Where, Why and How
Wednesday, May 15, 2:30 p.m. – 6:00 p.m.

Course Overview:
While much of the information posted on Twitter is difficult to interpret without adequate context or knowledge about the user or issue, the sheer magnitude of users and Tweets has provided a wealth of information to researchers on important social topics. Some researchers have approached the “mining” of this Twitter data using a variety of qualitative and quantitative methods. Others, noting the lack of solid information about exactly who Tweets and who does not, see Twitter as a non-representative source of information and have decided to ignore the stream of data altogether.

This course will identify the strengths and weaknesses in utilizing Twitter for social research, and how the advent of this social network is impacting traditional research approaches. Using a framework for approaching an analysis of this large-n text stream, part of this course will step through, using open source tools, how social media data are collected, parsed and analyzed, and the decisions that are required to be made during this process that impact both analysis and data understanding.

Instructors:
Joe Murphy is a survey methodologist at RTI International with more than 15 years of experience researching the causes and solutions for issues related to survey quality and managing survey projects. His research focuses on the implementation of new data collection processes, new data sources and analytic techniques to maximize data quality, increase response and reduce costs. His recent work has been centered on data sources and techniques such as Internet search patterns, social media data analysis (e.g., Twitter), data visualization, crowdsourcing and social research in virtual worlds. Mr. Murphy is the director of the program on digital technology and society in RTI’s Survey Research Division.

Carol Haney is vice president and head of product marketing at Toluna with more than 15 years of experience working with respondent data and other related, large data sets. Before joining Toluna, Ms. Haney was the thought leader in the development of Research LifeStreaming at Harris Interactive, an industry-changing method of doing data collection by panel-based social media listening. Ms. Haney has served in leadership roles at TNS, SPSS and the National Opinion Research Center.

Course Objectives:
Attendees will learn the differences between Twitter and other social networking data, how Twitter may be a useful resource for public opinion research, the trade-offs involved in using social media data for public opinion research, and how one can, using open source tools, technically and analytically read the Twitter pipe and process the data. An overview of the type of tools available, current text analytic techniques and limitations will be covered as well.

Who Should Attend:
Survey practitioners and students interested in social media data and potential uses of incorporating social media data into their own research. No previous experience in these areas is required.

Course 3

Mail and Telephone Data Collection With Address-Based Samples
Thursday, May 16, 8:00 a.m. – 11:30 a.m.

Course Overview:
Within the last decade, address-based sampling (ABS) has led to a transformation of household sampling and data collection methods in the United States. Researchers have used the ABS frame in a variety of ways with face-to-face, telephone and mail modes of data collection. While ABS provides a new opportunity, it also presents challenges of how best to use the new sampling frame to conduct surveys.

This course begins with a quick overview of the data on the ABS frame and how it can be used to sample households in the United States. One of the key attributes of the ABS frame is the ability to link telephone numbers to a subset of the addresses, enabling different approaches to data collection. We focus on mail and telephone data collection modes, with only a brief mention of face-to-face and Web modes and ABS. For the mail and telephone modes, we present models of data collection that make varying uses of the modes. For each of these data collection models we describe the comparative advantages and disadvantages and provide examples of surveys that used the models. We also compare these models of data collection with dual frame (landline and cell) telephone samples.

Instructor:
Mike Brick is a vice president of the survey methods group at Westat and is a research professor in the Joint Program in Survey Methodology at the University of Maryland. He has a doctorate in statistics from American University, is a Fellow of the American Statistical Association and an elected member of the International Statistical Institute. He has published in a variety of journals on topics including sampling, weighting and nonresponse bias, including articles on telephone and address-based sampling and data collection. Brick served on AAPOR’s Executive Council (2007-2009), as associate editor of Survey Methodology (2001-present) and on the editorial board of Public Opinion Quarterly (2003-present).

Course Objectives:
This half-day short course is intended to introduce researchers to the alternative methods of mail and telephone data collection strategies with ABS. The participants should come away with a good understanding of the advantages and disadvantages of the different modes with ABS and should be able to apply these concepts to their survey needs.

Who Should Attend:
The course is aimed at survey researchers and sponsors who have to decide on the methods of collecting data and wish to consider sampling from the ABS frame.

Course 4

Advanced Focus Group Moderator Techniques
Thursday, May 16, 8:00 a.m. – 11:30 a.m.

Course Overview:
The literature is rich on aspects of design, implementation and analyses of focus groups. Missing, however, for practicing professionals and researchers, are a theoretical framework, guidelines and educational opportunities on how to moderate a focus group. In this experiential learning environment, participants will learn and apply techniques to sustain discussion among participants and elicit more than one-person answers; maximize time to get through all questions; and build an environment where opinions are tolerated by other participants.

These techniques are life skills useful not only in focus groups but in community forums, committee and board meetings, and teaching. Mock focus groups will be conducted. Participants will learn the theoretical basis for moderating focus groups and its implications on the knowledge a researcher is seeking; four focus group process objectives for directing group dynamics; and twenty practical techniques to create and maintain focus group discussions.

Instructor:
Nancy Ellen Kiernan, PhD, has been a program evaluator and faculty member at Penn State University for 30 years with extensive experience in evaluating an array of programs as reflected in research publications in Evaluation Review, Evaluation and Program Planning, Journal of Nutrition Education, American Journal of Industrial Medicine and International Quarterly of Community Health Education. Her field work includes moderating focus groups among diverse groups such as Pennsylvania Amish, at-risk urban youth, greenhouse managers, the elderly, community coalition members, landscape architects, food stamp recipients, veterinarians and farmers.

Dr. Kiernan received the Excellence in Program Evaluation Award and the Sustained Excellence in Evaluation Award from the American Evaluation Association’s Extension Evaluation Group, and the Achieving Woman Award from the Penn State University Commission on Women for innovative approaches to evaluation. She has taught courses on focus group moderation at Stanford University, University of California-Berkeley School of Public Health, PAPOR, and the American Evaluation Association Conferences, to name a few.

Course Objectives:
Participants will improve techniques using practical exercises and examples; role play with other participants in a focus group; and evaluate a moderator on the use of these techniques in a mock focus group, or, use techniques as a moderator.

Who Should Attend:
The course is ideal for researchers, staff, faculty, community evaluators, agency administrators and advanced students.

Course 5

How to Publish in Survey Research: Strategies, Venues, Opportunities and Errors to Avoid
Thursday, May 16, 8:00 a.m. – 11:30 a.m.

Course Overview:
All survey research involves writing reports of findings. A report can simply be written for the people who conducted the survey, but most often reports are written for external consumption. Publication of reports in academic journals is vital for our work to have constructive impact on society and on the profession of survey research. This short course is designed to help professionals to improve their abilities to write effective reports that will be accepted for publication. Survey research papers have the potential to be published in the journals of many different disciplines.

This course will provide practical guidance addressing the following issues:

How to decide whether a piece of research is ready for publication.
What a paper needs to achieve in order to be publishable.
How to write an introduction, method, results and discussion section.
How to revise a manuscript for resubmission after an initial review.
How to deal with the ego blow of rejection.
How to revise a manuscript after it has been rejected.
How to resist the urge to say that reviewers don’t know what they’re talking about.

The instructors will present the results of a survey of editors of peer reviewed journals publishing survey research papers, together with bibliometrics data and other information generally not readily available. The findings from this survey will explain what journal editors say are the most common mistakes they encounter in submitted papers and how to avoid making them.

Instructors:
Mario Callegaro is survey research scientist at Google, London. Mario holds a master’s degree and doctorate in survey research and methodology from the University of Nebraska, Lincoln. He is associate editor of Survey Research Methods, on the editorial board of the International Journal of Market Research, and reviews papers for survey research journals. Mario has published numerous papers and book chapters, and presented at international conferences on survey methodology and data collection methods.

Jon A. Krosnick is Frederic O. Glover Professor in Humanities and Social Sciences, professor of communication, professor of political science, and (by courtesy) professor of psychology at Stanford University, Senior Fellow at the Woods Institute for the Environment at Stanford, research psychologist at the U.S. Census Bureau, and research professor at the Survey Research Laboratory at the University of Illinois. He is a world-recognized expert in survey methodology who has published extensively in academic journals.

Course Objectives:
Attendees should acquire a full understanding of the options and appropriate strategies (which differ from discipline to discipline).

Who Should Attend:
Public opinion scholars and practitioners looking to publish their research in peer-reviewed journals.

Course 6

Cellular Telephone Methodology: Sampling, Targeting and Dispositioning
Sunday, May 19, 8:00 a.m. – 11:30 a.m.

Course Overview:
Dual-frame surveys (landline and cellular telephones) are a ubiquitous feature of the present-day survey research landscape. Cellular telephones present a major challenge in that they contain very little meta-data. For decades, such meta-data (name, address, even probable race, age and a host of other demographic metrics) have afforded great power in sampling landlines to target specific sub-populations. Cell phones, however, have very little comparable data. How then does one oversample for respondents of specific geographies or demographics, gain an understanding of the expected incidence and coverage, or sample at the local level? What are the prospects for listed information on cellphones?

Beyond sampling, cellular telephones are distinct from landline telephones in many attributes associated with how they are dialed, the result of dialing, and how one might disposition the outcome of those dialing attempts. For example, refusal conversions take on a different color when being attempted on a personal, rather than household, communication device. These differences lead to a number of special considerations regarding how one most effectively “works” cell phone sample and in the calculation of response rates for a cell phone sample.

The course will look briefly at the history of telephony and the development of cell phones, then focus on cell phone owner mobility, the nature and utility of cellular switch points and cellular rate centers, the efficacy and application of billing zip codes and recent activity flags, the use of cellular listed databases, trends in cellular response, the potential of only dialing cell phones, the treatment of voice mails, refusal conversions and callbacks on cell phones, and considerations in the dispositioning of cell phones and the calculation of cell phone dual-frame response rates.

Instructor:
David Dutwin is vice president and chief methodologist of SSRS/Social Science Research Solutions. He is co-chair of the AAPOR Special Task Force on Survey Refusals, a current member of the Standards Committee, and student paper winner in 2002. In addition to his position at SSRS, he is an adjunct professor at West Chester University and methodologist for JPAR/Jewish Policy and Action Research. David has a doctorate from the University of Pennsylvania, Annenberg School for Communication.

Course Objectives:
This half-day short course provides the most recent lessons surrounding the sampling, dialing and targeting of cell phones in survey research. Attendees should be able to apply these lessons in designing their studies and assessing the work of others.

Who Should Attend:
The course is aimed at both producers of survey data as well as those who commission survey research projects.