Transparency Initiative

Jump To:

What is the Transparency Initiative?

AAPOR’s Transparency Initiative is designed to promote methodological disclosure through a proactive, educational approach that assists survey organizations in developing simple and efficient means for routinely disclosing the research methods associated with their publicly-released studies.

The Transparency Initiative is an approach to the goal of an open science of survey research by acknowledging those organizations that pledge to practice transparency in their reporting of survey-based research findings. In doing so, AAPOR makes no judgment about the approach, quality or rigor of the methods being disclosed.

Join the TI! You will be in great company!

Membership in the Transparency Initiative will identify your organization as one that is willing to publicly disclose its basic research methods and make them available for public inspection. It marks your organization as one that values openness and has been formally recognized for your commitment to transparency in research methods.

Your organization can apply for Transparency Initiative membership online. We’ve tried to make the application process as easy as possible.

Please reach out to Co-chairs Krista Jenkins and Jennifer Benz For more details regarding the Transparency Initiative, view our Terms & Conditions.

How to Join the TI

  • Review the materials about the TI:
    • Disclosure Elements
    • Certification Agreement
    • Detailed Application Process
  • Choose one or two points of contact in your organization or subunit of your organization.
  • Select two example survey reports that adhere to the Disclosure Elements.
  • Sign the Certification Agreement, which assures that employees in your organization who are responsible for releasing public survey reports — in addition to your points of contact — have been educated about the TI; e.g., with these Educational Materials:
    • Transparency Initiative Information Modules Informational Module #1
      “Promoting Transparency in Survey Research”
    • Informational Module #2: “How to be a Transparent Research Organization”
  • Submit your application materials online, including the example survey reports and certification agreement. You may need to set up an account for this step.
  • The TI will appoint two committee members to review your application, which will take about a month. If the reviewers suggest changes to your survey reports, you will get a chance to make those changes and then resubmit your reports for another review.
  • After your application is accepted, the TI committee will get in touch once a year to confirm that you still want to be a TI member. Every two years, the TI committee will request to recent reports to recertify that your organization continues to be TI-compliant.
  • If you have any questions about the application process, please check out the Frequently Asked Questions below or get in touch at [email protected].

Transparency Initiative Coordinating Committee
Co-Chairs: Krista Jenkins & Ruth Igielnik

For questions about the Transparency Initiative please contact AAPOR Staff at [email protected].

Frequently Asked Questions

Joining the Transparency Initiative will provide your organization with formal public recognition by AAPOR of your voluntary pledge to abide by our disclosure standards. It will also provide you with the opportunity to advertise your commitment to transparency by using AAPOR’s Transparency Initiative logo in your promotional materials. In addition, it will afford you with the opportunity to differentiate yourself from competitors who have elected not to participate.

All Transparency Initiative submissions will remain confidential and will be used only for the purpose of reviewing for membership in the TI. No submitted samples/information will be publicly released without your written permission.

No. Survey subunits of larger organizations are eligible to join the Transparency Initiative without committing other components of the organization to adhere to its requirements.

Your organization can apply for Transparency Initiative membership online. We’ve tried to make the application process as easy as possible.

Yes, for-profit organizations are welcome and encouraged to apply for membership in the Transparency Initiative. We can also consider for membership an organization that only commissions research for private clients if they meet our requirements (see the Terms and Conditions documentation on the website).

All survey-related organizations – domestic or international – may apply for membership in the Transparency Initiative if they are willing to abide by the transparency guidelines. (See the Terms and Conditions documentation on the website).

Nothing. In 2017, the AAPOR Executive Council formally decided to waive any membership fees for organizations to join the Transparency Initiative.

Once your organization has submitted their reports and Transparency Certification Agreement, we will work to complete the initial review process as quickly as possible, but final approval and admission to the Transparency Initiative may take some time, depending on the completeness of your submission. A team of Transparency Initiative reviewers will work with your designated representative to complete the review process as quickly as possible.

If you have submitted your application, you should have received a confirmation email with your application materials. If you have questions or need assistance, please email [email protected]

Adherence to Transparency Initiative principles includes insuring that staff are trained and prepared to practice transparency when reporting survey findings and insuring that all related reports are consistent with the Transparency Initiative reporting requirements. There’s no question that compliance with Transparency Initiative principles will take some time and effort on your organization’s part. To the degree that your organization is already fully disclosing your research methods on a regular basis, less effort should be needed to become Transparency Initiative compliant.

In the majority of situations where the Transparency Initiative reviewers find problems, they will provide assistance in helping the applicant organization revise their current disclosure procedures in order to become compliant with the TI requirements. The Transparency Initiative’s goal is to work with your organization to achieve our common goal of transparency and openness.

No. Organizations will be expected to continue reporting methodological information using the modality and format they consider being most appropriate for their purposes. The Transparency Initiative requires only that the disclosure elements be made publicly available at the same time that survey findings are publicly released, and that additional methodological details be made available within 30 days of any request for that information. The specific format in which that information is to be shared is at the discretion of the organization. While organizations will always have freedom to tailor the format in which they make this information publicly available, it is expected that in the majority of cases this information can be easily posted on your organization’s website.

In those cases when your organization does not collect the survey data, you will be expected to request this information from the fieldwork subcontractor so it is available in the event the data is made public.

In those cases where your organization participates as a fieldwork contractor only, you must agree to provide the same information to the study’s sponsor. It’s understood that whether clients elect to disclose that information or not is beyond your control. But insuring that they have the information is your responsibility.

No. The Transparency Initiative disclosure requirements are prospective only. So, for example, reports that provide time trend data will only require methodological details for the new primary survey data being reported.

No. The Transparency Initiative does not attempt to judge the quality of member’s survey methods. Rather, it is concerned with insuring that research methods are consistently disclosed so that research consumers have available the information necessary to reach their own conclusions regarding survey quality.

AAPOR has long championed the importance of transparency in the public reporting of public opinion and survey research. The Transparency Initiative’s disclosure standards are those found in the AAPOR Code of Professional Ethics and Practices, the Code that all AAPOR members have pledged to uphold. Encouraging adherence to these standards is consistent with our long-held values of research transparency and openness.

Public invitations to join the Transparency Initiative were first announced in October 2014 and extended to all survey organizations.

Yes; any organization that wants to show additional support of the TI may donate funds to the Initiative and hold SPONSOR status with the TI. For more information on becoming a TI Sponsor, contact AAPOR Staff at AAPOR HQ

For more details regarding the Transparency Initiative, view our Terms & Conditions.

How does the TI help the public evaluate and understand survey-based and other research findings?

Members of the Transparency Initiative are required to disclose a range of important details about how they conducted their research. Providing this critical information is important for assessing the rigor and appropriateness of the underlying methodology.

Members who are in good standing with the Transparency Initiative are reviewed every two years with checks done on how they are describing their methodology to the public, or their clients if the data is proprietary.

The following is a list of required elements that must be disclosed with all publicly and/or proprietary research products.

Items for Immediate Disclosure

Describe the data collection strategies employed (e.g. surveys, focus groups, content analyses).

Name the sponsor of the research and the party(ies) who conducted it. If the original source of funding is different than the sponsor, this source will also be disclosed.

Measurement tools include questionnaires with survey questions and response options, show cards, vignettes, or scripts used to guide discussions or interviews. The exact wording and presentation of any measurement tool from which results are reported as well as any preceding contextual information that might reasonably be expected to influence responses to the reported results and instructions to respondents or interviewers should be included. Also included are scripts used to guide discussions and semi-structured interviews and any instructions to researchers, interviewers, moderators, and participants in the research. Content analyses and ethnographic research will provide the scheme or guide used to categorize the data; researchers will also disclose if no formal scheme was used.

Survey and public opinion research can be conducted with many different populations including, but not limited to, the general public, voters, people working in particular sectors, blog postings, news broadcasts, an elected official’s social media feed. Researchers will be specific about the decision rules used to define the population when describing the study population, including location, age, other social or demographic characteristics (e.g., persons who access the internet), time (e.g., immigrants entering the US between 2015 and 2019). Content analyses will also include the unit of analysis (e.g., news article,  social media post) and the source of the data (e.g., Twitter, Lexis-Nexis).

The description of the methods of sampling includes the sample design and methods used to contact or recruit research participants or collect units of analysis (content analysis).

  • Explicitly state whether the sample comes from a frame selected using a probability-based methodology (meaning selecting potential participants with a known non-zero probability from a known frame) or if the sample was selected using non-probability methods (potential participants from opt-in, volunteer, or other sources).
  • Probability-based sample specification should include a description of the sampling frame(s), list(s), or method(s).
    1. If a frame, list, or panel is used, the description should include the name of the supplier of the sample or list and nature of the list (e.g., registered voters in the state of Texas in 2018, pre-recruited panel or pool).
    2. If a frame, list, or panel is used, the description should include the coverage of the population, including describing any segment of the target population that is not covered by the design.
  • For surveys, focus groups, or other forms of interviews, provide a clear indication of the method(s) by which participants were contacted, selected, recruited, intercepted, or otherwise contacted or encountered, along with any eligibility requirements and/or oversampling.
  • Describe any use of quotas.
  • Include the geographic location of data collection activities for any in-person research.
  • For content analysis, detail the criteria or decision rules used to include or exclude elements of content and any approaches used to sample content. If a census of the target population of content was used, that will be explicitly stated.
  • Provide details of any strategies used to help gain cooperation (e.g., advance contact, letters and scripts, compensation or incentives, refusal conversion contacts) whether for participation in a survey, group, panel, or for participation in a particular research project. Describe any compensation/incentives provided to research subjects and the method of delivery (debit card, gift card, cash).

Include a description of all mode(s) used to contact participants or collect data or information (e.g., CATI, CAPI, ACASI, IVR, mail, Web for survey; paper and pencil, audio or video recording for qualitative research, etc.) and the language(s) offered or included. For qualitative research such as in-depth interviews and focus groups, also include length of interviews or the focus group session.

Disclose the dates of data collection (e.g., data collection from January 15 through March 10 of 2019). If this is a content analysis, include the dates of the content analyzed (e.g., social media posts between January 1 and 10, 2019).

  1.  Provide sample sizes for each mode of data collection (for surveys include sample sizes for each frame, list, or panel used)
  2. For probability sample surveys, report estimates of sampling error (often described as “the margin of error”) and  discuss whether or not the reported sampling error or statistical analyses have been adjusted for the design effect due to weighting, clustering, or other factors.
  3. Reports of non-probability sample surveys will only provide measures of precision if they are defined and accompanied by a detailed description of how the underlying model was specified, its assumptions validated, and the measure(s) calculated.
  4. If content was analyzed using human coders, report the number of coders, whether inter-coder reliability estimates were calculated for any variables, and the resulting estimates.

Describe how the weights were calculated, including the variables used and the sources of the weighting parameters.

Describe validity checks, where applicable, including but not limited to whether the researcher added attention checks, logic checks, or excluded respondents who straight-lined or completed the survey under a certain time constraint, any screening of content for evidence that it originated from bots or fabricated profiles, re-contacts to confirm that the interview occurred or to verify respondent’s identity or both, and measures to prevent respondents from completing the survey more than once. Any data imputation or other data exclusions or replacement will also be discussed. Researchers will provide information about whether any coding was done by software or human coders (or both); if automated coding was done, name the software and specify the parameters or decision rules that were used.

All research has limitations and researchers will include a general statement acknowledging the unmeasured error associated with all forms of public opinion research.