Methodology

The American Trends Panel survey methodology

Overview

The American Trends Panel (ATP), created by Pew Research Center, is a nationally representative panel of randomly selected U.S. adults. Panelists participate via self-administered web surveys. Panelists who do not have internet access at home are provided with a tablet and wireless internet connection. Interviews are conducted in both English and Spanish. The panel is being managed by Ipsos.

Data in this report is drawn from ATP Wave 147, which was conducted from April 29 to May 19, 2024, among a sample of U.S adults who have never had children and are unlikely to have children in the future. The sample was comprised of 8,826 panelists who had previously indicated that they did not have children and, for those under 50 years old, had also said they were unlikely to have children in the future. At the start of the survey, respondents were screened and considered eligible for inclusion if they confirmed that they had never been a parent or guardian to any children, living or deceased, including biological or adopted children. To be considered eligible, respondents under 50 also had to say that they were not too or not at all likely to have children in the future. Of the 7,304 respondents who completed the screening questions, there were 3,312 who were eligible and went on to complete the full survey, including 1,779 respondents from the ATP and an additional 1,533 from Ipsos’ KnowledgePanel (KP). The survey has a response rate of 82% (AAPOR RR3).⁶ The cumulative response rate accounting for nonresponse to the recruitment surveys and attrition is 2%. The break-off rate among panelists who logged on to the survey and completed at least one item is 1%.

Weighting and analysis were performed separately for respondents ages 18 to 49 and those 50 and older. The margin of sampling error for the sample of 770 respondents ages 18 to 49 is plus or minus 5.3 percentage points. The margin of sampling error for the sample of 2,542 respondents ages 50 and older is plus or minus 2.4 percentage points.

Panel recruitment

The ATP was created in 2014, with the first cohort of panelists invited to join the panel at the end of a large, national, landline and cellphone random-digit dial survey that was conducted in both English and Spanish. Two additional recruitments were conducted using the same method in 2015 and 2017, respectively. Across these three surveys, a total of 19,718 adults were invited to join the ATP, of whom 9,942 (50%) agreed to participate.

In August 2018, the ATP switched from telephone to address-based sampling (ABS) recruitment. A study cover letter and a pre-incentive are mailed to a stratified, random sample of households selected from the U.S. Postal Service’s Delivery Sequence File. This Postal Service file has been estimated to cover as much as 98% of the population, although some studies suggest that the coverage could be in the low 90% range.⁷ Within each sampled household, the adult with the next birthday is asked to participate. Other details of the ABS recruitment protocol have changed over time but are available upon request.⁸

We have recruited a national sample of U.S. adults to the ATP approximately once per year since 2014. In some years, the recruitment has included additional efforts (known as an “oversample”) to boost sample size with underrepresented groups. For example, Hispanic adults, Black adults and Asian adults were oversampled in 2019, 2022 and 2023, respectively.

Across the six address-based recruitments, a total of 23,862 adults were invited to join the ATP, of whom 20,917 agreed to join the panel and completed an initial profile survey. Of the 30,859 individuals who have ever joined the ATP, 11,897 remained active panelists and continued to receive survey invitations at the time this survey was conducted.

The American Trends Panel never uses breakout routers or chains that direct respondents to additional surveys.

Sample design

This survey was designed to cover the population of noninstitutionalized persons ages 18 and older living in the U.S., including Alaska and Hawaii, who have never had any children and are unlikely to have any in the future. All active ATP members who previously said in Wave 133 (fielded in August 2023) that they did not currently have any children, including any living outside their household, and were either 1) 18 to 49 years old and not too or not at all likely to have children in the future, or 2) 50 years old or older were invited to participate in this wave.

The ATP was supplemented with oversamples of adults ages 50 and older who previously reported that they did not currently have any children living either inside or outside of their household. Initially, an oversample composed of Hispanic, non-Hispanic Black and non-Hispanic Asian KP panelists and further stratified by gender was drawn, and a total of 242 were invited to participate an initial soft launch. Because eligibility rates for this oversample were lower than anticipated, no additional cases in this sample were invited. Instead, a new oversample – consisting of adults of any race, ages 50 and older and with no children, stratified by education and gender – was drawn. KnowledgePanel respondents in the final data include both those from the initial soft launch and the second oversample.

Because the group of potentially eligible adults who were sampled is broader than the target population, respondents were asked screening questions at the beginning of the survey to confirm their eligibility. Respondents were only considered eligible if they said they had never been a parent or guardian to any children, living or deceased, including any biological or adopted children. Additionally, respondents who were under 50 years old were only considered eligible if they also said they were not too likely or not at all likely to have children in the future.

KP respondents who screened ineligible for the main survey were asked a subset of demographic questions needed for weighting purposes and are only considered to have completed screening if they answered both the screening and demographic question sets.

Questionnaire development and testing

The questionnaire was developed by Pew Research Center in consultation with Ipsos. The web program was rigorously tested on both PC and mobile devices by the Ipsos project management team and Pew Research Center researchers. The Ipsos project management team also populated test data that was analyzed in SPSS to ensure the logic and randomizations were working as intended before launching the survey.

Incentives

All respondents were offered a post-paid incentive for their participation. ATP respondents could choose to receive the post-paid incentive in the form of a check or a gift code to Amazon.com or could choose to decline the incentive. Incentive amounts ranged from $5 to $15 depending on whether the respondent belongs to a part of the population that is harder or easier to reach. Differential incentive amounts were designed to increase panel survey participation among groups that traditionally have low survey response propensities.

Ipsos operates an ongoing modest incentive program for KnowledgePanel to encourage participation and create member loyalty. The incentive program includes special raffles and sweepstakes with both cash rewards and other prizes to be won. Typically, panel members are assigned no more than one survey per week. On average, panel members complete two to three surveys per month with durations of 10 to 15 minutes per survey. An additional incentive is usually provided for longer surveys.

Data collection protocol

The data collection field period for this survey was April 29 to May 19, 2024. Postcard notifications were mailed to a subset of ATP panelists with a known residential address on April 29.⁹

ATP invitations were sent out in separate launches: soft launch and full launch. Sixty ATP panelists were included in the first soft launch, which began with an initial invitation sent on April 29. After reviewing the first soft launch, a minor change was made to the screening question “Have you ever been a parent or guardian to any children, living or deceased, including any biological or adopted children?” The original question asked about “legal” guardianship and included a reference to stepchildren. Sixty panelists were included in the second soft launch, which began with an initial invitation sent on May 2. The second soft launch confirmed that the updated program was working as it should. The ATP panelists chosen for both soft launches were known responders who had completed previous ATP surveys within one day of receiving their invitation. All remaining English- and Spanish-speaking sampled ATP panelists were included in the full launch and were sent an invitation on May 3.

KP invitations were sent out in separate launches: soft launch and full launch. A total of 242 KP panelists were included in the first soft launch, which began with an initial invitation sent on April 29. After the first soft launch, the same wording edit was made to the parental status screening question and the sample design for KP panelists was changed (refer to the Sample Design section). A total of 245 panelists were included in the second soft launch, which began with an initial invitation sent on May 9. The second soft launch confirmed that the updated program was working as it should. All remaining English- and Spanish-speaking sampled KP panelists were included in the full launch and were sent an invitation on May 10.

All panelists with an email address received an email invitation and up to three email reminders if they did not respond to the survey. All ATP and KP panelists who consented to SMS messages received an SMS invitation and up to three SMS reminders.

Data quality checks

To ensure high-quality data, the Center’s researchers performed data quality checks to identify any respondents showing clear patterns of satisficing. This includes checking for whether respondents left questions blank at very high rates or always selected the first or last answer presented. As a result of this checking, three respondents were removed from the survey dataset prior to weighting and analysis.

Weighting

This survey was weighted in a multistep process that accounts for multiple stages of sampling and nonresponse that occur at different points in the survey process. First, each panelist begins with a base weight that reflects their probability of selection for their initial recruitment survey. These weights are then rescaled and adjusted to account for changes in the design of ATP recruitment surveys from year to year. Base weights for KP respondents were provided by Ipsos.

Because there are no benchmarks for the population of U.S. adults without children, weighting parameters were based on the full set of ATP members who were potentially eligible for inclusion in the sample prior to any screening. First, the base weights for all ATP members who responded to the 2023 Annual Profile Survey (ATP Wave 133) were calibrated to align with the population benchmarks in the accompanying table to create a full-panel weight.

This full-panel weight was then used to compute weighting benchmarks for the universe of adults who met the criteria for being included in the sample. Weighting was performed separately using separate targets for respondents ages 18 to 49 and those ages 50 and older.

Respondents ages 18 to 49

For ages 18 to 49, weighting parameters were calculated based on all ATP members who said in Wave 133 that they did not have any children and were not too likely or not at all likely to have children in the future. These parameters include age, gender, education, race/ethnicity, place of birth, years lived in the U.S., identification as Black (alone or in combination), census region, metropolitan status, frequency of internet use, religion, party affiliation, voter registration and volunteerism.

For respondents in this group who completed the screening questions, including those who screened ineligible, the full-panel weight was then calibrated to align with the parameters listed above and trimmed at the 2nd and 98th percentiles to reduce the loss in precision stemming from variance in the weights. This trimming is performed separately among non-Hispanic Black, non-Hispanic Asian, Hispanic and all other respondents.

Respondents ages 50 and older

For ages 50 and older, weighting parameters were calculated based on all ATP members who said in Wave 133 that they did not have any children. These parameters include race/ethnicity by gender, age by gender, education by gender, age by education, race/ethnicity by education, born inside vs. outside the U.S. among Hispanics and Asian Americans, years lived in the U.S., identification as Black (alone or in combination) by Hispanic ethnicity, census region by metropolitan status, frequency of internet use, religion, party affiliation by race/ethnicity, and party affiliation by voter registration.

Because this group includes both ATP and KP respondents, base weights for respondents from each sample were rescaled and combined to account for the inclusion of multiple samples and their designs. The combined base weight was then calibrated to align with the parameters listed above and trimmed at the 2nd and 98th percentiles to reduce the loss in precision stemming from variance in the weights. This trimming is performed separately within cells defined by the cross classification of gender by education.

After the weighting was performed for screener respondents from both age groups, individuals who screened ineligible were removed from the data, with no further weighting adjustments.

Sampling errors and tests of statistical significance take into account the effect of weighting.

The following table shows the unweighted sample sizes and the error attributable to sampling that would be expected at the 95% level of confidence for different groups in the survey.

Sample sizes and sampling errors for other subgroups are available upon request. In addition to sampling error, one should bear in mind that question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of opinion polls.

Dispositions and response rates

Secondary data methodology

The analysis in Chapter 4 of this report is based on data from the U.S. Census Bureau’s 2021 and 2022 Survey of Income and Program Participation (SIPP). The population represented in the SIPP is the civilian noninstitutionalized population living in the United States. It is a nationally representative survey that focuses on the income of U.S. households and their participation in government programs. The SIPP also collects extensive data on many factors of economic well-being, and it includes a comprehensive section on fertility history, which we use to identify individuals who have ever had biological children and those who have not.

The SIPP is a longitudinal survey that interviews participants multiple times over a four-year period. Each household in the SIPP is interviewed at yearly intervals over this four-year period. The reference period for the interview questions is the preceding twelve-month calendar year. In this analysis, we use only information relative to the twelfth reference month, as it is the most recent measure.

© Pew Research Center 2024