Skip to main content

How to think about Base Sizes and Statistical Significance

Jane Franco avatar
Written by Jane Franco
Updated over 3 weeks ago

Base size refers to the number of participants included in your study or analysis. Choosing the right base size helps ensure your findings are credible, interpretable, and appropriate for your research goals.

This article explains:

  • What a base size is

  • How recruitment affects base size

  • How many participants to use for different study types

  • How to think about base sizes for concept testing

  • When statistical significance is relevant


What is a base size?

Your base size is the number of participants contributing data to a result, insight, or metric.

You need at least 5 interviews to begin generating insights.

The right base size depends entirely on what you’re trying to learn.


How to think about your base size and recruitment

Your ability to reach a given base size depends heavily on how easy the audience is to recruit.

Niche or low-incidence audiences

Higher base sizes may be difficult or impractical.

Examples:

  • Ages 18–24 living in the US; people who purchased a car in Ohio; customers who bought a Kia at a specific dealership

In these cases, smaller base sizes may still be appropriate especially for qualitative research.

Broad or high-incidence audiences

Larger base sizes are usually easier to achieve.

Example:

  • Past 3‑month candy eaters (non‑chocolate)

Broad audiences make a study more feasible and efficient to to field.


What influences sample size?

Several factors affect how large your base size should be, including:

  • Research objectives

  • Audience incidence (IR)

  • Number of segments

  • How you want to cut the data


How many participants do Outset studies usually have?

It depends entirely on the study type and research objective.

Across Outset, we see studies ranging from 5 to 1,000+ participants, but most commonly we see 20–50 participants, depending on audience incidence and methodology.


Sample size guidelines

Use the guidelines below to choose a base size based on what you want to learn.

Outset Qualitative Research

Where the interview has a majority of conversational questions, less quantitative questions.

Research purpose

Typical base size

What this supports

Early exploration

20+ total participants

Initial themes, language, and hypothesis generation

Theme saturation

30+ total participants

Core themes start to recur

Pattern confidence

40-50 total participants

Emerging confidence in relative prominence and consistency of themes

How to choose: Use theme saturation when you want to know what themes exist. Use pattern confidence when you want to understand which themes appear more often than others.


Outset Quantitative research (directional)

Where the interview has a majority of quantitative questions (Rating, Stack Rank, Multiple Choice).

Research purpose

Recommended base size

What this supports

Directional quantification – standard AI testing & usability (NOT concept testing)

20–30+ total participants

Directional interpretation of quantitative metrics

Directional quantification – concept testing

10–20 per concept

Directional comparison of concepts without statistical certainty


Outset Quantitative research (statistical significance)

Where the interview has a majority of quantitative questions (Rating, Stack Rank, Multiple Choice).

Research purpose

Recommended base size

What this supports

Statistical quantification of quant metrics only (NOT concept testing)

50+ total participants

*recommended if audience is broad (high incidence) but not if audience is niche (low incidence)

Statistically significant data

Statistical quantification of quant metrics only – concept tests

50+ per concept

*recommended if audience is broad (high incidence) but not if audience is niche (low incidence)

Statistically grounded comparison of concepts


How should I think about base sizes for concept testing?

Concept testing base size depends on:

  • The number of concepts being tested

  • Whether concepts are tested monadically or sequentially

Key definitions

  • Monadic testing: Each participant sees only one concept

  • Sequential testing: Participants see multiple concepts in sequence


What do I need for statistical significance / analysis?

Statistical analysis typically applies to quantitative questions / analysis.

Statistical analysis does not typically apply to qualitative themes or open-ended responses.

As a rule of thumb:

  • Plan for 50+ total participants (or 50+ per concept) for basic statistical significance / analysis


Final note

These guidelines are intended to support planning and interpretation, not replace internal research standards. We recommend aligning base size decisions with your internal practices and research goals whenever possible.


These guidelines are meant to support planning and interpretation, not replace your internal standards. Always align base size decisions with your research goals and audience constraints.

Hope this helps! If you have any further questions, please reach out to our team at [email protected] or via chat.

Did this answer your question?