When to Run a Simulated Study Before Recruiting Real Research Participants
A head of product or research lead sequencing a roadmap or design decision faces two bad defaults: skip any directional check and ship on an untested assumption, or recruit real participants for a question a faster experiment could have narrowed first. Run a controlled simulated study before recruiting when the goal is to sharpen the question, not to answer it.
The cost of getting the sequence wrong
Recruiting, scheduling, and running a study with real participants is slow. Teams that skip a directional check first often build a feature or flow around an assumption nobody tested, then find the problem after engineering and design time is spent. Teams that recruit before sharpening the question often spend that time on a broad or poorly framed study, then have to recruit again once the real question becomes clear.
Neither failure is about research effort; both are about sequencing: which check happens first, and whether it changes what gets asked next.
What a simulated study can and cannot do
A simulated study runs a controlled experiment on a decision, using a modeled population instead of recruited humans. It can compare which of several concepts or flows changes stated intent for a target segment, and surface where that response is likely to diverge across segments.
What people say they would do and what they actually do under real conditions are different questions; a simulated study answers the first one. Independent reviewers of AI-generated research participants draw the same line: participant simulation can approximate a stated response, but it does not substitute for watching how someone actually uses a product (Nielsen Norman Group, "Synthetic Users: AI 'Participants'").
Where each method fits
| Method | Question it answers | What it cannot answer |
|---|---|---|
| Simulated study | Which concept or flow shifts stated intent for a segment, and where that response diverges across segments | What a person actually does when using the product |
| Usability testing | How a person actually behaves when using the product | Whether a different concept would have changed their intent |
| Ethnographic interview | Why a behavior happens in its real context | Whether a proposed change moves the outcome at scale |
Running a simulated study first does not remove the need for the other two methods. It changes what the team asks once real participants are in the room.
A sequence that keeps the causal question intact
- Define the decision: which action or flow, for which target segment, is being compared.
- Run a controlled experiment on that decision in a simulation before recruiting anyone.
- Use the result to sharpen, not replace, the study plan for real participants: which segments to prioritize, which alternatives are worth a moderated session, and which questions are already answered.
- Move the same causal question into real-human validation. Subconscious tests or validates studies with real participants, so the team carries the decision forward instead of redesigning the study from scratch.
This sequence matters most when the target segment is expensive or slow to reach: a narrow B2B buyer role, a regulated population, or a segment spread across geographies. In those cases, a controlled study against a person-level audience graph covering 800 million real people can narrow the question before a recruiting budget is spent.
Limitations and failure conditions
A simulated study estimates causal effects on stated choices and intent; it is not a substitute for usability testing or ethnographic interviews. Treat results as directional until validated with real participants, especially where the gap between what someone says and what they do is the entire risk being managed. Real-human validation confirms or corrects the simulated answer; it does not turn a stated-intent test into an observed usability session, a clinical trial, or automatic proof of market performance.
Practical next step
If the target segment is reachable and the decision is narrow, a moderated study alone may be enough. If the segment is hard to reach, the question is still broad, or the team cannot afford to recruit twice, run the controlled simulated study first, then carry the same question into real-human validation. Teams weighing this sequence against a specific roadmap or launch decision can review prior case evidence, read the research methodology, or book time to walk through a live decision.
