Simulated vs Recruited Research: Choose the Evidence Your Decision Needs
Choose the method by the consequence of being wrong. Use a simulated experiment to compare actions and narrow a directional question. Use real-human research when a consequential decision needs human validation, documented sampling, or observed behavior. When the stakes justify both, carry the same causal question from simulation into a human study.
Match the method to the claim
The useful distinction is the claim each method can support, not simulated versus real.
| Research path | Best matched decision | What it can establish | What it cannot establish alone |
|---|---|---|---|
| Simulated experiment | Which concept, message, audience, or action deserves further investment | Directional differences between controlled alternatives | How real people will behave in market |
| Recruited human study | Whether a consequential launch, price, or public claim holds with real participants | Human responses with a documented sample | Revealed behavior unless the study observes a real action |
| Simulation followed by human validation | Which action to take when the decision is both exploratory and consequential | A narrowed causal question followed by a human check | Automatic proof of market performance |
A recruited survey still records stated answers. It becomes behavioral evidence only when the design observes a click, purchase, abandonment, or another real action. Stated preferences can diverge from revealed choices, so a simulated answer should not be presented as observed behavior. The study compares stated and revealed preferences directly.
Use simulation to discriminate between actions
A simulated experiment compares controlled alternatives before committing capital: concept rounds, message variants, audience exploration, competitive framing, pre-research scoping.
The practical advantage is discrimination. A randomized experiment holds the decision context steady, varies an action, and estimates which alternative changes the outcome. That beats asking a modeled buyer for an unstructured opinion.
Hard-to-reach buyer roles can still be represented in an early experiment, but representation is not recruitment: a modeled senior buyer helps form and test a hypothesis, not prove one participated.
Do not treat recruitment as a truth certificate
Real participants matter, but the recruitment label does not guarantee clean evidence. Nonprobability surveys face documented risks from fraudulent respondents, bots, inattentive answers, and professional survey-taking. NORC's literature review summarizes these quality threats and the controls researchers use to detect them.
Judge a recruited study by its sample, screening, quality controls, instrument, and connection to the decision. Use it when:
- A single launch, pricing, or positioning call is expensive to reverse.
- The market context is novel or has changed enough that prior patterns may not hold.
- The result will become a public statistic and needs documented human sampling.
- The study can connect stated reasons to analytics, sales data, or an observed action.
A recruited study is not ground truth when its sample or instrument is weak.
Carry one causal question through both stages
The strongest sequence does not run two unrelated studies. A simulated experiment sharpens the alternatives, outcome, and target population, then hands that same causal question to a human validation study.
Subconscious supports controlled causal experiments on a simulation of the market and can validate them with real human participants. The question carries over unchanged, so a decision-maker can compare the simulated result with a human baseline instead of reconciling different instruments.
The replication record provides a defined benchmark for that comparison. Subconscious reports 93% replication accuracy against real human study outcomes across 350+ published studies in 20+ domains. Replication accuracy does not guarantee a new study will predict market performance, and it does not remove the need for decision-specific validation. Read the paper and its methodology.
Subconscious can also run controlled studies against a person-level audience graph covering 800 million real people. That figure describes population definition and audience reach, not a recruitable pool of study participants.
Set the evidence gate before the study starts
Before choosing a method, write down the action, the outcome, and the evidence required to release capital.
- For a reversible directional choice, run the controlled simulated comparison and state its boundary.
- For a hard-to-reverse decision, require real-human validation of the same causal question.
- For a behavioral claim, observe behavior rather than treating stated preference as a proxy.
- For a public number, require documented sampling and methods.
- For either path, define failure conditions before seeing the result.
The leaderboard is useful when replication evidence and failure modes matter to the procurement decision. If the decision is already defined, bring the action, alternatives, and required evidence to a working session.