How to Use Synthetic Audiences Without Losing Credibility
Credibility requires clear disclosure and evidence appropriate to the decision. Before a synthetic-audience study runs, identify the question, response source, intended use, and validation criteria. A generated preference, a recruited hypothetical choice, and an observed action must be described accurately.
Agree on the evidence required for the proposed action before reading the result. Exploration, controlled comparisons and independent validation have different purposes; a public descriptive claim may instead rely on verified records or a cited source.
Why the disclosure question is not optional
Synthetic and AI-assisted methods now sit inside ordinary research workflows: hypothesis generation, first-pass analysis, message screening. That shift raises the exact question a compliance-minded buyer has to answer before adopting any of it: what gets disclosed, and to whom.
The 2025 ICC/Esomar International Code emphasizes disclosure of methods, data sources and limitations, fit-for-purpose research, and human oversight. Use that standard alongside the requirements applicable to the actual study; a tier label alone does not establish scientific validity.
Should the gate be set before the study or after the finding?
The gate is not a judgment call made once a result looks convincing. It is a decision made in advance, based on what the finding will be used for:
- Exploration only. Hypothesis generation, objection mapping, early message screening. No launch, pricing, or public claim depends on the result alone.
- Directional comparison. Ranking concepts, framing options, or positioning statements against each other to narrow a shortlist before a more expensive study.
- Decision-specific evidence. Specify the source and method required for the intended claim. Use an identified comparison for an intervention effect, verified data for a description, and relevant human or live evidence when the decision endpoint or evidence requirements demand it.
Write the tier down before the study runs. A team that decides the tier after seeing the result will tend to upgrade a convenient finding to "good enough to act on."
How do you run a directional comparison?
Exploration can use open-ended prompts to suggest hypotheses. A controlled directional comparison requires a specified design, assignment and response measure.
Define eligible profiles and their grounding sources. Hold nuisance wording and information conditions stable while changing the intended alternative. Randomize assignment to the intervention and baseline, or state the assumptions of another justified design. Record model/prompt versions, context handling, repeated-response dependence and uncertainty; constant wording alone does not guarantee attribution.
For a Subconscious comparison, agree on the generated-choice endpoint, design, estimator and uncertainty method. It estimates a response contrast within those conditions rather than automatically establishing a market effect.
Why put a human between the output and the claim?
A synthetic comparison produces output. Deciding whether that output is valid evidence for the decision in front of the team is a human step, not an automatic one.
Review the raw response trace, population source and coverage, assignment and instrument, model/prompt versions, estimator, uncertainty, contrary results, and independent validation. A human review is useful when those checks are concrete and recorded.
Validate the intended response
A generated study cannot directly observe actual customer behavior. Additional evidence depends on the endpoint, coverage, cost of error and any applicable study requirements. Verified descriptive data may support a public description; a consequential causal or market claim needs evidence that supports that claim.
If generated and human results will be compared, align eligibility, alternatives, information, assignment and endpoint where appropriate. Agree on fielding scope and acceptance criteria separately, then record differences. A hypothetical human-choice task and an observed purchase answer different questions.
Include inconclusive and contrary findings in the study deliverable. Independent validation can confirm, contradict or remain unresolved. Decide beforehand how each outcome affects the action, and do not present human hypothetical choices as automatic proof of market performance.
Disclose the tier in the deliverable
Disclose whether responses were generated, recruited, or observed; the AI role and model/provider/version; prompt and run records; audience sources and coverage; endpoint; estimator and uncertainty assumptions; and independent validation actually completed. Hiding the source is one failure mode. Transparent labeling does not repair bias, poor identification or an endpoint mismatch.
For a hypothetical concept screen, a disclosure might read: "Selections were generated for described small-business purchasing leads; no participants were recruited. Model, prompt and run identifiers accompany the methods record. The endpoint was choice among the shown concepts. Reported variation is conditional on this generated setup; no independent human or sales validation was completed. Use the screen to nominate concepts for further study, not forecast conversion."
A starting checklist
Before a synthetic-audience study begins, confirm:
- The business decision the finding will inform, written in one sentence.
- The question type, response source, intended use, and predeclared acceptance criteria.
- Eligibility, population provenance, coverage gaps, instrument, assignment and model/prompt records.
- The owner who checks source fidelity, design, analysis and contrary findings.
- The independent human or live comparison needed, if any, its endpoint, fielding owner, and action if validation fails.
Bring the intended action, grounding sources, response measure, and disclosure requirements to a study review. Review aggregate method evidence separately from validation of the proposed study.