AI Personas or Causal Experiments: Where Research Agencies Should Use Each

Edited by Avi Yashchin, CEO and Co-Founder · August 7, 2026

Research agency and consultancy leaders face one recurring decision: which engagement stage can run on an AI-driven method, and which requires a controlled experiment or real-participant research before a client acts on it. Get that sequencing wrong and a directional AI read gets presented as a defensible finding, the failure mode that costs agencies client trust.

Why this decision matters

Clients now know some research work can move faster than a traditional agency timeline. Agencies that show exactly where that speed comes from, without diluting rigor, win the engagement; agencies that overpromise what a fast method can prove, or under-deliver on turnaround, don't.

The risk sits at the handoff point. A hypothesis from an open-ended AI persona conversation narrows what to test. It is not a finding a client can act on with confidence: a conversation with a simulated respondent does not tell you which of two alternatives would actually change behavior. Presenting the first as the second damages credibility under scrutiny.

What causes the failure

The failure is not "using AI in research." It is skipping the step that turns a directional impression into a causal answer. Traditional qualitative work already has this two-step structure: a handful of interviews surface themes, and a larger validated study confirms which theme actually predicts the behavior. Open-ended AI persona sessions suit the first step. They were designed to generate plausible responses in conversation, not to compare defined alternatives under controlled conditions and report which one moved the outcome.

Where each method fits in an engagement

The pattern: anything that shapes what to test can run on fast, exploratory methods; anything the client will use to commit budget or defend a strategy decision needs a method built to compare alternatives and report a result.

Evidence: two different questions, two different tools

An open-ended AI persona session answers "what does this simulated respondent say when asked?" That is a generative, conversational output, useful for hypothesis discovery in the same way early qualitative work is.

A controlled discrete-choice experiment answers a different question: "when a defined population is exposed to alternative A versus alternative B, which one they choose, and with what confidence." Subconscious runs this second kind of study: a controlled experiment comparing defined alternatives across a defined population, returning causal effects with confidence intervals, for the stages of an engagement that need a defensible, client-ready result rather than a directional read.

The distinction is not about which tool sounds more sophisticated. It is about which question was actually asked.

Recommended decision process

1. Name the client decision the deliverable supports, and what it costs the agency if the finding turns out wrong under scrutiny.
2. If the deliverable narrows hypotheses or pre-tests an instrument, an open-ended AI persona exploration is appropriate.
3. If the deliverable is the finding a client acts on, budget for a controlled experiment or recruited real-participant validation before it ships.
4. State explicitly, in the deliverable, which category it falls into: a client who knows a finding is directional will not mistake it for validated.
5. When a directional read and a validated study disagree, trust the validated study and say so.

Where Subconscious fits

For the stages of an engagement that need a defensible, client-ready result, Subconscious runs controlled discrete-choice experiments that compare defined alternatives across a defined population and return causal effects with confidence intervals, per the replication paper. That fits engagements where a recommendation has to survive a client's own stakeholders asking "how do you know."

An agency can also move from a simulated experiment to real-human validation without changing the underlying causal question, which matters when a client wants the same comparison confirmed with recruited participants before a launch decision. See how Subconscious structures a study and examples of engagements that used a validated result.

Limitations and failure conditions

A controlled causal experiment does not replace moderated qualitative interviews, recruited real-participant fieldwork, or the interpretive and strategic expertise an agency brings to a client relationship. It answers which alternative moves a defined outcome; it does not run a discovery conversation, do the strategic synthesis a client is paying for, or offer open-ended persona chat sessions.

Audience reach and recruited human validation are also not the same claim. The scale of a simulated population an experiment can run against is a different fact from how many real participants were recruited into a study, and an agency reporting either should be precise about which one it is describing.

If the underlying research question is really about generating early-stage hypotheses, cheaply and quickly, a controlled experiment is the wrong tool for that job too. Matching the method to the actual decision, in both directions, is the discipline this whole question depends on.

What clients ask, and how to answer honestly

Clients increasingly ask an agency directly whether a deliverable is "real research or AI." The honest answer names which method produced the finding and what that method can and cannot support. A client who hears "this stage used a fast exploratory method to shape the questions, and this stage used a controlled comparison to validate the answer" trusts the agency more than one who hears a blended answer that avoids the distinction. That value, interpretation, judgment about which questions matter, accountability for the recommendation, does not disappear when part of the work runs faster. It is what keeps a client paying for the engagement instead of running the tool themselves.