A Research Operating Model for Matching Evidence to Decision Risk
A research leader should route each question by the cost of being wrong. AI-assisted exploration can surface hypotheses. A controlled experiment on a simulated audience can compare actions directionally. An expensive or public decision may require the same causal question to be validated with real people. Human review governs all three.
Route the question before choosing the method
A plausible answer is not always strong enough to move a pricing, message, or launch decision. The research leader has to decide what evidence would change the action, what a wrong choice would cost, and whether the result will stay inside the team or support an external claim.
AAPOR’s best practices connect survey design to a defined objective, target population, and pretesting. Evidence routing should begin with those study requirements and the decision’s cost of error.
Before selecting a method, state the decision and the evidence sufficient to act. An employment projection does not establish an evidence policy.
What are the three evidence tiers and the review gate?
A practical planning model has four layers: exploration, directional testing, human review, and validation. For decision routing, human review is a gate across three evidence tiers rather than a separate grade of proof.
| Evidence tier | Question it can answer | Appropriate use | What it cannot establish |
|---|---|---|---|
| AI-assisted exploration | What hypotheses, objections, or alternative framings should the team examine? | Defining the question and expanding the option set | Which action will cause the desired behavior |
| Directional causal testing | Which controlled alternative changes simulated audience choice? | Narrowing messages, prices, concepts, or launch options | That the result will replicate with real people or in market |
| Real-human validation | Does the same causal comparison hold with human participants? | Expensive, public, or hard-to-reverse decisions | Automatic proof of market performance |
The gate is constant. A researcher checks the question, audience definition, alternatives, business context, and caveats before any output reaches a stakeholder.
How does Subconscious keep the causal question intact?
Subconscious belongs between idea generation and external proof. It runs controlled causal experiments on simulated audiences to compare actions directionally. The team defines the audience, alternatives, and outcome before the comparison begins. Naming what a directional result cannot do is what lets a buyer check it. The result narrows the action set. It is not a verdict.
When the decision relies on a modeled result, define the matched human population, alternatives, treatment, endpoint and agreement criterion. Confirm recruitment, fieldwork ownership and deliverables with the provider. The study process should record adaptations and remaining limits; a change from stated choice to observed purchases measures a different outcome.
Put the review gate where claims change hands
The human review step does not disappear when the method changes. Research judgment owns question framing, audience definition, neutral alternatives, interpretation, and the threshold for escalation. A fluent answer can become misleading when a deck removes the label that distinguished a hypothesis from a causal comparison or a validated result.
Carry the evidence label into the decision artifact:
- "Hypothesis from AI-assisted exploration: untested."
- "Directional causal comparison: narrows actions, not validated with people."
- "Human agreement test: passed, failed, or inconclusive under the predeclared criterion; task and population recorded."
These labels keep the escalation path visible and prevent confident writing from turning a directional result into proof.
What the operating model cannot delegate
A method's limits belong on the record right next to where it gets used. Subconscious is one tier inside an evidence system. Matched human choice evidence checks the specified task. Clinical outcomes, observed usability and market performance require designs and measurements suited to those endpoints.
The research team still decides which questions need exploratory work, which can use a directional comparison, and which require validation with people. That decision cannot be delegated to the tooling.
Run a one-month routing exercise
Choose a recurring decision and record the minimum meaningful effect, cost of error, and evidence trigger. In an invented pricing example, a rollout risks $200,000 in margin. An optional stated-choice screen requires at least a +2 percentage-point choice contrast; a task-supported interval from −1 to +5 points does not clear that screening threshold. Separately, authorizing rollout requires at least 3% incremental contribution margin in a live purchase pilot, using observed purchase rates, prices and costs against a holdout. A choice interval cannot be compared directly with the margin threshold. Record passed, failed or inconclusive for each criterion and any remaining transport risk. The amounts and thresholds are illustrative.
Use the research record to frame the method and its boundaries. If a current decision does not fit cleanly into one tier, discuss the study design before treating an output as evidence.