Real Panel or Causal Experiment First: Deciding Before You Field a Survey
A consumer insights or growth marketing lead facing a concept, message, or feature question has two starting moves: recruit a real-consumer panel and field a survey, or run a controlled experiment first to see which comparisons are worth fielding. The second move exists because the first is expensive to get wrong.
What a consumer panel answers
Pollfish is a consumer survey panel platform. A buyer defines a target audience by demographic and behavioral criteria, and Pollfish's network of mobile apps reaches those respondents. That workflow returns real human answers to the exact question fielded, recruited from an existing operational pipeline.
The strength of a real panel is also its constraint. Once a survey is written and fielded, changing the question means re-fielding it. Each round trip carries real recruitment cost and real calendar time. That is fine when the question is already scoped. It is expensive when the question is still being worked out.
The cost of guessing wrong before you field
There are two ways to lose money on this decision. Field a full panel survey on a poorly scoped question, and the budget produced an answer to the wrong question. Or skip human validation on a decision that needs verified respondent behavior, and the business ships an unconfirmed claim. Neither failure is about the panel vendor's quality. Both are about sequencing: what gets tested before a team commits real recruitment budget.
Narrowing the question before you recruit anyone
Subconscious runs randomized experiments on a simulation of your market, validated against real human behavior, to show why people choose and which action drives the outcome. Run ahead of a fielded panel study, that experiment does one job: compare candidate messages, concepts, or features against each other and estimate which move the outcome, before any real respondent is recruited.
That narrows; it does not substitute for the panel. Subconscious can test or validate studies with real human participants, so a team can move from the simulated comparison to a real-human check on the same question without redesigning the study. The practical advantage: panel budget goes toward the two or three comparisons that survived the first pass, not the full list of things someone in the room thought might matter.
Comparing the two starting points
| Panel-first (e.g. Pollfish) | Experiment-first (Subconscious) | |
|---|---|---|
| Respondent source | Real consumers, recruited per study | Simulated market, no recruitment |
| What it answers | What this specific fielded question returns from real people | Which of several candidate actions is worth fielding |
| Cost of a revised question | A new recruitment and fielding round | Re-running the comparison in the same simulation |
| Where it's strongest | The question is already scoped and the decision needs verified human sample data | The question set is still being narrowed |
| Where it falls short | Iterating on an unscoped question gets expensive fast | Does not itself recruit or field real respondents |
Where the two combine
A team can run the causal experiment first to narrow ten candidate messages or concepts down to the two or three that show a real effect, then field only those on a real panel like Pollfish for the confirmation the decision requires. That sequencing keeps panel spend attached to comparisons that already showed a signal, not spread across every idea in the room.
Where this breaks down
Subconscious does not recruit or field real consumer respondents, and it does not replace a real-panel survey when the decision requires verified human sample data, not just a directional estimate. The research library documents where simulated results have and have not replicated human outcomes; that scope, not a general claim of interchangeability, should guide the call.
Making the call
Before recruiting anyone, ask which constraint binds: an unscoped question that needs narrowing, or a scoped question that needs a verified human answer. The first calls for an experiment first. The second calls for fielding directly. Most research programs need both moves, in that order, more often than they need to pick one tool and stop.
Check current leaderboard replication results before weighting a simulated comparison, or book time to talk through where a given question falls.