Which Validation Check Should a Product Manager Run Before Engineering Starts?
Most teams engage with only 6% of the features that ship, according to product-usage benchmark data from Mind the Product. Before a feature reaches that fate, a product manager makes a validation decision: run a quick simulated read, put a prototype in front of real users, or run a controlled experiment comparing variants. Picking the wrong check is how engineering time lands on a feature nobody uses.
Three checks, three different questions
A simulated read on a concept, prototype, or feature description surfaces early objections cheaply, before a final design exists. A check earns trust when its limits are stated plainly. It tells a PM what stands out or confuses, not what a buyer will actually choose when the alternative is doing nothing.
A moderated or unmoderated usability session with recruited users tells a PM whether people can complete a task and where they get stuck. This method's limits belong on the record next to its uses. It does not isolate which of two competing feature variants, price points, or messages caused a change in behavior.
A controlled causal experiment puts two or more variants in front of the same target audience under the same conditions and estimates which one moves the outcome. It answers the question the other two leave open: which specific change caused the difference.
Matching the check to the decision
| Decision in front of the PM | Best-suited check | What it cannot tell you |
|---|---|---|
| Does this concept land, or confuse people, before a prototype exists? | Simulated read on a concept or PRD excerpt | Which variant a buyer would actually pick |
| Can a real user complete the flow without help? | Usability session with recruited participants | Which price, framing, or feature variant drives the behavior change |
| Which of two pricing tiers, feature variants, or messages actually changes buyer choice? | Controlled causal experiment across variants | Whether the interface itself is confusing, a usability question, not a causal one |
The first two checks are exploratory: fast, cheap, and good for narrowing options. The third is confirmatory: slower to set up, and built for the moment a PM has to commit engineering time to one variant over another.
When does a controlled experiment change the launch decision?
New Age Floral used a series of controlled experiments to test pricing before a launch decision. Across five iterative experiments with consumer behavior modeled across 125 participants, the research identified $60 as the price that maintained market share, with sales expected to drop off above $100, and avoided an estimated $65,000 in traditional research cost in the process (case study). Finta ran a similar pricing test and moved to a $99-per-month or $950-per-year structure, closing five new customers at the new price point.
Subconscious runs this kind of test as randomized experiments on a simulation of the target market, validated against real human behavior. It answers "which action" questions: which price, which feature variant, which message changes what a buyer chooses. It fits once exploratory or qualitative work has narrowed the field to a short list of variants, when the remaining question is which one to ship.
What does a causal read not do?
Naming a method's limits is what lets a buyer check it before relying on it. A simulated experiment does not replace watching a real person try to use the product, nor does it run a recruitable panel of human participants from an audience graph. When a launch decision is high-stakes, run the simulated comparison first, then validate with real human participants before committing, without changing the underlying causal question (approved methodology).
Before the ticket goes to engineering
A concept that has not been shaped yet needs a fast exploratory read. A flow that might confuse people needs real users watching it. A feature variant, price, or message that will only ship once needs a controlled comparison. Talk to Subconscious about setting up that comparison before the engineering ticket gets written.
