Should You Run a Fake Door Test, or Test the Concept First?
A fake door test tells a product manager whether people click a button for a feature that doesn't exist yet. It doesn't tell them why those people clicked, what they expected to happen next, or whether they'd actually adopt the feature once it shipped. That's the real cost of building the fake door before testing the underlying decision: design and engineering time goes into an experiment whose single aggregate click rate can't say which audience segment reacted, what expectation drove the click, or whether the signal would hold up against real behavior.
What a fake door click rate actually measures
Fake door testing puts a button, landing page, or menu item in front of users for a feature that hasn't been built, then counts clicks as a proxy for demand (Amplitude). It's cheap relative to shipping the real feature, but it still requires copy, design, tracking instrumentation, and enough live traffic to produce a signal worth trusting (Chameleon). A click is also a single number. It doesn't separate the segment that clicked because the framing matched their workflow from the segment that clicked out of curiosity, and it can't be re-run against a different value proposition without re-instrumenting the whole test.
Running a controlled experiment before you build anything
A causal experiment on the feature concept asks a narrower, more useful question: does a specific framing or variant of this feature change stated adoption intent, and by how much, with a confidence interval attached. That analysis exists before a single line of tracking code goes into the product: no button, no landing page, no live-traffic requirement.
| Fake door click test | Controlled concept experiment | |
|---|---|---|
| What it measures | Aggregate click rate on a live button or page | Causal effect of a specific framing/variant on stated adoption intent, with a confidence interval |
| Segment detail | None by default; requires separate cohort instrumentation | Built into the study design |
| Build cost before running | Design, copy, and tracking instrumentation | None; runs on a described concept |
| Traffic requirement | Needs real visitors to reach significance | Runs on a study population instead |
| What it can't tell you | Why people clicked, or whether they'd follow through | Whether people would actually click a live door, absent a real one to click |
Neither method replaces the other. The controlled experiment tells a team whether a concept is worth the cost of instrumenting a live test at all; the fake door tells them whether real visitors, in the wild, behave the way the concept study predicted.
Reading the result: build, refine, or kill
Three outcomes come out of a concept experiment, and each points to a different next step:
- A clear, consistent effect across the framing tested. The team has a defensible case for spending the design and engineering time on a live fake door, and the study already indicates which framing to lead with.
- A mixed or inconsistent effect. The concept needs a sharper problem statement or a different framing before it's worth instrumenting anything live.
- No meaningful effect, or a negative one. The team can retire the idea before spending a sprint on it.
This is the same build/kill logic a fake door test produces from click data. The difference is where the signal arrives, and that a causal estimate says which specific variant moved the number rather than reporting one blended rate.
Moving from a concept study to a live test
When a concept clears a controlled experiment, the same causal question can be checked with real human participants rather than only a simulated study population. Subconscious can test or validate studies with real human participants. That step doesn't change what's being measured; it changes who is answering.
What this doesn't replace
A controlled concept experiment does not build, host, or instrument a live fake door, landing page, or in-product click funnel, and it does not report actual website click-through analytics. Its output is a causal effect estimate from a controlled study, a different signal than observed click behavior. Treat it as a step that precedes or complements a live fake-door test, not a substitute for one. Review methodology detail on the research page, or see how this fits into a broader evaluation workflow in case studies.
If a feature concept is close to a build decision, the fastest way to see whether a causal concept test changes the call is to book a session and run it against a real feature on the roadmap.
