Which Causal Method Fits Your Data When a Randomized Trial Isn't Possible
A marketing or analytics leader wants to know whether a pricing change, a campaign, or a launch actually caused a shift in customer behavior. Randomized controlled trials give the cleanest answer, but a live GTM decision often can't wait for one, and sometimes randomization isn't available at all. When a team reaches for existing observational data instead, the question becomes which method fits the data shape in front of them, and what happens if the identifying assumption behind that method silently fails.
Reading a causal effect out of data you already have
Causal claims are strongest when they come from randomization: differences in outcomes then reflect the treatment, not some hidden confounder that varies between test and control groups. In practice, randomization is frequently out of reach.
Two examples make the constraint concrete:
- TV advertising: nobody can flip a coin to decide which households watch a given linear TV spot, but a business still needs to know what that spot did to customer behavior.
- Environmental proximity: households cannot be randomly assigned distance from an industrial facility, yet the causal health consequence of that proximity is a real policy question.
Quasi-experimental methods exist for exactly this gap. When their identifying assumptions hold, they support a causal claim from observational data without an experimental randomization step. CausalPy, an open-source Python package, packages four of these designs into one Bayesian API.
Four designs, four data shapes
Each method below applies to a different data structure. Picking the wrong one doesn't produce an error: it produces a plausible-looking estimate resting on an assumption your data can't support.
Synthetic control applies when multiple units exist and only one receives the treatment. It blends the untreated units into a single weighted composite standing in for the missing counterfactual, then measures the treated unit against that composite once the intervention begins. The gap is the estimated effect. Typical use: evaluating a change rolled out in one region, country, or business unit while others were left alone.
Interrupted time series applies when only one unit was ever treated, all you have on it is a running series of measurements over time, and there's no other group to compare against. It fits the pre-intervention trend, extrapolates it forward as the counterfactual, and compares that projection to what actually happened after the intervention. Typical use: assessing a policy change, product launch, or platform update against one tracked metric over time.
Difference in differences fits situations with one before-and-after reading on each side: a group that got the treatment and a group that didn't. It takes the before-to-after change within each group, then credits the treatment with whatever separates the two. Typical use: evaluating a program when pre/post data exists for both treated and untreated populations.
Regression discontinuity applies when treatment is assigned by a threshold on a continuous running variable, often something other than time, such as an age or score cutoff. A sharp break in the outcome right at the threshold counts as evidence of a causal effect, because confounders that vary smoothly across the running variable can't plausibly produce a sharp jump. Typical use: an eligibility rule or policy line drawn at a cutoff, whether that's an age limit, a passing test score, an income ceiling, or a geographic border.
A published example of this last design: Carpenter and Dobkin's study of the US legal drinking age used regression discontinuity to estimate the effect of turning 21 on all-cause mortality, finding a discontinuity in the mortality rate right at the threshold (NBER working paper; published version, American Economic Journal: Applied Economics). The logic: no treatment was randomly assigned, but if mortality breaks sharply exactly at age 21 and nowhere else, a smoothly varying confounder can't explain it.
Where the assumption isn't automatically checked
None of these four methods verify their own assumption automatically. Synthetic control assumes the untreated units can be weighted into a valid counterfactual, checked with in-space and in-time placebo tests. Interrupted time series assumes the pre-intervention trend would have continued unchanged, checked with placebo-date tests. Difference in differences assumes parallel trends between groups absent the treatment, checked with pre-trend and event-study tests. Regression discontinuity assumes nothing else changes discontinuously at the same threshold, and that units can't sort precisely around the cutoff, checked with a McCrary density test. Each diagnostic can catch a broken assumption, but none of them run on their own: skip the check, and the method still returns a confident-looking estimate that attributes an unrelated shift to the action you took, wasting the budget decided on that estimate.
Running the controlled experiment instead
Where quasi-experimental methods extract a causal estimate from data that already exists, Subconscious runs the controlled experiment upfront: pricing, messaging, or launch actions get tested against a person-level audience graph before the decision ships. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people: that's audience reach for a controlled study, not a recruitable panel of 800 million participants.
When the causal question calls for it, the same study design can move from simulation to real-human validation. That matters because a quasi-experimental analysis estimates a causal effect on realized market behavior, while a designed choice experiment estimates a causal effect on stated preference within the choice task itself, and stated preference tends to run high relative to realized behavior (hypothetical bias). The two answer different questions with different risk profiles.
What this doesn't replace
Subconscious does not implement synthetic control, interrupted time series, difference-in-differences, or regression discontinuity as product features, and moving to real-human validation doesn't turn a controlled causal test into a clinical trial, a usability session, or an automatic guarantee of market performance. Quasi-experimental methods remain the right tool when the data already exists and no new study is possible. The choice in front of a buyer is whether that's the situation they're actually in, or whether the decision is big enough to justify running a controlled test before committing the budget.
Where to go next
If you're deciding between reading a causal effect out of existing data and running a controlled test before you commit budget, Subconscious's research covers how the controlled-study approach works, and how we work covers what a study looks like end to end.
