Why a Bayesian Marketing Mix Model Still Needs Calibration Before You Reallocate Budget
A Bayesian Marketing Mix Model (MMM) can tell you which channel looks most effective. It cannot tell you, on its own, whether that estimate is trustworthy enough to move tens of millions of dollars. That gap is where reallocation decisions go wrong.
The decision a CMO actually faces
An MMM produces a channel-effectiveness estimate and a recommended budget split. The question in front of a CMO or VP of Marketing Analytics isn't whether to trust the model. It's whether to act on the MMM's output as-is, or require calibration against an incrementality experiment (a holdout or geo-experiment) before shifting spend.
Getting this wrong is expensive in a specific way. If the model's estimated effect for a channel is inflated because spend was allocated in anticipation of demand, or by adstock and saturation assumptions that don't hold in practice, the team reallocates spend toward that channel expecting a lift that isn't there. The shortfall only shows up after the budget has already moved.
What is the structure of a Media Mix Model?
A Bayesian MMM regresses an outcome, typically revenue or conversions, on media spend across channels, plus controls for seasonality, pricing, and other demand drivers. Two structural choices matter most for the estimate:
- Adstock: media spend rarely converts in the week it runs. Adstock functions model the carryover effect of an impression over following weeks.
- Saturation: each channel's return diminishes as spend increases. Saturation functions capture that diminishing return so the model doesn't extrapolate a straight line past the point where it breaks down.
The Bayesian treatment adds priors over these functions and produces a distribution over the channel-effectiveness estimate rather than a single point value. The model's correlation problem is published here so a buyer can weigh the channel-effectiveness estimate correctly. It does not, by itself, resolve the deeper problem: media spend across channels is often correlated, and a correlational model can attribute an effect to the wrong channel when two channels move together.
Why is calibration the step that closes the gap?
Google Research published a methodology for calibrating Bayesian MMM priors against experimental results, using incrementality experiments to constrain what the model is otherwise unable to distinguish from spend patterns alone. The logic is straightforward: a well-designed incrementality experiment (a geo holdout, for example) produces an estimate of a channel's effect at the tested spend level, with its own uncertainty. Feeding that estimate into the MMM as an informed prior narrows the range of channel-effectiveness values the model will accept, instead of leaving it to infer the answer purely from historical correlation.
Marketing measurement practitioners describe the same requirement from the buyer's side: an MMM without calibration is a starting hypothesis, not a validated budget plan.
What calibration does not fix
Naming this limitation here is what lets a buyer check the model before spend moves. MMM remains a correlational, aggregate-level model even after calibration. Calibration reduces the bias that comes from endogenous spend allocation and unconstrained adstock or saturation curves. It does not eliminate it, and it does not turn the model into a causal method on its own. A calibrated MMM is still an estimate built from historical spend and outcome data, refined by whatever experimental evidence was available to anchor it. It does not test a new, specific reallocation before you commit to it.
Where does causal testing fit before the budget moves?
Subconscious runs controlled discrete choice experiments to test how a specific action, such as a candidate budget shift or channel reallocation, changes the choice-relevant behavior it's designed to measure, and compares scenarios with quantified uncertainty rather than a single point estimate. That complements MMM calibration rather than replacing it: calibration improves the model's estimate of what already happened; a configured causal study tests respondent behavior under the specific reallocation being considered, not the resulting aggregate media response.
Where the decision depends on validating a result against real behavior, a team can move from a simulated causal study to a study with real human participants while keeping the same causal question, This limitation sits next to the method's strengths so a buyer can check it before acting. Though internal validity in the simulated study is not evidence of external validity to in-market media response.
Practical checklist before reallocating
- Confirm the MMM's adstock and saturation assumptions were fit against your own channel mix, not a template setup.
- Check whether any channel-effectiveness estimate has been calibrated against an incrementality experiment. If not, treat it as a hypothesis.
- Before moving spend on the strength of an uncalibrated estimate, test the specific reallocation as its own causal question, ideally as a holdout or geo-experiment; a configured discrete choice study can test the underlying behavioral assumptions but does not substitute for market-level incrementality evidence, rather than reading the MMM output as a finished answer.
Testing a specific budget reallocation as a controlled causal question, rather than reading it directly off an MMM, is one way to see whether the estimate holds before spend moves. Learn how Subconscious runs causal studies.