When a Flat Choice Model Misrepresents Cannibalization
On this page
A consumer-goods pricing or revenue-growth leader planning a new product launch needs to know one thing before committing trade spend: will this product mostly take share from close substitutes in its own segment, or will it pull incremental volume from across the whole category? Get that wrong and the launch budget goes to the wrong shelf.
The proportional-substitution trap
A basic discrete choice model treats sales as the outcome of many individual purchase decisions. Each product has a utility, driven by attributes like price and shelf availability (All Commodity Volume, or ACV). Utilities convert to purchase probabilities through a softmax function that includes an outside (no-purchase) option, and the model estimates each attribute's association with sales volume.
p_i = \operatorname{softmax}(u_i) = \frac{\exp(u_i)}{\sum_{j} \exp(u_j)}Forward simulations from a fitted choice model support causal interpretation only when identification assumptions are justified. In observational sales data, price and availability can correlate with unobserved demand shocks, so changing them in a prediction does not by itself identify an intervention effect.
The trap is in what the flat version of this model assumes about the error term, the catch-all for unobserved reasons a customer picks one product over another (a coupon, a craving). A flat multinomial logit model assumes those unobserved preferences are independent across products. The consequence is mechanical: when a new product enters, the model predicts every existing product loses market share in exact proportion to its starting share.
If buyers substitute more strongly among kids' toothpaste than between kids' and adult products, proportional substitution can misrepresent the launch. The direction and size of the error need category evidence; a flat model is not necessarily wrong for every choice set.
How does nesting products by segment change the substitution prediction?
A nested (hierarchical) logit model addresses this by grouping products into nests, typically by segment or brand, that are meant to capture the unobserved factors driving substitution. The nested logit model computes the overall choice probability in two steps: the probability of selecting a nest, then the conditional probability of selecting a specific product within that nest.
P(\text{choose product}) = P(\text{choose nest}) \cdot P(\text{choose product} \mid \text{nest})An estimated within-nest scale can concentrate substitution among nest members. At the flat-logit limit, proportional substitution returns. Whether the proposed nests improve prediction should be checked against relevant holdouts and substitution evidence.
Why does the nest structure have to be chosen, not assumed?
The nested model's realism depends on getting the nest structure right. An analyst has to decide which grouping (segment, brand, sub-brand, or some hierarchy of these) reflects how consumers actually substitute; an unsuitable grouping can misrepresent cannibalization along a different axis. Building this kind of hierarchical structure to arbitrary depth while keeping estimation numerically stable is itself a nontrivial part of the modeling work, detailed in PyMC Marketing's documentation on nested logit and non-proportional substitution patterns (PyMC Marketing, "Nested Logit and Non-Proportional Patterns of Substitution").
Nest choice is a hypothesis requiring validation. An updated portfolio matrix can be useful when its data, category structure, uncertainty, diagnostics and drift monitoring are maintained. The methods example does not establish that Subconscious supplies an automated, always-current portfolio matrix. See Sinha's empirical cannibalization study.
Where does this fit in a launch decision?
Subconscious's causal experimentation can compare launch alternatives: test a proposed product's positioning, price, or segment against holdout conditions and estimate directional causal effects for a defined population before committing capital. A segment-aware substitution matrix answers a narrower question than a general launch comparison, so it should be scoped as its own study design, with the nest structure proposed and checked against category knowledge, rather than treated as a standing report a platform generates automatically.
For launch confirmation, specify the matched human choice or market substitution endpoint, audience, alternatives and agreement criterion. Confirm recruitment, data access, fieldwork ownership and deliverables with the provider. Review the study workflow when defining that scope.
Limitations and failure conditions
- Multinomial logit imposes independence-of-irrelevant-alternatives and proportional substitution assumptions. Check them against the relevant choice sets and held-out substitution patterns.
- Nested logit's substitution behavior depends on the grouping, scale and fitted parameters. Compare proposed structures with held-out substitution evidence; an unsuitable hierarchy can leave substantial forecast error.
- Flat and nested choice models can use individual purchase records or experimental choices as well as aggregates. This sales example uses product-level aggregates; the PyMC-Marketing notebook illustrates individual choices and discusses omitted heterogeneity.
- A regularly updated matrix needs maintained choice sets, relevant data, uncertainty and drift checks; specify whether that deliverable is included in the proposed study.
Next step
Use the public research record for aggregate choice-parameter-rank evidence. To compare a proposed launch's segment fit and price, scope the audience, choice set and substitution validation. Applied cases provide context rather than a live comparison interface.
