Models for calculating preference shares
A senior buyer evaluating a simulated market-share forecast needs to know which model turned utilities into shares before greenlighting a launch. Four aggregation rules do this job: First Choice, Share of Preference (logit), Randomized First Choice, and Purchase Likelihood, the menu Sawtooth's Lighthouse Studio built its category around. Picking among them is a second-order decision: it changes how already-estimated utilities get aggregated, not whether those utilities reflect real choice behavior. The first-order question is whether the utilities behind the forecast came from a randomized choice experiment, or from an unvalidated survey or LLM standing in for one. Pick the wrong aggregation rule and the ranking of alternatives shifts a little. Feed any of them unvalidated utilities and the output is a confident-looking guess, not a causal estimate of market behavior.
- The four standard preference-share models (First Choice, logit Share of Preference, Randomized First Choice, Purchase Likelihood) all aggregate individual-level utilities; which one you pick matters only after those utilities are validated.
- Randomized First Choice was built specifically to fix the red-bus/blue-bus failure of flat logit models and outperformed rival share rules in Sawtooth's own 2016 holdout competition, a vendor-run test on Sawtooth's own 21 holdout tasks (Sawtooth Software, Randomized First Choice).
- Utilities almost always come from Hierarchical Bayes estimation on discrete choice, Mixed Logit, or ICLV data today; those are estimators, not causal methods, so causal identification depends on the randomized manipulation in the underlying experiment, not the estimator.
- LLM-generated synthetic respondents skip the randomized experiment step entirely, so their utilities carry no causal identification no matter how sophisticated the downstream share model is.
- On Subconscious's causal-fidelity benchmark, the best-performing configuration reaches 87% of the measured human ceiling in one study (0.832 against a human-to-human ceiling of 0.959); the mean across the 43 studies that pass design filters is 0.73 (causal-fidelity paper).
What are the models for calculating preference shares?
The four models in common use all take the same input, a set of individual-level part-worth utilities, and differ only in how they turn those utilities into predicted market shares. First Choice assigns each simulated respondent's entire share to whichever alternative has the highest utility, a winner-take-all rule. Share of Preference, the standard logit form, splits each respondent's share probabilistically across alternatives in proportion to relative utility, which makes it fast but binds it to the independence-of-irrelevant-alternatives (IIA) assumption. Randomized First Choice adds simulated random error to each respondent's utilities before running repeated first-choice draws, which relaxes IIA at the cost of more computation. Purchase Likelihood scores each product independently on a purchase-probability scale rather than forcing a fixed pool of share across the competitive set. In Sawtooth's own 2016 modeling competition, a vendor-run test using 21 of its own out-of-sample holdout tasks of varying product similarity, RFC outperformed the other standard simulation models on holdout prediction accuracy (Sawtooth Software, Randomized First Choice).
| Model | How it aggregates shares | IIA-sensitive? | Best for |
|---|---|---|---|
| First Choice | Winner-take-all per respondent | Less sensitive at the individual level, but volatile with small samples | Simple competitive sets where you mainly need a directional winner |
| Share of Preference (logit) | Probabilistic split by relative utility | Yes, fully bound by IIA | Fast, well-separated product sets with little overlap between alternatives |
| Randomized First Choice | Simulated error plus repeated first-choice draws | No, built to relax IIA | Markets with near-duplicate or highly similar alternatives |
| Purchase Likelihood | Independent purchase-probability score per product | No, it scores products independently, not as a fixed-pool share split | Category-expansion or new-product-entry questions, not fixed-pool share splits |
Why is the share formula a second-order decision?
Because the formula never touches raw human behavior, only whatever utilities were already estimated from your data. If a McFadden discrete choice model, a Mixed Logit, or an ICLV specification was fit on responses from a randomized experiment, the resulting utilities carry causal information about how people traded off attributes under real, incentive-relevant choice conditions. If those same estimators were fit on an unvalidated online panel, a leading survey, or LLM-simulated responses with no randomized manipulation behind them, the utilities carry whatever bias, narrowness, or say-do gap was baked into the input, and no downstream share rule removes it. DCE, Mixed Logit, and ICLV are estimators, not causal methods on their own; causal identification comes from the randomized manipulation in the experiment design that produced the choice data, not from the statistical machinery used to fit it afterward.
Does Randomized First Choice fix the red-bus/blue-bus problem?
Yes, partially, and only for the aggregation step, not for whatever came before it. Standard logit-based share models satisfy IIA, which means the ratio of any two products' predicted shares stays fixed no matter what else enters the choice set. Add a near-identical alternative, a "blue bus" next to an existing "red bus," and a flat logit model has it pull share proportionally from every competitor instead of primarily from its closest substitute (Sawtooth Software, The Red-Bus/Blue-Bus Problem). Randomized First Choice was introduced by Orme (1998) and refined with Huber and Miller (1999) specifically to relax this, by injecting simulated error into individual utilities before running repeated first-choice draws, and it was the top performer across Sawtooth's own 2016 holdout competition, run on Sawtooth's own 21 holdout tasks (Sawtooth Software, Randomized First Choice). That fix operates entirely on the aggregation side. If the utilities feeding RFC came from an unvalidated source, RFC still relaxes IIA correctly and still produces a share estimate with no demonstrated connection to real market behavior.
Where do the utilities come from, and why does that decide the outcome?
Individual-level utilities today almost always come from Hierarchical Bayes estimation run on discrete choice, Mixed Logit, or ICLV data, because HB borrows strength across respondents to produce stable individual-level part-worths from limited per-person data. HB is a statistical technique for recovering parameters from a dataset; it says nothing about whether that dataset reflects real choice behavior. That determination is made earlier, at the experiment design stage, by whether respondents faced a randomized manipulation of attributes and prices under conditions that resemble a real decision, or answered a survey (human or LLM-simulated) with no randomization and no incentive to respond as they would in market conditions. Any willingness-to-pay figure pulled from these utilities should also be read with hypothetical bias in mind: stated WTP tends to run high unless the underlying design is incentive-aligned.
Can LLM-generated synthetic respondents replace a randomized choice experiment?
No, not on current evidence. An LLM standing in for a respondent was never subject to a randomized manipulation of price and attributes, so its output carries no causal identification, no matter how sophisticated the downstream share model applied to it is or how tight the resulting confidence interval looks. A confidence interval computed from a simulated experiment covers the estimated effect within that simulated population; it does not bound the real market unconditionally, and an unvalidated point estimate stays unvalidated no matter how tight the interval around it looks.
This is the reason validation against a real human baseline matters more than which share rule sits downstream of it. Subconscious's causal-fidelity paper (causal-fidelity paper) compares simulated study results against real, held-out human study results and reports the gap as a ratio, never a bare percentage, because a raw accuracy number is meaningless without the ceiling it's measured against.
Full methodology and per-study results: causal-fidelity paper.
Two caveats belong next to that number, not below it. First, it's a validation result on studies run to date, not a guarantee for a new, unstudied market. Second, published human studies can sit inside a language model's training data, which is exactly what a held-out replication protocol is designed to detect and control for, not a reason to treat the comparison as settled.
How should a senior buyer decide?
Validate the utilities before evaluating the formula. Ask any vendor for a holdout replication score measured against an independent human baseline, stated as a ratio with its denominator, the way the 87%-of-ceiling figure above is stated. Check whether the underlying choice data came from a randomized manipulation of real alternatives or from an unrandomized survey or LLM simulation. Only after that is settled does the share-rule choice matter: use RFC when your competitive set includes near-duplicate alternatives where the red-bus/blue-bus problem is live, use flat logit Share of Preference when alternatives are well separated and IIA is a reasonable assumption, and treat Purchase Likelihood as a separate question about category expansion rather than a fixed-pool share split. Public, study-by-study replication scores are posted on the leaderboard, and the reasoning behind the validation protocol is covered in more depth in the methods and validation hub.
Take whatever preference-share tool you're currently evaluating and ask for its holdout accuracy against a real, independent human sample, stated with the denominator. If the vendor can't produce that number, the share-formula question is premature. For a read on your specific market, talk to the team.