Choosing Between MaxDiff, Conjoint, and a Controlled Experiment
A research team choosing a study design before fieldwork has to answer one question: does the business decision need a preference ranking, an importance score, a satisfaction metric, or a causal estimate of what drives behavior? Picking the wrong instrument produces data that cannot support the decision it was meant to inform.
The decision this instrument choice actually serves
Before writing a single question, define three things: who should answer, what they are being asked to evaluate, and what the team will do with the result. A pricing decision, a message test, and a feature-tradeoff decision are not interchangeable research problems and do not share one correct method.
The failure mode is not a badly worded question. It is choosing a method whose output shape does not match the decision. A ranking tells a team what people prefer, not what would happen if the team changed the offer.
How do MaxDiff, conjoint, satisfaction metrics, and a controlled experiment compare?
These four instrument families answer different questions, and none substitutes for the others.
| Method | What it produces | What it cannot tell you | Typical use |
|---|---|---|---|
| MaxDiff (best-worst scaling) | A relative importance or preference ranking across a list of items | Whether changing one attribute would change a choice or an outcome | Prioritizing features, claims, or messages by stated importance |
| Conjoint analysis | Utility values for attributes and levels, derived from tradeoff choices | Evidence of real-world market behavior beyond the tested choice task | Pricing, packaging, and feature-tradeoff design |
| Satisfaction metrics (NPS, CSAT, SUS) | A single-number snapshot of sentiment or usability at one point in time | The reason behind the score, or what would move it | Tracking sentiment over time, flagging usability problems |
| Controlled discrete choice experiment | A causal effect estimate, with a confidence interval, for a specific action | Real market behavior beyond the stated-choice task, and answers outside the tested population and set of actions | Deciding which action to take among defined options |
MaxDiff and conjoint analysis are both tradeoff methods: MaxDiff ranks items by relative importance, while conjoint analysis estimates the tradeoffs buyers make between attributes like price and features through a randomized-design choice task (Sawtooth Software). Neither produces a causal estimate of real-world market behavior beyond the tested choice task.
When is a preference ranking or satisfaction score the wrong tool?
A ranking or importance score is directional: it says people rate a feature or claim highly relative to alternatives. Treating a directional read as a causal estimate is a common way research gets misapplied: the study answers a different question than the one the business is asking.
Satisfaction metrics have the same limit. NPS or CSAT can flag that something is wrong, but they do not identify which lever moved the number, because they were never designed to isolate cause from correlation.
When does the decision need a causal answer?
If the question is which action to take among a defined set of options, and the cost of choosing wrong is a launch, a price change, or a message that does not land, the study needs to isolate cause, not just rank preferences.
A controlled discrete choice experiment varies all attributes simultaneously through a randomized design and reports a causal effect, averaged across the other attributes, with a confidence interval. Subconscious runs this kind of study against a population modeled on the affected buyers when the decision is which action drives an outcome.
A short framework for choosing
- State the decision in one sentence: which action, priced how, framed how.
- Ask whether the answer needs to be causal (an estimate of what would change behavior) or directional (a relative preference or a sentiment snapshot).
- If the answer is directional, MaxDiff, conjoint attribute exploration, or a satisfaction metric can fit.
- If the answer needs to hold up under scrutiny as a causal claim, use a controlled experiment designed to isolate that one variable.
- Decide, before fieldwork, what evidence would change the recommendation. A study designed after the fact to justify a decision already made is not evidence.
Limitations
A controlled discrete choice experiment does not replace early qualitative exploration, and it does not set the target-group definition; that judgment call stays with the research team. The experiment does not certify representative statistics for a regulatory or public claim. Subconscious can test or validate studies with real human participants to check the study's stated choices against real behavior, since stated preference runs high relative to what people actually do (hypothetical bias), and that step is required before any claim from the study goes external.
Subconscious can run controlled studies against a person-level audience graph covering 800 million real people, a targeting frame rather than a probability sample of respondents. That reach describes the population available for a controlled study design, not a recruited sample delivered without further validation.
Next step
Teams that have used MaxDiff or conjoint analysis to narrow a set of options, and now need a causal answer, can see how a controlled study is scoped in case studies, or book a working session to map a pricing, message, or feature decision to the right method.
