Choosing Between MaxDiff, Conjoint, and a Controlled Experiment
A research team choosing a study design before fieldwork has to answer one question: does the business decision need a preference ranking, an importance score, a satisfaction metric, or a causal estimate of what drives behavior? Picking the wrong instrument produces data that cannot support the decision it was meant to inform.
The decision this instrument choice actually serves
Before writing a single question, define three things: who should answer, what they are being asked to evaluate, and what the team will do with the result. A pricing decision, a message test, and a feature-tradeoff decision are not interchangeable research problems and do not share one correct method.
The failure mode is not a badly worded question. It is choosing a method whose output shape does not match the decision. A ranking tells a team what people prefer, not what would happen if the team changed the offer.
MaxDiff, conjoint, satisfaction metrics, and a controlled experiment compare differently
These four instrument families answer different questions, and none substitutes for the others.
| Method | What it produces | What it cannot tell you | Typical use |
|---|---|---|---|
| MaxDiff (best-worst scaling) | A relative importance or preference ranking across a list of items | Whether changing one attribute would change a choice or an outcome | Prioritizing features, claims, or messages by stated importance |
| Conjoint analysis | Utility values for attributes and levels, derived from tradeoff choices | A causal estimate for a specific real-world intervention outside the tested attribute set | Pricing, packaging, and feature-tradeoff design |
| Satisfaction metrics (NPS, CSAT, SUS) | A single-number snapshot of sentiment or usability at one point in time | The reason behind the score, or what would move it | Tracking sentiment over time, flagging usability problems |
| Controlled discrete choice experiment | A causal effect estimate, with a confidence interval, for a specific action | Answers outside the tested population and the tested set of actions | Deciding which action to take among defined options |
MaxDiff and conjoint analysis are both tradeoff methods: MaxDiff ranks items by relative importance, while conjoint analysis estimates the tradeoffs buyers make between attributes like price and features (Sawtooth Software). Neither produces a causal estimate of what would happen if a team changed one input and left the rest of the market alone.
Where a preference ranking or satisfaction score is the wrong tool
A ranking or importance score is directional: it says people rate a feature or claim highly relative to alternatives. Treating a directional read as a causal estimate is a common way research gets misapplied: the study answers a different question than the one the business is asking.
Satisfaction metrics have the same limit. NPS or CSAT can flag that something is wrong, but they do not identify which lever moved the number, because they were never designed to isolate cause from correlation.
When the decision needs a causal answer
If the question is which action to take among a defined set of options, and the cost of choosing wrong is a launch, a price change, or a message that does not land, the study needs to isolate cause, not just rank preferences.
A controlled discrete choice experiment holds everything but the tested attribute constant and reports a causal effect with a confidence interval. Subconscious runs this kind of study against the affected buyers when the decision is which action drives an outcome.
A short framework for choosing
- State the decision in one sentence: which action, priced how, framed how.
- Ask whether the answer needs to be causal (an estimate of what would change behavior) or directional (a relative preference or a sentiment snapshot).
- If the answer is directional, MaxDiff, conjoint attribute exploration, or a satisfaction metric can fit.
- If the answer needs to hold up under scrutiny as a causal claim, use a controlled experiment designed to isolate that one variable.
- Decide, before fieldwork, what evidence would change the recommendation. A study designed after the fact to justify a decision already made is not evidence.
Limitations
A controlled discrete choice experiment does not replace early qualitative exploration, and it does not set the target-group definition; that judgment call stays with the research team. The experiment does not certify representative statistics for a regulatory or public claim. Subconscious can test or validate studies with real human participants without changing the underlying causal question, and that step is required before any claim from the study goes external.
Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. That reach describes the population available for a controlled study design, not a recruited sample delivered without further validation.
Next step
Teams that have used MaxDiff or conjoint analysis to narrow a set of options, and now need a causal answer, can see how a controlled study is scoped in case studies, or book a working session to map a pricing, message, or feature decision to the right method.
