Does a simulated vaccine-preference study match a published one? A Kreps et al. replication check
A head of insights weighing a simulated discrete choice experiment against a full human-recruited study needs one answer: does the simulated ranking of preferences track how real people order the same attributes? Greenlighting a launch, price, or messaging call on a result that doesn't track wastes the study budget and the credibility of whoever recommended it.
The published benchmark
Kreps, Prasad, Brownstein, and colleagues ran a discrete choice experiment on US adults weighing COVID-19 vaccine attributes: efficacy, duration of protection, side effects, FDA approval status, and political endorsement, among others (Kreps et al., JAMA Network Open, 2020). The published paper reports which attributes respondents weighted most and least heavily: a ranking, not a single number.
The replication check
Subconscious ran a simulated version of the same choice task and compared the resulting attribute ranking to the one reported in the published paper. The two rankings correlated closely: a Spearman rank correlation of rs = .8734 (p < .001) (Kreps et al., JAMA Network Open, 2020).
That correlation describes one documented replication of one published study's attribute ordering, not a general accuracy percentage.
What this does and doesn't establish
- It establishes that, for this vaccine-attribute preference task, a simulated ranking tracked a peer-reviewed human study's ranking closely enough to be directionally useful before a full field study.
- It does not establish that the correlation holds for every domain. Health and vaccine-attribute preference research is one case; a different category of decision needs its own check.
- It does not mean Subconscious recruited or fielded the original human sample. That sample belongs to the Kreps et al. study; Subconscious's role was running the comparable simulated task and comparing the resulting order.
- It does not substitute for real-human validation when a decision's stakes call for it. A simulated ranking pressure-tests attribute priorities early, before a team decides whether a full human-recruited study is worth the spend.
Where this fits in a research plan
Run the simulated discrete choice experiment first, compare its output against any relevant published or prior study, and use that comparison to decide whether the priority order is stable enough to act on or needs a human-recruited study first. Subconscious supports moving from a simulated experiment to real-human testing without changing the underlying causal question: see how we work for that path, and research or case studies for other documented results. Teams that want to see current comparative performance can also check the leaderboard.
Limitations
This is a single replication against one published study's attribute rankings. It is not a benchmark across studies, domains, or populations, and it is not a guarantee that any other simulated task will correlate this closely with a comparable human study.
