Checking a simulated immigration-attitudes study against Hainmueller & Hopkins 2015
A research or policy team weighing whether to trust a simulated discrete-choice result on a politically sensitive topic needs a prior human benchmark, not a general accuracy claim. This entry compares a Subconscious-simulated discrete-choice run to a published, peer-reviewed study on U.S. immigration attitudes.
The published study behind the comparison
Hainmueller and Hopkins ran a conjoint experiment asking 1,407 U.S. adults to evaluate hypothetical immigrant profiles varying by education, English-language skills, country of origin, job experience, and other attributes, then rank which profiles they preferred (The Hidden American Immigration Consensus: A Conjoint Analysis of Attitudes toward Immigrants, American Journal of Political Science; also available via the Immigration Policy Lab). Subconscious ran a simulated version of the same discrete-choice question, testing preferences over immigrant-profile attributes including education, English skills, country of origin, job experience, work plans, reason for migration, and prior U.S. trips, and compared the ordering of preferences against the published human result.
What the correlation says
The simulated and human orderings correlated at r_s = .5406, p < .001. That is a Spearman rank correlation between how the simulation ranked immigrant-profile attributes and how the real study's respondents ranked them, a measure of relative ordering agreement, not a percentage-accuracy score.
A correlation in this range is moderate, not a claim of equivalence. It says the two rankings tend to agree more than they disagree; it does not say which specific attribute rankings matched and which diverged. Only one summary statistic is available for this comparison; there is no reported sample size, per-attribute breakdown, or confidence interval on the Subconscious side.
What this comparison does not establish
One moderate correlation on one topic, U.S. immigration attitudes as measured in 2015, does not establish accuracy for a different topic, population, or decision. Do not round r_s = .5406 up.
This entry also compares a simulation to a prior published study, not to a freshly recruited human panel. When a decision turns on live respondent behavior rather than a decade-old benchmark, Subconscious can validate the same causal question with real human participants.
| Hainmueller & Hopkins 2015 (human) | Subconscious simulation | |
|---|---|---|
| Sample | 1,407 U.S. adults, conjoint experiment | Simulated discrete-choice run, same attribute set |
| Attributes tested | Education, English skills, origin, job experience, work plans, migration reason, prior U.S. trips | Same attribute set |
| Result compared | Published preference ranking | r_s = .5406, p < .001 correlation to the published ranking |
Reading this comparison before a fielding decision
Before treating this single comparison as grounds for a policy-messaging or attitudes-research decision, check three things: whether the topic and attribute set match, whether a moderate correlation suits the stakes, and whether a 2015 benchmark still reflects current attitudes on a topic that shifts with policy and news cycles. See the full leaderboard for how this entry sits alongside other published comparisons, and research for the underlying replication method.
A team deciding whether to trust a simulated result for its own attitudes or policy-preference question should look for the closest topic match on case studies and confirm the comparison method fits the decision at hand. Teams that need current-population confidence can move to how Subconscious works with real human participants on the same causal question.