Checking a simulated immigration-attitudes study against Hainmueller & Hopkins 2015
On this page
Immigration-attitudes research needs to separate preferences about particular applicants from preferences about total immigration. A simulation of applicant choices can help explore the first question. It cannot establish the second question or prove that current voters share a historical sample's attitudes.
The human study's question
Hainmueller and Hopkins's 2015 American Journal of Political Science study used a two-wave population-based survey experiment. Respondents evaluated hypothetical immigrants with nine attributes varied simultaneously. The outcome concerned support for admitting applicants. The study identified agreement across respondent groups about the kinds of applicants they preferred, while distinguishing those preferences from views about how many immigrants to admit. Immigration Policy Lab's study record.
Evaluating paired profiles is not the same measurement as ranking a list of attribute levels. An analyst estimates effects of those randomized attributes from the responses. A matched simulation should preserve the questionnaire, alternatives, outcome coding and estimand before comparing estimates.
What this page can establish
The human paper documents its experiment. This page has no public run artifact substantiating a study-specific Subconscious correlation against that experiment. It therefore makes no numerical replication claim. Any other immigration comparison must retain its own study, matching convention and metric; its score cannot be transferred to this study.
| Required match | Why it matters |
|---|---|
| Applicant attributes and levels | A changed profile changes the experimental question |
| Admission-support outcome | Approval, forced choice and an attribute ranking differ |
| Population and period | Current attitudes can differ from a historical sample |
| Estimator and baseline | Different reference levels can change comparisons |
| Attribute-level effects | An aggregate coefficient can conceal specific failures |
Inspect politically sensitive differences
Before relying on a simulation, inspect effects concerning language, education, origin and immigration history separately. A similar overall ordering can hide a sign reversal on an attribute central to the policy question. The analysis should show uncertainty and disagreements rather than treating a positive correlation as sufficient evidence.
Also check whether prompts introduce additional applicant characteristics when an attribute changes. Explicit alternatives reduce ambiguity but do not prove that a language model has held all background assumptions constant. That assumption needs a sensitivity check and a matched human comparison.
Choose the next study from the decision
For questionnaire development, simulation can help locate confusing or incomplete profiles. For a claim about current public support, collect current human evidence for the same conditions. For a claim about enacted policy or voting behavior, an admission-preference experiment alone is insufficient.
The public evidence record explains the distinction between parameter-rank agreement and stronger validation tasks. A decision review can scope the population, alternatives and human check needed for a particular policy or communications question. The objective is a defensible inference about that question, with its limitations visible.