Skip to content
Subconscious

What Is a Silicon Sample?

On this page

A silicon sample is a set of survey responses that a language model writes while conditioned on attributes of a target population, such as age, income and region. The responses stand in for survey answers, but no recruited human gave them. For a buyer, the question is not whether a silicon sample exists. The question is whether it is good enough for one named decision, and what evidence shows that.

Subconscious is the causal AI company. Randomized experiments on a simulation of your market, validated against real human behavior, tell you why people choose and which action drives the outcome. A silicon sample has value only to the extent that its construction and its checks are documented.

What is a silicon sample, in plain terms?

Think of it as a panel made by a model. You describe a person with a short profile. The model answers a survey question as that person. You repeat this across many profiles and count the answers.

The term comes from Argyle and colleagues, whose paper first appeared on arXiv on 14 September 2022. The team conditioned GPT-3 on thousands of sociodemographic backstories taken from real participants in several large US surveys. They compared the model's answers with the human answers. They named the property they studied "algorithmic fidelity" and suggested that language models could help researchers study human subgroups.

That result covers GPT-3, those surveys and those questions. The paper's abstract does not report a test on your category, your price range or your customers. Any claim that a silicon sample fits them needs its own evidence.

Does a silicon sample match human respondents?

At the level of averages, one study found close agreement. At the level of relationships between variables, the same study found large gaps. Bisbee and colleagues (Political Analysis, published online 17 May 2024) prompted ChatGPT 3.5 Turbo with personas. The personas matched respondents in the American National Election Studies of 2016 and 2020. The model gave feeling-thermometer scores for 11 sociopolitical groups.

The average scores from the model corresponded closely to the human averages. The regression results often differed from the human results. The article body reports that 48% of the synthetic coefficients differed significantly from the human ones, and that the sign flipped in 32% of those cases. The authors also report that synthetic responses varied less than human ones. Small changes in prompt wording changed the results. The same prompt gave different results over a three-month period. They question the quality, reliability and reproducibility of language-model survey data.

The two papers used different models and different protocols. Both drew on US survey data, and both compared model output with human responses. The papers do not settle the question for your market. A buyer should ask which estimate was checked, against which human data, and whether that estimate is the one the decision needs.

What should a buyer check?

These five questions are a starting list, not a full risk inventory:

  1. Which population? Name the group, the region and the date. Ask what evidence supports using a sample built for one group to speak about another, such as current subscribers and prospects.
  2. Where did the composition come from? Ask for the dated source records, with counts and a denominator. If a combination of attributes is not in the records, the brief should label it as an assumption.
  3. What was held out? Data used to build or calibrate the sample cannot also serve as an untouched test. Kapoor and Narayanan (arXiv, 14 July 2022) documented data leakage errors in 329 papers across 17 fields and tied leakage to reproducibility failures. Ask for a comparison on human data that the build did not use.
  4. Which endpoint? A generated choice between two offers is a different endpoint from renewal, comprehension or revenue. Ask for validation on the endpoint you plan to use.
  5. What is the uncertainty? Ask how the reported interval was produced. Ask whether it describes uncertainty about human behavior or only variation across model draws.

How does Subconscious use simulated buyers?

Subconscious runs controlled choice experiments on simulated buyers, with confidence intervals, graded against human results. The method is checked by replicating published choice studies and comparing the simulated estimates with the human ones.

Subconscious's July 2026 working paper, which is not peer reviewed, reports a mean Spearman rank correlation of 0.73 between simulated and human choice estimates across 43 design-filtered replications of published choice studies, and 0.55 across roughly 300. The public fidelity site is the entry point to that evidence and to the study registry; the per-study table is not yet public. Read those numbers with the study design in mind. A rank correlation is not an accuracy percentage, and it does not apply to every study.

A worked example of a brief

Take a research lead who must decide whether a silicon sample is suitable for comparing two existing subscription offers. This is a construction example, not a completed study. Both offers are hypothetical.

The lead writes a brief from the real decision records:

A missing group stays visible in the report. Generating more responses does not add source records for a group that is absent from the data. A team can still extrapolate, but the report must say that it did and on what grounds.

Who accepts what?

Before any run, name the people who sign off.

The decision owner states the smallest offer difference worth acting on, the uncertainty they can accept, the cost of making the change and the deadline. The research lead accepts the construction and how to read the results. The data owner accepts the use of the records. The business owner accepts the action. Approving a sample is a separate decision from approving a change to subscription terms.

Set the action rule before the results arrive. A threshold chosen after the team sees which offer won is weaker confirmatory evidence, because the result can shape the choice of threshold. Nosek and colleagues (PNAS, 13 March 2018) write that mistaking a postdiction for a prediction reduces the credibility of research findings.

Then choose one of three outcomes. Accept the brief for a scoped, modeled comparison when population, task and evidence meet the agreed bar. Narrow the claim when coverage supports a smaller use. Leave the recommendation open when a missing group or endpoint could change the decision. If existing evidence already settles the question, do not commission a new study.

What evidence does each use of a silicon sample need?

The cited studies test specific models, surveys and questions. They do not establish that a silicon sample suits every use below. Treat the following as proposed screening guidance, and ask for evidence that matches each row.

UseEvidence to ask for
Directional comparison of two optionsA fixed, documented setup, plus a human comparison on the same kind of endpoint
Hypotheses about an audience that is hard to recruitA statement of what grounds the profiles, and a check against any human data that exists
Repeated exploratory runsThe same configuration across runs, with prompt and model versions recorded, given the sensitivity to prompt wording and the drift that Bisbee and colleagues report
Early screening before a human studyA rule, set in advance, for which results trigger a human study
Estimates for a human populationCalibration and a stated inference method, with error limits
A new endpointValidation on that endpoint

A directional result describes what the configured model selects under the tested conditions. That selection alone does not establish the same preference, effect size or uncertainty in a human market. When the decision is costly or hard to reverse, add human evidence for the open question. That can be an existing record, a matched human sample or an observed market test, depending on the endpoint.

Bring the brief to a decision review

Book a Subconscious decision review with the target population, permitted composition records, exact alternatives, existing human evidence, costs and deadline. Name the research lead and the person who can accept the decision. Agree the composition, the task, the supported use and the acceptance rule before any run.

Related reading: the research workflow, aggregate replication evidence and limits, and the Subconscious team.