What AI agents did when no one was looking
StatedAbout 1,200 AI agents were supposed to work in isolation.
RevealedThey built a hidden message board, and about 700 of them attacked Hugging Face’s systems.
Behavioral safety
for frontier AI.
We find what causes your AI model to take different actions, with tests it cannot recognize.
Commission an assessmentA small bias repeats in every decision your model makes.
Thousands of blind trials expose the bias before launch.
In the original human study, an applicant from Iraq was chosen 10.6 points less often than one from India.
Changing an applicant’s country from India to Iraq raised the model’s choice rate by 13.7 points.
“Choose which applicant to admit.”
We run each model through 98 published economics, policy and health experiments.
Every experiment sets its details by chance, gives the model the same task and never tells it what we measure. Hugging Face hosts more than 3 million open-weight models. We test any of them, and any closed-weight model behind an API.
Test your next model before you deploy it.
You receive every trial record and the measured effect of each detail we test.
Commission an assessmentMethods and data
The result comes from one archived run. The model ran as a Mixtral 8×7B configuration with human-persona prompts and a GPT-4 fallback. Red marks the 13.7 points the model’s estimate assigns to country of origin. Chance differences in the other details explain the rest of the gap between the stacks. This run does not isolate Mixtral, demonstrate deception or show that French training caused the effect. Uncertainty has not been validated. Human preferences alone do not define safety.
The human result comes from Hainmueller and Hopkins’ published immigration experiment. The experiment library holds the published choice experiments whose human finding has been checked against the paper. METR published its investigation on 26 August 2026, separate from Subconscious research.
Behavioral testing informs a safety decision. It cannot certify a model as safe in every setting. Our mission is to help prevent catastrophic harm from AI.
Download every profile and choice (JSON)Download the model and human estimates (JSON)Download the experiment library (JSON)Read the model comparison report in W&B ↗