AI-Generated Models That Run vs. Models You Can Trust
The decision: trust the output, or gate it first
An AI coding agent hands back a statistical model. It ran. No errors, a finished script, a results object. Does that mean the model is ready to inform a pricing decision, a risk score, or a clinical read?
A benchmark study of Claude Code building Bayesian models offers a direct answer: no. Code that executes and a model fit to inform a decision are two different claims. A model with divergent MCMC chains, a mis-specified prior, or label-switching in a mixture model can look identical to a sound one until someone checks the diagnostics. The failure does not show up as a bug report. It shows up later, as a wrong call that nobody can trace back to its source.
What the benchmark measured
A published benchmark study tested two conditions with Claude Code: a base agent working from general training knowledge, and the same agent with a written domain-knowledge document injected into its system prompt, covering current best practice for model specification, parameterization, sampling configuration, and convergence diagnostics (Agent Skills, Anthropic). Both conditions used identical prompts, tools, and CLI flags.
Five tasks escalated in difficulty: a hierarchical model, ordinal regression, a stochastic-volatility model, a Gaussian mixture model, and a sparse variable-selection model. Each targets a point where a wrong implementation choice, not a syntax error, produces code that runs but gives unreliable answers.
Rather than score every run on one number, the study split evaluation into two stages:
| Stage | Question it answers | What it checks |
|---|---|---|
| 1. Viability gate (pass/fail) | Did the model produce output worth examining at all? | Sampling completed with a usable posterior, convergence diagnostics inside an acceptable range, and no degenerate estimates |
| 2. Quality scoring (0–5 per criterion) | Among the models that pass, how good is this one? | Completeness of the output, convergence quality, whether the model choice fits the problem, adherence to modern best practice, how many rewrite cycles it took, and how many turns it used |
That two-stage design is the transferable lesson, independent of which tool produced the model. A single quality average hides failure: if a third of runs never clear the viability bar, their absence quietly inflates the reported average for everyone left standing.
Where base knowledge runs out
Both conditions passed the hierarchical model and ordinal regression tasks at the same rate, because those patterns are common enough in training data that domain augmentation adds little.
The gap opened on the harder, less-documented tasks:
- On stochastic volatility, the unaided agent produced no viable run in three attempts, reaching for a manually built autoregressive parameterization that never converged. With the domain document available, it used a purpose-built random-walk structure and converged in most attempts.
- On the horseshoe variable-selection task, the unaided agent passed roughly a third of the time; with the document, nearly every attempt passed.
- On the mixture model, the unaided agent's convergence score more than doubled once the ordering constraint that prevents label-switching was applied, rather than sorted after the fact as a cosmetic fix.
The pattern holds across tasks: domain knowledge did not make the agent smarter about statistics. It made the agent more consistent about reaching for the parameterization a domain expert would already know to use.
What the code actually differs on
On the sparse variable-selection task, the two conditions produced structurally different models. Without the domain document, Claude built a standard horseshoe prior with a centered parameterization and manually ordered cutpoints:
with pm.Model() as model:
tau = pm.HalfCauchy("tau", beta=1)
lambda_i = pm.HalfCauchy("lambda", beta=1, shape=n_predictors)
sigma_beta = tau * lambda_i
beta = pm.Normal("beta", mu=0, sigma=sigma_beta, shape=n_predictors)
eta = pm.math.dot(X, beta)
trace = pm.sample(2000, tune=1000, target_accept=0.95, random_seed=42,
chains=4, return_inferencedata=True)With the document available, it built a regularized horseshoe with a slab component and a non-centered parameterization:
with pm.Model(coords=coords) as model:
X_data = pm.Data("X", X_scaled, dims=("obs", "features"))
tau = pm.HalfStudentT("tau", nu=2, sigma=1)
lam = pm.HalfStudentT("lam", nu=5, dims="features")
c2 = pm.InverseGamma("c2", alpha=1, beta=1)
z = pm.Normal("z", mu=0, sigma=1, dims="features")
lam_tilde = pt.sqrt(c2 / (c2 + tau**2 * lam**2))
beta = pm.Deterministic("beta", z * tau * lam * lam_tilde, dims="features")
idata = pm.sample(draws=1000, tune=1000, chains=4, nuts_sampler="nutpie",
random_seed=42, target_accept=0.95, init="adapt_diag")Both versions "run." Only the second adds a slab term that keeps shrinkage from producing implausibly large coefficients, decouples the coefficients from the shrinkage scale for easier sampling geometry, and labels its dimensions so the diagnostics are interpretable later.
The lesson generalizes past this one benchmark
This is a third-party benchmark of one coding agent building Bayesian models with the PyMC package, not a Subconscious study, and its specific pass rates and costs describe that setup, not any general guarantee. What generalizes is the discipline: any AI-assisted analysis that will inform a real decision needs an explicit pass/fail gate before a quality score, and a documented rubric for what "good" means once something clears that gate.
That is the same discipline behind causal experiment design: a result is not useful because a model produced a number, it is useful because the design, the diagnostics, and the uncertainty around that number can be checked. Subconscious's own method runs controlled discrete choice experiments and reports causal effects with confidence intervals rather than a single point estimate, so a decision-maker has something to check before acting on it. Details on how those experiments are designed and validated are on Subconscious's research page and in its published leaderboard.
Where this comparison stops applying
A viability gate catches models that fail outright or produce degenerate estimates. It does not catch a model that passes every diagnostic while answering the wrong business question, and no automated check replaces a domain expert reviewing whether the model specification matches the decision it will inform. The benchmark's own quality scoring needed a rubric written by someone who knew what "appropriate" looked like for each task; a gate without that judgment behind it is just a lower bar dressed up as assurance.
Next step
Treat "it ran" as the start of the checklist, not the end of it. For how a documented, checkable process applies to causal decisions specifically, see how Subconscious runs an experiment or read more about the team behind the method.