Likelihood Approximations Through Neural Networks: A Validation Checklist Before You Trust the Output
A behavioral model that reports a choice probability or a causal effect is only as trustworthy as the likelihood behind it. When that likelihood has no closed form, a team has to decide whether to accept the model's output on faith or check it against the simulator that generated its training data first.
The safer answer is: check it. This walkthrough, adapted from a PyMC tutorial on likelihood-free inference, shows what that check looks like for a model with an intractable likelihood, and the Bayesian workflow that follows once it passes.
Why some models have no likelihood to evaluate
Standard Bayesian inference needs a likelihood function that scores how probable the observed data are under a given set of parameters. Many models used to analyze reaction times and choices don't have one in closed form.
The example model is a Drift Diffusion Model (DDM), a common framework for joint reaction-time-and-choice data. A DDM describes a decision as a random walk: evidence accumulates at a drift rate toward one of two boundaries, and whichever boundary is crossed first determines both the choice and the reaction time. The standard DDM has a known analytical likelihood, but realistic variants do not. One example is a variant where the decision boundary narrows as a deadline approaches. Those variants remain easy to simulate; you can generate synthetic trials from them in a few lines of code. They're just hard to score directly.
That gap between "easy to simulate" and "hard to score" is what simulation-based inference (SBI) is built to close.
The three-step pipeline
Simulation-based inference replaces the missing analytical likelihood with a learned approximation, trained entirely on simulated data:
\text{simulate} \rightarrow \text{train an approximator} \rightarrow \text{validate against the simulator}- Simulate. Generate synthetic trials across many parameter settings using the model's simulator. The walkthrough this is adapted from draws 1000 trajectories at a time from a DDM simulator to build training data.
- Train an approximator. Fit a neural network on the simulated data so it learns to score data the same way a true likelihood would. The source example trains two specialized networks, one scoring the reaction-time density for observed responses and one scoring how likely a response was withheld, and combined the pair spans the full likelihood.
- Validate against the simulator. Before the trained network goes anywhere near a real inference run, its output has to be checked against the simulator that generated its training data.
Step three is where most of the risk in this workflow lives, and the easiest step to skip under time pressure.
What "validate against the simulator" actually means
Validation here is not a summary statistic. It's a direct visual and numerical comparison between what the trained network predicts and what the simulator itself produces when run at the same parameter values.
For the reaction-time network, that means plotting the network's predicted likelihood curve directly against a histogram of reaction times the simulator generated at the same parameter setting, for several settings across the plausible parameter range. For the choice-probability network, it means overlaying the network's predicted probability against the simulator's own observed choice frequencies. Both checks are described in Fengler et al.'s work on likelihood approximation networks for cognitive neuroscience, the peer-reviewed source for this class of approximation.
This is the checkpoint that separates "the network converged during training" from "the network's output means what I think it means." A network can report a low training loss and still produce a likelihood surface that diverges from the simulator in the regions of parameter space that matter for a specific dataset.
The broader case for this approach, using a trained approximator instead of an intractable likelihood and treating the approximator as something to validate rather than trust by default, is laid out in Cranmer, Brehmer, and Louppe's survey of simulation-based inference.
Wiring the validated network into a Bayesian model
Once a network's output has been checked against the simulator, it can be dropped into a probabilistic model. In the source workflow, the trained networks are wrapped as custom PyTensor operations and injected into a PyMC model as potential functions, a mechanism for adding a custom log-likelihood term rather than using a built-in distribution. From there, standard Bayesian machinery applies: priors go on the model's parameters, and Markov chain Monte Carlo sampling produces a posterior distribution rather than a single point estimate.
The example model draws 500 trials under one experimental condition and 500 under the other and trains its two networks for 10 and 20 epochs respectively. Those specific counts are the tutorial's own settings for building training data quickly on a local machine, not a floor or ceiling for how much simulation or training a given problem needs. A production model on a wider parameter range would generally need more of both. The point is the pipeline's shape, simulate, approximate, validate, then infer, not the specific trial or epoch counts.
Recovering the true generating parameters from synthetic data, checking the posterior lands where you know the answer because you generated the data yourself, is the same closing step. It's a necessary check before the pipeline runs on data whose true answer is unknown, though recovering parameters from a single synthetic dataset doesn't by itself confirm the pipeline is correct across the board.
Where this discipline matters beyond one tutorial
The specific failure mode this validation step guards against is general: any model whose output feeds a business decision but whose fit was never checked against the simulator or dataset that generated it is a black box wearing a probability distribution. The number in the answer is real; whether it means what the reader thinks it means is not.
A related discipline applies to how Subconscious reports results: causal effects come from the randomized design of the experiment, and Subconscious reports them with confidence intervals rather than a single simulated prediction. A confidence interval on its own doesn't validate the model, but an effect checked against structure the model didn't just invent, paired with that interval, holds up to scrutiny that a bare point estimate doesn't. A team evaluating how a decision workflow gets from a simulated estimate to something checked against the simulator or process that generated it should expect that checkpoint to exist regardless of which modeling approach produced the first number.
Limitations
This is a methods walkthrough on likelihood-free Bayesian inference for drift diffusion models, built around open-source PyMC tooling. It's not a description of a Subconscious product pipeline, and Subconscious does not ship this exact neural-network-likelihood workflow or DDM-specific tooling. The general lesson, approximate a missing likelihood and then validate that approximation against the simulator's own output before it informs a decision, is the part worth carrying forward regardless of which models or software a team uses.