Predicting Swinging Strikes with Bayesian Additive Regression Trees
On this page
Bayesian additive regression trees predict swinging strikes by summing many shallow trees fit through MCMC sampling over pitch-tracking features like velocity and spin rate, producing a whiff probability with a credible interval attached. A model that returns one number is asking for trust it hasn't earned. It earns that trust with an uncertainty band, checked against data the model never saw. A worked example from baseball pitch-tracking data shows what that check looks like and why a decision-maker should demand it before any go/no-go call.
The decision this example stands in for
Approving a launch, a price change, or a messaging shift on a model's output means trusting that output's error range, whether the model states one or not. A point estimate with no stated uncertainty hides risk: if the real result lands outside the range nobody wrote down, the team has no record of how much uncertainty it ignored to get there. The fix is a habit of checking calibration and holdout performance before treating any number as decision-ready.
Christopher Fonnesbeck’s worked pitch example
The details below come from Christopher Fonnesbeck’s February 2026 PyMC Labs BART article. Consult that case and its implementation for the dataset extraction and evaluation configuration; the foundational BART paper is a methods source, not the source of these baseball results.
The pitch-quality problem in baseball illustrates the check cleanly because the raw data is public and dense. MLB's Statcast system, using Hawk-Eye cameras installed in all 30 ballparks, captures velocity, spin rate, movement, and release-point geometry for every pitch thrown. The pybaseball library turns that into a single function call returning roughly 700,000 pitch-level observations per season with over 90 tracked columns.
The target: whiff rate, the share of swung-at pitches a batter misses, removes take decisions from the denominator, leaving a measure that still reflects which pitches batters chose to swing at. Four physical features, standardized to z-scores, form the baseline model:
| Feature | Description |
|---|---|
| Velocity | How fast the ball is traveling the instant it leaves the pitcher's hand |
| Vertical Break | Vertical deviation the spin imparts on the pitch's flight |
| Horizontal Break | Side-to-side drift toward the arm side or the glove side |
| Spin Rate | The ball's total rotation speed, measured in RPM |
Velocity and spin can interact: the relationship between spin and whiff probability may depend on velocity. BART learns such structure flexibly. A linear-in-parameters model can also represent interactions or nonlinear basis terms when they are explicitly included; a simple additive specification leaves them out.
Why use a Bayesian additive regression tree instead of a gradient-boosted tree?
Default mean or classification objectives in gradient boosting return point predictions. XGBoost also supports quantile objectives directly; bootstrap and conformal calibration are other approaches. Quantile outcome intervals differ from posterior uncertainty about a probability. BART’s foundational paper describes Bayesian tree inference; its posterior intervals still depend on the model and require appropriate calibration checks.
| Aspect | XGBoost | BART |
|---|---|---|
| Tree construction | Grows trees one at a time via greedy, sequential optimization | Draws trees by MCMC sampling from a posterior |
| Regularization | Explicit penalties on depth, leaf count, and λ | Implicit, enforced through priors over tree shape |
| Output | Point prediction or conditional quantiles, depending on objective | Posterior distributions conditional on model and prior |
| Uncertainty | Quantile objectives, bootstrap, or conformal methods with distinct interpretations | Posterior credible intervals; calibration requires separate checks |
BART's tree-structure prior favors shallow trees: the probability that a node at depth d splits follows
P(\text{split at depth } d) = \frac{\alpha}{(1 + d)^{\beta}}With the library's default values (α = 0.95, β = 2), a depth-0 node splits with 95% probability, but a depth-2 node splits only about 10% of the time, an implicit brake against the overfitting that plagues greedy tree growth. This BART variant, implemented within the PyMC probabilistic-programming framework, combines a specialized particle Gibbs sampler for the tree component with standard gradient-based samplers for any linear pieces in the same model, such as venue-level random effects.
The check that actually matters: calibration, then holdout
Fitting the model is the easy half. The example runs two checks before trusting the output:
Calibration. Pitches the model assigns a 10% whiff probability should whiff about 10% of the time in the data. A calibration report that only shows the hits is marketing copy. The baseline model calibrated reasonably well across most of its range, with visible deviation only at the extreme high end, where small sample sizes make any estimate noisy.
Twenty pitchers were held out of training and scored afterward in the source example. The reported calibration deviations include small-sample bins and mild underprediction at low whiff rates. That supports evaluation on unseen pitchers in this dataset; it does not isolate intrinsic pitch quality from batter, context, or swing-selection effects.
Neither check is optional for a model whose output will drive a decision. A model that calibrates well on its own training data but has never been scored against holdout cases has only proven it can describe the past.
Adding features changes the uncertainty, not just the number
A second version of the model added release-point features (height, side, extension), an aerodynamic proxy called axis differential, a same-handed-matchup indicator, and venue-level random effects, growing from 100 to 150 trees. Comparing the two models by Widely Applicable Information Criterion, a Bayesian estimate of out-of-sample predictive accuracy that avoids running full cross-validation, showed the richer model with a better (lower) WAIC than the simpler one, though WAIC estimates leave-one-observation-out accuracy for the pitchers and venues already in the data, not generalization to new pitchers, and can be unreliable for hierarchical models with influential observations.
The enhanced model's uncertainty bands were wider than the baseline's, on the same pitchers. That's not necessarily the model performing worse: added variance components, weaker per-parameter identification, and more trees can widen posterior intervals on their own, and the widening may also reflect genuine execution variability, such as a pitcher's release height or extension shifting from throw to throw. A model reporting a narrower interval than the data justifies is the riskier of the two, not the more admirable one.
What this means for evaluating a causal-experimentation vendor
The specific numbers in this example (baseline calibration quality, the WAIC improvement from adding features, any pitcher-level leaderboard) describe a public baseball dataset, not a Subconscious result, benchmark, or case study. What transfers is the standard: a claim that a predictive model is trustworthy should come with a calibration check and a holdout test against cases the model didn't see; a causal treatment effect requires separate identification evidence, since counterfactual outcomes are never observed to check against, not a single number asserted with confidence.
For a Subconscious study, request the uncertainty procedure and matched human evidence for the treatment, population and endpoint used in the decision. Confirm recruitment and validation deliverables for the engagement. See the research workflow and published evidence on the leaderboard.
Where does this method not apply?
Predictive calibration checks probability agreement, and held-out scoring examines generalization for those cases. Neither alone establishes posterior interval coverage or causal identification. A matched human experiment tests a business intervention for its specified population and task; the result may disagree or remain inconclusive.
The question to ask before trusting a number
Before a launch, pricing, or messaging decision rests on a model's output, ask two things: has this been checked for calibration, and has it been tested on cases the model never saw. A vendor, or a method, that can't answer both is asking for trust on a point estimate alone.