Research results

PDF report · HTML report

Learning to write statistical models

In the coral task, GLM-5.3-Flash ran a program in PyMC, a Python library for probability models, and submitted a linear regression to predict the maximum diameter of brain coral at reef sites (coral training example).

An attempt at a task, which we call an episode, can include data requests and a program submission. A query budget caps how many observations the agent can request. The program is a probabilistic model of how the data arose, and it predicts the rows withheld from the agent. SynthStats is designed to train language models to write such programs, the rewards updating their choice of queries as well as their programs.

The verifier checks the syntax and safety of a program before fitting it, and checks the built model under a strict PyMC profile, which excludes custom densities, arbitrary additions to the log density, priors and branches that depend on PyMC data containers. An accepted program receives a utility of 1.5 + q if its held-out expected log predictive density (ELPD) is finite. ELPD measures the predictive density assigned to withheld observations; q is the hyperbolic tangent of its improvement over the task's baseline per held-out row. A refused program or an episode without a submitted program receives zero.

What the demonstrations show

Both tasks come from BoxingGym, a benchmark of simulated scientific environments, and in each task the language model received a description in English together with 80 observations of one simulated dataset, the other 20 being held out for scoring.

In both demonstrations, dugong growth and disease spread, the agent was the language model Qwen3.6-35B-A3B, which wrote eight programs for each: four single attempts and four proposals from a two-round loop in which a critic gives feedback on fitted predictions. We wrote the reference knowing how the simulator generates its data. All eight dugong-growth programs scored above the floor model (which uses the same predictive distribution whatever the input), three of them with point estimates above the reference program on this simulated dataset of 20 held-out points. The Monte Carlo standard error (MCSE) measures the error from estimating a score with 200 posterior draws. Single attempts 2 and 4 scored higher by 1.4059 (MCSE 0.1048) and 1.3761 (0.1051), each more than two MCSEs. Loop proposal 4 also scored above the reference, by 0.0907 (MCSE 0.1355), within two MCSEs. Six of the eight disease-spread programs received scores; all six beat the floor model and scored below the reference. Each figure shows the reference next to the first single attempt, that is, the program written in a single call to the model without any feedback.

Figure 1. Dugong growth. We plot the simulated observations as circles where the agent saw them, and crosses where we held them out. Growth slows with age in the fits of both the reference and the first single-attempt program. We draw a line at the predictive median, with a band containing the central 90% of predictive draws. Panel scores are summed held-out log predictive densities (higher is better); coverage counts the held-out observations inside the 90% predictive interval.

Figure 2. Disease spread. We fitted the first single-attempt program to the simulated demonstration data: the agent saw the circles, and we held out the crosses. Its fit follows the rise in infected counts. It scores below the reference. Lines give the predictive medians, and bands the central 90% intervals. Panel scores are summed held-out log predictive densities (higher is better); coverage counts the held-out observations inside the 90% predictive interval.

Demonstration fitting and generation

Consider the dugong reference at age 0.16. We fitted it in a single chain, with 200 tuning iterations followed by 200 retained posterior draws, and repeated each retained draw 20 times for prediction, which gives 4,000 simulated lengths at that age; its 90% predictive interval runs from the 5th to the 95th percentile of those lengths, 0.69 and 1.55. We fitted and sampled every other scored demonstration program the same way. We calculated MCSEs from batch means of each draw's delta-method influence on the total held-out log score, over ten consecutive batches of 20 draws; MCSEs of differences combine the two fits in quadrature, treating them as independent.

The programs that the language model wrote came from calls of at most 2,048 output tokens each, in the critic loop as in the single attempts (the demonstration records give the model calls, tokens and timings for each activity).

Principal findings

We tested four constructed forgeries against one verifier version: all four were refused at model build, and an honest control scored. Our UAI 2026 paper on likelihood hacking, Likelihood hacking in probabilistic program synthesis, appears in the UAI 2026 proceedings.

The training comparisons

We trained the larger model, GLM-5.3-Flash, on 220 packages, with four episodes on each, as in the coral task above. For each task, the package contains its prompt and data, the withheld rows and reference scores. Two checkpoints of that run are compared with the untrained model on 103 evaluation packages from 39 scenario groups held out from training: the checkpoint after three updates, and the final checkpoint after fourteen updates. On each package, we subtract the untrained model's mean utility from the trained model's, which gives the paired difference. Each 95% interval comes from 10,000 resamples of the 39 whole groups.

Checkpoint after three updates

Final checkpoint after fourteen updates

Judge-reward comparison

The fit-only and judge-assisted comparisons sampled their untrained episodes separately and used different verifier versions.

Small-agent training

The smaller model is Qwen3.5-0.8B, used as a tool-using agent. We first gave it a supervised warm start, which trains it to imitate positively rewarded episodes from teacher language models. From that one warm start, we trained it with Subtrajectory Balance (SubTB) in two arms, at learning rates of 1e-5 and 1e-6 (one tenth of the first rate), and with group relative policy optimization (GRPO) in a third, also at 1e-5, each over eight updates on 220 training packages in 81 scenario groups. SubTB is computed on segments of the agent's sequence of actions, and GRPO compares an episode's reward with those of the other episodes on the same task. Every arm is measured against the warm start it began from.

Full supporting results.