Repository navigation
Description
#1384 (merged as 29bfc85) landed seven differential pull requests. Five of them add check kinds that parity_checks/nightly/baselines/differential.json does not have yet. A row missing from the baseline is reported as new, and a new row never alerts. Until the baseline records these rows, a regression in any of the new checks goes unreported.
On #1384's own nightly (run 40904665252) the new kinds add 14,066 rows: 28,949 in all, against the baseline's 14,883, with none missing.
| PR | issue | kinds | rows each |
|---|---|---|---|
| #1316 | #1305 | rename, reorder, rescale_time, split |
1,353 |
| #1318 | #1307 | restart, reuse, batch |
1,353 |
| #1319 | #1309 | stable |
1,353 |
| #1320 | #1311 | steady_vs_run, newton_vs_run |
1,353 |
| #1322 | #1310 | ssa_cc_vs_interp, ssa_backend_as_asked |
268 |
#1317's generated models are not part of the nightly. #1323's NF arm has its own baseline, differential_nf.json, committed in that PR.
Gated on: a scheduled nightly on main at or after 29bfc85. The next one is 2026-10-30 08:17 UTC. A workflow_dispatch of nightly-parity on main gives one sooner.
To do:
- Take that run's
nightly-differentialartifact (verdicts.json) and diff it against the committed baseline withparity_checks/nightly/verdicts.py. Expect the twelve kinds as new, and no alert. - Before recording them, look at the new kinds' DIFF and EXCEPTION rows. Recording one makes it a known failure, so each needs a reason in the pull request, or an issue of its own. On run 40904665252 there were 17:
reorder: BIOMD0000000908 DIFF, 1.09e+134 of the trajectory's peak with species, reactions and parameters reversed. MODEL1812040005 EXCEPTION (CV_REPTD_RHSFUNC_ERRat t = 1).restart: BIOMD0000000827 EXCEPTION (CV_CONV_FAILUREat t = 106).stable: 10 DIFF, each "contested" by differential: attribute a disagreement with a third arm, and flag answers that flip with the grid or tolerance #1309's design (BIOMD0000000620, 621, 755, 1043, 1060; MODEL1109160000, MODEL1911100005, MODEL1912090001, MODEL2310250001, MODEL7908934508). 4 EXCEPTION (BIOMD0000000336, 763, 810; MODEL1006230034).
- Copy
verdicts.jsonover the baseline in a re-baseline pull request. Baseline-only pull requests skip the nightly sweep by the paths filter.
Overlap with #1371. #1371 re-records the same file for the four klu_vs_dense rows #1295 fixed, and is gated on the same kind of run. One re-baseline pull request from one nightly can close both.
Done when the re-baseline pull request is merged and the next nightly reads the twelve kinds as unchanged rather than new.
Activity
nightly: re-record differential.json from the 2026-10-30 nightly (#1395…
Merge pull request #1423 from Montana/nightly/1395-rebaseline-differe…