Skip to content

A fit specification, with a command and a notebook over it - #196

Merged
remislp merged 1 commit into
masterfrom
fit-specification
Sep 11, 2026
Merged

remislp merged 1 commit into
masterfrom
fit-specification

Conversation

@remislp

@remislp remislp commented Sep 11, 2026

Copy link
Copy Markdown
Member

Step 4 of the fitting layer. HJCFitter and FitResult (#194) made a fit expressible in Python; this makes it expressible in a file, so the same description drives a notebook, a command, a batch runner, and — one day — a desktop interface, which is a way of editing a fit specification.

hjcfit template -o my-fit.toml
hjcfit check my-fit.toml             everything a fit does except the search
hjcfit fit my-fit.toml -o out.json   the search, and a result with provenance
  • fitspec.py — the description. Imports only the standard library and executes nothing; a test asserts that from a subprocess, because the point of separating description from run is that anything can read one without acquiring a mechanism library.
  • runner.py — the half that executes: records through dcio, mechanism through scalcs, search through HJCFitter, result as JSON with provenance.
  • cli.py — the command, also python -m HJCFIT.likelihood.cli.
  • examples/fit_template.ipynb + examples/CH82.toml — the same fit as a notebook.

TOML rather than the YAML the plan named

A specification is mostly rate constants, and PyYAML implements YAML 1.1, whose resolver wants a signed exponent. Measured:

written yaml.safe_load tomllib
1e8 '1e8' (str) 1e8
1.0e8 '1.0e8' (str) 1e8
1e+8 '1e+8' (str) 1e8
1.0e+8 1e8 1e8

A rate constant arriving silently as a string is the worst failure this file could have. TOML has one number syntax and is in the standard library from 3.11; tomli covers 3.10 and is in the [fitting] extra. (The .yaml in scalcs/samples is a third thing again — a pickled object graph needing unsafe_load — so YAML in this stack already means something else.)

Two silent defects this surfaced

Both in microscopic reversibility, both guarded in build_mechanism:

  • mr naming a rate outside its cycle. SCALCS warns on stderr and carries on, which is worse than refusing: the rate leaves the free-parameter list while the cycle's constraint stays where it was, so it is neither fitted nor computed. On CH82 with mr = "k(-1)", eight free parameters became seven with k(-1) stuck at its guess.
  • A second rate of a cycle left flagged. A cycle determines exactly one rate and set_mr does not clear the previous one — update_mr carries the comment TODO: check for consistency between cycle.mrconstr and rate.mr. CH82's sample already has 2k*(-2) under MR, so mr = "beta1" gave seven free parameters rather than eight, with 2k*(-2) stuck at 0.411.

Rates that end on a limit are now reported

A rate outside its limits is reset before the likelihood sees it, as HJCFIT did, and nothing announced it — so the value read like an estimate when it is a statement that the model forbids where the likelihood wanted to go. SCALCS gives every rate default limits, so this happens with none set.

log10(L) = 2288.7861   1154 evaluations in 1.3 s
  beta1                5
  ...

Rates that ended on a limit -- these are not estimates:
  beta1                5  at its upper limit of 5

The notebook checks its own curves

A predicted curve over an observed histogram is the one plot where a wrong curve looks like a finding about the fit rather than a bug, so each is checked against a quantity it did not come from.

The open density integrates to 1.00000 with a mean of 5.705 ms against 5.703 observed. The shut density has to be conditioned on [t_res, t_crit], and that is not cosmetic: a burst fit never saw a shut time above t_crit, only 31.6% of the predicted density lies below it, and the fitted mechanism's slowest shut component is 23 800 s in a record two minutes long — a component a burst fit does not identify and was never asked to. Conditioned, its mean is 0.1944 ms against 0.1938 observed.

CI runs the whole tests directory now

tests/test_fitting.py — the 36 tests written for the fitting layer in #194 — was run by no job from the day it was added. The step named two files. A list of test files is one more piece of declared metadata that nothing exercises, which is the fifth time that pattern has cost something in this stack.

Verified

  • The fit through the specification reproduces the one in fitting.rst: 2286.9746 at the guess → 2289.1326 in 1313 evaluations.
  • load_records gives groups identical element for element to read_idealized_bursts, so the two callers of that dcio pairing cannot drift.
  • A wheel built by scikit-build-core from this pyproject.toml writes hjcfit = HJCFIT.likelihood.cli:main into entry_points.txt, with tomli>=1.1; python_version < "3.11" and extra == "fitting" intact.
  • 164 tests pass. The one exception is the console-script test, which needs the package installed from this pyproject.toml and so passes only in CI.

🤖 Generated with Claude Code

Step 4 of the fitting layer. HJCFitter and FitResult made a fit expressible
in Python; this makes it expressible in a file, so that the same description
drives a notebook, a command, a batch runner and anything else -- including,
one day, a desktop interface, which is a way of editing a fit specification.

  hjcfit template -o my-fit.toml
  hjcfit check my-fit.toml            everything a fit does except the search
  hjcfit fit my-fit.toml -o out.json  the search, and a result with provenance

fitspec.py is the description and executes nothing: it imports only the
standard library, and a test asserts that from a subprocess. runner.py is the
half that executes -- records through dcio, mechanism through scalcs, search
through HJCFitter. cli.py is the command; examples/fit_template.ipynb and
examples/CH82.toml are the same fit as a notebook.

TOML rather than the YAML the plan named. A specification is mostly rate
constants, and PyYAML implements YAML 1.1, whose resolver wants a signed
exponent: yaml.safe_load reads 1e8 as the string '1e8', and so does 1.0e8,
and so does 1e+8. Only 1.0e+8 is a float. TOML takes all of them and is in
the standard library from 3.11; tomli covers 3.10 and is in the extra.

Two defects this surfaced, both silent, both guarded in build_mechanism:

* mr naming a rate outside its cycle. SCALCS warns on stderr and carries on,
  which is worse than refusing: the rate leaves the free-parameter list while
  the cycle's constraint stays where it was, so it is neither fitted nor
  computed. On CH82 with mr = "k(-1)", eight free parameters became seven with
  k(-1) stuck at its guess.
* a second rate of a cycle left flagged. A cycle determines one rate and
  set_mr does not clear the previous one -- update_mr carries the comment
  "TODO: check for consistency between cycle.mrconstr and rate.mr". CH82's
  sample already has 2k*(-2) under microscopic reversibility, so mr = "beta1"
  gave seven free parameters rather than eight with 2k*(-2) stuck at 0.411.

Also reported rather than swallowed: rates that end on a limit. A rate outside
its limits is reset before the likelihood sees it, as HJCFIT did, and nothing
announced it -- so the fitted value read like an estimate when it is a
statement that the model forbids where the likelihood wanted to go. SCALCS
gives every rate default limits, so this happens with no limits set.

The notebook draws the predicted distributions over the observed histograms and
checks each curve against a quantity it did not come from, because a wrong
curve there looks like a finding about the fit rather than a bug. The open
density integrates to 1.00000 with a mean of 5.705 ms against 5.703 observed.
The shut density has to be conditioned on [tres, tcrit]: a burst fit never saw
a shut time above tcrit, only 31.6% of the predicted density lies below it, and
the fitted mechanism's slowest shut component is 23 800 s in a record two
minutes long. Conditioned, its mean is 0.1944 ms against 0.1938 observed.

CI now runs the whole tests directory rather than two named files.
tests/test_fitting.py -- the 36 tests written for the fitting layer in #194 --
was run by no job from the day it was added. A list of test files is one more
piece of declared metadata that nothing exercises, which is the fifth time
that pattern has cost something here.

Verified: the fit through the specification reproduces the one in fitting.rst
(2286.9746 at the guess, 2289.1326 in 1313 evaluations); load_records gives
groups identical, element for element, to read_idealized_bursts; and a wheel
built by scikit-build-core from this pyproject writes the hjcfit console script
into entry_points.txt with the marked tomli requirement intact.

164 tests, all passing except the console-script one, which needs the package
installed from this pyproject and so passes only in CI.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@remislp
remislp merged commit 7b8dda1 into master Sep 11, 2026
15 checks passed
@remislp
remislp deleted the fit-specification branch September 11, 2026 20:43
@remislp remislp mentioned this pull request Sep 26, 2026
remislp added a commit that referenced this pull request Sep 26, 2026
1.1.0 published the fitting layer. This publishes a fit written as a file,
and Bayesian inference over the same fit.

  #196  A fit specification (TOML) and the hjcfit command: template, check,
        fit, show.
  #197  HJCFIT.likelihood.mcmc: the posterior of a mechanism's rate
        constants by MCMC, following Epstein et al. (2016). LogPosterior,
        priors, the pilot and adaptive samplers, Chain, ESS and split-R-hat,
        and the Gaussian approximation from a numerical Hessian. Also
        HJCFitter(solver=...), the [likelihood] and [mcmc] tables, and
        hjcfit sample.

288 tests passing.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant