Repository navigation
solver_stats: count the RHS evaluations a difference-quotient Jacobian costs (#1333) - #1403
Merged
Merged
Conversation
…n costs (lanl#1333) n_rhs_evals is CVODE's count of the integration's own evaluations, so the ones bngsim's dense and colored quotients spend forming a Jacobian, lanl#1302's jump test included, showed in no counter. n_jac_rhs_evals counts them, and CVODE's own quotient's; n_jac_formations is the number of Jacobians formed, beside n_jac_evals, which counts setups that can reuse one. The nightly carries and compares n_jac_rhs_evals. On jacobian="fd" runs of the ODE perf suite they are 0.5-12x n_rhs_evals.
lanl#1333) CVODE zeroes most of its counters at CVodeReInit, but the linear solver's (CVodeGetNumJacEvals, CVodeGetNumLinRhsEvals) only at the first CVode call after it (cvLsInitialize, from cvInitialSetup). A segment that a re-init opens and nothing steps in still holds the segment before, which reinit_cvode had already banked, so record_solver_stats read it twice. The 10-species chain of test_jac_rhs_evals.py with `E1: at (time > 10): X0 = 10`, run over (0, 10) with jacobian="fd", forms 4 dense quotients (20, 15, 18 and 20 evaluations, traced in the callback) and reported n_jac_formations = 8: 9.1 evaluations a formation for a quotient that takes a column per species, 10, and lanl#1302's test past them. Over (0, 20) the event's restart has steps after it and the two agree. A segment with no f call yet has had no CVode call, so its linear-solver counts are taken as 0. n_jac_evals, the linear-solver setups, is cv_nsetups and CVodeReInit zeroes it; its comment in result.hpp said "Jacobian evaluations", which n_jac_formations now is. The tests: the dense quotient's evaluations per formation are held in (n, 4n] on the restarting run and on the one re-initialized at its end (fails without this: 9.125); the colored count must exceed the formations, since a compiled run's base evaluation alone made it equal to them.
…n step across restarts (lanl#1333) Found by a blind review of the branch. CVODE counts a Jacobian (nje) when it sets jcur and calls the Jacobian function, before it checks the return (cvodes_ls.c cvLsLinSys, cvLsSetup), so n_jac_formations counts the Jacobians it asked for, one a failed f call cut short included, not the ones "actually formed". The docs, the docstring, result.hpp and the changelog say so now. CVODE's own quotient never runs: every path installs a callback of bngsim's before the linear solver is set (setup_linsol_and_jac), so CVodeGetNumLinRhsEvals is 0. The user-facing text no longer lists it as a source of n_jac_rhs_evals; the code still reads it, for a path that one day leaves CVODE to its own quotient. The per-formation bound was 4n, and the restarting run reads 38 a formation for n = 10 with the closed segments' formations dropped from SegmentCounters::operator+=, so that mutant passed it. The chain's law is smooth, so the jump test takes one point past each column and no more: 3n bounds it, and the mutant now fails four tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses #1333, its second part: counting what #1302 costs. The first part, the step of the steady-state solvers' own quotients (Step 4), is a corpus measurement this PR does not make.
What was missing
solver_stats["n_rhs_evals"]is CVODE's count of the integration's own right-hand-side evaluations. Where the Jacobian is formed by difference quotients (jacobian="fd", or a closed form declined), bngsim forms it itself, through #1302's dense quotient and #1254's colored one. Those evaluations went to no counter. #1302's jump test added one to three per tested column, and the nightly's deterministic work counters could not see any of it.The change
n_jac_rhs_evalscounts the right-hand-side evaluations spent forming Jacobians:dq_side_point);CVodeGetNumLinRhsEvals, banked across restarts with the other segment counters).A counter on
Impl, reset withclosed_segmentson both run paths, is reached throughCvodeUserData::jac_rhs_evals. The count is 0 where every Jacobian is closed-form.n_jac_formations(CVodeGetNumJacEvals) is the number of Jacobians actually formed.n_jac_evalscounts the linear solver's setups, and a setup can reuse the last Jacobian, so the existing counter could not say what a formation costs.The nightly (
verdicts.py,_core/work.py, kept in lockstep) carriesn_jac_rhs_evalsand alerts on it (floor +50). A baseline that predates it has no value to compare, so it is skipped until a baseline is re-recorded.The docs:
Result.solver_statsandsolvers.mdsay what each count is.Measured
jacobian="fd", codegen=False,t_span=(0, 100), the ODE perf suite #1302 measured:n_rhs_evalsn_jac_rhs_evalsAbout two evaluations per species per Jacobian formed: its column, and #1302's point past it. On the large models, the right-hand-side work of an FD run was understated by up to 13×.
A traced 10-species chain checks the count exactly: 5 Jacobians formed at 15, 18, 20, 20 and 20 evaluations, which sum to the 93 reported.
Tests (
test_jac_rhs_evals.py, 7)n_speciesper formation (its columns, then the jump test), and at most four times that;Mutations (C++, each rebuilt)
Checked
parity_checks/tests: 958 passed and 1 failed,test_bngsim_provenance, which fails the same way without this change.pre-commit run --all-filesandsphinx -Ware clean.Not in this PR
fd_jacobian.hpp), and The differenced state Jacobian steps every species by 1.5e-8 of the largest concentration: dY_ss/dp with jacobian="fd" is 7% off beside an unrelated species at 1e7 and 44% at 1e8, with no error #1002 added the own-step and half-step columns for a species far below the state's scale since. Whether an error-weight floor saves the march or KINSOL iterations is a corpus measurement of its own, and this PR does not make it.