Repository navigation
ci: run the whole suite nightly with every model through the MIR JIT (#1217) - #1426
Merged
Merged
Conversation
…1217) BNGSIM_CODEGEN_JIT=mir BNGSIM_CODEGEN_THRESHOLD=1 over python/tests is the run that found #1208, #1212 and #1357. mir.yml runs a hand-picked list of codegen files, so a bug outside that list went unseen until someone ran the whole suite by hand. #1350 made the run a signal: the tests that cannot pass under it skip with declared reasons. This makes it a job. forced-jit.yml: nightly at 07:53 UTC, on workflow_dispatch, and on a pull request that edits it. ubuntu-latest and macos-15, two shards each. The build is python-tests.yml's uv sync --extra dev, with MIR added through SKBUILD_CMAKE_DEFINE as sanitizers.yml adds its flags. A guard fails a build without MIR. The run is strict on skips and replaces crashed workers (--max-worker-restart=500). A failure opens or comments on one tracking issue, as sanitizers.yml does.
…o compiler (#1217) test_the_compiled_right_hand_side_flips_where_the_interpreter_does skipped whenever the compiled backend was not cc, with the reason "no C compiler". Under BNGSIM_CODEGEN_JIT=mir the compiler is there and MIR was selected. conftest declares that reason LOCAL_ONLY, so in the forced-JIT job's strict audit the skip failed the run ("!! 2 no C compiler: ..."), with every test passing (17,180 passed, exit 1, macOS arm64). The check holds under MIR: the JIT's run ends where the interpreter's does, to rel 1e-12, on both schedules. It now runs on either compiled backend. Whole suite, BNGSIM_CODEGEN_JIT=mir BNGSIM_CODEGEN_THRESHOLD=1, strict: 17,182 passed, 180 skipped, exit 0. The file by default: 82 passed.
wshlavacek
added
the
changelog exempt
…e it is the last digits (#1217) The forced-JIT job's first run failed one test on ubuntu-latest: test_a_march_the_step_cannot_stop_returns_the_state_the_residual_test_passed got residual 4.6e17, with a species at -8.5e5. The test pins the march's fallback: a march taken on past the state the residual test passes returns that state. It needs a march that passed the test. Reproduced on x86-64 Linux from 15ba102's wheel (gcc, no MIR), steady_state(criterion="residual"): interpreted, closed form converged residual 3.9e-10 (the default) interpreted, fd runs away residual 2.6e8 compiled, compiled form runs away residual 4.6e17 (forced codegen) compiled, fd converged residual 1.8e-10 The state has an eigenvalue with real part +0.13, so whether a march passes the residual test before it runs away is decided in the last digits. An interpreted march runs away too, so this is not a compiled-path bug. On macOS arm64 all forced runs passed it. The test now asks for the interpreted march with codegen=False, which is the march the test describes ("main's"), and it keeps running in the forced job. Linux, BNGSIM_CODEGEN_THRESHOLD=1, codegen=False: residual 8.3e-10, min -3.5e-8, run measure 1.2e15, and the four assertions hold. macOS arm64: the file gives 228 passed by default and 227 passed, 3 skipped under BNGSIM_CODEGEN_JIT=mir BNGSIM_CODEGEN_THRESHOLD=1.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The scheduled job #1217 asked for, now that #1350 made the run a signal.
What
.github/workflows/forced-jit.ymlruns the whole Python suite withBNGSIM_CODEGEN_JIT=mir BNGSIM_CODEGEN_THRESHOLD=1: every model compiled, through the MIR JIT. It runs nightly at 07:53 UTC, betweensanitizers.yml(06:37) andnightly-parity.yml(08:17), onworkflow_dispatch, and on a pull request that edits the file. That last trigger is why this PR runs it.ubuntu-latestandmacos-15, two shards each, aspython-tests.ymlshards. steady_state(method='integration') on the compiled RHS and Jacobian refuses a state at residual 2e-39 and does not retry on difference quotients #1357's compiled march ended at exactly 0 on x86-64 Linux and within 1e-52 of it on arm64 macOS, so one platform is not enough.python-tests.yml'suv sync --extra dev, with MIR added throughSKBUILD_CMAKE_DEFINE, assanitizers.ymladds its flags. A guard fails a build without MIR, where the run would be a false green.BNGSIM_SKIP_AUDIT=strictand--max-worker-restart=500. A JIT bug can kill a worker (mir: move to upstream master 439a90a2 and drop the #413 carry; fix four MIR codegen holes #1208'sexit), and by default xdist stops replacing crashed workers after 4 per worker, silently dropping the rest of the run.sanitizers.ymldoes.A test fix the first local run needed
test_the_compiled_right_hand_side_flips_where_the_interpreter_doesskipped whenever the compiled backend was notcc, as "no C compiler". Under the JIT the compiler is there.conftest.pydeclares that reason local-only, so the strict audit failed the run with every test passing. The check holds under MIR, to rel 1e-12 on both schedules, so it now runs on either compiled backend.Verified (macOS arm64, a MIR build of this branch)
test_stops_on_the_evaluators_flip.py, defaulttest_ci_*,test_pre_push_pytest,test_changelog_fragments,test_lapack_dense_ci_coverage)This PR's own
forced-jitrun is the first on Linux.