Repository navigation
[train] main train 2026-11-04: 8 reviewed changes from cekees/proteus - #189
Merged
Merged
Conversation
…har code
The csubgridError tau kernels (A_tau, HJ_tau, ADR_tau, ADR_tau_sd) take a C
`char` ('1' or '2'). Under Python 3 a str or bytes argument is rejected by the
Cython wrapper, so every ASGS subgrid-error class built with the default or an
explicit string stabFlag raised "TypeError: an integer is required" on its first
residual. 61 decks construct these classes.
Measured (calculateSubgridError_HJ_tau, one point, dt=1, |dH|=0):
before: '2' -> TypeError, b'2' -> TypeError
after: ord('2') -> tau = 1.0e4 (the '2' branch)
test/ci/test_poisson.py and test/levelset/rotation/test_rotation2D.py: 6 passed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…. (2013) Port of the continuous Galerkin path of erdc/proteus-gwvd at 14961d2 (2012-07-31), the code behind Povich, Dawson, Farthing & Kees (2013), Comput. Geosci. 17:529: - proteus/gwvd/GwvdCoefficients.py: VariableDensity_mod2_FlowEquationCoefficients (freshwater head, non-divergence form, buoyancy pointwise or through the Knabner-Frolkovic consistent velocity) and VDGroundwater_mod2_TransportCoefficients (mass fraction, Hamilton-Jacobi form, driven by the flow's post-processed mass flux). The C evaluators from gwvdCoefficients.c are reimplemented in numpy. - proteus/gwvd/Gwvd_consolidated_classes.py: Fixed_dt_controller_systemit, NonConservativeBackwardEuler, ISO_fixed_MinModelStep (iterated coupling). - test/gwvd/2dbox: the 2-D static stratified box decks, with GWVD_CV, GWVD_PP, GWVD_RHO_SW and GWVD_T_DAYS overrides (2012 values are the defaults), and plot_fig2.py for the paper's Fig. 2 comparison. One deliberate change: NonConservativeBackwardEuler reports c_t = 0 until its time history is initialized. Current proteus spins up the flow model before the transport history exists, so the flow source omega (drho/dc) c_t / rho saw c_t = omega rho c / 1.0 = 360 in the salt layer. 2-D box, NoCV-LN, 32x64 nodes, dt = 1e4 s, 1 day: before the c_t fix: head range 1.0e5 m after spin-up, c mixed to 0.38 in one step after: head 39.07-43.10 m (hydrostatic), c in [-0.013, 1.010], mean 0.5000 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…days) Used for the 100-day comparison: 10-90% transition at x = 10 m is 8.18 m (NoCV-PE), 1.58 m (CV-PE), 1.46 m (NoCV-LN, CV-LN) against 1.44 m for the eps = 0 run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…options in pro run; every layout checked driver="per-rank" has every rank write its own <name>_p<rank>.h5 with plain HDF5 and no communication; per-rank files hold per-rank datasets, so it turns global_sync off and says so. document= chooses where the grids are described: "global", one <name>.ymf whose every step is a spatial collection of the ranks' grids, each referencing its rank's file, built by master from the gathered grids and rebuilt by readArchiveDomain from master's file (the roster's layout B); or "per-rank", one <name>_p<rank>.ymf per rank, each step a single grid, each rank appending its own with no collective step (layout C). Every rank's file carries its own metadata rows and attributes, so a per-rank hot start reads nothing but its own file. pro run takes --archive-mode, --archive-driver and --archive-document alongside parun; RunOptions carries them. check_archive is the one integrity check every layout goes through: each document validates, each collection has its steps in the expected shape, every data item names a dataset that exists in the file it says with the dimensions it declares, the files on disk are the expected ones, and a per-rank-file global document is rebuilt from master's file. It runs over the single-file drivers x both dataset layouts x 1-3 ranks and the per-rank driver x both documents x 1-3 ranks, plus the per-rank hot start and running documents, and the invalid combinations are refused by name. pro run's options are checked end to end. Landed on cekees/main 9fca0b9 (decision 1 of the ymf mesh plan, C7.1), cherry-picked from archive-modes with no redesign. Two conflicts, both with work merged since the commit was written: the metadata writer now sizes its dataset through _dataset (the hot-start reuse/replace helper), so the per-rank row count feeds that helper instead of the old create-or-open; and the archive is format 2 (time outside, space inside), so the new tests read collections through archive_collections instead of domain["time_collections"]. The live document's per-instant records already branched on uniformSteps, so per-rank documents append single grids per space. Measured (macOS arm64, MPICH, FI_PROVIDER=tcp, ymf proteustoolkit main d466645): test_archive_parallel_metadata.py 87 passed (rank script at 1, 2, 3 ranks; 66 before this commit) archive/hotstart/pro CLI/read paths 292 passed, 1 skipped parun -n 2 ladr_2d (KSP), per-rank driver: document=global ladr_2d.ymf + _p0.h5 + _p1.h5, 11 steps x 2 grids, 154/154 data items resolve; rebuilt from _p0.h5 document=per-rank _p0/_p1.ymf, 11 steps x 1 grid each, 77/77 resolve Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t way On 2026-10-31 a #138 run sat 64 min in the suite (normal 11) with three ranks of `cube_mg_mpi.py hex 3 2 mgm` spinning in PMPI_Waitall under MatMultTranspose_MPIAIJ. Whatever the cause, the launcher in test_cube_multigrid.py had no timeout, so retrying_run (test/conftest.py) could not scale one and never set PROTEUS_TEST_DUMP_AFTER: the helper's mpi_failfast never dumped the ranks' stacks and the job held the runner until GitHub's limit. Ten launches in test/multilevel had no timeout (test_cube_multigrid, test_hierarchical_mesh, test_multigrid, test_parallel_cube_meshes, test_parallel_mgm, test_parallel_nested_iteration, test_parallel_smoothers x2, test_petsc_pcmg, test_vertex_interpolation, test_launch_retry). Each gets the budget its neighbours already use: 600 s where a multigrid cycle runs, 300 s for the mesh/transfer checks, 120 s for the launch-retry probe. retrying_run scales it by PROTEUS_TEST_TIMEOUT_SCALE and dumps every rank's stack at 80% of it. The other cube_mg_mpi.py launchers (test_coarse_solvers, test_cube_q2, test_partition_arrays, test_without_petsc4py) already had one. test_launchers_have_timeouts.py parses every file in test/multilevel and fails on a subprocess.run/call/check_*/Popen with no timeout keyword, so the next launcher cannot regress. (Outside test/multilevel, 43 of 79 launches have none; not this commit's territory.) Measured, macOS arm64, conda MPICH 5.0.1, FI_PROVIDER=tcp, -n 6 xdist, --import-mode=importlib: the 11 changed files, 151 passed in 105 s; the slowest test 44.6 s (two launches), so the budgets have >= 7x headroom before PROTEUS_TEST_TIMEOUT_SCALE. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The gwvd handoff put the slow Larson-Niklasson (pwl) runs down to pwl refactoring its node-star systems on every call. Measured, it does not: the stars are factored once (dgetc2 in getConservationJacobianPWL, guarded by updateConservationJacobian, which only a moving domain resets) and each later call back-solves (dgesc2). The time was getElementwiseFlux, a diagnostic (its element_flux is read only by test_elementwise_flux.py) that getConservationResidualPWL calls twice per post-processing call, as a Python loop over elements, faces, points and coordinates. Under cProfile it took 3.03 s of 3.11 s for five calls. It is now vectorised over elements and summed in the old loop's order (face, point, coordinate), so each element's flux is the same floating-point sum: element_flux is bit-identical to the old loop (pwl, pwl-bdm on 8,192 elements, pwl-bdm2 P2 on 2,048), and nothing else reads it. The -opt variants never called it. Per post-processing call, serial, vpp_poisson_2d: pwl 8,192 elements P1 0.371 s -> 0.0034 s pwl-bdm 8,192 elements P1 0.378 s -> 0.0050 s pwl-bdm2 2,048 elements P2 0.111 s -> 0.0019 s gwvd 2dbox (#175 deck, GWVD_PP=1 = pwl, 32 x 64 nodes, serial) to 30 days: 138.5 s -> 18.5 s, all 27 archived datasets bit-identical; 1000 days extrapolates from about 77 min to about 10 min. Tests: test/post_processing/test_velocity_postprocessing.py 31/31 and test_elementwise_flux.py 1/1 (Mac, env proteus-bc-261025). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
conda-forge published pip 26.2.2 build _1 (pyh145f28c_1 16:34 UTC, pyh8b19718_1 16:38 UTC, 2026-11-02). It drops pip's bundled cacert.pem and patches certifi.where() to return sys.prefix/ssl/cacert.pem. `python -m build` installs the build requirements with `pip --python <isolated venv>`, so pip runs inside the venv, where that file does not exist, and "Build sdist" failed with FileNotFoundError in truststore on every leg that relaxes the file's pip =24.0 to a floor. The fork's CI overlay fixed it in 4ad703d; this is the same change on main's workflow, which removes one of the overlay's differences from main. Measured: on the fork's dev run 41639026962 (overlay with --no-isolation) "Build sdist" and "Install sdist with default build isolation" pass on all nine legs that reached them, macOS Intel and arm64, linux-64 py3.12/3.13, linux-aarch64. proteustoolkit main run 41631333572 (2026-11-03 11:54 UTC) passed without the change because its legs resolved pip 26.2.2 pyh145f28c_0; build _2 followed _1 at 22:43 UTC on 11-02. So main is not failing today; it is one resolver choice away from it, and the sdist never needed an isolated environment: the next step still installs it with default build isolation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The previous commit worded the comment differently from the fork overlay's 4ad703d, and the two edits overlap, so #178 could not merge into dev (no CI) and the hourly fork-sync would have hit the same conflict merging main into dev after the train lands. With these lines byte-identical to dev's, `git merge-tree HEAD cekees/dev` is clean and the overlay's hunk disappears once this reaches main. The leading "Fork overlay:" words can be dropped from main in a later commit, which then merges into dev cleanly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…red too C3.7. The coverage leg (linux-64 py3.12) measured only its main suite and kept coverage.xml as an artifact nobody read. Now: - both suites of that leg run under coverage, and each artifact carries the raw .coverage data (include-hidden-files); the FSI one is named "-fsi"; - a `coverage` job, after the matrix, combines the two through [tool.coverage.paths], publishes the per-subpackage table and the 40 modules with the most untested statements in the run summary, keeps the combined XML/HTML as `coverage-combined`, and fails if a module's statement coverage is below its floor; - .github/coverage/floors.json holds a floor for each of the 140 measured modules, today's value rounded down to 0.1%, and ratchet.py checks them. A module may sit below its floor by max(0.5 pp, two statements' worth) before it counts as a drop. New modules are listed, not checked; a missing FSI artifact reports without enforcing (--partial). Lowering a floor is an edit to floors.json in the pull request that lowers it; --write-floors raises them from a combined report and never lowers one. Floors are from proteustoolkit main 9fca0b9, run 41631333572, main suite only: 87,237 statements, 52.5% covered (the plan's 50.9% was an older run). The fork's run 41642966289 of the same commit gave an identical report, module for module (0 of 140 differ), so the measurement is deterministic and the tolerance only has to absorb code edits, not noise. Checked locally: the floors pass against their own report; raising one floor by 5 pp (MeshTools) exits 1 and names it. The FSI suite's contribution and the combine step are first measured by this PR's own run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and against the exact flux
dg (RT0) and dg-bdm (BDM1) project the interior-penalty numerical flux of a DG solution, which balances every element
by construction. Nothing tested them; dg-bdm's face velocities were not its own, for three reasons:
- getElementBoundaryBDM1velocityValuesLagrangeRep (postprocessing.c) zeroed the right slot and accumulated into
ebq_velocity[eN*ebN*nq*nd + nB*nq*nd + ...], reading the shape functions at [ebN*nq*iq*nDOF + k];
- its wrapper passed ebq_v.shape[2] (face quadrature points) as the number of shape functions;
- with DG P2 it evaluated the linear representation with the quadratic basis (vt.ebq['v']); it now uses the P1 test
space's traces, and every face's ebq_global (and ebqe) velocity is its left element's trace, where before only
exterior faces were written ("tjp hack" comment, removed).
The element velocities were right throughout (BDM face moments match the numerical flux to 1e-16).
Measured (vpp_poisson_2d, 13 x 13 nodes unless noted), dg-bdm before -> after:
element residual from the face velocities P1 34.2 -> 1.2e-13 P2 48.9 -> 1.6e-13
max face-flux error against the exact flux P1 34.2 -> 4.3e-3 P2 48.9 -> 1.7e-13 (25 x 25: 1.29 -> 2.9e-13)
dg balanced already (1.2e-13 P1, 1.6e-13 P2). L2 velocity error, 7 / 13 / 25 nodes:
dg P1 0.126 0.0629 0.0313 (rate 1.0) dg P2 0.124 0.0621 0.0311 (1.0, RT0)
dg-bdm P1 0.263 0.132 0.0662 (1.0) dg-bdm P2 0.0143 0.00359 0.000897 (2.0)
Element partition, one layer of overlap, pinned, MUMPS: 2 and 4 ranks match serial to 4.1e-12 for both variants and
both orders; residuals 1.4e-13.
Harness: VPP_SPACE=dg|ldg picks DG P1/P2 with SIPG or LDG (element partition, pinned like the node partitions);
summary.json records the linear solver's converged reason, the L2 velocity error and the largest face-flux error
against -a grad u, and (VPP_PROBLEM=lowk, used by the next commit) the flux through a low-K block's faces. _run
asserts a positive converged reason: with CG/ASM every DG run on 2-4 ranks stopped at 5000 iterations (reason -3)
and "differed" from serial by up to 2.2, the documented harness trap. DG systems are solved directly.
The VPP chooser's guard for an element partition without overlap called exit(), which ends every rank with status
0; it now raises (test: 2 ranks, C0 P1 with element partitioning at overlap 0).
Tests: test_velocity_postprocessing.py 46/46 (31 + 15 new), Mac, env proteus-bc-261025, FI_PROVIDER=tcp.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…K block, serial and on 2-4 ranks pwl-ib-fix-0 (the low-K "hack": faces where the conductivity jumps by more than 1e4 to below 1e-5 become no-flux) had its own copy of VPP_PWL_RT0's node-star setup (359 lines) and neither of the two parallel exchanges round 1 added to pwl (#149). It is now a subclass of VPP_PWL_RT0 (191 lines, omitFluxBoundaryNodes=False as before) that keeps its interface flagging, its no-flux face values and its three C calls, and calls the owner-to-ghost star exchange after the star solve and the cut-face exchange after the residual, with the flagged faces zeroed first. Serial output is bit-identical to the old class. Its per-face print() is a level-3 logEvent; the P2 branch tests its own component (solutionTestSpaceIsNotPWL.get(ci)). Problem: vpp_poisson_2d_p VPP_PROBLEM=lowk, a = 1 with a block (1/3, 2/3)^2 of conductivity k = 1e-8, u = 1 / 0 on x = 0 / 1, no flux on y = 0, 1; 13 x 13 nodes, 32 block faces. Measured: flux through the block faces pwl 0.037; pwl-ib-fix-0 0 (exact flux is O(k)) element imbalance, serial pwl 3.5e-15; pwl-ib-fix-0 3.8e-9 (0.38 k at k = 1e-6, 1e-8, 1e-10) pwl-ib-fix-0 vs serial, before -> after: 2 ranks overlap 0 0.30 -> 2.7e-8 (block faces leaked 2.4e-2 -> 0) 4 ranks overlap 0 0.44 -> 3.3e-8 (3.6e-2 -> 0) 2 ranks overlap 1 7.3e-3 -> 3.3e-8 4 ranks overlap 1 1.1e-2 -> 5.0e-8 The parallel answer matches serial to the conductivity's order, not to round-off (4 ranks: 4.5e-6, 5.0e-8, 3.9e-10 at k = 1e-6, 1e-8, 1e-10). Its stars are solved unpinned (calculateConservationFluxPWL_noNeumannFix); a star the flagged faces split is a singular system whose parts do not balance by O(k), so dgesc2's answer depends on the order of the star's elements, which differs between ranks. Round-off agreement would need a pin per connected part of each star chosen by global element number (C, not done). Tests: test_velocity_postprocessing.py 51/51 (5 new), Mac, env proteus-bc-261025. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…and the LDG solve found wrong
point-eval-gwvd's postprocess is a no-op: the velocity is what the transport's LDG (mixed-form) flux writes into
q[('velocity',ci)] (Transport.calculateElementResidual, sparse-diffusion branch, "added for ldg coupling tjp"), and
no face velocity is set. With any other flux nothing writes that array, so it handed on zeros (measured with SIPG
DG P1: velocity 0). It now raises unless the flux is a Diffusion_LDG (or subclass) with sparse diffusion.
evaluateElementVelocityField read q[('velocity_dofs',ci)], which nothing in proteus sets (the 2011 code's
calculateVelocityQuadrature_MixedForm2_vdof_sd, d258a11, was never wrapped): a KeyError, now a RuntimeError that
says so. No deck in the tree, in #175 (gwvd uses pwl, pwl-opt and point-eval) or on main uses it.
Tested on vpp_poisson_2d with DG P1 and Diffusion_LDG (VPP_SPACE=ldg):
q velocity minus the LDG velocity 0 (bit-identical)
face fluxes from the face velocities 0 (none set; the elements do not balance through them)
L2 velocity error against -a grad u 1.7e2 (strict xfail; RT0 from DG SIPG gives 0.063)
The LDG solve itself is wrong: max |u - (x^2 + y^2)| = 13.2 (also 7.1 for the Laplacian with Dirichlet data
everywhere, where SIPG gives 2.2e-3); more Newton iterations diverge (1, 5, 30: 7.7, 2.6e2, 5.6e12); and three
identical runs gave 13.2174, 13.2199, 13.2169, so something in the LDG path reads uninitialised memory. Its Python
arrays are all zero-initialised; the C mixed-form routines are the next place to look. LDG is outside this territory
(NumericalFlux.py) and has no test anywhere in the tree; not fixed here. No parallel comparison: the variant does no
communication, and the transport's answer is not deterministic.
Tests: test_velocity_postprocessing.py 53 passed, 1 xfailed (2 new + the xfail), Mac, env proteus-bc-261025.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…o longer swallowed
generateMesh wrapped both the choice of options and the meshing in one bare
except, and on any exception meshed again with the numerics module as the
options. A real error from the mesher (a bad triangle flag, a missing file)
was hidden behind whatever the retry did: success with other options, or a
different, later error. The choice already exists alone as
chooseMeshOptions (domain options unless they give no size, else the n
module with p's genMesh), so generateMesh now chooses with it and meshes
once.
Test (test/mesh_tests/test_generate_mesh_errors.py, a spy on _generateMesh
that fails the first call):
before: generateMesh returned the second try's mesh, 2 calls (DID NOT RAISE)
after: RuntimeError("the mesher failed") raised, 1 call
options with no size still fall back to the n module: passes before and after
Mesh-dependent suites (mesh_tests, ci, poisson_2d, pro CLI, solver_tests,
domain_ymf, adr_problem, griffiths_lane_6, cylinder2D): 338 passed, 1 skipped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he class sets
The non-partitioned branch read domain.polyfile, which MeshTetgenDomain
never sets (it stores its fileprefix as meshfile, as the -F branch above
already uses), so every MeshTetgenDomain run without -F stopped with
AttributeError before reading a file.
Test (3x3x3 cube written as tetgen files, read through _generateMesh):
before: AttributeError: 'MeshTetgenDomain' object has no attribute 'polyfile'
after: the mesh read back has the source's 27 nodes, 48 tetrahedra and
bit-identical nodeArray
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd fileprefix
The DMPlexDomain branch of _generateMesh was a copy of the tetgen branch:
under -F (generatePartitionedMeshFromFiles) it tested
meshOptions.triangleOptions and then partitioned tetgen files named by
fileprefix, which is never assigned in that branch. A DMPlex domain has no
tetgen files to partition, so the branch now says so and points at the
path that works (no -F: build from the DMPlex, partition in memory).
Test (a 2x2x2 hexahedral DMPlex box; hexahedra because a simplex box needs
a tetrahedral generator, which conda-forge PETSc lacks):
before: TypeError: argument of type 'NoneType' is not iterable (default
triangleOptions; with 'f' and 'ee' set it reached the unbound
fileprefix, UnboundLocalError)
after: ValueError naming the DMPlex domain and the -F option
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tests moved from proteus/tests to test, and five Makefile lines still
named the old root: the serial and 4-rank partition checks
(proteus/tests/ci/test_meshPartitionFromTetgenFiles.py) and three py.test
targets with their --ignore lists. proteus/tests still exists only as a
stray tracked deck directory (proteus/tests/richards), so the py.test
targets collected nothing useful rather than failing.
Test (test/test_makefile_paths.py: every test path the Makefile names exists):
before: 8 missing, e.g. proteus/tests/ci/test_meshPartitionFromTetgenFiles.py,
proteus/tests/solver_tests, proteus/tests/periodic/*test_periodic.py
after: 0 missing
test/ci/test_meshPartitionFromTetgenFiles.py: 2 passed serial and on 2 ranks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… names The C10a round-1 docstring said no deck or test uses p1-nc. test_single_phase_gw.py::test_ncp1 does (it passes, CI and Mac), and three more in-tree decks name it; nothing in air-water-vv (55 conservativeFlux decks: pwl-bdm-opt 37, point-eval 18) or the manuscript decks on 1.9.x and main does. Measured on vpp_poisson_2d with NC_AffineLinearOnSimplexWithNodalBasis (a scratch copy of the harness, not committed): element residual from the face velocities 2.5e-16, 1.7e-16, 1.1e-16 on 7, 13, 25 nodes L2 velocity error against -a grad u 0.115, 0.0594, 0.0303 (first order) 2 and 4 ranks, overlap 0 and 1, pinned serial answer to 9.0e-14; residual 2.2e-16 The removal flag stays (cekees's decision). The two misspelled names it cited, f_ci_wieght (non-sparse-diffusion branch) and cpostprocesssing (second-potential branch), are corrected: neither branch is reached by the default sparse-diffusion single-potential path, which is why test_ncp1 passes, and either would raise NameError. Tests: test_single_phase_gw.py 3/3; test_velocity_postprocessing.py 53 passed, 1 xfailed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…refuses int() of a 1-d array
generateFromPlex finds, for each face of a cell, the cell's node not on
that face, as a one-element array, and converted it with int(). NumPy
deprecated int() of an array with ndim > 0 and 2.5 makes it a TypeError,
so every DMPlex domain failed on the first cell, whatever the plex. The
index is now taken with .item(). Found writing the DMPlex importer test
(C7.1): that test is added here, a round trip of a tetrahedral mesh
through PETSc and back.
Test (3x3x3 cube, 48 tets, handed to DMPlex.createFromCellList, interpolated,
read through a DMPlexDomain; NumPy 2.5.2, PETSc 3.25.4):
before: TypeError: only 0-dimensional arrays can be converted to Python scalars
after: nodeArray bit-identical; the same 48 tetrahedra as node sets (the
order within a tet is proteus's, node i opposite face i); 120 faces,
48 exterior faces, 98 edges, as the source mesh
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n none match The first run of the coverage job (#178, run 41682330352) passed without checking anything. `coverage combine` maps the runner's site-packages onto the checkout, and the XML it writes names each file relative to its <source> (/home/runner/work/proteus/proteus/proteus), so modules came out as "ADR.py" and "mprans/RANS2P.py"; no floor named one, every module was "new", and nothing was enforced. A single run's report names files by their absolute site-packages path, which is what the floors were recorded from, so the local checks agreed with themselves. ratchet.py now joins each filename to the report's <source> and keeps the path from the last "/proteus/" (the package has no subdirectory of that name), and it exits 1 when fewer than half of the floors name a module in the report, so a naming mismatch can no longer pass as "nothing dropped". Measured on that run's combined artifact: 140 of 140 floors matched, 87,237 statements, 52.6% covered (main suite alone 52.5%), no module below its floor, and one at least a point above it: the FSI suite takes proteus/mprans/RANS2P.py from 83.7% to 87.2%. The proteustoolkit main report still passes (exit 0); the same combined report with its <source> rewritten exits 1 with "only 0 of 140 floors name a module". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t family The #163 harness keyed its report by model name, so a multi-component model reported only its last component (RANS2P: v; RANS3PF: v), and it matched dofs by (x, y), so in 3D the dofs that share x and y were compared in arbitrary order. Report name[i] for component i and match by (x, y, z). New cases, serial against 2 ranks, largest relative difference over every model component (macOS arm64, partition pinned with -mat_partitioning_type current): | deck | models | atol | 2 ranks | |---|---|---|---| | damBreakLevelSet, t=0.5 | RANS2P, VOF, NCLS, RDLS, MCorr | 1e-10 | 9.6e-15 (3 ranks 6.0e-14; deck tolerances 3.1e-14) | | damBreakLevelSet interface=1, t=0.5 | RANS2P, CLSVOF | 1e-10 | 2.4e-13 | | marin (3D), he=0.5, t=0.1 | CLSVOF, RANS3PF projection | 1e-10 | 2.4e-10 (before the z fix: 1.6 in w) | | moses (3D, surface tension), t=0.1 | CLSVOF, RANS3PF projection | 1e-10 | 7.6e-12 (before: 9.5e-1) | The four #163 decks with every component: damBreak 5.5e-13, TwoDimBucklingFlow 2.5e-11, fillingTank 2.0e-6, risingBubble 2.3e-12 (constant removed); the RANS3PF u component was not compared before and agrees as well. Harness options: MAXITS raises maxNonlinearIts (RDLS stalls at ~1e-7 in 50 iterations for atol 1e-10); DIR + SO run an so-file deck; DIRECT=1 replaces a serial-only LU by a PETSc direct KSP in both runs. damBreakLevelSet.py gets interface=1 (RANS2P with CLSVOF); its default is unchanged. damBreak_solver_options.py is not covered: it uses the removed TwoPhaseFlowProblem(ns_model=...) constructor and does not run (no test uses it). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…losed tank pressureincrement_p.py set fixNullSpace=True with openTop=True. With the top open the Dirichlet condition there fixes the increment, so the serial pin of dof 0 changed the answer (its row became u[0] = 0), and in parallel PresInc's substitution of a constant null space projected a constant out of a nonsingular problem: the 2-rank run's Newton for PresInc diverged (residual x30 per iteration with a direct solve) and the run failed. Now fixNullSpace = not openTop, matching how pressureincrement_n.py already picks LU (open) or the null-space KSP (closed). Serial against 2 ranks (RANS3PF + VOF3P + NCLS3P + RDLS + MCorr3P, nd=2, atol 1e-10, PETSc direct solves): the 2-rank run failed -> runs; difference 2.0 in MCorr3P, 1.9e-2 in velocity (fixed in the next two commits). Whether PresInc should substitute the null space at all when the problem has Dirichlet conditions is part of the open pressure-normalization decision. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nodes take the owners' values Two nodal arrays were built by element loops and read at ghost nodes: - interface_locator (NCLS, NCLS3P): a node is marked when an element around it is cut; a ghost node sees only the elements this rank has, so it could stay unmarked while its owner is marked. RDLS freezes the marked nodes (freeze_interface_within_elliptic_redist) over all local dofs. CLSVOF already scattered its locator. - lumped_qx/qy/qz (RDLS, ELLIPTIC_REDISTANCING=2 and 3): a lumped L2 projection of the normal; a ghost node holds a partial sum, and the owned rows of the elliptic redistancing residual read it. Both are scattered from the owners after they are computed (NCLS3P and RDLS get the scatterDOFArray their siblings have). Measured with a ghost-consistency check (every dof-shaped array, ghost entry against the owner's): both arrays consistent after the fix. RDLS's residual at its first Newton iterate, serial against 2 ranks: 1.1e-2 -> 5.6e-17 (owned rows, same u). surface_tension/rising_bubble_rans3p (nd=2, atol 1e-10), serial against 2 ranks: 2.0 -> 1.1e-2 (MCorr3P), velocity 1.9e-2 -> 1.4e-5. The rest is the entropy-viscosity normalization (next commit). Serial: bit-identical. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NCLS3P (EXPLICIT_METHOD), VOF3P and VOF (STABILIZATION_TYPE 1) and TADR (TaylorGalerkinEV) set, per element,
numDiff = min(cMax h |v|max, cE h^2 |R_entropy|max / norm_factor), norm_factor = max(|Emax - Emean|, |Emean - Emin|)
over the domain. Each kernel took the mean, max and min over its local elements, ghosts included, with no
reduction, so every rank had its own norm_factor and the ghost elements' numDiff disagreed with their owners'
(measured: ghost-element q[('numDiff', 0)] off by 2.5e-6 in NCLS3P from the third step).
The kernels now accumulate the statistics over owned elements only and hand them out (entropy_stats) with each
element's two viscosities (element_viscosity); the numDiff they write is only read by later calls (the residual uses
q_numDiff_u_last), so in parallel the new EntropyViscosity.reduce_numDiff reduces the statistics over the ranks and
rewrites q_numDiff_u. In serial nElements_owned == nElements_global and nothing is rewritten: bit-identical
(rising bubble serial against main: 0.0 on every field).
surface_tension/rising_bubble_rans3p, nd=2, atol 1e-10, serial against 2 ranks, with the two previous commits:
| field | before | after |
|---|---|---|
| MCorr3P | 1.1e-2 | 5.5e-14 |
| VOF3P | 1.2e-5 | 8.6e-17 |
| NCLS3P / RDLS | 5.7e-6 / 5.3e-6 | 1.9e-16 / 1.9e-16 |
| velocity u / v | 1.1e-5 / 1.4e-5 | 1.7e-15 / 1.4e-15 |
| PresInc / pressure | 8.8e-6 / 1.7e-7 | 5.4e-15 / 1.3e-16 |
(SuperLU_DIST 6.9e-14.) The deck is now a case of test_twophaseflow_partition_independence.py.
VOF and TADR get the identical change; their Taylor-Galerkin paths are not measured in parallel: no in-tree test
uses TADR's (it diverges on the disk of test/TADR without FCT, serial and parallel alike), and the TwoPhaseFlow VOF
decks use other stabilizations. Their serial stored-solution tests pass (test/TADR, test/LS_with_edgeBased_EV).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ed it
C3.4. 39 test files (and 16 decks/helpers) import importlib.reload, most of
it copied from before tests stopped sharing state. Measured before editing,
on ceds-model in proteus-ci-main (the CI conda env: Open MPI 5.0.11, py3.12,
PETSc 3.26.1, ymf@main), each file in its own pytest process:
- every reload made a no-op (a plugin replacing importlib.reload): 23 of
the 39 files still pass, 16 fail. In the 16 the first test passes and
the later ones fail: each test sets Context options and reloads its p/n
decks so they read them again at import. Those keep their reload.
- the 23 were then edited and all 39 files run in ONE process (what an
xdist worker sees), in list order and reversed, against the unedited
tree. Two of the 23 do need reload across files and are left alone:
test_rotation2D (wrong values, 340/340 entries, after another file
changed default_s/so/p/n) and test_single_phase_gw (NameError
connectionList from a deck that is not re-executed). HotStart_3P is left
alone too: its comment documents a dependency between differently
configured runs that passing does not disprove.
The remaining 20 lose 71 lines: test_bochev_pressure_stabilization imported
reload and never called it; reload(PETSc) (AddedMass x2,
test_nse_RANS2P_step) reloads a compiled extension, which CPython does not
re-initialise, so it never did anything; the rest reloaded decks or
default_* modules that no other test in the file or the set changes.
After the edit, all 39 files in one process: forward 165 tests, 0 failed
(unedited: 0); reversed, 33 failed, the identical set the unedited tree
fails (test_gauges, poisson_2d weak cases, TADR, VOF, NCLS, vortex,
vortex2D, CLSVOF with RANS2P). That reversed-order failure is main's
today, with every reload in place: reload does not make these files
order-independent, and CI passes only through how xdist deals them out.
The weekly multi-way run (next commit) is what watches that.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… fails C3.3. The suite has to pass however pytest runs it, not only as sdist.yml runs it (-n auto --dist load, file order). That property regresses silently, so it gets a weekly canary rather than a per-PR gate: pytest-ways.yml, Sundays 03:41 UTC and on demand, one linux-64 leg on the committed lock (environment-openmpi-dev.yml, as sdist.yml's locked leg), the suite run as serial -p no:xdist, file order serial-random -p no:xdist, pytest-randomly seeded by the run number load -n auto --dist load (the control: CI's way) load-random -n auto --dist load, random order loadfile -n auto --dist loadfile forked -n auto --dist load --forked with sdist.yml's ignore list and the two FSI IBM simulations deselected. A scheduled run with a failing way opens "Weekly pytest-ways canary failed", or comments on it if one is open (issues: write, GITHUB_TOKEN). --dist loadgroup is not one of the ways: pytest 9.2 broke it (pytest-xdist#1386) and proteustoolkit#1298 removed its groups. It will not be green on its first run. Measured with the previous commit's setup (ceds-model, CI env): the 39 test files that import reload pass in one process in file order and fail 33 tests in reverse order, the same with or without that commit. Expect serial-random and load-random to find those. Checked: the six argument sets parse with pytest 9.1.1, xdist 3.6.1, forked 1.7.5 and pytest-randomly 3.x on a toy suite; the workflow has not run (a schedule runs from the default branch only, so its first run is after the train). sdist.yml gets a comment that run_like_ci.sh and this file repeat its ignore list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… unreadable mesh files raise
A PSLG run with triangle not on PATH (an unactivated env) failed in the
partitioner with an IndexError, two steps from its cause (C32's note,
from the vdft session's report): writeMeshFiles ran triangle through the
shell and ignored exit 127, and the triangle reader, given no files,
printed "cannot open file" and returned a mesh with no nodes.
writeMeshFiles now runs triangle and tetgen through _runMesher, which
raises with the command and its output on a nonzero exit (gmsh with
check=True), and cmeshTools.generateFromTriangleFiles /
generateFromTetgenFiles(Parallel) raise IOError when readTriangleMesh /
readTetgenMesh report failure.
Tests:
generateFromTriangleFiles / generateFromTetgenFiles("nothere"):
before: an empty mesh, no error after: IOError naming nothere.node
writeMeshFiles on a box PSLG with PATH holding no triangle:
before: returns, no mesh files after: RuntimeError, exit status 127
Mesh-dependent suites (as in the first commit): 342 passed, 1 skipped;
test_meshPartitionFromTetgenFiles: 2 passed on 2 ranks.
C32's test_meshPartitionEmptyMesh.py expects 'empty' from the
partitioner on this path; the reader now raises before it gets there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…leaving them waiting
With the mesher's exit status now checked (previous commit), a failure
on rank 0 raised there while the other ranks waited at the barrier for
its files: under parun the exception aborts the job, but under pytest or
any caller that does not abort, the run hung. _generateMesh's PSLG and
PLC branches now call writeMeshFiles through _writeMeshFilesOnMaster,
which broadcasts rank 0's failure so every rank raises it. Seen while
checking the previous commit: test_meshPartitionFromTetgenFiles under
mpiexec -n 2 hung once, because its teardown runs on both ranks and one
rank can delete meshNoVessel.poly before the other's tetgen reads it (a
pre-existing race in that test; before the exit-status check tetgen's
failure there was silent).
Test (mpiexec -n 2, writeMeshFiles replaced by one that raises on rank 0):
before: timeout at 300 s, both ranks still running
after: rank 0 raises the mesher's error, rank 1 raises "mesh generation
failed on rank 0: RuntimeError: ..."; 0.5 s
Mesh-dependent suites: 343 passed, 1 skipped. test_meshPartitionFromTetgenFiles
under mpiexec -n 2, 4 runs: no hang; 3 of 4 show the teardown race as one
rank's FileNotFoundError (that test's file, reported to C32).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ened test/step_retry_mpi.py ran TwoPhaseFlow decks only and forced failures only through Newton.solve. Now: - DIR + SO run another deck (an so file, p and n files, or "--SWEs deck.py"); FT and OPTS set its options; MAXITS raises maxNonlinearIts (RDLS at atol 1e-10). - FAILSTEP=n fails the model's n-th model step at solveMultilevel, for solvers that do not go through Newton.solve (the explicit shallow-water solvers, under the base MultilevelNonlinearSolver). - the accepted times are recorded and REPLAY works for Sequential_MinModelStep too (SWFlow), not only the adaptive controller; - retries.txt counts the restores, and the test requires one: a TwoStageNewton ignores a failed first stage (its second stage resets failedFlag), so a FAILAT on a first stage silently forced nothing; - dofs are matched by x, y and z; a replay takes every option of the case except the failure; the history-file check applies where RANS2P/RANS3PF write them. No change to the three existing cases (5.6e-13, 5.6e-13, 3.8e-13 against their replays). Limit found: REPLAY steers choose_dt_system, but the first step's dt comes from initialize_dt_system, so a failure in the first step cannot be replayed; cases fail a later step. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he dofs restoreSystemStepState put back each model's dofs and then rebuilt the nonlinear solver's unknown vector (uList) from them with setFreeDOF. A model's Newton starts from uList, and uList need not equal the dofs at the start of a step: RDLS's dofs are changed after its last solve (the level set is copied back), so an uninterrupted step started RDLS from the uList of its last solve and the retry from the dofs. saveSystemStepState now keeps a copy of uList per level and the restore puts it back. Found by restoring every float array and number of every model on a retry (BRUTE restore: retry = replay to 1.4e-13) and bisecting the keys down to redist_p0.ti.u, which is RDLS's uList. surface_tension/rising_bubble_rans3p (RANS3PF + VOF3P + NCLS3P + RDLS + MCorr3P, nd=2, atol 1e-10), one forced failure against a replay of the same accepted times, largest relative difference over every model: | failed model | main | after | |---|---|---| | MCorr3P, RANS3PF or PresInc | 7.6e-1 | 2.7e-13 | | Pres | 7.6e-1 | 1.1e-5 (next commit) | | VOF3P, NCLS3P, RDLS (second stage) | not measured | 0.0 | The new case is in test_step_retry.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PresInc's postStep writes the flow model's coefficients.ebqe_velocity_last (the boundary velocity RANS3PF reads as its star velocity in the next step). When Pres failed after PresInc's postStep, the retry read the failed attempt's boundary velocity. It is now part of RANS3PF.Coefficients.state() (so a hot start archives and restores it as well; an archive without it restores the rest as before). Found by comparing every model's arrays at the retried step against the replay (identical after RANS3PF's preStep except PresInc's boundary fluxes) and restoring the RANS3PF coefficients by name. surface_tension/rising_bubble_rans3p, Pres failure against the replay: 1.1e-5 -> 2.7e-13 (with the previous commit; on main 7.6e-1). fillingTank (CLSVOF + RANS3PF projection), Pres failure: 5.6e-13. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A failed CLSVOF solve leaves interface_locator set by its last residual, and the frozen-interface loop of the next solve (preRedistancing stage) reads it before the kernel recomputes it. It joins u_dof_old and quantDOFs in CLSVOF's state arrays, so the retry starts from the step's locator. damBreakLevelSet interface=1 (RANS2P + CLSVOF, atol 1e-10, t=0.1), CLSVOF failure against the replay: 1.1e-3 -> 0.0. fillingTank (CLSVOF + RANS3PF): 2.0e-13 for CLSVOF, RANS3PF and PresInc failures (unchanged). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n be retried The shallow-water RKEV integrators go through their stages in place: the stage counter, the stage values, the transport's *_dof_old arrays (each stage's starting solution) and dLow (computed by choose_dt from the step's initial state for the first stage) all change during a step. They had no state(), so a step that failed in a middle stage was retried with the counter past the last stage: "assert self.lstage > 0 and self.lstage <= self.timeOrder" in updateStage. Both RKEV classes now carry these in state()/set_state(); state_contract_inventory moves them from unmeasured to state. SWFlow circular_damBreak (t=0.1), forced failure against a replay of the same accepted times: | case | main | after | |---|---|---| | SW2DCV failing in a middle stage (FAILAT=6) | AssertionError | 0.0 (2.7e-7 without dLow) | | SW2DCV failing after a whole step (FAILSTEP=4) | 1.6e-15 | 0.0 | | GN_SW2DCV failing after a whole step (FAILSTEP=4) | 0.0 | 0.0 | GN's explicit solver cannot be failed in a middle stage by the harness; its class gets the identical state(). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (gauge plan step 2)
A hot-started run reopened every gauge file and the legacy RANS2P/RANS3PF histories with "w", so a run continued
from an archive lost every row before the restart instant (point and line gauges kept them only after a PUMI
adapt). Now one rule for all of them, in proteus.Gauges:
- reset_history_files(hotStart) is called by NumericalSolution before any model opens a file;
- open_history_file(path) opens "w", or on a hot start keeps an existing file ("a+") and says so;
- each writer registers how to cut its files back (on_hot_start): Gauges by the time column (keep_rows_up_to: the
header and the rows at or before t; the sample grid is re-anchored on the first kept row and nothing is written
again at or before t), RANS2P/RANS3PF by step count through timeHistory.txt (keep_steps_up_to: rows per step per
file, one per boundary flag for RANS2P's per-flag files);
- NumericalSolution calls resume_history_files(t) with the restart instant (the archived step it restarts from) once
the initial conditions are read and before the gauges take their initial sample.
The gauge cut is collective over the gauge communicator (rank 0 reads the file and broadcasts the kept times).
RANS3PF cylinder of test/HotStart_3P, 14 steps, hot start at step 8, against the uninterrupted run:
| file | main | after |
|---|---|---|
| timeHistory/forceHistory_p/_v/moment/wettedArea | 6 of 15 rows | byte-identical |
| point gauge, every step | 8 of 17 rows | byte-identical |
| point gauge, sampleRate 0.012 (not a multiple of dt) | 4 of 7 rows, grid restarted at t=0.04 | byte-identical |
| line-integral gauge | 8 of 17 rows | byte-identical |
test/test_hotstart_history_files.py runs it (driver option HS_GAUGES) and checks the two cutting helpers directly.
Not done here (gauge plan step 3, decisions open): boundary-integral gauges, headers for the legacy files, series in
the archive. The rule for which instant is the restart instant is C17's (archived_times, hotStartTime).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit and 180d400 (pushed with a placeholder message by mistake; it added the script) are the local-run recipe the 2026-11-01 plan asks for: a local full-suite run showed 53 failed / 8 errors that were "baseline", and the fleet's direct nightly shows 7-31 failures per node on commits CI passes. run_like_ci.sh runs the suite as sdist.yml does: -n auto --dist load, --import-mode=importlib from a scratch directory, sdist.yml's ignore list and FSI deselection, one BLAS/OpenMP thread, PROTEUS_TEST_TIMEOUT_SCALE=auto, FI_PROVIDER=tcp under MPICH, OMPI_MCA_pml=ob1, the env's own bin first on PATH. It prints what decides agreement first: the python, the proteus it imports, the ymf it actually imports (and its commit) next to ymf@main, PETSc's version and external packages (warning when MUMPS is missing), and the mpiexec. --inplace runs this checkout; --workers N sets -n. Measured, ceds-model, proteus-ci-main (the CI conda env: Open MPI 5.0.11, py3.12, PETSc 3.26.1 with MUMPS), ymf@main d466645 on PYTHONPATH, this branch at 180d400, --inplace --workers 3: 2170 passed, 2 skipped, 1 xfailed, 0 failed, 29:49 under a co-tenant load of 130-260 on 192 cores. CI's linux-64 py3.12 main suite on this PR (run 41682330352): 2163 passed, 2 skipped, 1 xfailed. Every one of CI's 2166 node ids is in the local run with the same outcome; the 7 extra are test_import_hygiene cases over files that a checkout has and an installed package does not (now said in the header). Not run on a Mac in full: the Mac was at load 97-137 from other sessions; a two-file run there behaves. Why the fleet nightly disagrees with CI (nightly f392a15, 2026-11-03, logs read on ceds-model and spark2): none of it is code. - no MUMPS in the pip PETSc of dev_pip: 19 tests (test_adr_problem's null-space, periodic and Taylor-Hood solves, test_pro_cli's null space, two test_rank_interface_flux cases); dev_pip_mumps drops them; - 8 left on every node with MUMPS, all partition-dependent under that pip PETSc 3.26.0/ParMETIS build: 6 dgp1 multilevel cases on 3 ranks ("at least one of the 3 ranks owns none"), the RANS2P no-overlap step (1.2e-5 against rtol 1e-6), SW2DCV's circular dam break on 2 ranks; - spark2 adds 4 exact-zero mat-vec checks (8.9e-16 on aarch64 with the node's own compiler; CI's conda aarch64 build gets 0); - and the nightly runs Python 3.10/3.12, serially with --forked, without FI_PROVIDER, against whichever ymf the node's env has. The script's banner shows each of these before the first test runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the first error when both fail
The first commit of this PR meshed once with the chosen options. CI
(linux-aarch64, 14 failures) showed the retry is load-bearing: decks whose
domain MeshOptions pass chooseMeshOptions' size test but cannot mesh their
domain, and that meshed only through the bare-except retry with the n
module:
- CLSVOF/with_RANS2P: a RectangularDomain with triangle options and no
node counts (TypeError: %i format ... NoneType in the box branch);
- tadr/ncls/subdomain-mesh-size partition tests: a box PSLG whose
domain-options triangle run leaves no box.1.node (now an IOError from
the reader, commit 6; before, an empty mesh that failed later).
generateMesh now tries the chosen options, and on failure (when they were
the domain's) logs the error and meshes with the n module as before. What
was swallowed is fixed: when the retry fails too, the first error is
raised, with the retry's attached as its context, instead of the retry's
error alone.
Tests (spy on _generateMesh):
first fails, retry fails: before: the retry's AttributeError, first lost
after: the first RuntimeError, retry's as __context__
first fails, retry succeeds: before and after: the retry's mesh
Locally: the 14 CI failures' files (test_tadr/ncls partition independence,
test_subdomain_mesh_size, CLSVOF) 32 passed; mesh-dependent suites plus
mcorr/vof partition and TwoPhaseFlow: 359 passed, 1 skipped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ymf (cekees/ymf#7, C42) lets a discretization split the model into sub-problems solved in order each step: two-phase level-set flow as flow, VOF, level set, redistancing and mass correction. ymf_run solves the model as one ADR system. For a model that classifies as one system, running a split branch that way would solve a different discretization under the branch's name, silently. blockers() now refuses such a branch, naming its sub-problems, until they are mapped onto Proteus's models (C5's dispatcher). Nothing changes for an unsplit branch. Measured: test_pro_cli.py 79 passed, 11 skipped (was 78; one new test), macOS arm64, env proteus-bc-261025, against both ymf main (0624788) and cekees/ymf coef-unknowns (bf566e4); the new test needs no ymf schema change, since blockers reads the in-memory entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cekees's decisions on the modules the coverage table shows at or near zero (plan section 5, decision 6), applied to the floors: - proteus/m_comp_co2 (2,984 statements, 0%) is not legacy (proteustoolkit#1282); it gets a test from a small deck Arnob will provide. Until then it has no floor and is listed as "test pending". - proteus/Viewers.py (1,617 statements, 6.6%) is excluded from the floors until C46 (runtime visualization, a new roster placeholder) decides what happens to it. - proteus/SimTools.py keeps its floor at today's 11.9%; C5 replaces it. - proteus/InputTranslators.py keeps its 2.1% floor here. It is deleted in 2.0 in a separate pull request; once that lands, the ratchet lists the floor under "modules no longer measured", which does not fail. To express "measured, not checked", a floor may now be a string, the reason. Such modules are shown in their own table with the reason and are never compared; --write-floors leaves them alone, so making one a number is a reviewed edit. Checked against this PR's combined report (run 41682330352): exit 0, the three entries listed with their reasons; --write-floors on a copy keeps them and raises only RANS2P.py (83.7 -> 87.2) and TwoPhaseFlowProblem.py (94.6 -> 95.0), and the rewritten file passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cekees decided (2026-11-04, plan section 5, decision 6): delete
proteus/InputTranslators.py in 2.0 and keep it on 1.9.x.
The module holds three readers of old external formats: GF (a vessel
geometry description, to a PLC), Ipars and ADH_metfile, 744 lines. Nothing
in proteus, its tests or its docs uses them. Its users were:
- proteus/__init__.py, which listed it in __all__ (a star import pulled
it in);
- scripts/gf2poly, installed by setup.py, which is still Python 2
("print args") and so has not run since the 2to3 transition;
- scripts/ipars2pyadh, not installed.
All three go, and gf2poly leaves setup.py's scripts list. The API docs are
generated by sphinx-apidoc from the package, so the module's page goes
with it; no hand-written page names it.
Coverage: the module was 2.1% covered (11 of 511 statements) on main
9fca0b9; the C3.7 ratchet (#178) will list its floor as a
module no longer measured, which does not fail.
Checked on macOS arm64 (proteus-bc-261025, extensions seeded from a build
of the same C sources): `from proteus import *` binds all 54 remaining
__all__ names; test/test_import_hygiene.py 161 passed; `git grep` finds no
InputTranslators, gf2poly or ipars2pyadh outside the issue archive.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… ranks (decision 2) In a closed tank PresInc, Pres and PresInit are fixed only up to a constant, and each run picked its own: PresInc pinned dof 0 in serial and used a constant null space in parallel, PresInit took its solver's constant. Pressures, pressure gauges and per-flag pressure forces depended on the rank count. Decided by cekees 2026-11-03: zero mean. Each of the three coefficients classes declares zeroMean, and NumericalSolution.postStep shifts the model to zero integral mean (proteus.mprans.PressureNormalization) before the model's own postStep, so before any other model reads it: the dofs, the values at the quadrature points and the solver's unknowns (uList, which the next solve starts from; all dofs are free here). The mean is mass-weighted, sum_i m_i p_i / sum_i m_i with m_i = int phi_i over the owned dofs and reduced over the ranks: the integral mean of the discrete pressure, independent of the mesh grading (a plain dof average is not). It is applied only when the model has no Dirichlet condition anywhere (no weak Dirichlet face, no strong Dirichlet dof, checked once over all ranks); an open top or a prescribed outlet pressure already fixes the pressure and is left alone. risingBubble (closed tank, refinement=1, t=0.05, atol 1e-12), serial against 2 ranks, nothing removed: | field | before (a constant, on a 1142 Pa range) | after | |---|---|---| | PresInc | 8.1 | 4.0e-12 | | Pres | -28 | 4.8e-14 | | PresInit | -63 | 3.0e-15 | The strict xfail from #163 is now a pass. Serial against this branch before: the pressures move by a constant only (-8.4, 201, 242; non-constant part <= 5.4e-13), velocity and CLSVOF by <= 8.6e-16. Open domains decline (zeroMean resolves False for all three models) and are bit-identical to before: damBreak, fillingTank, surface_tension rising bubble (open top). HotStart_3P (manufactured solution, pure-Neumann pressure) is normalized and its hot start stays bit-identical (test_hotstart_rans3pf, test_hotstart_history_files); shifting the dofs without uList broke that (4e-16), which is why the shift lives in NumericalSolution, which holds uList. PresInc's serial dof-0 pin and parallel null space are unchanged: whatever constant the solve picks is removed after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nge needed on main) Reported from the VDFT work (abarua-ce/proteus VDFT_barua_kees, test/salinity/comp_geoscience_reproduce): TADR FCT with STABILIZATION_TYPE 5 differed from serial by 0.077 and 3.8% of the mass on 4 ranks, because the limiter ratios Rpos/Rneg were not exchanged. On main FCTStep has scattered uLow, the nodal porosity, Rpos/Rneg and the limited solution since C0 rounds 2 and 3, and the STAB=5 hook goes through the same FCTStep. Measured here on the Zalesak disk of test/TADR (problem=1, refinement=2, t=0.08, Newton atol 1e-10), serial against N ranks: | ranks | max |u - u_serial| | total mass, relative | |---|---|---| | 2 | 3.6e-11 | 3.3e-13 | | 4 | 3.5e-11 | 3.8e-13 | | 2, scatters off (control) | 4.9e-1 | 8.3e-3 | | 4, scatters off (control) | 5.2e-1 | 1.5e-3 | So the reported defect is that of a branch without those scatters. tadr_partition_mpi.py gets STAB=5 (backward Euler + Newton in the n-file) and pins equal strips along the same cut direction on more than 2 ranks (the 2-rank cut is unchanged; 4 ranks used to leave ranks without elements). The 2-rank case joins test_tadr_edge_based_on_two_ranks_is_the_serial_answer with its no-scatter control. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tory gauge plan step 3)
Decisions (cekees, 2026-11-04): one CSV per gauge with a header row; the legacy RANS2P/RANS3PF history files stay
behind a flag, on by default for one release; the moment reference point is set per tag in the gauge definition.
(Series in the .h5 archive are step 5, not here.)
proteus.Gauges.BoundaryIntegralGauges(tags, quantities, moment_points, sampleRate, activeTime, units, fileName) is an
auxiliary variable of the flow model. Tags are {name: boundary flag > 0} (or a list of flags); quantities are
wetted_area, force_pressure, force_viscous, force (their sum) and moment, per component (x, y[, z]; the moment's z in
2D). It reads the per-flag integrals the kernel already computes over owned faces and reduces over the ranks, after
each accepted system step, so a failed and retried attempt writes no row. The moment about a declared point is
M_b + (b - point) x F from the kernel's moment about the flag's barycenter b; a tag without a point keeps b. The file
is "time,tag,quantity,component,units,value", one row per sample, tag, quantity and component, with SI units per unit
depth in 2D unless overridden; sampleRate uses the PointGauges grid; a hot start keeps the rows up to the restart
instant (the step-2 helpers). No row at t0: no residual has run then.
RANS2P/RANS3PF Coefficients.legacyHistoryFiles (default True) turns the old timeHistory/forceHistory/momentHistory/
wettedAreaHistory/particle_* files off.
Measured (macOS arm64), every tag x quantity x component on the tank flags 1-4:
| check | result |
|---|---|
| damBreak (RANS3PF projection, atol 1e-12, t=0.1), serial vs 2 ranks | 1.2e-11 relative (largest on values ~1e-18) |
| damBreakLevelSet (RANS2P, atol 1e-10, t=0.5), serial vs 2 ranks | 1.8e-11 |
| fillingTank, one forced CLSVOF failure vs its replay | same rows, one sample per accepted step (34), values 2.4e-13 |
| HotStart_3P cylinder, hot start at step 8 vs uninterrupted | byte-identical |
| legacyHistoryFiles = False (damBreak, damBreakLevelSet) | no legacy files, gauge rows written |
Tests: test_boundary_integral_gauges.py (values, the moment shift, refused definitions), new cases in
test_twophaseflow_partition_independence.py, the fillingTank retry case and the hot-start file test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On ceds-model, 2026-11-04 00:18 CST, a rank of bc_transfers_mpi.py (user gh-runner, PPID 1) had been running for 10 h 43 min. It came from cekees/proteus run 41700828360 (linux-64 py3.12 on ceds-model-2), cancelled at 19:35:16 UTC on 11-03: the runner's orphan sweep terminated prterun 456810 but not the rank 456826 it had just forked, and the rank, alone, held up stream D's quiet-window check for hours. The question for afb9a7e's timeouts was whether they end such a tree. Measured: they did not, under Open MPI. - Open MPI 5.0.11 (the CI Linux env): prterun puts each rank in a process group of its own, inside prterun's session. subprocess.run's timeout SIGKILLs prterun alone; all three ranks of a 3-rank job stayed alive, reparented to init. Killing prterun's process group does not reach them either. - MPICH 5.0.1 (macOS): each rank takes a session of its own, but dies with its hydra proxy, which stays in the launcher's process group. So test/conftest.py now runs every launch in a session of its own (run_in_own_session, a subprocess.run that retrying_run wraps), and when a launch times out, is interrupted or exits non-zero, end_session terminates the launcher's process group and then every process still in its session, waits up to 5 s, then kills what is left. Once the launcher has exited, a process it left that still holds the output pipes is no longer waited for (communicate would have waited for its EOF until the timeout, or for ever without one). A SIGTERM to a pytest process, as at the start of a cancelled job, ends the launches still running before the process exits. It never acts on pytest's own session. Tests (test/test_mpi_launch_retry.py): a launcher that leaves a child in a process group of its own (Open MPI's layout, no MPI needed), timed out and exiting 3; three never-finishing ranks under mpiexec, timed out; stdin given to a launch that outlasts the first one-second wait. Measured: - ceds-model, Open MPI, the CI env: the three new launch tests fail against the previous conftest (3 failed, leaving 2 children and 3 ranks with PPID 1) and pass with this one (3 passed, 19 s). - full suite through test/run_like_ci.sh --inplace --workers 4: 2174 passed, 2 skipped, 1 xfailed, 0 failed in 15:03 (CI's 2166 tests, the 7 checkout-only import-hygiene cases, these 4 tests). (A first run found the stdin bug fixed here: input passed again on the second wait, "Cannot send input after starting communication", in two test_pro_cli shell tests.) - macOS arm64, MPICH: test/multilevel + test_mpi_launch_retry.py 643 passed in 187 s; test_mpi_launch_retry.py + test_pro_cli.py 105 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The orphan of 2026-11-03 (previous commit) came from a cancellation, not a timeout. At the end of a job the runner terminates the processes that carry the job's RUNNER_TRACKING_ID, from one list it makes once; run 41700828360's log shows that list ending in prterun 456810, and the rank 456826 that prterun forked a moment later was not on it. A cancelled pytest may never run test/conftest.py's clean-up, so the workflow gets a backstop. The new last step of the test job runs on self-hosted Linux whether the job passed, failed or was cancelled (if: always()). It finds every process whose environment holds this job's RUNNER_TRACKING_ID (grep -z over /proc/*/environ), skips its own shell and that shell's ancestors, sends SIGTERM, then SIGKILL after 3 s, and repeats until a sweep finds nothing (at most five rounds). It is best effort (continue-on-error) and prints what it ended. GitHub-hosted runners are discarded after the job and skip it. Checked on ceds-model with the step's script: an orphan (PPID 1, a process group of its own) carrying a test id was ended in round 1; a decoy with another id and the script's own shell were left alone. The step is outside the fork overlay's hunks: `git merge-tree` with cekees/dev is clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner
Author
|
@claude review |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Additional findings (outside the current diff — GitHub can't attach inline comments there):
-
🔴
proteus/mprans/EntropyViscosity.py— Parallel runs with explicit entropy viscosity can get NaN numerical diffusion when the entropy is uniform, which serial runs and the base branch do not. With E constant,norm_factoris 0 and an element with zero entropy residual gives 0/0 = NaN.np.minimumpasses NaN through, but the C kernels usefmin, which drops it and keeps the linear viscosity. Fix: compute numDiff the way the kernels do, e.g.np.fmin(element_viscosity[:, 0], element_viscosity[:, 1] / norm_factor)undernp.errstate(divide='ignore', invalid='ignore'), so the 0/0 and x/0 cases both fall back to the linear viscosity.Why this was flagged
The trigger is a run on more than one rank with STABILIZATION_TYPE == 1 (TADR, VOF, VOF3P) or EXPLICIT_METHOD (NCLS3P), starting from a field whose entropy is the same everywhere. A tracer that starts at zero is a common example. Then max, min and mean of E are equal, so
norm_factorat EntropyViscosity.py:28 is 0, and every element withmaxEntRes == 0haselement_viscosity[:,1] == 0. Line 33 computes0/0 = NaN.np.minimumreturns NaN for those elements, and line 34 writes NaN intoshockCapturing.numDiff[0], which the next residual reads, so the solution goes NaN. The kernels compute the same ratio (for example VOF.h:1250-1254) but applyfmin(linear_viscosity, entropy_viscosity), and Cfminreturns the non-NaN argument. Serial runs never reach this code because of thecomm.size() == 1early return, and on the base branch the kernel's own value is kept, so only parallel runs are affected after this change.Verification: With E uniform, norm_factor at proteus/mprans/EntropyViscosity.py:28 is 0. Line 29 then gives 0/0 = NaN,
np.minimumpasses it through, and line 30 writes it into q_numDiff_u. The kernels usefmin(TADR.h:1567-1571) and keep the linear viscosity, so serial runs and the base branch are fine.reduce_numDiffreturns early only when comm.size()==1.
…back, gauge headers on a hot start Three defects found by review of proteustoolkit#1335 (main train 2026-11-04), each confirmed against the code and covered by a test that fails without the fix. 1. EntropyViscosity.reduce_numDiff (from #184). With a uniform entropy, norm_factor is 0 and an element with zero entropy residual gave 0/0 = NaN. np.minimum kept the NaN and wrote it into q_numDiff_u, so on 2+ ranks the solution went NaN. Serial runs and the kernels use C fmin, which keeps the linear viscosity. It now uses np.fmin under errstate, the kernels' rule. test/test_entropy_viscosity_reduce.py: uniform entropy gives NaN before, the linear viscosity after. 2. Archiver.readArchiveDomain (from #176). When <name>.h5 was missing it fell back to <name>_p0.h5 without reading the document layout. With document="per-rank" that file holds only rank 0's subdomain, so gatherTimes silently wrote a _complete.ymf covering rank 0 alone. It now raises, naming the per-rank files, when the document is per-rank and other ranks' files exist. check_archive in test_archive_parallel_metadata.py asserts the error: layout C at 2 and 3 ranks assembled rank 0 alone before, and raises after. 3. Gauges.resumeAt (from #184, gauge plan step 2). A hot start kept the old file's header as it was, so with moved, added or reordered gauges under the same file name, new rows sat under the old labels. The header is now built by headerLine() and compared with the kept one. If they differ, the file starts over with this run's header and a log line. Two tests in test_hotstart_history_files.py: same gauges keep the rows up to t, other gauges start the file over. macOS arm64, proteus-bc-261025, FI_PROVIDER=tcp, -mat_partitioning_type current: test_entropy_viscosity_reduce.py 2 passed test_hotstart_history_files.py 6 passed test_archive_parallel_metadata.py -k per_rank_file_layout 6 passed test_archive_read_paths.py 21 passed test_tadr_partition_independence.py 10 passed Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fork CI on the combination before the proteustoolkit pull request merges. Each change below was reviewed, and passed CI, as its own pull request on cekees/proteus. This branch merges them
onto
mainat 9fca0b9 with one merge commit per pull request, in the order listed.Merge with "Create a merge commit", not squash or rebase, so the commits keep their hashes and the fork's pull requests close themselves.
Diff against
main: 107 files changed, 4117 insertions(+), 1364 deletions(-).🤖 Generated with Claude Code