You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
TestProject's array_* builders (src/simlin-engine/src/test_common.rs) accept a subscript that names a dimension the fixture never declares, and nothing between the builder call and the compiler objects.
parse_array_declaration splits the bracket contents on commas and pushes the trimmed strings verbatim into Equation::ApplyToAll(dims, ..) / Equation::Arrayed(dims, ..); build_datamodel copies self.dimensions and self.variables into the Project without cross-checking them. So this builds without complaint:
TestProject::new("x").indexed_dimension("D",2).array_aux("pp[D,2]","1")// no dimension named `2` exists
pp then fails to compile with BadDimensionName, and whatever the test asserts is asserted over a model that never compiled.
How it bit
On main, allocate_target_per_element_partials_decline in src/simlin-engine/src/db/ltm_rank_decline_tests.rs (line 275) built exactly that fixture. The test asserted that ALLOCATE's per-element LTM partials decline. They did "decline" -- because the whole model failed at BadDimensionName, not because the LTM decline logic fired. The test passed on a model that does not compile, and the behaviour it claimed to pin was never exercised.
PR #1057 (branch salsa-roofline-engine) fixed that one fixture (pp[d,xp] with indexed_dimension("xp", 4)) and added an assert_model_compiles guard to every test in that file. The builder itself is unchanged, so the same class recurs on the next fixture someone writes with a typo'd or forgotten dimension.
Why it matters
This is the failure mode the root CLAUDE.md rule "A test that hand-builds its inputs proves nothing about the inputs production supplies" is about: the fixture supplied a datamodel production can never produce (a variable dimensioned over a dimension that does not exist in the same project), and the test passed on an input that does not occur. The guard in ltm_rank_decline_tests.rs protects one file; there are ~2300 array_* call sites across ~60 test files in simlin-engine and libsimlin, none of which are protected.
A check in build_datamodel (not in each array_* method: builder chains declare dimensions and variables in arbitrary order, and build_datamodel is the single funnel every helper -- run_vm, error_diagnostics, flow_exprs, assert_compile_error_vm, ... -- goes through):
For every variable whose equation is ApplyToAll(dims, ..) or Arrayed(dims, ..), every entry of dims must match a dimension in self.dimensions (covers indexed_dimension, indexed_subdimension, named_dimension*, and with_dimension).
Match by canonicalize(name), not ==. The engine resolves dimension names canonically (variable::get_dimensions, GH engine: bare arrayed name inside nested PREVIOUS() in an apply-to-all equation fails to compile #541), and existing fixtures rely on it -- e.g. arrayed_helper_resolves_capitalized_dimension in tests/integration/ltm_array_agg.rs declares Region and the helper references it as region. An exact-match check would reject valid fixtures.
Panic with the variable name and the offending dimension name, e.g. fixture "pp" is dimensioned over undeclared dimension "2" (declared: ["d"]).
Skip when datamodel_override is set (from_datamodel wraps a converter-produced project; the builders are documented no-ops there).
About 20 lines. The cost is the sweep: this runs on every fixture in the workspace, so the change has to land with a cargo test --workspace pass that finds every fixture currently relying on the omission. A grep for tests that deliberately assert BadDimensionName through the builder found none -- the one test that wants an undeclared dimension (test_unknown_element_subscript_skips_unresolved_dimension in db/diagnostic_tests.rs) already builds its datamodel::Project directly via its own helper rather than through TestProject, which is the right shape for a fixture that needs an invalid model. If the sweep does turn up builder-based fixtures that need an undeclared dimension on purpose, they should move to that shape rather than get an opt-out flag on the builder.
Context
Identified during the PR #1057 review of ltm_rank_decline_tests.rs, where the hand-built fixture masked a compile failure as the LTM verdict the test was named for.
Problem
TestProject'sarray_*builders (src/simlin-engine/src/test_common.rs) accept a subscript that names a dimension the fixture never declares, and nothing between the builder call and the compiler objects.parse_array_declarationsplits the bracket contents on commas and pushes the trimmed strings verbatim intoEquation::ApplyToAll(dims, ..)/Equation::Arrayed(dims, ..);build_datamodelcopiesself.dimensionsandself.variablesinto theProjectwithout cross-checking them. So this builds without complaint:ppthen fails to compile withBadDimensionName, and whatever the test asserts is asserted over a model that never compiled.How it bit
On
main,allocate_target_per_element_partials_declineinsrc/simlin-engine/src/db/ltm_rank_decline_tests.rs(line 275) built exactly that fixture. The test asserted that ALLOCATE's per-element LTM partials decline. They did "decline" -- because the whole model failed atBadDimensionName, not because the LTM decline logic fired. The test passed on a model that does not compile, and the behaviour it claimed to pin was never exercised.PR #1057 (branch
salsa-roofline-engine) fixed that one fixture (pp[d,xp]withindexed_dimension("xp", 4)) and added anassert_model_compilesguard to every test in that file. The builder itself is unchanged, so the same class recurs on the next fixture someone writes with a typo'd or forgotten dimension.Why it matters
This is the failure mode the root
CLAUDE.mdrule "A test that hand-builds its inputs proves nothing about the inputs production supplies" is about: the fixture supplied a datamodel production can never produce (a variable dimensioned over a dimension that does not exist in the same project), and the test passed on an input that does not occur. The guard inltm_rank_decline_tests.rsprotects one file; there are ~2300array_*call sites across ~60 test files insimlin-engineandlibsimlin, none of which are protected.Component
src/simlin-engine/src/test_common.rs--TestProject::{array_const, array_const_with_units, array_aux, array_stock, array_flow, array_with_ranges, array_flow_with_ranges, array_with_default_and_overrides, array_aux_direct, array_with_ranges_direct}andbuild_datamodel.Proposed fix
A check in
build_datamodel(not in eacharray_*method: builder chains declare dimensions and variables in arbitrary order, andbuild_datamodelis the single funnel every helper --run_vm,error_diagnostics,flow_exprs,assert_compile_error_vm, ... -- goes through):ApplyToAll(dims, ..)orArrayed(dims, ..), every entry ofdimsmust match a dimension inself.dimensions(coversindexed_dimension,indexed_subdimension,named_dimension*, andwith_dimension).canonicalize(name), not==. The engine resolves dimension names canonically (variable::get_dimensions, GH engine: bare arrayed name inside nested PREVIOUS() in an apply-to-all equation fails to compile #541), and existing fixtures rely on it -- e.g.arrayed_helper_resolves_capitalized_dimensionintests/integration/ltm_array_agg.rsdeclaresRegionand the helper references it asregion. An exact-match check would reject valid fixtures.fixture "pp" is dimensioned over undeclared dimension "2" (declared: ["d"]).datamodel_overrideis set (from_datamodelwraps a converter-produced project; the builders are documented no-ops there).About 20 lines. The cost is the sweep: this runs on every fixture in the workspace, so the change has to land with a
cargo test --workspacepass that finds every fixture currently relying on the omission. A grep for tests that deliberately assertBadDimensionNamethrough the builder found none -- the one test that wants an undeclared dimension (test_unknown_element_subscript_skips_unresolved_dimensionindb/diagnostic_tests.rs) already builds itsdatamodel::Projectdirectly via its own helper rather than throughTestProject, which is the right shape for a fixture that needs an invalid model. If the sweep does turn up builder-based fixtures that need an undeclared dimension on purpose, they should move to that shape rather than get an opt-out flag on the builder.Context
Identified during the PR #1057 review of
ltm_rank_decline_tests.rs, where the hand-built fixture masked a compile failure as the LTM verdict the test was named for.