Skip to content

test: TestProject array_* builders accept a subscript over an undeclared dimension, so a fixture can pass on a model that never compiles #1058

Description

@bpowers

Problem

TestProject's array_* builders (src/simlin-engine/src/test_common.rs) accept a subscript that names a dimension the fixture never declares, and nothing between the builder call and the compiler objects.

parse_array_declaration splits the bracket contents on commas and pushes the trimmed strings verbatim into Equation::ApplyToAll(dims, ..) / Equation::Arrayed(dims, ..); build_datamodel copies self.dimensions and self.variables into the Project without cross-checking them. So this builds without complaint:

TestProject::new("x")
    .indexed_dimension("D", 2)
    .array_aux("pp[D,2]", "1")   // no dimension named `2` exists

pp then fails to compile with BadDimensionName, and whatever the test asserts is asserted over a model that never compiled.

How it bit

On main, allocate_target_per_element_partials_decline in src/simlin-engine/src/db/ltm_rank_decline_tests.rs (line 275) built exactly that fixture. The test asserted that ALLOCATE's per-element LTM partials decline. They did "decline" -- because the whole model failed at BadDimensionName, not because the LTM decline logic fired. The test passed on a model that does not compile, and the behaviour it claimed to pin was never exercised.

PR #1057 (branch salsa-roofline-engine) fixed that one fixture (pp[d,xp] with indexed_dimension("xp", 4)) and added an assert_model_compiles guard to every test in that file. The builder itself is unchanged, so the same class recurs on the next fixture someone writes with a typo'd or forgotten dimension.

Why it matters

This is the failure mode the root CLAUDE.md rule "A test that hand-builds its inputs proves nothing about the inputs production supplies" is about: the fixture supplied a datamodel production can never produce (a variable dimensioned over a dimension that does not exist in the same project), and the test passed on an input that does not occur. The guard in ltm_rank_decline_tests.rs protects one file; there are ~2300 array_* call sites across ~60 test files in simlin-engine and libsimlin, none of which are protected.

Component

  • src/simlin-engine/src/test_common.rs -- TestProject::{array_const, array_const_with_units, array_aux, array_stock, array_flow, array_with_ranges, array_flow_with_ranges, array_with_default_and_overrides, array_aux_direct, array_with_ranges_direct} and build_datamodel.

Proposed fix

A check in build_datamodel (not in each array_* method: builder chains declare dimensions and variables in arbitrary order, and build_datamodel is the single funnel every helper -- run_vm, error_diagnostics, flow_exprs, assert_compile_error_vm, ... -- goes through):

  • For every variable whose equation is ApplyToAll(dims, ..) or Arrayed(dims, ..), every entry of dims must match a dimension in self.dimensions (covers indexed_dimension, indexed_subdimension, named_dimension*, and with_dimension).
  • Match by canonicalize(name), not ==. The engine resolves dimension names canonically (variable::get_dimensions, GH engine: bare arrayed name inside nested PREVIOUS() in an apply-to-all equation fails to compile #541), and existing fixtures rely on it -- e.g. arrayed_helper_resolves_capitalized_dimension in tests/integration/ltm_array_agg.rs declares Region and the helper references it as region. An exact-match check would reject valid fixtures.
  • Panic with the variable name and the offending dimension name, e.g. fixture "pp" is dimensioned over undeclared dimension "2" (declared: ["d"]).
  • Skip when datamodel_override is set (from_datamodel wraps a converter-produced project; the builders are documented no-ops there).

About 20 lines. The cost is the sweep: this runs on every fixture in the workspace, so the change has to land with a cargo test --workspace pass that finds every fixture currently relying on the omission. A grep for tests that deliberately assert BadDimensionName through the builder found none -- the one test that wants an undeclared dimension (test_unknown_element_subscript_skips_unresolved_dimension in db/diagnostic_tests.rs) already builds its datamodel::Project directly via its own helper rather than through TestProject, which is the right shape for a fixture that needs an invalid model. If the sweep does turn up builder-based fixtures that need an undeclared dimension on purpose, they should move to that shape rather than get an opt-out flag on the builder.

Context

Identified during the PR #1057 review of ltm_rank_decline_tests.rs, where the hand-built fixture masked a compile failure as the LTM verdict the test was named for.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions