Skip to content

Read all four submatrices in QuadMatrix::read, not just in Debug - #82

Merged
d-torrance merged 1 commit into
Macaulay2:masterfrom
d-torrance:quadmatrix-read
Sep 2, 2026
Merged

d-torrance merged 1 commit into
Macaulay2:masterfrom
d-torrance:quadmatrix-read

Conversation

@d-torrance

Copy link
Copy Markdown
Member

QuadMatrix::read wrapped the reads of topRight, bottomLeft and
bottomRight in #ifdef MATHICGB_DEBUG:

  const auto topLeftModulus = topLeft.read(file);
#ifdef MATHICGB_DEBUG
  const auto topRightModulus = topRight.read(file);
  const auto bottomLeftModulus = bottomLeft.read(file);
  const auto bottomRightModulus = bottomRight.read(file);
#endif
  MATHICGB_ASSERT(topLeftModulus == topRightModulus);
  ...

In a Release build MATHICGB_ASSERT discards its argument entirely, so the
three modulus comparisons disappear and the three variables they name become
unused — and the reads that produced them were deleted along with the warning.
SparseMatrix::read is not a query: it consumes bytes from the FILE* and
fills the submatrix. So a Release build reads topLeft, stops, leaves three
quadrants empty, and leaves the file offset sitting after the first submatrix.
The reduction then runs on that.

What it looks like

Generating matrices with mgb gb cyclic5 -reducer 26 -storeMatrices 1, then
mgb matrix cyclic5-1 under each build:

build .brmat .rbrmat
Release 20 B 20 B
Debug 228 B 228 B

cmp differs at byte 1 of both. The Release result is not subtly wrong — it is
empty, because only topLeft was ever read.

The fix

Read all four unconditionally and mark the three moduli [[maybe_unused]].

This is the remedy the same commit already chose 60 lines away. 320c4a5
(2022-01-04) is titled "changes so code compiles with only a few compiler
warnings. But [[maybe_unused]] is used, which we might have to revert"
, and
it fixed the identical warning in SparseMatrix::read with

-  const auto colCount = readOne<uint32>(file);
+  [[maybe_unused]] const auto colCount = readOne<uint32>(file);

while taking the #ifdef route here. Both are consuming functions; only one of
the two sites kept its work. (The commit message's worry is moot — stdinc.h
has required C++17 since PR #60.)

The test

src/test/QuadMatrix.cpp gives each quadrant a distinct entry, writes, reads
back and compares all four, so an empty quadrant and a quadrant holding
another's data are both visible.

Release Debug
without the fix FAILED PASSED
with the fix PASSED PASSED

That asymmetry is the bug stated as a test. Note the gap was never the build
matrix — the pre-PR #80 cmake job set no CMAKE_BUILD_TYPE at all, so it built
this broken path too. Nothing exercised QuadMatrix::read.

Verification

  • Without the fix the new test fails in Release and passes in Debug; with it
    both pass, 247/247 each.
  • Release and Debug produce byte-identical .brmat and .rbrmat on
    cyclic5-1, -2, -5 and -9 — six comparisons, none differing.
  • The fixed Release output matches byte-for-byte what Debug produced before the
    fix, so Debug was the correct side.
  • The Release build compiles with no warnings, and autotools make distcheck
    passes with the new test in the tarball.

Scope

320c4a5 added exactly two #ifdef MATHICGB_DEBUG blocks. The other, at
src/test/gb-test.cpp:251, wraps Reducer::ReducerType(reducerType) — a pure
cast, so nothing is lost when it compiles out. It is left alone. The remaining
#ifdef MATHICGB_DEBUG blocks under src/mathicgb/ wrap asserts and debug-only
helpers, not work the release build needs.

This is reachable only through mgb matrix, which aborted before it could show
any of this until PR #74 — which is why a 2022 bug surfaces now.

🤖 Generated with Claude Code

QuadMatrix::read wrapped the reads of topRight, bottomLeft and
bottomRight in #ifdef MATHICGB_DEBUG. In a Release build MATHICGB_ASSERT
discards its argument entirely, so the three asserts that compare the
moduli disappear, the three variables they name become unused -- and the
reads that produced them were deleted along with the warning. A Release
build therefore reads topLeft, stops, and leaves the other three
quadrants empty, with the file offset still sitting after the first
submatrix.

The reduction then runs on that, so a Release mgb matrix and a Debug one
disagree about what a stored matrix reduces to. On cyclic5-1.qmat:

  build     .brmat   .rbrmat
  Release     20 B      20 B     (empty: only topLeft was read)
  Debug      228 B     228 B

with cmp reporting a difference at byte 1 of both files.

Read all four unconditionally and mark the three moduli
[[maybe_unused]], which is what 320c4a5 did for colCount in
SparseMatrix::read -- the same commit, "changes so code compiles with
only a few compiler warnings", that introduced this in 2022-01-04. It
silenced the warning by removing the work instead.

src/test/QuadMatrix.cpp is a regression test for it: give each quadrant
a distinct entry, write, read back, compare all four. It fails in a
Release build without the fix and passes in a Debug one, which is the
bug stated as a test.

Verified: without the fix the new test fails in Release and passes in
Debug; with it both pass, 247/247 in each. Release and Debug also
produce byte-identical .brmat and .rbrmat on cyclic5-1, -2, -5 and -9,
the fixed Release output matches what Debug produced before the fix, the
Release build compiles with no warnings, and autotools make distcheck
passes.

Reachable only through mgb matrix, which aborted before it could show
any of this until PR #74, which is why it surfaced now and not in 2022.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@d-torrance
d-torrance merged commit 53bad6b into Macaulay2:master Sep 2, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant