Skip to content

Validate user input where it enters the library, rather than asserting on it - #85

Merged
d-torrance merged 4 commits into
Macaulay2:masterfrom
d-torrance:assert-on-user-input
Sep 18, 2026
Merged

d-torrance merged 4 commits into
Macaulay2:masterfrom
d-torrance:assert-on-user-input

Conversation

@d-torrance

Copy link
Copy Markdown
Member

Three checks on user input were in the wrong place: two aborted a Debug build
on input a Release build reported and declined, and the third sat where no
test could reach it. PR #66 and PR #69 settled the reasoning for the first
shape; PR #83 said the third would want its own PR.

-divisorLookup and -monomialTable

staticMonoLookupMake's switch asserted before it threw, and nothing
validated the code earlier:

$ mgb gb cyclic5 -divisorLookup 99      # Release
Unknown code for monomial data structure

$ mgb gb cyclic5 -divisorLookup 99      # Debug
mgb: StaticMonoMap.hpp:431: Assertion `false' failed.
Aborted (core dumped)

Deleting the assert would make the two agree, but it would leave the value
unchecked until it reached a switch four call levels down. PR #69 faced the
same choice and took the other route — it put the modulus check in
F4Reducer's constructor and left the asserts downstream, noting that
"establishing the bound here lets [them] assert it". So this checks the code
at the two entry points that take it from outside, MonoLookup::makeFactory
and ModuleMonoSet::make, and leaves the switch alone.

The error arrives when the factory is made rather than when it is first used,
and it lists the codes:

ERROR: Unknown code 99 for the monomial data structure.
The supported codes are:
  1   list, using divmasks
  2   KD-tree, using divmasks
  3   list
  4   KD-tree

-monomialTable reaches ModuleMonoSet::make, a second path with the same
abort, and is covered too.

The streaming interface

MATHICGB_STREAM_CHECK asserted before it threw at twenty sites in the public
streaming interface, so a Debug build aborted on exactly the caller protocol
violations the macro exists to report — calling idealBegin() twice, and so
on. Nothing had ever caught it: every test that drives an ideal stream follows
the protocol, so no check had fired, and before PR #80 no CI cell built this
code with assertions at all.

This also removes the 19 clang warnings PR #81's CI turned up in the macOS
debug cells. They came from the assert(("message", condition)) idiom, whose
left operand genuinely has no effect; deleting the assert removes the idiom.

The matrix file's modulus

PR #74's primality check lives in src/cli/MatrixAction.cpp, which is the one
place it cannot be tested — mathicgb-unit-tests links no src/cli object
and there is no CLI-level harness.

It moves into SparseMatrix::read, beside the size check PR #83 added, where
the modulus is still intact and a test can reach it. QuadMatrix::read needs
nothing of its own: it reads its four submatrices through
SparseMatrix::read, so all four are checked. The move also covers a read the
CLI never checked at all — the reference .rbrmat that a second run compares
its output against.

What stays in the CLI is the file name, which read cannot know, having only
a FILE*:

$ mgb matrix composite.brmat
While reading composite.brmat:
ERROR: The modulus 100 is not prime. MathicGB only supports prime fields.

PR #83's oversized-modulus error gets the file name the same way.

Four commits

  1. 41dce00 — MonoLookup::displayCodes and ModuleMonoSet::displayCodes
    had byte-identical bodies; both now read one table next to the switch that
    enumerates the same codes. No functional change.
  2. ac4054b — validate the code at the two entry points, with
    src/test/MonoLookup.cpp covering both.
  3. ecd68c4 — MATHICGB_STREAM_CHECK throws without asserting, plus the
    first test to exercise any of those twenty checks.
  4. 2f12500 — the primality check moves into SparseMatrix::read, with a
    test at each entry point.

Each builds and passes standalone.

Verification

  • 253/253 in cmake Release and Debug, no build warnings from GCC 13.3.
    Test count 248 → 253.
  • autotools make distcheck passes.
  • clang 18.1.3 on src/mathicgb.cpp: 19 warnings before, 0 after.
  • Each new test fails without the check it covers.
  • mgb gb -divisorLookup 99 and mgb sig -monomialTable 99 now report
    identically in Debug and Release.
  • mgb matrix refuses a composite-modulus .brmat with exit 255; a prime one
    reduces and writes its output as before.

One thing this does not fix

mgb gb -monomialTable 99 still exits 0. That option is offered by
GBCommonParams, which both actions share, but only SigGBAction reads it,
so for the gb action any value is accepted because none is used. That is a
separate defect from the one here and wants its own change.

🤖 Generated with Claude Code

d-torrance and others added 4 commits September 2, 2026 08:39
MonoLookup::displayCodes and ModuleMonoSet::displayCodes had
byte-identical bodies listing codes 1 through 4, and the switch in
staticMonoLookupMake enumerates the same four. The two displayCodes now
read a table next to that switch.

The duplication was never two independent facts. Both MonoLookup and
ModuleMonoSet dispatch through staticMonoLookupMake, and both Make
functors take the same <UseKDTree, AllowRemovals, UseDivMask>, so a code
is valid for one exactly when it is valid for the other and means the
same thing in both. Diverging would mean one of them stopping using
staticMonoLookupMake, which is a restructure rather than a drift.

The strings were carried over verbatim, and checked rather than assumed.
Against the switch, with UseKDTree documented as selecting mathic::DivList
or mathic::KDTree:

  code  instantiates      UseKDTree  UseDivMask  string
  1     Functor<0, ?, 1>  DivList    yes         list, using divmasks
  2     Functor<1, ?, 1>  KDTree     yes         KD-tree, using divmasks
  3     Functor<0, ?, 0>  DivList    no          list
  4     Functor<1, ?, 0>  KDTree     no          KD-tree

All four describe what the switch builds. AllowRemovals is absent from
the descriptions because it is a separate runtime argument, not part of
the code.

<ostream> and <utility> are added because the new function signature
names std::ostream and the table names std::pair. Both were arriving
through <vector> on libstdc++, which is not something to rely on.

No functional change: mgb help gb prints byte-identical code listings,
and the suite passes 248/248 in cmake Release.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
staticMonoLookupMake's switch asserts before it throws on an unknown
code. The code comes from -divisorLookup and -monomialTable, which
nothing validated, so a Debug build aborted on input a Release build
reported and declined:

    $ mgb gb cyclic5 -divisorLookup 99      # Release
    Unknown code for monomial data structure

    $ mgb gb cyclic5 -divisorLookup 99      # Debug
    mgb: StaticMonoMap.hpp:431: Assertion `false' failed.
    Aborted (core dumped)

Removing that assert would make the two agree, but it would leave the
value unchecked until it reaches a switch four call levels down, and it
would give up an assertion that is correct everywhere the code really is
internal. PR #69 faced the same choice and took the other route: it put
the modulus check in F4Reducer's constructor and left the asserts in
F4MatrixBuilder, F4MatrixBuilder2 and F4MatrixReducer, noting that
"establishing the bound here lets [them] assert it".

So check the code at the two entry points that take it from outside --
MonoLookup::makeFactory and ModuleMonoSet::make -- and leave the switch
alone. The assert downstream is then guarding an invariant, which is
what it is for.

Two things follow. The error arrives when the factory is created rather
than when it is first used, and it lists the codes:

    ERROR: Unknown code 99 for the monomial data structure.
    The supported codes are:
      1   list, using divmasks
      2   KD-tree, using divmasks
      3   list
      4   KD-tree

And -monomialTable is covered as well as -divisorLookup. It reaches
ModuleMonoSet::make, a second path with the same abort, which the review
had not noticed.

src/test/MonoLookup.cpp covers both entry points, rejecting 99 and
accepting each of 1 through 4. The second half is deliberate: it is what
fails if the table and the switch ever drift apart. Testing this at all
is only possible because these entry points are in libmathicgb --
mathicgb-unit-tests links no src/cli object, so the same check in the
CLI could not be tested, which is why section 18 groups these findings.

Verified: 250/250 in cmake Release and Debug, autotools make distcheck
passes, no build warnings, and both builds now report a bad
-divisorLookup and a bad -monomialTable identically.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
MATHICGB_STREAM_CHECK asserted and then threw, so in a Debug build the
assert fired first and the twenty checks in the public streaming
interface aborted where the throw would have reported the problem.

What they report is caller error -- calling idealBegin() twice,
appending more terms than were promised -- not a bug in the library, so
throwing is the whole of the right answer and the assert is simply in
the way.

Nothing had ever caught this. Every test that drives an ideal stream
follows the protocol, so no check had fired, and before PR #80 no CI
cell built this code with assertions at all. The new test constructs a
StreamStateChecker directly and walks it into three violations, which
is the first time any of these twenty checks has been exercised.

The message drops "Assert expression:" for "Failed check:", there being
no assert left to name.

This also clears the 19 clang warnings that PR #81's CI turned up in the
macOS debug cells. They came from the assert(("message", condition))
idiom, which puts a comma operator with a no-effect left operand into
every expansion; deleting the assert removes the idiom. clang 18.1.3
reports 19 warnings in this file before the change and 0 after, and GCC
never warned.

One thing deliberately left out: StreamStateChecker's constructor
catches its own check without rethrowing, so with the assert gone a
composite modulus double-frees instead of aborting. That path is
unreachable through the public interface -- the modulus comes from a
GroebnerConfiguration that has already rejected a composite one -- and
the real fix is to stop owning the Pimpl by raw pointer. Left for its
own item, which is why no composite-modulus test appears here.

Verified: 251/251 in cmake Release and Debug, no build warnings from
GCC 13.3, and the new test passes in both configurations.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PR #74 put the primality check in MatrixAction.cpp, which is the one
place it cannot be tested: mathicgb-unit-tests links no src/cli object
and there is no CLI-level harness, so the check shipped uncovered and
could not be given a test where it stood.

SparseMatrix::read is where the modulus comes off the file, and it
already rejects the file for the other thing that can be wrong with
that field -- a value too large for Scalar, from PR #83 -- through the
same error path. The primality check belongs beside it.

QuadMatrix::read needs nothing of its own. It reads its four
submatrices through SparseMatrix::read, so all four are checked, and
the section of the review that asked for this assumed two changes
where one does.

Moving it also covers a read the CLI never checked at all: the
reference .rbrmat that a second run compares its output against.

What the CLI keeps is the file name, which read() cannot know, having
only a FILE*. readMatrixFile opens the file, reads it, and on an error
says which file it was before letting the exception past:

    While reading composite.brmat:
    ERROR: The modulus 100 is not prime. MathicGB only supports prime
    fields.

Rethrowing rather than composing a new message matters: mathic's
reportError has already put "ERROR: " into what(), so folding that text
into a second reportError prints the word twice. This way PR #83's
oversized-modulus error gets the file name too, for free.

The three call sites each lose their CFile boilerplate to the same
helper, and mgb matrix takes a list of files, so naming the file is
worth keeping.

Two tests, one per entry point, both of which fail without the check.
A composite modulus needs no patching to get into a file, unlike an
oversized one -- write() does not check either.

Verified: 253/253 in cmake Release and Debug, no build warnings, and
mgb matrix refuses a composite-modulus .brmat with exit 255 while a
prime one reduces and writes its output as before.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@d-torrance
d-torrance merged commit f6fd877 into Macaulay2:master Sep 18, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant