Skip to content

Maintain the invariant that no empty block sits behind the front block - #21

Merged
d-torrance merged 2 commits into
Macaulay2:masterfrom
d-torrance:fix-growcapacity-empty-block
Aug 29, 2026
Merged

d-torrance merged 2 commits into
Macaulay2:masterfrom
d-torrance:fix-growcapacity-empty-block

Conversation

@d-torrance

Copy link
Copy Markdown
Member

guardPoint() documents an invariant:

if (block().hasPreviousBlock()) {
  // The previous block is not empty as then it would have been
  // deallocated, so it is safe to take the guard point from the
  // previous block.
  MEMT_ASSERT(!block().previousBlock()->empty());

freeTopFromOldBlock() relies on it too. Nothing maintained it. There are two
independent ways to end up with an empty block behind the front block, and
this fixes both — one commit each, with a regression test each.

1. restoreToGuardPoint emptied blocks without deallocating them

It reset the position of every block newer than the guard point but never
freed them. Two blocks inside a guard are needed to show it, since with one
the only empty block is the front block, which is allowed:

memt::Arena arena;
void* a = arena.alloc(1);
{
  memt::Arena::Guard guard(arena);
  arena.alloc(arena.getMemoryUse() + 1);
  arena.alloc(arena.getMemoryUse() + 1);
}
arena.freeTop(a); // a is the only allocation that is still live

The commit also drops the _debugAllocs entries for the blocks being freed
before freeing them, so no freed address is used afterwards.

2. growCapacity chained an empty front block behind the new one

alloc() grows the capacity whenever a request does not fit in what is left
of the front block — including when that block is empty. No guard point is
involved
, so this is reachable from plain alloc/freeTop:

memt::Arena a;
void* p = a.alloc(1);                     // block A
void* q = a.alloc(a.getMemoryUse() + 1);  // block B, A behind it
a.freeTop(q);                             // B is empty now
void* r = a.alloc(a.getMemoryUse() + 1);  // block C, B empty behind it
a.freeTop(r);
a.freeTop(p);                             // p is in A, but B is examined

Fixed in growCapacity rather than in MemoryBlocks, as BufferPool shares
the latter and has no such requirement.

Symptoms

Both cases behave identically. With MEMT_DEBUG, Arena.cpp aborts on
Assertion 'previous->isInBlock(ptr)' failed in freeTopFromOldBlock.
Without it, the position of one block is set to an address inside another and
getAllocatedMemoryUse() wraps around to a huge value. Using a Guard
instead of the final freeTop trips !block().previousBlock()->empty() in
guardPoint() instead.

Case 2 is the more serious of the two: it needs no guard points, and it
corrupts the block chain silently in ordinary release builds.

Verification

26/26 under cmake -DCMAKE_BUILD_TYPE=Debug, cmake default, and autotools.
make distcheck passes with and without --enable-debug. Clean under
AddressSanitizer and UndefinedBehaviorSanitizer.

Each regression test fails before its fix and passes after, in both debug
and release builds, so neither depends on MEMT_DEBUG being defined.

A fuzzer doing random valid operations (nested guards, released guards, LIFO
freeTop, freeAndAllAfter, block-forcing allocations) runs clean over 10
seeds x 400 trials in both builds, and reproduces the bug on 5 of 5 seeds
against master.

The two commits are independent and either can be dropped without affecting
the other.

🤖 Generated with Claude Code

d-torrance and others added 2 commits August 29, 2026 19:27
restoreToGuardPoint emptied every block that was allocated after the
guard point was taken, but never deallocated them.  guardPoint() and
freeTopFromOldBlock() both require that a block behind the front block
is not empty, so this left the arena in a state that they mishandle.

It takes two blocks inside the guard to show up, as with only one the
sole empty block is the front block, which is allowed:

  memt::Arena arena;
  void* a = arena.alloc(1);
  {
    memt::Arena::Guard guard(arena);
    arena.alloc(arena.getMemoryUse() + 1);
    arena.alloc(arena.getMemoryUse() + 1);
  }
  arena.freeTop(a); // a is the only allocation that is still live

That aborts in freeTopFromOldBlock when MEMT_DEBUG is defined.  Without
it the position of one block is set to an address inside another, and
getAllocatedMemoryUse() then wraps around to a huge value.

Free them as freeAndAllAfterFromOldBlock already does.  The front block
is kept, as it is kept there too.  Drop the record of the allocations
in the blocks being freed before freeing them, so that no freed address
is used afterwards.

Note that this is not the only way to get an empty block behind the
front block.  growCapacity does it as well, by chaining an empty front
block behind a newly allocated one, and that is not fixed here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
alloc() grows the capacity whenever a request does not fit in what is
left of the front block, including when the front block is empty.  The
old front block was then chained behind the new one regardless, so an
empty block ended up behind the front block.  guardPoint() and
freeTopFromOldBlock() both require that this does not happen.

No guard point is needed to reach it:

  memt::Arena a;
  void* p = a.alloc(1);                     // block A
  void* q = a.alloc(a.getMemoryUse() + 1);  // block B, A behind it
  a.freeTop(q);                             // B is empty now
  void* r = a.alloc(a.getMemoryUse() + 1);  // block C, B empty behind it
  a.freeTop(r);
  a.freeTop(p);                             // p is in A, but B is examined

Every call there is allowed.  It aborts in freeTopFromOldBlock when
MEMT_DEBUG is defined, and otherwise sets the position of one block to
an address inside another, after which getAllocatedMemoryUse() wraps
around to a huge value.

Deallocate the old front block when it is empty.  This is done in
growCapacity rather than in MemoryBlocks, as BufferPool shares the
latter and has no such requirement.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@d-torrance
d-torrance merged commit 9634416 into Macaulay2:master Aug 29, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant