Skip to content

fix(master): pay back the running-pool gauge on task panic - #353

Merged
beinan merged 1 commit into
lance-format:mainfrom
beinan:fix/pool-gauge-panic-safe
Oct 8, 2026
Merged

beinan merged 1 commit into
lance-format:mainfrom
beinan:fix/pool-gauge-panic-safe

Conversation

@beinan

@beinan beinan commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator

Part of #333. Fixes review finding 5 (panic path) on #348 (credit: production review).

Problem

running.increment before spawn, running.decrement after run_task returns. A panic in run_task unwinds past the decrement: permits are released by Drop, the gauge is not, so master_task_pool_in_use{state="running"} reads as consumed forever.

Fix

GaugeHold(Gauge) with Drop → decrement, held across run_task. Three lines.

Validation

Type-level: the decrement is now in Drop, which runs on both normal return and unwind. No new test — asserting on a metrics::Gauge value needs a recorder harness this crate's tests do not have; the existing poller test (reserved_resident_merge_finishes_while_legacy_pool_is_saturated) passes against etcd, clippy -D warnings and fmt clean.

🤖 Generated with Claude Code

lance-format#348 incremented the running gauge before spawning the task and
decremented it after run_task returned. A panic inside run_task unwinds
past the decrement; the permits are released by Drop but the gauge
stays elevated, so the pool reads as consumed forever. Hold the
decrement in a Drop guard so unwinding pays it back.

Refs lance-format#333. Review finding 5 on lance-format#348.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@beinan
beinan merged commit 41ecd31 into lance-format:main Oct 8, 2026
11 checks passed
beinan added a commit that referenced this pull request Oct 8, 2026
…ted (#360)

Part of #333. Closes the `claiming` cancellation gap noted in review
round 4 (credit: production review).

The `claiming` gauge was incremented before the etcd claim and
decremented after. A poller aborted mid-claim (drain, shutdown, test
teardown) skipped the decrement. Same `GaugeHold` Drop guard as #353,
held across the await.

Clippy `-D warnings`, fmt clean; poller test passes against etcd.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant