Skip to content

fix: defer native catch-up busy admission without failure backoff - #320

Merged
beinan merged 1 commit into
lance-format:mainfrom
beinan:codex/native-busy-deferral
Oct 6, 2026
Merged

beinan merged 1 commit into
lance-format:mainfrom
beinan:codex/native-busy-deferral

Conversation

@beinan

@beinan beinan commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

When a persistent native publisher cannot claim its table for 30 seconds, it currently exits as a failure and earns at least 120 seconds of additional backoff despite doing no merge work. The executor now returns a typed pre-claim busy outcome; the CLI emits an attempt-bound JSON marker and exit 75, and the supervisor retries confirmed busy outcomes after 2–5 seconds without adding failure debt.

Claim RPC errors and unknown outcomes remain failures. A claim accepted across the retry deadline is delivered without cancellation. The supervisor joins the child, validates the exact marker and absence of claim/work evidence, preserves previous failure counts and durable deadlines, and retains the full operator/owner/attempt CAS. Old binaries and partial-work failures retain their existing behavior. Deployment requires both the updated binary and supervisor; production has not been changed.

Validation:

  • 16 supervisor process/real-etcd and marker-contract tests passed; eight Rust executor regressions passed.
  • Real-etcd actual-executor regression passed: a live claim and four pending WAL generations survived the full 30-second busy boundary; after release, merge completed and all four payloads matched.
  • Actual compiled CLI plus tracked supervisor and disposable etcd passed: 30.14s busy returned the matching attempt marker and exit75, recorded a ~4.5s deferral, retained previous failure debt and the competing owner, and joined on shutdown. This was local binary staging, not a production or Kubernetes rollout.
  • Formatting, Ruff and diff checks passed. CI results are recorded on this PR.

@beinan
beinan marked this pull request as ready for review October 6, 2026 03:08
@beinan
beinan merged commit 55456b8 into lance-format:main Oct 6, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant