Repository navigation
feat(pathways): resilient cluster start that never stalls the data pump - #1
Merged
Merged
Conversation
A failed startPump() or leader bootstrap left a leader that holds the lease with no running pump, so no other instance could take over. The skill now ships a production start runtime: retry the whole start with stopPump() + stopCluster() teardown, exit after the last attempt, exit on "Failed to bootstrap leader runtime after becoming leader", and a leader watchdog. Verified against @flowcore/pathways 2.10.4 and @flowcore/data-pump 0.23.3 source. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
skills/flowcore-pathways/references/resilient-startup.md: production start runtime for virtual pathways. It retries the whole start withstopPump()+stopCluster()teardown, exits after the last attempt, exits onFailed to bootstrap leader runtime after becoming leader(detected in the logger adapter), and runs a leader watchdog. It separates liveness from readiness and adds handler timeouts sized todeliveryTimeoutMs.SKILL.md: new rule 11 (a start error must never leave a silent stall). The minimal example now uses a single-attemptstartOnce()plus the runtime. Checklist items added.cluster-and-pump.md: why the naive start stalls. Fixes the PostgreSQL-outage row: the leader does not step down, becauserenewLeasethrows and the error is logged asLease loop error.Verification (source)
Every claim was checked against the
@flowcore/pathways2.10.4 source (builder.tsstartCluster/startPump/handleLeadershipChange/bootstrapLeaderPump,cluster-manager.ts,postgres-coordinator.ts,pathway-pump.ts) and the@flowcore/data-pump0.23.3 source (data-pump.tsredelivery and fetch loop). The patterns come from about 40 Flowcore workspace solution and outage fragments on pump stalls.Test plan
node scripts/validate-package.mjspasses (13 checks), and the self-test passes 33/33deno checkofpathways-logger.ts,pathways-runtime.ts, the SKILL.md example and the handler-timeout block against the pathways 2.10.4 source🤖 Generated with Claude Code