Repository navigation
Merge certified origin floors into a replication source's seq row, apart from its applied cursors - #3109
Merged
Conversation
…art from its applied cursors An end_txn can now carry originFloors: [nodeId, closedFloor, relayable][], an origin's closed timestamp floor as certified by its producer (the floor of harper#3085) and whether the source received that origin directly with full table coverage. The apply loop merges them by max into nodes[].closedFloor and nodes[].relayable on the connection's [seq, peer] row, kept apart from originLogKey: an applied position can sit above a transaction still open at the origin, a floor cannot, so only the floor is proof that resume may start there. A floor-only end_txn writes the row like a cursor-only one. harper-pro#922 item 2 is the consumer: its replication layer carries the floor between peers and resumes from it. Dispatch-Task: hp-922-phase1-origin-floors Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hhrw3yq9XSUgnNo3ymfe4B
…nite one, and make the floor test deterministic A stored floor is what resume trusts, so core now enforces for floors what it only documented for cursors: an end_txn's originFloors are merged only when the stream is tagged and passes onFailure, since an untagged or callback-less source reports a failed segment nowhere this loop can hold the floor back. An entry whose floor is not a positive finite number is skipped. The test's no-op frames are followed by a frame that writes, so the write count is checked after they have run rather than after a sleep. Dispatch-Task: hp-922-phase1-origin-floors Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hhrw3yq9XSUgnNo3ymfe4B
… it, and settle the DESIGN wording Dispatch-Task: hp-922-phase1-origin-floors Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hhrw3yq9XSUgnNo3ymfe4B
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces support for merging certified origin floors apart from applied cursors during transaction replication, updating the design documentation, replication apply loop logic, and adding corresponding unit tests. The review feedback recommends enhancing robustness by explicitly verifying that originFloors is an array in the admitsOriginFloors helper, and adding defensive checks during the iteration of originFloors to prevent potential runtime TypeErrors from malformed entries.
kriszyp
marked this pull request as ready for review
October 9, 2026 12:05
kriszyp
added a commit
to HarperFast/harper-pro
that referenced
this pull request
Oct 9, 2026
…anges; restore certified origin floors by pinning core at merged harper#3109 (#1017) * Refuse a core sync that drops the committed pointer's changes, and pin core at harper#3109 so certified floors reach the seq row again harper-pro#1011 merged with `core` pinned at its still-open companion harper#3109; the Sync Core it triggered (#1016) moved `core` to a harper main that does not contain that commit, removing the Table.ts merge of `event.originFloors` into `nodes[].closedFloor`. The receiver kept sending floors, core ignored them, and idleOriginFloorResume.test.mjs timed out on every Cluster 6/6 leg waiting for the first certificate. `sync-core.sh` now fetches the tracked tip, freezes its SHA and runs `core-sync-guard.sh` before moving anything: the committed pointer must be the tip, an ancestor of it, or merge into it without changing its tree (a squash- or rebase-merged companion), else the sync is refused with the diffstat. `core` moves to harper#3109 merged with current harper main, and a unit test drives a real core table with the receiver's end_txn shape so a core that ignores floors fails in seconds rather than 90 s per case on six legs. Refs #922 Depends-on: HarperFast/harper#3109 Dispatch-Task: ci-regression-harper-pro-1016-20261009 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LrZVGMv3XzudCdEkxX5yjX * Compare tree ids in the core sync guard, track `.` and the configured remote like submodule update does, and restore the worker flag after the seam test Review round 1 (Codex graded, Gemini, Cursor composer): `git diff --quiet` is a configurable comparison, so the no-op merge test now compares the merged tree id with the candidate's tree id; `submodule.core.branch=.` means the superproject's own branch and the fetch remote follows the checked-out core branch's remote, as `git submodule update --remote` did, with a fixture case; merge-tree's stderr is kept apart from the tree id; nested submodules are still updated after the checkout; the seam test resets `setMainIsWorker`; the fixture teardown is guarded; narrating comments trimmed. Dispatch-Task: ci-regression-harper-pro-1016-20261009 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LrZVGMv3XzudCdEkxX5yjX * Accept a core pointer whose own pull request merged with revisions, proven by the merge commit the Sync Core workflow looks up A coordinated PR merges with core pinned at its companion's branch commit; if the companion then merges with edits to the lines it added, the pointer's exact content is on no tip and the guard's merge test conflicts, stalling every sync after a merge that was legitimate. The workflow now resolves the committed pointer's merged pull request (`commits/{sha}/pulls`) and passes its merge commit as CORE_SYNC_SUPERSEDED_BY, which the guard accepts only when the candidate contains it. workflow_dispatch gains a drop_content input for the explicit override. The seam test's end_txn events match the receiver's shape (no timestamp field). Dispatch-Task: ci-regression-harper-pro-1016-20261009 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LrZVGMv3XzudCdEkxX5yjX * Take the superseding merge commit from a workflow input instead of looking it up, and unshallow from the selected remote Review round 2: the automatic lookup accepted any merged pull request that contained the pointer, so a stacked PR that dropped the lines would have passed with no human step, replaying #1016. The proof is now a workflow_dispatch input (superseded_by) a person supplies; the guard still requires the candidate to contain it. Also: the guard unshallows from the remote sync-core.sh selected, the merge-tree error file is removed on every exit, the revised-companion fixture compares tree ids, and the fixture file's header is one line. Dispatch-Task: ci-regression-harper-pro-1016-20261009 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LrZVGMv3XzudCdEkxX5yjX * Initialize an empty core before syncing, refuse a superseding commit that is behind the pointer, and stop on a detached superproject tracking `.` Review round 3: a plain clone leaves core/ empty, so the fetch ran against the superproject and the guard reported an unrelated comparison (the old command failed later, at the lock-file copy) — the submodule is initialized first, with a fixture case; a pasted CORE_SYNC_SUPERSEDED_BY behind the pointer can never be its merged pull request and is now refused; branch `.` on a detached superproject stops as `git submodule update --remote` does instead of fetching the remote's default branch; two narrating comments removed. Dispatch-Task: ci-regression-harper-pro-1016-20261009 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LrZVGMv3XzudCdEkxX5yjX * Never re-initialize a deinitialized core from the sync Round 4: `submodule update --init` after a deinit regenerates the module config without core.worktree and corrupts the git dir (AGENTS.md), so the sync initializes core only when it was never initialized and stops otherwise. Dispatch-Task: ci-regression-harper-pro-1016-20261009 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LrZVGMv3XzudCdEkxX5yjX * Reach the deinit refusal in the plain-clone fixture The first sync left the fixture's package.json modified, so the second run stopped at the dirty-manifest check before the deinit refusal it asserts. Dispatch-Task: ci-regression-harper-pro-1016-20261009 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LrZVGMv3XzudCdEkxX5yjX * Pin core to the merged origin-floor companion on main The squash-merged harper#3109 tree matches the previous companion merge exactly. The core sync guard accepts the divergent commit IDs by content. Dispatch-Task: hp-1017-repin-3109-merge Co-Authored-By: GPT-5 Codex <noreply@openai.com> * Make the core sync guard advisory: warn in the Sync Core PR, never refuse A companion that merges with different content than the pinned commit is indistinguishable from a dropped companion, and a refusal leaves Sync Core stuck until someone overrides it. The guard now prints a Markdown warning (dropped commits and diffstat) and always exits 0; sync-core.sh applies the validated SHA regardless and writes the warning to CORE_SYNC_WARNING_FILE, which the workflow puts at the top of the PR body and in the job summary. The superseded_by / drop_content overrides are removed. Detection is the seam test and the cluster shards. Dispatch-Task: hp-1017-advisory-guard Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013nCqStpkWerMFwdBvgTEA3 * Keep the core sync going when the warning file cannot be written An unwritable CORE_SYNC_WARNING_FILE no longer exits under set -e before the checkout; a failed mktemp reports the comparison as undecidable; git output is uncolored so it cannot corrupt the Markdown warning; the job summary ends the warning with a newline. Dispatch-Task: hp-1017-advisory-guard Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013nCqStpkWerMFwdBvgTEA3 --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: GPT-5 Codex <noreply@openai.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
⊙ Problem
A replication source can report, per origin, the highest log key it has durably applied (
originCursors, #3080), but no field on the[seq, peer]row can hold a proof that resume may start somewhere: an applied position can sit above a transaction still open at the origin, since a log is appended in commit order, not key order. harper-pro#922 item 2 carries the origin-closed timestamp floor that #3085 certifies (no later append below it) between peers, and needs core to persist it in the same row, under the same commit ordering and blob rule as the cursors.💡 Solution
An
end_txnmay now carryoriginFloors: [nodeId, closedFloor, relayable][], read byupdateRecordedSequenceIdonly from a tagged stream that passesonFailure— a stored floor is what resume trusts, so core enforces for floors what it documents for cursors (admitsOriginFloors). It merges each entry by max intonodes[].closedFloor, skipping a floor that is not a positive finite number, recordsnodes[].relayable(whether the source received that origin directly with full table coverage, which is what lets a relay forward the floor; an equal floor can turn it on), and treats a floor-onlyend_txnlike a cursor-only one: the row is written when a floor rose or a relayable flag turned on, and not otherwise. The outer dispatch gate admits anend_txnthat carries only floors by the same admission check, so an idle connection's floor-only updates reach the row.resources/DESIGN.mdrecords the field, the separation fromoriginLogKey, and the held-stream andonFailurerules.⚖️ Alternatives
✅ Verification
unitTests/resources/replicationSeqCursor.test.js: a new case drives floor-only rises at an unchanged scalar (each writes), an equal floor whose relayable flag turns on (writes once), a lower floor, a repeated one, a stream withoutonFailureand a NaN floor (no write, checked after a later frame that does write), and a second origin, and asserts the data cursor is untouched —npx mocha unitTests/resources/replicationSeqCursor.test.js: 6 passing.npm run typecheck(harper-pro, which compiles thiscore): 0 errors;npm run check:design-docs: index complete, budgets met; oxlint clean on the touched files.integrationTests/cluster/idleOriginFloorResume.test.mjson the companion PR exercises this merge through a live A–B–C cluster (floors stored per origin on the[seq, peer]rows, read back through a fixture), 3/3 passing twice.Refs harper-pro#922
🤖 Generated by Claude Fable 5.1; posted via @kriszyp.
Related PRs: #3080 overlaps (the
originCursorsmerge inupdateRecordedSequenceIdthis extends), 1 other independentComplexity: medium
Review-Coverage: authored=claude; ran=cursor-composer,gemini,codex; adjudicated=domain; declined=cursor-grok,cursor-kimi,cursor-muse; rounds=4; full=1 @ 7077108
Review-Attention: study ~13m (critical: Table.ts; decisions: enforce-floors-document-cursors, relayable-follows-latest-floor, floors-in-seq-row, do-less-alternative) @ 7077108