Repository navigation
feat(gateway): honour Idempotency-Key on POST /v1/harnesses, so a retried create replays instead of duplicating - #613
Conversation
…create replays instead of duplicating (HarnessRouter#582) The reservation goes through the control store /responses already uses, and is taken before _skills_prepare/_plugins_prepare — those write blobs, so reserving after them would duplicate the side effects even when the vertex is not duplicated. A create that fails before the vertex lands releases the key. Deploy to Cloud sends deploy:<hid>:<fingerprint> per logical create, so a deploy retried later replays the copy instead of making a second one. Protocol text in §5.1, conformance check F-10.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…tays in the caller's workspace, and makes the harness again once its harness is deleted (HarnessRouter#582) - A repeat while the first create is still writing waits for it (Tasks §6), instead of 404. - The key is scoped to the workspace the harness is made in and kept apart from /v1/responses' keys, so a workspace's key never answers with another workspace's harness. - A key whose harness was deleted names none, so a deploy whose Cloud copy was deleted makes it again instead of failing until the key expires. A claim left by a create that died is taken over after two minutes. - Kept 24 hours (Tasks §6), the capability reported only with a store, the key sent on the deploy's POST only. - Protocol: §5.1 states the window, the wait and the delete rule; errors.md lists idempotency_key_reused; the schema names the capability; F-10 cleans up after itself and checks the delete rule (suite 2026.10.19.post4); CHANGELOG. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
richard-epsilla
left a comment
There was a problem hiding this comment.
Thank you, @Jahanzeb-git. Claiming the key before the prepares is the right call, and hashing the body before they rewrite it caught a real trap. I pushed one commit on top (3bbcc43) for four things the tests didn't reach, plus the protocol pieces around them.
1. A repeat while the first create is still running got a 404. The repeat finds the claim, reads the harness, finds nothing yet, and answers harness_not_found. That's the timeout case the key exists for: a create with a large skill is still writing its blob when the client gives up and sends it again. Tasks §6 already settles it for tasks: "If the first request is still running, the server MUST wait for it and return its result rather than returning a partial or a conflict." The repeat now waits for the harness to land.
2. A key could answer with another workspace's harness. The claim was keyed by organization only. A workspace's key that reused a key string got the other workspace's harness back from the replay, which is the cross-workspace read _scope_keeps exists to refuse. The claim is now scoped to the workspace the harness is made in. It's also kept apart from /v1/responses keys, which share the store.
3. On the deleted-harness question you raised: I went with creating again rather than 404. Deploy to Cloud is what decided it. The person deletes Cloud's copy and deploys again with the form unchanged. The PUT answers 404, the deploy creates, and it sends the same deploy:<hid>:<fingerprint> key. With the mirror, that deploy failed with "not found" until the key expired. A response replay reads a result, so 404 fits there. A create's purpose is a harness that exists, so once its harness is deleted the key names none, and a repeat makes one. The invariant still holds: at most one live harness per key.
4. A claim left by a create that died (a process killed between the claim and the write) would hold the key for its whole window. After two minutes with no harness, a repeat takes it over. Release-and-reclaim goes through the atomic create, so one racer wins and the others wait on it.
Smaller ones:
- The key is kept 24 hours rather than 1, as Tasks §6 asks of a task's key. A deploy retried the next morning should still find its copy.
harness_idempotencyis reported only when there is a store to keep keys in. Without one, a keyed create is refused 503.- The deploy sends the key on the POST only, not the PUT.
- Protocol:
- §5.1 now states the 24-hour window, the caller's scope, the wait, and the delete rule as MUSTs.
- errors.md lists
idempotency_key_reused, and the 2026-10-19 schema names the capability. - F-10 deletes what it makes. As it was, it ran after F-07's cleanup, so its harness stayed on the server. It also checks the delete rule.
- The suite is now 2026.10.19.post4, with a CHANGELOG entry.
Tests: five new gateway tests (in progress, workspace, deleted, stale claim, responses key), each of which fails on 8921b23, and a deploy test where Cloud keeps keys and the first answer is lost. Full gateway suite 1512 passed, conformance tests 155 passed. I'll verify it on a release candidate before it ships as 0.37.11.
richard-epsilla
left a comment
There was a problem hiding this comment.
Verified on 0.37.11-rc.1 (sha256:9eb7543d…) on the test instance over real HTTP: two simultaneous creates with one key make one harness, a changed request gets 409 idempotency_key_reused, the key makes a new harness after a delete, one key string in two workspaces makes two harnesses that can't read each other, and no key behaves as before. Conformance F-10 passes live and leaves nothing behind. A deck turn completes on the candidate. Merging for 0.37.11.
|
All four are mine, and they are the same mistake: I tested the path through the
They are one question in four places: what the key means before the thing it What I would have caught with more care: the first by asking what the key is Now the part that is not about the diff. You fixed these rather than sending them back at me. That is the review load I 25 to 35 hours a week, as a contractor: the gateway and runner review queue, Pick one issue and I will read it and come back with a scope and a number for |
|
Thank you, @Jahanzeb-git. That is the right reading of all four, and each one now has a test of its own in On the offer: yes to the first step. The issue is #476, input requests in the end-the-turn shape. The shape is settled in the assessment there, and nobody is building it yet. It runs through every place you named:
Please post your scope as a comment on #476: how you would split it into PRs, what you would test for each clause, and what you would leave out. Once you have commented there, the issue can be assigned to you. @phanisarman, it is your proposal: if you were already planning the PR, say so on #476 and it stays yours. |
Closes #582
POST /v1/harnesses honours an
Idempotency-Key. The reservation uses the same control store/responsesalready uses, and is taken before_skills_prepare/_plugins_preparerun. Thosewrite blobs and packages, so reserving after them would duplicate the side effects even when the
Harness vertex is not duplicated.
409 idempotency_key_reusedDeploy to Cloud sends
deploy:<hid>:<fingerprint>per logical create, so a deploy retried later(a lost answer, a closed laptop) replays Cloud's copy instead of making a second one. A different
harness, or a changed form, is a different key.
Protocol text in
harnesses.md §5.1, capability advertised asharness_idempotency, conformancecheck
F-10.Tests
gateway/tests/test_harness_idempotency.py— 6 tests: replay, 409, no-key unchanged, keyreleased after a failed create, a >48KB skill still hashes the same after the prepare rewrite,
capability advertised
One choice worth a second opinion: if the key's harness has since been deleted, the replay answers
404 harness_not_found, mirroring_idem_replay. Releasing the key and creating again is thealternative — I picked the mirror for consistency, but it's a few lines either way.
All three parts from the Work section are in this PR rather than split, the protocol text and check are small enough that splitting them would have cost more review time than it saved.