Repository navigation
feat(artifact-delivery): stream Skill builds up to 64 MiB - #510
Merged
Merged
Conversation
A build read every Skill file into memory, then packed a second copy, so the limits had to stay at 10 MiB, 2 MiB a file and 900 files. #487 worked around that by leaving media out, which dropped the music latent-spaces/brag needs for its films. The build now streams the codeload archive, keeps only the Skill folder, hashes as it goes, and streams the archive into R2 with the digest R2 checks. Files the archive lacks come from raw.githubusercontent.com. Every file still matches its Git blob. Public limits rise to the CLI's 64 MiB and 2,000 files. A CLI that sends Skilld-Capabilities: linked-files gets larger files as linked files instead of omitted ones.
A sweep of the largest Skills found three limits the old 900 file ceiling hid: a check result with over 100 findings fails the schema, a path USTAR cannot hold failed while packing instead of blocking, and one compressed chunk of zeros can inflate to many megabytes at once. The signer also needs Node types now that it hashes as it streams.
Renumbers the linked files migration to 0145 after main took 0144, and makes the load options required: #495's requester fallback forwarded a load without them, so no CLI would have received a linked file.
1 of 6 tasks
harlan-zw
added
the
harlan-agent-review
harlan-github-agent
Bot
added
harlan-agent-review-required
Contributor
🤖 MERGED
GitHub merged this pull request.
Review completed. No finding above 80/100. Selected findings run after merge and open separate pull requests. |
harlan-github-agent
Bot
added
harlan-agent-ready
![harlan-github-agent[bot]](https://avatars.githubusercontent.com/in/4579275?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Harlan Agent Kit posted this automated review. AI open source policy.
Automated Review: READY.
1 of 6 tasks
harlan-zw
added this pull request to stack #513
October 6, 2026 17:22
harlan-github-agent
Bot
added
the
harlan-agent-running
#508 builds a Resolution in its own request. Keep its after-response build beside the linked-files header, and give its tests the linked_files column.
harlan-github-agent
Bot
removed
the
harlan-agent-ready
harlan-github-agent
Bot
removed
the
harlan-agent-running
harlan-zw
added a commit
that referenced
this pull request
Oct 7, 2026
* feat(registry): move Repositories that GitHub renamed or transferred ADR-0002 kept a moved Repository under its old name, so pages, sitemaps and run commands named Repositories GitHub no longer serves, and two renames left duplicate Repositories in the registry. Sync now follows GitHub: it stores the Repository ID, moves every row to the new name in one batch, and keeps the old name as an alias that answers 301. ADR-0013 replaces ADR-0002. * chore(scripts): add the moved Repository backfill Sync moves a Repository the next time it reads it. The script runs the same statements now for the 13 moves the 2026-10-07 run sweep found, with a read-only dry run as the default. * fix(registry): renumber the moves migration and parse the ID once Main gained its own 0144 migration, and two migrations may not share a number. The batched summary read narrowed the Repository ID with a check TypeScript could not carry into the parser, so the parser now owns it. * fix(registry): queue moved Repositories past the sync prefetch The hourly prefetch records an unchanged Repository in place without a sync job. A moved Repository with an unchanged tree never reached syncRepo, so its rows would never move. * fix(registry): refuse a move into a name another Repository holds * fix(scripts): refuse a backfill merge into another Repository's row The backfill now runs the held-row check sync runs, so it never merges into a row stored for a different Repository ID. It also stops with a clear message when the D1 lacks the moves migration. * chore: renumber the repository move migration and ADR #510 adds migration 0145 and ADR-0013, and #512 adds ADR-0014. This branch moves to 0146 and ADR-0015 so all three merge without a clash. * fix(scripts): reserve migration 0145 for pull request 510 --------- Co-authored-by: Harlan GitHub Agent <harlan-github-agent[bot]@users.noreply.github.com>
harlan-zw
added a commit
that referenced
this pull request
Oct 7, 2026
#510 lets a ready build serve only a CLI that reads what it holds, so the warm run passes the CLI's linked-files capability to findReusableBuild.
harlan-zw
added a commit
that referenced
this pull request
Oct 7, 2026
#510 loads a Skill through a streaming read and the signer hashes the stored object as a stream.
harlan-zw
added a commit
that referenced
this pull request
Oct 7, 2026
* perf(artifact-delivery): serve a warm run from its ready build A warm run requested a new Resolution, waited for the queue, then waited again while the build re-signed the same bytes: 5.6 to 9.2 s in 14 production runs on 2026-10-07. When a ready public build of the same Repository, commit, Skill folder and policy passes every reuse check, the POST now answers with that ready Resolution, as GET would, and queues nothing. A request that names no commit resolves on GitHub first, within 2.5 s, using the cached commit and tree reads. When no ready build fits, the build gets the request pinned to that commit and folder, as a registry name already is. The run sweep keeps building every check. * fix(artifact-delivery): emit the served event without an object spread * test: give the warm run tests the streamed build shape #510 loads a Skill through a streaming read and the signer hashes the stored object as a stream. --------- Co-authored-by: Harlan GitHub Agent <harlan-github-agent[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🔗 Linked issue
Related to #487. The CLI side of linked files is skilld-dev/skilld#209.
❓ Type of change
📚 Description
A public build held every Skill file in memory, then packed a second copy. Its limits stayed at 10 MiB, 2 MiB a file and 900 files, and only the 900 had a recorded reason. #487 kept larger Skills running by leaving files out, so
latent-spaces/braglost the music its films use.A build now streams the Repository archive from
codeload.github.comat the exact commit. It keeps the Skill folder's entries, hashes them as they pass, and stops after the last one. Each file must still match the Git blob SHA from the tree. A file the archive lacks comes fromraw.githubusercontent.comat the same commit. An archive over 16 MiB is read a second time and streams into R2, and R2 refuses it unless it matches the digest of the first read. The signer hashes the object as it streams.ADR-0013 measures the constraint behind each limit. Public limits rise to what the skilld CLI accepts:
Private builds keep the old limits, because they still read one blob per request and encrypt the archive in memory.
Over 64 MiB, a CLI that sends
Skilld-Capabilities: linked-filesgets the largest files as linked files. The signed statement lists each one by path, mode, size and Git blob SHA, and the CLI reads it from GitHub at that commit. Other CLIs keep the #487 rule at the new limits, and never see the field.A local sweep of the registry, with
mainat #487 on the left:latent-spaces/bragtt-a1i/archifyoaustegard/claude-skills/tree-sittingcareerhackeralex/visualize/evalthedivergentai/gd-agentic-skills/godot-masterthvroyal/kimi-skills/kimi-xlsxAcross all 5,949 registry Skills, none that built in the 2026-10-07 sweep of
mainfails on this branch.Changes the larger limits needed:
loadtakes its options as a required argument. feat(artifact-delivery): fall back to the requester's GitHub token #495's requester fallback forwardedloadwithout them.The policy moves to
2026-10-07.3. The signer still signs2026-10-07.2. A ready build under it that left nothing out, up to 16 MiB, is checked again from its stored bytes. The deploy workflow applies migration 0145 and deploys the signer before the site, which this order needs: the old signer refuses an Artifact over 10 MiB.