|
| 1 | +# ADR-0013: Artifact size limits and linked files |
| 2 | + |
| 3 | +Date: 2026-10-07 |
| 4 | + |
| 5 | +Amends [ADR-0005](0005-v3-cli-harness-artifact-delivery.md). |
| 6 | + |
| 7 | +## Context |
| 8 | + |
| 9 | +A public build read every Skill file into memory, then packed a second copy as the archive. |
| 10 | +Its limits were 10 MiB an Artifact, 2 MiB a file, and 900 files. |
| 11 | +Only the 900 had a recorded reason: one blob read per file, against 1,000 subrequests. |
| 12 | + |
| 13 | +#487 kept Skills over those limits running by leaving files out. |
| 14 | +`latent-spaces/brag` then ran without the four music files its films use. |
| 15 | +`oaustegard/claude-skills/tree-sitting` ran without eight of its parsers. |
| 16 | + |
| 17 | +Each limit now names the constraint that sets it, measured on 2026-10-07. |
| 18 | + |
| 19 | +## Constraints |
| 20 | + |
| 21 | +| Constraint | Real limit | Old limit | New limit | |
| 22 | +|---|---|---|---| |
| 23 | +| Worker memory | 128 MB per isolate, shared with site requests | 10 MiB of files plus their 10 MiB archive, held at once | None from size. A build holds one network chunk, one inflate write of at most 16 KiB, SKILL.md, and the archive when it is at most 16 MiB | |
| 24 | +| Worker CPU | 300 s for the queue consumer | Not measured | 1 GiB of archive per pass. Inflating and walking costs 4 to 5 ms per MiB, so about 5 s | |
| 25 | +| Wall time | 15 min for the consumer. The skilld CLI waits 60 s for one Resolution | Not measured | A pass stops after the last Skill file. Cold codeload served PostHog/posthog (191 MiB gzipped) in 26 s | |
| 26 | +| Subrequests | 1,000 per invocation | 900 files, one blob read each | No file limit. One or two archive reads, and at most 400 single-file reads for files the archive lacks | |
| 27 | +| R2 put | 5 GiB in one part | Bytes in memory | A stream of known length in one part. R2 checks it against the SHA-256 the scan found | |
| 28 | +| D1 row | 2,000,000 bytes | Not measured | A file list of 448 KiB. The Resolution row holds about 3.34 statements plus the check results | |
| 29 | +| Signer | 128 MB per isolate | 10 MiB, read whole | 64 MiB, hashed as it streams | |
| 30 | +| skilld CLI 3.0 to 3.6.2 | 64 MiB download, 2,000 files, 6 MiB statement. Before 3.6.2 a download gets 30 s, so 64 MiB needs 2.2 MB/s | 10 MiB, 900 files | 64 MiB and 2,000 files. These two now bind | |
| 31 | +| USTAR | A path of at most 256 bytes | Checked after packing | Refused before any byte is read | |
| 32 | +| [GitHub](https://github.com) | 100 MiB a file | Not applicable | Linked files of at most 256 MiB in all | |
| 33 | + |
| 34 | +A private build keeps 10 MiB, 2 MiB a file, and 900 files. |
| 35 | +It still reads one blob per request and encrypts the archive in memory. |
| 36 | + |
| 37 | +## Decision |
| 38 | + |
| 39 | +### A build streams the Repository archive |
| 40 | + |
| 41 | +A public build reads `https://codeload.github.com/{owner}/{repository}/tar.gz/{commit}`. |
| 42 | +That is one subrequest and spends no REST quota. |
| 43 | +It inflates the archive in 16 KiB writes and keeps only the Skill folder's entries. |
| 44 | +It stops reading after the last Skill file. |
| 45 | + |
| 46 | +The Git tree still names every file, its mode, its size, and its blob SHA. |
| 47 | +Each file's bytes are hashed as they pass and must match that blob SHA. |
| 48 | +The archive is only a byte source, as before. |
| 49 | + |
| 50 | +A file the archive lacks, or holds with another size, is read from `raw.githubusercontent.com` at the same commit. |
| 51 | +Git LFS pointers, `export-ignore`, and `export-subst` produce those cases. |
| 52 | +A file whose archive bytes keep their size and fail the digest is read the same way on a second pass. |
| 53 | + |
| 54 | +Artifact order is the UTF-8 byte order of paths. |
| 55 | +It is the order a Git archive lists a tree in. |
| 56 | +All 487 archives of the 2026-10-07 sweep listed their files in that order. |
| 57 | + |
| 58 | +The first pass learns the archive digest and size, the file inventory, and the check results. |
| 59 | +An archive of at most 16 MiB stays in memory, and R2 stores it from there. |
| 60 | +A larger one is read from GitHub a second time and streams into R2 under the digest from the first pass. |
| 61 | +R2 refuses the put if one byte differs. |
| 62 | +The store happens before the checks are recorded. |
| 63 | +The CLI gives each stage after the checks 15 seconds. |
| 64 | + |
| 65 | +### Limits |
| 66 | + |
| 67 | +A public Artifact holds at most 64 MiB and 2,000 files. |
| 68 | +A file has no limit of its own. |
| 69 | + |
| 70 | +When the files exceed a limit, the largest leave the archive first. |
| 71 | +For a CLI without linked files, only files the Skill does not read leave, and the `omitted-files` check lists them. |
| 72 | +That is the #487 rule at the new limits. |
| 73 | +More files than 2,000 leave the same way, so linked files do not change the file count. |
| 74 | + |
| 75 | +### Linked files |
| 76 | + |
| 77 | +A skilld CLI that sends `Skilld-Capabilities: linked-files` on `POST /api/v1/resolutions` may get linked files. |
| 78 | +The Resolution stores that as `linked_files`, and the request fingerprint names it. |
| 79 | + |
| 80 | +When the archive would pass 64 MiB, the largest files other than SKILL.md become linked files, up to 256 MiB. |
| 81 | +The signed statement lists each one: |
| 82 | + |
| 83 | +```json |
| 84 | +{ |
| 85 | + "linkedFiles": [ |
| 86 | + { "path": "scripts/KimiXlsx", "mode": 420, "size": 77001601, "gitBlobSha": "<40 hex characters>" } |
| 87 | + ] |
| 88 | +} |
| 89 | +``` |
| 90 | + |
| 91 | +The CLI verifies the attestation first. |
| 92 | +It then reads each linked file from `https://raw.githubusercontent.com/{owner}/{repository}/{commitSha}/{skillPath}/{path}`, each path segment percent-encoded. |
| 93 | +It refuses the Skill unless every file has the declared size and Git blob SHA. |
| 94 | +It installs the file with the declared mode, beside the archive's files. |
| 95 | +The bytes never pass through skilld.dev, and the signature still covers them through the blob SHA. |
| 96 | + |
| 97 | +The field is present only when it is not empty. |
| 98 | +A CLI without linked files never receives it, because it refuses a statement field it does not know. |
| 99 | +A build with linked files serves only a CLI that reads them. |
| 100 | +A build that left nothing out serves both kinds. |
| 101 | + |
| 102 | +## Consequences |
| 103 | + |
| 104 | +`latent-spaces/brag` runs with its music: 289 files, 16.03 MiB, nothing left out. |
| 105 | +`tt-a1i/archify` runs whole at 10.36 MiB. |
| 106 | +`thedivergentai/gd-agentic-skills/godot-master` runs with 1,728 files. |
| 107 | +`thvroyal/kimi-skills/kimi-xlsx` runs its 73 MiB binary as a linked file for a CLI that reads one. |
| 108 | + |
| 109 | +A first run of a Skill late in a large archive can take longer than the CLI's 60-second wait. |
| 110 | +The build still finishes, and the next run reuses it. |
| 111 | + |
| 112 | +The previous policy `2026-10-07.2` stays signable. This change closes no safety gap. |
| 113 | +A ready build under that policy that left nothing out packs the same files, so it is checked again from its bytes. |
| 114 | +A stored archive over 16 MiB is never read back whole. It loads from GitHub. |
0 commit comments