Repository navigation
fix(registry): spend less GitHub quota on the sync and page views - #494
Merged
Merged
Conversation
GITHUB_TOKEN is a personal token, so the registry shares its hourly quota with every other tool on that account. Measured 2026-10-06: about 1,650 of 1,900 daily sync jobs only advanced the freshness cursor, about 630 /gh views a day re-asked GitHub for a repository it had already said was missing, and the owner and commit reads went out anonymous and failed. The hourly task now reads candidate summaries in GraphQL batches of 100 and records unchanged repositories in place. Sync and page tree reads go by tree SHA. The repository page caches 404s and tree scans at the edge, answers 404 for a private repository, and falls back to ungh.cc metadata when GitHub refuses. Owner and commit reads authenticate through the ETag client, so a repeat read answers 304 for free. Sync pauses name the bucket.
harlan-zw
added
the
harlan-agent-review
Collaborator
Author
|
Agent check, outside CI:
|
harlan-github-agent
Bot
added
harlan-agent-review-required
Contributor
🤖 MERGED
GitHub merged this pull request.
975035bd-b2f9-448d-91a2-4e08566a1fb7 Selected findings run after merge and open separate pull requests. |
harlan-github-agent
Bot
added
harlan-agent-running
![harlan-github-agent[bot]](https://avatars.githubusercontent.com/in/4579275?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Harlan Agent Kit posted this automated review. AI open source policy.
Automated Review: READY.
1 of 6 tasks
harlan-github-agent
Bot
removed
the
harlan-agent-ready
1 of 6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🔗 Linked issue
Related to #484
❓ Type of change
📚 Description
GITHUB_TOKENis a personal token, and GitHub counts its quota per account. The registry shares that bucket with every other tool on the account, so each read it skips leaves room forskilld runand CI. Production logs and D1 for the 24 hours to 2026-10-06 15:00 UTC showed where the registry spent reads it did not need:syncReporuns. Only the rest get a job. Discovery claims, repositories a job still holds, Skill-less repositories and anything GitHub gave no summary for keep their job. If the batch fails, every candidate is queued as before.syncRepoand the asset refresh now read the tree at the summary's head tree SHA instead of the branch name. The tree matches the summary, and a repeat read answers 304 from the ETag cache./api/repos/:owner/:repo. About 630 views asked GitHub for a repository it could not find, and nothing remembered the answer. A 404 is now cached at the edge for an hour. A registry repository is scanned at its synced tree SHA, and the scan is cached by that SHA. A private repository now answers 404: the token can read private repositories, and this page is public. When GitHub refuses with 403, 429, 5xx or a timeout, the page reads its metadata from ungh.cc, without the Skill file list. ungh.cc never serves build content or trees./api/orgs/:owner,/api/skills/featuredand/api/skill-relatedcalled GitHub with noAuthorizationheader. They shared the Worker IP's anonymous allowance, and about 330 commit reads failed. They now go through the ETag client with the token. This adds a request toGITHUB_TOKENthe first time each owner or commit list is read; a repeat read answers 304 for free, and both stay behind their existing D1 and edge caches.nuxt/ui: 163 graphql requests remaininginstead of leaving the bucket unnamed. The 200 request reserve is now a named constant with its sizing.