Skip to content

perf(artifact-delivery): run ten builds at once - #507

Merged
harlan-zw merged 1 commit into
mainfrom
perf/build-queue-concurrency
Oct 7, 2026
Merged

harlan-zw merged 1 commit into
mainfrom
perf/build-queue-concurrency

Conversation

@harlan-zw

@harlan-zw harlan-zw commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

🔗 Linked issue

None. Launch prep for skilld run latency. Harlan approved this production config change.

❓ Type of change

  • 📖 Documentation
  • 🐞 Bug fix
  • 👌 Enhancement
  • ✨ New feature
  • 🧹 Chore
  • ⚠️ Breaking change

📚 Description

The skilld-artifact-build consumer ran at most two builds at once. Builds took 4 to 7 s in production traces on 2026-10-07, so two consumers finish about 20 a minute. A larger burst waits in the queue, and the CLI stops waiting for a Resolution after 60 s. Queue delivery already took 1.2 s at the median and 3.4 s at worst in 20 timed runs. This raises max_concurrency to 10 and keeps every other setting.

Limit at 10 builds at once Numbers
Batch size Stays 1. A large build spends up to 900 of its invocation's 1,000 subrequests, so two builds cannot share an invocation.
D1 A build makes 11 to 14 queries since #496, each 0.13 to 0.49 ms at the median and at most 11 ms at p99 in the last 24 h. Ten builds of 4 s each make about 35 queries a second, about 14 ms of D1 time each second. The database spent 1,709 s on 429,993 queries in those 24 h, 2% of the time.
GitHub The read App installation on skilld-dev has 5,000 requests an hour. #493 measured 7 requests for a first run pinned to a commit, 8 on a branch, and 0 for a repeat. That is about 700 first builds an hour, whatever the concurrency. Two consumers running 10 s cold builds already make about 12 a minute, or 5,000 requests an hour. At 10, a burst spends it sooner, then builds fail RATE_LIMITED with a retry time, and signed-in runs fall back to their own token (#495).

🤖 AI disclosure: Harlan Agent Kit modified this description. My AI open-source policy.

Two build consumers finish about 20 builds a minute. A larger burst
waits in the queue, and the CLI stops waiting after 60 s. Ten
consumers run each build as before, one per invocation, so a large
build keeps its subrequest budget. D1 and the read App's GitHub quota
both hold at ten; the pull request states the numbers.
@harlan-zw harlan-zw added the harlan-agent-review Pull request triage requires an adversarial Review. Adding this label manually always forces one. label Oct 6, 2026
@harlan-github-agent harlan-github-agent Bot added harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. harlan-agent-running An Agent holds a Task on this issue or pull request right now. and removed harlan-agent-review Pull request triage requires an adversarial Review. Adding this label manually always forces one. labels Oct 6, 2026
@harlan-github-agent

harlan-github-agent Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

🤖 READY · 88/100

Harlan Agent Kit posted this automated review. It is not Harlan's personal review or approval. AI open source policy. A person still decides the merge.

  • Merge gate: Passed.
  • Review gate: Passed. No material issues.
  • CI gate: Passed.

@harlan-github-agent harlan-github-agent Bot added harlan-agent-ready The automated Review passed every gate on this head commit. and removed harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. labels Oct 6, 2026

@harlan-github-agent harlan-github-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Harlan Agent Kit posted this automated review. AI open source policy.

Automated Review: READY.

Read the Review results

@harlan-zw
harlan-zw merged commit 62d0f07 into main Oct 7, 2026
7 checks passed
@harlan-zw
harlan-zw deleted the perf/build-queue-concurrency branch October 7, 2026 01:00
@harlan-github-agent harlan-github-agent Bot removed the harlan-agent-ready The automated Review passed every gate on this head commit. label Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant