Conversation
added 2 commits
September 18, 2026 09:36
The Rust job on master ran past twenty minutes with no end in sight. It was not wedged on anything this repository compiles: clippy had been running since 02:33 where the same step takes 65 seconds warm, because a cache entry never existed for it. Every cache in the repository is scoped to `refs/pull/211/merge` and keyed `Darwin-arm64`, which is the macOS panel job. Caches are branch scoped and a branch can only read its own and its base's, so nothing on master ever had one to read, and now that #211 is merged those entries are unreachable anyway. Neither job declared `timeout-minutes`, so the ceiling was GitHub's default of six hours. That is the part that turns a slow step into an invisible one: the run does not fail, it just never finishes, and the queue backs up behind it. Both jobs now have a bound sized to their warm time with room for a cold graph. The sidecars are the other two and a half minutes. Both build outside the workspace target directory, which is what `rust-cache` keys on, so neither was ever cached, and both are the least likely things in the tree to change: the proxy is a fixed-version crates.io install, and the reader is pinned to the v2-era WorkTable it must never move off. They are now cached as the produced binaries, keyed on the manifests, scripts, reader sources and GUI schema that decide them, since the reader's build script derives its schema copy from that. `runner.os` is in the key, so a Linux runner cannot pick up the host-target-suffixed macOS binaries sitting in a developer's checkout.
It was the longest step in the Rust job: 65 seconds warm, minutes cold. `--all-targets` type-checks the test and bench targets on top of the lib and bins, which is close to double the compile work, and `cargo test` on the next line then does nearly all of it again. A lint is worth most while the code is still open, which is the machine doing the writing, not one reporting six minutes later. It stays wired up here behind `workflow_dispatch` with a `clippy` input, so the machine's answer is one manual run away when someone wants it. `inputs.clippy` is undefined on push and pull_request, so the step is skipped there without needing a second condition. The exact local command is in the comment above the step, because it was documented nowhere: not in a gates file, not in CLAUDE instructions, and the only mention of clippy in `docs/` is one review from July.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Rust job on master ran past twenty minutes. Not wedged on anything we compile: clippy had been running since 02:33 where the same step takes 65s warm.
What was wrong
No cache for the Rust job. Every cache entry in the repo is scoped to
refs/pull/211/mergeand keyedDarwin-arm64(the macOS panel job). Caches are branch-scoped and a branch can only read its own and its base's, somasternever had one. With #211 merged, those entries are unreachable anyway. The Linux Rust job compiles the full dependency graph from scratch on every run.No
timeout-minuteson either job, so the ceiling was GitHub's six-hour default. That is what turns a slow step into an invisible one: the run does not fail, it just never finishes and the queue backs up behind it.The sidecars rebuilt every run, ~2m30s combined. Both build outside the workspace target directory, which is what
rust-cachekeys on, so neither was ever cached.What this does
timeout-minutes: 20on Rust (5m warm, ~15m cold),6on Frontend (1m20s warm).runner.osis in the key so a Linux runner cannot pick up host-target-suffixed macOS binaries.The sidecars are the safest things to cache: the proxy is a fixed-version crates.io install, and the reader is pinned to the v2-era WorkTable it must never move off.
Expected effect
First run on master populates
rust-cachefor the branch; subsequent runs read it. Steady state should be well under the 5m17s that the last warm branch run took.