Skip to content

Perf: Verify api-simulator wheel caching in CI #2135

Closed

Description

@Trecek

Problem

api-simulator is a private Rust/PyO3 crate compiled from source. It's pinned to git rev 8f711f9... and only used in 3 test files (guarded by pytest.importorskip). Cold Rust compilation takes 3-8 minutes.

Status: CONFIRMED — Wheel is NOT Cached

Investigation completed 2026-05-07 via diagnostic PR #2161 (run 25503753214). The api-simulator wheel recompiles from Rust source on every CI run (45–64s). Evidence collected across 6 runs — zero showed cache reuse.

Root Cause: Three Compounding Failures

1. Cache poisoning by preflight job
The preflight job runs setup-uv@v7 and only executes uv lock --check (no install, no compilation). It saves the uv cache in its Post Install uv step — this cache contains metadata (82MB) but no compiled wheels. The test job restores this same cache (same key), compiles the wheel during uv sync (cache grows to 493MB with the 3.6MB wheel), but GitHub Actions skips saves on exact key matches: Cache hit occurred on key ..., not saving cache. The wheel is built and then discarded every run.

2. No restore-keys fallback
setup-uv uses a single exact cache key hashing uv.lock. Any uv.lock change → total miss, no partial restore. uv.lock changes 36–51 times per busy day from version bumps (733 of 2,444 total commits).

3. uv-version parameter silently ignored
setup-uv@v7 renamed uv-version to version. The workflow uses uv-version: "0.9.21", which produces ##[warning]Unexpected input(s) 'uv-version' and is silently ignored. CI runs uv 0.11.11 instead of the pinned 0.9.21.

Diagnostic Evidence (Run 25503753214)

Pre-install — cache hit is empty of wheels:

Cache hit: true
=== Looking for api-simulator in uv cache ===
(empty — no files found)
=== uv cache size === 82M

During install — uv explicitly reports uncached, compiles from Rust:

DEBUG Identified uncached distribution: api-simulator @ git+...
Building api-simulator @ git+... (60 seconds)

Post-install — wheel exists but won't be saved:

api_simulator-0.1.0-cp39-abi3-linux_x86_64.whl (3.6M)
uv cache size AFTER install: 493M
Post Install uv: "Cache hit occurred on key ..., not saving cache."

Evidence Across 6 Runs

Run ID Branch/Event Build Time Cache Hit? Compiled?
25503753214 PR #2161 (diagnostic) 60s Yes (poisoned) YES
25502884634 merge_group (pr-2159) 59s Yes (poisoned) YES
25503302498 merge_group (pr-2160) 46s Yes (poisoned) YES
25502299857 PR (auto-research-8-2) 57s Yes YES
25502299816 PR (auto-research-9-1) 64s Yes YES
25500503978 merge_group (pr-2143) 60s Yes YES

Suggested Fixes

Fix 1: Eliminate cache poisoning (primary)

Either (a) disable setup-uv caching in preflight (add enable-cache: false), or (b) add a separate actions/cache@v4 step in the test job keyed on the api-simulator rev + platform, bypassing the setup-uv built-in cache.

Fix 2: Fix uv-version → version

Change uv-version: "0.9.21" to version: "0.9.21" in both jobs so the pin is respected.

Fix 3: Remove Rust toolchain from preflight

uv lock --check validates the lockfile — it doesn't build packages. The dtolnay/rust-toolchain@stable step in preflight is unnecessary.

Fix 4 (from original issue): Pre-build or sccache

If the setup-uv cache fix alone doesn't achieve cache reuse (due to uv.lock churn), consider:

  1. Pre-build wheel via PyO3/maturin-action@v1 in a separate job
  2. Add sccache for Rust compilation caching independent of uv.lock
  3. Separate integration job for the 3 api-simulator tests

Files

  • .github/workflows/tests.yml
  • pyproject.toml — api-simulator dependency spec

Investigation Report

Full report: .autoskillit/temp/investigate/investigation_api-simulator-cache-2135_2026-05-07_145900.md
Diagnostic PR: #2161

Implementation Notes

Add a .github/CLAUDE.md with concise instructions about CI workflow constraints — setup-uv cache key behavior (exact-match save rule, no restore-keys), the version: vs uv-version: rename, when dtolnay/rust-toolchain is actually needed, and api-simulator dep constraints (auth, platform gate, maturin). Any agent editing .github/workflows/ will pick these up automatically via the CLAUDE.md hierarchy.

Activity

  1. added
    needs-human-reviewRequires human review before implementation
    in-progressIssue is being actively worked on
    stagedImplementation staged and waiting for promotion to main
    and removed
    in-progressIssue is being actively worked on
    on May 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-human-reviewRequires human review before implementationrecipe:implementationRoute through implementation recipestagedImplementation staged and waiting for promotion to main

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions