Skip to content

ci: trim CI artifacts and caches - #7942

Merged
PastaPastaPasta merged 5 commits into
dashpay:developfrom
knst:ci-compressions
Oct 27, 2026
Merged

PastaPastaPasta merged 5 commits into
dashpay:developfrom
knst:ci-compressions

Conversation

@knst

@knst knst commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Issue being fixed or feature implemented

A develop run uploads about 4.1 GB of artifacts and the repository cache sits at 7.9 GB of its 10 GB limit (run https://github.com/dashpay/dash/actions/runs/39637558287). Much of that is data nobody reads:

  • Every build bundle carries test_dash, bench_dash, fuzz, test_dash-qt and the generated qrc_*.cpp sources. The functional test job deletes them right after extracting the bundle (ci/dash/slim-workspace.sh), and in a local build they are about two thirds of the bundled binaries.
  • Functional test logs are compressed at zstd level 5. They are repetitive text where higher levels are nearly free.
  • ccache entries are written at level 1 and then every build, including pull request builds that never save the cache, recompresses the whole cache with ccache -X 9. The remote ccache only ever receives the level 1 bytes.
  • The guix workflow uploads .tar.gz, .zip and .dmg outputs through upload-artifact's default zip compression.
  • The macOS SDK took nine workflow steps, a workflow input/output pair and a slot in the repository cache for a 200 MB input that only changes with the Xcode version.

What was done?

One commit per item:

  • ci: bake the macOS SDK into the CI runner image: download, verify and extract the SDK while building ci.Dockerfile, the way ci: Cache more stuff in the ci images: msan, iwyu, pip, sdks bitcoin/bitcoin#27028 does, and point depends at it through SDK_PATH. Removes the cache lookup, download, artifact handoff and restore steps from build-depends.yml and build-src.yml.
  • ci: stop bundling the test binaries that the test job deletes on arrival: bundle-artifacts.sh excludes the test binaries and generated qrc sources, and slim-workspace.sh no longer lists them. dash-qt and dash-tx stay in the bundle because the win64 and mac bundles have no test job and exist for people to download product binaries.
  • ci: compress functional test logs with zstd level 9: on a 160 MB log sample level 9 is 13% smaller than level 5 at 0.3 s instead of 0.2 s. Level 19 would be 32% smaller but over a hundred times slower, which does not scale to the multi-gigabyte sanitizer logs.
  • ci: write ccache entries at zstd level 9 instead of recompressing later: CCACHE_COMPRESSLEVEL=9 in ci/test/00_setup_env.sh and the ccache -X 9 pass dropped from build-src.yml. On 1 GB of object files level 9 is 7% smaller than level 1; level 5 gains nothing over level 1.
  • ci: upload guix outputs without recompressing them: compression-level 0 on the guix artifact upload.

How Has This Been Tested?

Note that pull_request_target runs the workflow YAML from develop, the removal of the SDK workflow steps, the removal of ccache -X 9 from build-src.yml had been tested by PR in the fork: knst#11

CI run of this PR: https://github.com/dashpay/dash/actions/runs/40204419499 (all jobs green), compared against the develop run https://github.com/dashpay/dash/actions/runs/39637558287.

This PR reduces artifact volume by 38 percent and simplifies the mac SDK handling; it does not change billed minutes.

Artifact sizes, develop vs this PR:

artifact develop PR change
build-linux64 313 MB 129 MB -59%
build-linux64_asan 421 MB 181 MB -57%
build-linux64_tsan 423 MB 237 MB -44%
build-linux64_sqlite 316 MB 141 MB -55%
build-linux64_nowallet 277 MB 127 MB -54%
build-linux64_multiprocess 258 MB 158 MB -39%
build-aarch64-linux 286 MB 117 MB -59%
build-mac 171 MB 113 MB -34%
build-win64 378 MB 372 MB -2% (keeps dash-qt)
build-linux64_fuzz 76 MB 17 MB -77%
test_logs-linux64 264 MB 216 MB -18%
test_logs-linux64_asan 274 MB 221 MB -19%
test_logs-linux64_tsan 345 MB 282 MB -18%
test_logs-linux64_sqlite 238 MB 195 MB -18%
test_logs-linux64_nowallet 79 MB 66 MB -17%

Step timings, baseline vs the two warm runs

  • Bundle artifacts: 7-8 s to 3-5 s on the asan and sqlite builds, 2-3 s elsewhere.
  • Download build artifacts: nowallet 13 s to 6 s, tsan 9 s to 5 s, multiprocess 8 s to 3-7 s.
  • Bundle test logs: 3-5 s more for the higher zstd level (linux64 21 s to 23-26 s, tsan 10 s to 15 s).
  • mac-build and the depends check-cache jobs no longer have the SDK restore/prepare/upload steps (4 s in the baseline mac job). The mac depends and mac build jobs built against the
    SDK baked into the image: the image sets SDK_PATH and depends/Makefile only defaults it when unset. The mac job went from 5.3 min to 4.1 and 3.6 min; most of that is a 44 s ccache
    restore in the baseline against 1-2 s here.
  • Save ccache cache ran on all 11 targets of the develop run; the saved entries are 128-203 MB each, one full build at level 9, against 556-577 MB on dashpay.

Breaking Changes

None for users. The build-* artifacts no longer contain test_dash, bench_dash, fuzz or test_dash-qt; anyone who downloaded those from CI must build them locally.

Checklist:

  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have assigned this pull request to a milestone

knst added 5 commits October 25, 2026 21:46
The guix output directory holds .tar.gz, .zip and .dmg files, so the default zip compression of upload-artifact only burns CPU.
Entries were written at ccache's default level 1 and then every build, including pull request builds that never save the cache, ran ccache -X 9 to recompress them. Writing at level 9 from the start removes that pass and also shrinks what gets uploaded to the remote ccache, which only ever saw the level 1 bytes.
On 1 GB of object files level 9 is 7% smaller than level 1, and level 5 gains nothing over level 1.
Test logs are highly repetitive text, and on a 160 MB sample level 9 is 13% smaller than level 5 while taking 0.3 s instead of 0.2 s.
The highest Level 19 would be 32% smaller but more than a hundred times slower, which does not scale to the multi-gigabyte sanitizer logs.
Level 9 seems as a good balance of CPU / storage
slim-workspace.sh removes test_dash, bench_dash, fuzz, test_dash-qt and the generated qrc_*.cpp right after extracting the build bundle, so the functional test job pays to download them and thte build job pays to compress and upload them for nothing. In a local build these files are about two thirds of the binaries in the bundle. The unit tests run before bundling, so leaving them out of the archive changes nothing else.

dash-qt and dash-tx stay in the bundle: the win64 and mac bundles have no test job and exist so people can download the product binaries.
The mac depends and build jobs fetched the SDK at job time: a cache lookup in check-cache, a download from bitcoincore.org on a miss, an artifact handoff for runs whose cache token is read-only, and a restore-or-download-or-fail step triplet in each consuming job. That is nine workflow steps and a workflow output/input pair for a 200 MB input that only changes when depends/hosts/darwin.mk bumps the Xcode version, and it took a slot in the 10 GB repository cache that the ccache generations already fill.

Download, verify and extract the SDK while building ci.Dockerfile instead, the way bitcoin#27028 moved the SDK into its CI images, and hand its location to depends through SDK_PATH, since depends/Makefile defaults that to depends/SDKs inside the checkout. The depends and ccache cache keys already hash ci.Dockerfile, so an SDK bump there invalidates them as before.

contrib/containers/guix/scripts/setup-sdk stays for the Guix container.
@knst knst added this to the 24.1 milestone Oct 25, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review 🔄 Running since 2026-10-25T14:54:27.885388Z 289c7f8 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@thepastaclaw

Copy link
Copy Markdown
Collaborator

Guix Automation has started building this PR as v24.1.0-devpr7942.289c7f8a. A follow-up comment will be posted when the build finishes.

@thepastaclaw

thepastaclaw commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Final review complete — no blockers (commit 289c7f8) · triage: normal

@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2eacd635-c3c5-4b77-91b0-ca5c852ebe9d
Commits

Reviewing files that changed from the base of the PR and between 6dedab5 and 289c7f8.

Files selected for processing: 9
  • .github/workflows/build-depends.yml
  • .github/workflows/build-src.yml
  • .github/workflows/build.yml
  • .github/workflows/guix-build.yml
  • ci/dash/bundle-artifacts.sh
  • ci/dash/bundle-logs.sh
  • ci/dash/slim-workspace.sh
  • ci/test/00_setup_env.sh
  • contrib/containers/ci/ci.Dockerfile
Files with no reviewable changes: 4
  • .github/workflows/build.yml
  • .github/workflows/build-src.yml
  • ci/dash/slim-workspace.sh
  • .github/workflows/build-depends.yml

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 2 remain after this review.

Walkthrough

The CI Docker image now downloads, verifies, and installs a macOS SDK. Build workflows remove SDK artifact and cache handling. CI scripts change build archive exclusions and workspace cleanup targets. Log and Guix artifact compression settings change, and the CI environment adds a default ccache compression level.

Priority: Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: Minimal · up to 289c7

No actionable CI failure or artifact regression is established; the change is suitable for normal validation.

Pre-merge checks: Passed 5
Check name Status Explanation
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 3…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Title check Passed The title clearly and concisely summarizes the main changes to reduce CI artifacts, caches, and related storage or compression overhead.
Description check Passed The pull request description clearly explains the CI artifact, cache, compression, and macOS SDK changes described in the changeset.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 3 files. (2 skipped: 2 unsupported.)

Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@knst knst changed the title ci: trim CI artifacts and cachesxx ci: trim CI artifacts and caches Oct 25, 2026

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final validation — Phase 1 + Phase 2

The artifact-trimming changes correctly remove the listed test binaries for Unix-like targets, but the exclusions do not cover the .exe names produced by the Windows build. Consequently, the win64 bundle still contains the test executables that the PR claims to remove. The CI snapshot also includes a failed aarch64 build, but no failure details are available to establish a separate issue.

🟡 1 suggestion(s)

Review provenance

Source: reviewer 1: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: dash-core-commit-history); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

  • Triage: normal by gpt-6.1-sol (effort low) — This is a cross-cutting CI and build-infrastructure change affecting artifact contents, compression settings, caching behavior, and the macOS SDK provisioning workflow, but it does not modify a critical code surface.
  • Phase 1 reviewers: muse-spark-1.3-contributor — general (completed, effort xhigh); agent phase1-reviewer
  • Phase 1 model: muse-spark-1.3-contributor — not quota-gated; passed over gemini-3.8-flash-high (antigravity quota lookup failed), glm-5.3-flash (not used above high effort; tier asks max)
  • Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
  • Phase 2 reviewers: gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — dash-core-commit-history (completed, effort high); agent phase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `ci/dash/bundle-artifacts.sh`:
- [SUGGESTION] ci/dash/bundle-artifacts.sh:45-52: Exclude Windows .exe test binaries from bundles
  The exclusion entries use extensionless names, while Automake appends `$(EXEEXT)` for the Windows target. The win64 build therefore produces `src/bench/bench_dash.exe`, `src/qt/test/test_dash-qt.exe`, `src/test/fuzz/fuzz.exe`, and `src/test/test_dash.exe`. Because `build-src.yml` runs the unit tests and then bundles the build for this target, these files exist when the script runs, and the extensionless patterns do not match them. The win64 artifact consequently still contains the test binaries that this PR says are no longer bundled.

Comment on lines +45 to +52
# Test binaries and generated sources that only the unit tests, run
# before bundling, need
"src/bench/bench_dash"
"src/qt/qrc_bitcoin.cpp"
"src/qt/qrc_dash_locale.cpp"
"src/qt/test/test_dash-qt"
"src/test/fuzz/fuzz"
"src/test/test_dash"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: Exclude Windows .exe test binaries from bundles

The exclusion entries use extensionless names, while Automake appends $(EXEEXT) for the Windows target. The win64 build therefore produces src/bench/bench_dash.exe, src/qt/test/test_dash-qt.exe, src/test/fuzz/fuzz.exe, and src/test/test_dash.exe. Because build-src.yml runs the unit tests and then bundles the build for this target, these files exist when the script runs, and the extensionless patterns do not match them. The win64 artifact consequently still contains the test binaries that this PR says are no longer bundled.

Suggested change
# Test binaries and generated sources that only the unit tests, run
# before bundling, need
"src/bench/bench_dash"
"src/qt/qrc_bitcoin.cpp"
"src/qt/qrc_dash_locale.cpp"
"src/qt/test/test_dash-qt"
"src/test/fuzz/fuzz"
"src/test/test_dash"
# Test binaries and generated sources that only the unit tests, run
# before bundling, need
"src/bench/bench_dash"
"src/bench/bench_dash.exe"
"src/qt/qrc_bitcoin.cpp"
"src/qt/qrc_dash_locale.cpp"
"src/qt/test/test_dash-qt"
"src/qt/test/test_dash-qt.exe"
"src/test/fuzz/fuzz"
"src/test/fuzz/fuzz.exe"
"src/test/test_dash"
"src/test/test_dash.exe"

source: muse-spark-1.3-contributor (phase1-reviewer: general); gpt-6.1-sol (phase2-reviewer: general, dash-core-commit-history)

@thepastaclaw thepastaclaw added the pastaclaw:commented thepastaclaw's latest review was comment-only label Oct 25, 2026
@thepastaclaw

Copy link
Copy Markdown
Collaborator

@PastaPastaPasta
PastaPastaPasta merged commit 0844304 into dashpay:develop Oct 27, 2026
60 of 73 checks passed
@thepastaclaw thepastaclaw removed the pastaclaw:commented thepastaclaw's latest review was comment-only label Oct 27, 2026
PastaPastaPasta added a commit that referenced this pull request Oct 29, 2026
…the hosted queue backs up

29cbd9d ci: run linux64 build and functional tests on the fast path runner (pasta)
50f1b75 ci: pick a self-hosted runner for a linux64 fast path (pasta)

Pull request description:

  ## Issue being fixed or feature implemented

  Most of the time a Core PR spends in CI goes to waiting for a GitHub-hosted runner, not to building or testing. Over the last week of `build.yml` runs, the linux64 critical path looked like this:

  | job | queue p50 | queue p90 | run p50 |
  |---|---|---|---|
  | `x86_64-pc-linux-gnu / Build depends` (ran in 14 of 24 runs) | 5.5 min | 84 min | 17 min |
  | `linux64-build` | 7.4 min | 43 min | 6 min |
  | `linux64-test` | 2.7 min | 28 min | 16 min |

  On average that path spends about 41 min queued against about 32 min of work. The pre-lint jobs already move to our own hardware when the hosted queue backs up (#7901, #7902, #7942). This PR does the same for the linux64 build and its functional tests.

  ## What was done?

  - `select_dynamic_runner.py` gains a `runner_fast` output. When the repository variable `RUNNER_SELFHOSTED_FAST` names a label, it picks that label using the pre-lint rung's rules:
    - some hosted job is queued;
    - no `blacksmith-ci` label overrides the choice;
    - the label isn't saturated: at most 2 jobs queued for it (`SELFHOSTED_FAST_MAX_QUEUED`). These jobs hold a runner for minutes rather than seconds, so the cap is lower than pre-lint's 6.

    Otherwise it falls back to the amd64 runner. Trust is the existing `SELFHOSTED_LINT_AUTHORS` allowlist, plus every push to `develop` or `master`: those carry merged code and already require write access. Fork PRs from anyone else stay on the hosted ladder.
  - The pre-lint selection now takes a precomputed "allowed" flag, so both rungs share one function. Its decisions and reason strings don't change.
  - `build.yml` sends the linux64 depends build job (cache misses only), `linux64-build` and `linux64-test` to `runner-fast`. They replace their hosted runs rather than run alongside them, so required check names stay the same and no hosted capacity is used twice.
  - `build-depends.yml` gains `runs-on-build` for the build job only. `runs-on` still forms the depends cache key, so a fast-path build saves the same entry the hosted asan, fuzz and sqlite builds restore.
  - `ci/README.md` documents the gate, the fallback and what a runner for the label must provide.

  Nothing changes until `RUNNER_SELFHOSTED_FAST` is set; unsetting it turns the path off again. Runner setup (a dedicated single-use, Docker-capable label) is handled outside this repository.

  `linux64-test` is the test job that moves. It consumes the same `linux64` bundle, so the fast path needs no extra build, and it is the broadest functional run (previous releases, `--extended`). A trimmed subset would have meant a new job and new required-check names.

  ## How Has This Been Tested?

  - `python3 .github/workflows/test_select_dynamic_runner.py`: 44 tests pass, 9 of them new. They cover the variable being unset, trusted and untrusted PRs (author, actor and cross-fork head), develop and master pushes by a non-allowlisted actor, other pushes and tags, no backlog, a measurement error, the saturation cap at 2 and 3 queued jobs, the Blacksmith label and amd64 inheritance, and fast-label jobs not counting as hosted backlog. The existing pre-lint tests pass unchanged.
  - `actionlint` 1.7.12 reports nothing new against `develop`. `test/lint/lint-github-actions.py` (zizmor), `lint-whitespace.py` and `lint-files.py` are clean.
  - `pull_request_target` runs the base branch's workflows, so this PR's own CI does not exercise the change. A push of this branch to a fork ran it with the variable unset: `Check skip conditions` succeeded, and the linux64 depends build and `linux64-build` resolved through `runner-fast` to `ubuntu-24.04`, the same runner as before. The self-hosted path itself can only be tested once `RUNNER_SELFHOSTED_FAST` is set.

  ## Breaking Changes

  None. Behaviour is unchanged unless `RUNNER_SELFHOSTED_FAST` is set.

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [x] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

  🤖 Generated with [Claude Code](https://claude.com/claude-code)

Top commit has no ACKs.

Tree-SHA512: e71bcf9fb1bb4febe33af9e07478949d225d865a39a47e69a79d2302e92e31138990edeb33571d04cbb807873f3555e092840dc7ae327cd83cb30bf2e30225c8
PastaPastaPasta added a commit that referenced this pull request Oct 29, 2026
…image)

7312925 ci: bake the macOS SDK into the CI runner image (pasta)

Pull request description:

  ## Issue being fixed or feature implemented

  Pull requests into `v24.0.x` run develop's workflow files (`pull_request_target`). Since #7942 (289c7f8, "ci: bake the macOS SDK into the CI runner image"), those workflows no longer fetch the macOS SDK at job time. They expect it in the CI image built from the PR's own `contrib/containers/ci/ci.Dockerfile`, found through `SDK_PATH`. `v24.0.x`'s Dockerfile does not do this, so on every depends cache miss `x86_64-apple-darwin / Build depends` fails with `configure: error: could not find a working compiler`: `depends/SDKs` is empty. This affects every PR into `v24.0.x` (seen on #7967), independent of what the PR changes.

  ## What was done?

  Partial backport of #7942: only the `contrib/containers/ci/ci.Dockerfile` hunk of 289c7f8, picked with `-x`. It is identical to develop's. The image downloads the Xcode 26.1.1 (17B100) SDK from bitcoincore.org, verifies its SHA256 and extracts it to `/opt/SDKs`, with `ENV SDK_PATH=/opt/SDKs`.

  Left out:
  - 289c7f8's workflow hunks (`build.yml`, `build-depends.yml`, `build-src.yml`), so this branch's own push workflows keep downloading the SDK as before. With `SDK_PATH` set in the image, depends uses the baked copy either way.
  - The other four #7942 commits (artifact and cache trimming), which are not needed here.

  The Xcode version matches this branch's `depends/hosts/darwin.mk` (`XCODE_VERSION=26.1.1`, `XCODE_BUILD_ID=17B100`), and `depends/Makefile` here already has `SDK_PATH ?= $(BASEDIR)/SDKs`.

  ## How Has This Been Tested?

  - Downloaded `Xcode-26.1.1-17B100-extracted-SDK-with-libcxx-headers.tar` and checked it against the SHA256 in the Dockerfile: OK.
  - The CI image itself is built by this PR's CI (`Build container`), and the mac depends job is the test.

  ## Breaking Changes

  None. CI only.

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone

  🤖 Generated with [Claude Code](https://claude.com/claude-code)

Top commit has no ACKs.

Tree-SHA512: 47adbaef6388bf076e66b286c28e4de2ad8870334d6cb29d8a66fd6bf52b15330d9c2f595f27d36cfa8ec5519209b026a8632ca24e7453e0e115984137778799
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants