Skip to content

recipes: fork independent key self-test sections - #789

Merged
juliankang4 merged 2 commits into
midagedev:mainfrom
juliankang4:dgx-spark-cut-the-check-recipes
Oct 24, 2026
Merged

juliankang4 merged 2 commits into
midagedev:mainfrom
juliankang4:dgx-spark-cut-the-check-recipes

Conversation

@juliankang4

Copy link
Copy Markdown
Collaborator

check-recipes의 입력 key self-test를 독립 구간으로 나눠 병렬로 실행해 전체 시간을 줄입니다.

What and why

tools/check-recipes.sh spends most of its wall time in key_self_test in tools/recipes.py. That test runs four independent contracts one after another in a single job: the input classes, the fixture tier, the variants and the model scope. Each contract already works on its own scratch tree and manifest, so nothing forces them to run serially.

This PR registers them as four separate jobs in the existing _forked runner (key inputs, key fixtures, key variants, key scope). Each job gets its own scratch tree, manifest and temporary HOME, so the Reach cache it writes disappears with it. section=None still runs the old sequential order for diagnosis. The self-test also prints a time per key section next to the existing slowest-section line. A first commit reuses probe results the test had already computed instead of recomputing them in later assertions.

No assertion was dropped. The expect calls inside key_self_test are the same 94 before and after, compared as syntax trees. The product code that computes and verifies keys is unchanged.

Release notes: internal | the recipes check finishes sooner because its key self-test sections run in parallel.

Code shape

  • No second copy. No new helper or function. The change reuses _forked and unittest.mock.patch.
  • Model code holds only architecture facts. Not a model change.
  • Dead code is gone. Nothing was left unused.
  • No silent failure. An unknown section value trips an assertion, and _forked collects each job's failures as before.
  • Comments state what is true now.
  • No relaxed gate. No case, band or tolerance changed. The shared base assertions now run once in each of the four jobs instead of once in total.
  • cargo fmt and cargo clippy: no Rust is touched, so I did not run them.

Duplication found or removed (path:line), or "none": none.

What I ran

The check ran on a four-core x86 Linux box (AMD Ryzen 5 PRO 6650H), pinned to four logical CPUs, with CPython 3.12.15 and RUSTFLAGS='-C target-cpu=x86-64-v3'. Both arms ran back to back in one sitting: this branch first, then upstream main. Each arm was:

bash tools/check-recipes.sh --jobs > jobs.tsv
python3 tools/static-sched.py run --max 4 --out blocks jobs.tsv
bash tools/check-recipes.sh --report blocks

Wall time of check-recipes on one run per arm:

Arm Wall (s) Blocks run Failures
main 971.40 147 0
this branch 770.03 147 0
  • Wall time: the branch run reports wall= 770.03 and the main run reports wall= 971.40.
  • Blocks: both arms list the same 147 block names, and the report line is "147 blocks ran, 0 skipped by a green input key" on each side.
  • Self-test: both arms end with "self-test: ok (227 gate recipes, 0 failures)".
  • Key sections on the branch: key inputs 704 s, key fixtures 410 s, key variants 244 s, key scope 458 s, key reach 547 s. The sections overlap, so they do not add up to the wall time.

The saving comes from running the sections in parallel. The work itself is the same, and the extra jobs compete for the same four CPUs. On an earlier comparison of the same change the key reach section grew from 362 s to 557 s. The check is still shorter overall, but it is one run per arm on one machine, so I make no claim about significance or about other CPU counts. The next candidates are listed below.

I also compared the expect calls of key_self_test before and after as syntax trees (94 identical), ran the real dispatch to confirm each of the four sections is called exactly once, and checked three source mutants against that test: dropping a section, losing the section=section binding, and deleting a base assertion. All three were rejected.

This is Python tooling for the check only, so the GPU path is untouched. Anything beyond the runs above is left to CI.

Hardware (if the change is for a machine)

Not applicable. CPU-only tooling change. The timing conditions are listed above.

What I ran into

This does not block the PR; these are candidates for follow-up work.

  • The largest remaining section is key inputs. Its global-input loop prepares the analysis five times in one place. Splitting that loop from the rest keeps every assertion, but it adds fixture setup and more CPU contention, so it needs its own full-check measurement.
  • Restoring an input to its earlier bytes after another corpus overwrote the Reach cache forces a second analysis. A bounded cache that keeps the corpus, the raw bytes, the code identity and full schema validation could avoid it. I have not measured how much it would save.
  • The self-test now starts 19 children instead of 16. Ordering or limiting them might reduce the contention, but I have not measured it.

AI help

AI 코딩 에이전트가 코드를 작성한 후 설명을 썼고, 테스트를 마쳤습니다. diff 리뷰와 PR 초안 리뷰는 제가 직접 했습니다.

@github-actions github-actions Bot added the tools Tools, recipes, CI and the toolchain: tools/, justfile, .github/, Cargo label Oct 24, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: CLEAN: The four section jobs retain the existing assertions and propagate failures through _forked.

Checked

  • Duplicates: 0 new functions, types or constants. The change reuses _forked; no dryer pairs were reported.
  • Complexity: no Rust functions changed. Python growth adds four section-selection branches to the existing test.
  • Silent failure: 3 cached skip-reason sites followed to their assertions; no new error suppression.
  • Tests: all 94 key and 6 reach assertion conditions match after expanding cached expressions. The dispatch expression, evaluated with a stubbed section test, invokes inputs, fixtures, variants and scope exactly once with the intended bindings. _forked collects job failures and exceptions; self_test returns 1 when failures remain.
  • Gates: no assertion, tolerance or pin was removed or relaxed.
  • Escape catalog: C1–C4 do not match. Separate scratch trees and temporary homes prevent cross-job mutation; section=None retains the sequential path.

Notes

Dryer and crapper did not run because no Rust file changed. Validation used syntax-tree comparison and the stubbed dispatch check; the full recipes suite was not rerun.

🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)

@juliankang4

Copy link
Copy Markdown
Collaborator Author

The review is done and the contribution check passed, so I'm merging this.

@juliankang4
juliankang4 merged commit c076199 into midagedev:main Oct 24, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tools Tools, recipes, CI and the toolchain: tools/, justfile, .github/, Cargo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant