fix: test-performance batch (#1105, #1120, #1121, #1123)

.config/nextest.toml

@@ -6,6 +6,17 @@
 # the workflow runs those separately via `cargo test --doc`, mirroring the
 # `test` / `test-doc` split in the Makefile.
 
+# nextest's built-in default is `fail-fast = true`, which would report only
+# the first failing test. `cargo test` — what `make test` ran before #1120 —
+# reported every failure in the first failing binary, so leaving the default
+# in place would have made local runs strictly less informative than the
+# command they replaced. That matters most for the bulk-snapshot refreshes
+# described in AGENTS.md, where a grammar bump or a metric change is
+# *expected* to move hundreds of tests at once and the whole point is to see
+# the full set.
+[profile.default]
+fail-fast = false
+
 [profile.ci]
 # Collect the full failure set in a single run rather than stopping at the
 # first failure, so the JUnit report is complete after one invocation.

AGENTS.md

@@ -199,8 +199,12 @@ Before considering a change done, run `make pre-commit` from the repo
 root. It is the canonical entry point for the full validation gate
 and runs, in one parallel pass: the cargo trio (`cargo fmt --check`,
 `cargo clippy --workspace --all-targets -- -D warnings` in both
-default-features and `--all-features` flavours, `cargo test
---workspace --all-features`), `cargo doc --no-deps --workspace
+default-features and `--all-features` flavours, the full test suite
+via `make test` + `make test-doc` — `make test` runs `cargo nextest
+run --workspace --all-features` when `cargo-nextest` is on PATH and
+falls back to `cargo test --workspace --all-features --lib --bins
+--tests` otherwise, and `make test-doc` covers the doctests nextest
+cannot run), `cargo doc --no-deps --workspace
 --all-features` with `RUSTDOCFLAGS="-D warnings"`,
 `cargo +nightly udeps`, the markdown /
 TOML / shell / Makefile / GitHub Actions lint families, the man-page
@@ -265,8 +269,9 @@ with `make self-scan-write-baseline-headroom` in the commit that moved
 the metric. A red gate on `main` traces directly to skipping this step.
 
 If GNU Make 4 or any of the optional tools (`taplo`, `rumdl`,
-`shellcheck`, `shfmt`, `checkmake`, `actionlint`, `ruff`, `mypy`, `pyright`,
-`maturin`) are unavailable, fall back to the raw cargo commands:
+`shellcheck`, `shfmt`, `checkmake`, `actionlint`, `cargo-nextest`,
+`ruff`, `mypy`, `pyright`, `maturin`) are unavailable, fall back to the
+raw cargo commands:
 
 ```bash
 cargo fmt --all -- --check

CHANGELOG.md

@@ -87,6 +87,19 @@ for historical reference.
 
 ### Changed
 
+- `make test` now runs the suite through `cargo-nextest` when it is
+  available, matching CI, and falls back to `cargo test` otherwise
+  (#1120). nextest schedules every binary's tests into one global pool
+  rather than finishing each test binary before starting the next. Set
+  `NEXTEST=` to force the fallback, or point it at a specific binary.
+  nextest's default profile disables fail-fast so a local run still
+  reports the whole failure set, as `cargo test` did.
+- Corpus snapshot tests now assert an exact per-corpus file count, and
+  separately that each resolved file reached its snapshot assertion
+  (#1123). A traversal or glob change that silently analyzed fewer files
+  previously still passed while verifying less than it claimed. A
+  deliberate corpus bump must update the expected count alongside the
+  snapshots.
 - **(breaking)** `FuncSpace`, `Ops`, `AstNode`, `wire::FuncSpace`, and
   `wire::Ops` now implement `Drop` (#1056), so fields can no longer be
   moved out of one by value: `let m = space.metrics;` becomes
@@ -140,6 +153,27 @@ for historical reference.
 
 ### Performance
 
+- `#[cfg(...)]` predicate classification is now linear in the attribute
+  body rather than `O(len²)` on deeply nested predicates such as
+  `all(all(all(…test…)))` (#1105). Every operand previously rescanned the
+  whole remaining tail probing for a top-level comma; commas are now
+  bucketed by paren depth in one forward pass and each region queries its
+  own bucket. This path is reached from every Rust attribute under
+  `--exclude-tests` — which this repository's own `bca.toml` enables — so
+  a machine-generated or adversarial source file was a denial-of-service
+  vector. Classification behaviour is unchanged, verified against the
+  previous implementation over millions of generated predicates. The
+  depth-50 000 regression test drops from ~73 s to ~0.04 s.
+- Dependencies now build at `opt-level = 1` under the dev and test
+  profiles (#1121). The tree-sitter runtime and every grammar are C/C++
+  libraries compiled by `cc-rs`, which forwards Cargo's `OPT_LEVEL`, so
+  they were previously parsing at roughly half speed in test builds.
+  Workspace members are unaffected and stay fully debuggable; the cost is
+  a one-time dependency rebuild and a slower cold build. Deliberately 1
+  rather than 2, which measured slower on the unit suite.
+- Corpus snapshot tests size their worker pool from
+  `available_parallelism()` instead of a hardcoded 4 jobs, which had left
+  three consumer threads analyzing DeepSpeech's 1042 files (#1123).
 - The ancestor chain now reaches the per-language metric bodies, retiring
   the last per-node `Node::parent` calls in the walk (#1096).
   `Getter::get_op_type` (and its `_with_code` variant), `Abc::compute`,

CONTRIBUTING.md

@@ -49,7 +49,8 @@ gate. In one parallel pass it runs, among other stages:
 - `cargo fmt --check`.
 - `cargo clippy --workspace --all-targets -- -D warnings` in both
   default-features and `--all-features` flavours.
-- `cargo test --workspace --all-features`.
+- The full test suite via `make test` (`cargo-nextest` when present,
+  otherwise `cargo test`) plus `make test-doc` for doctests.
 - `cargo doc --no-deps --workspace --all-features` with
   `RUSTDOCFLAGS="-D warnings"`.
 - `cargo +nightly udeps`.

Cargo.toml

@@ -366,6 +366,36 @@ serde_yaml = "^0.9"
 ciborium = "^0.2"
 toml = "^1.1"
 
+# Optimize dependencies in dev/test builds. The `"*"` glob excludes
+# workspace members, so the crates under `[workspace].members` stay at
+# opt-level 0 — quick to rebuild and to step through. It does cover the
+# vendored `tree-sitter-*` grammar crates, which are `[workspace].exclude`d
+# path dependencies rather than members; that is the point of the stanza.
+#
+# The motivation is specific to this workspace: the tree-sitter runtime and
+# every grammar are C / C++ libraries built by `cc-rs`, which forwards
+# Cargo's `OPT_LEVEL` for the package being built as the compiler's
+# optimization flag (`-O<n>` on GNU / Clang; cc maps 1 to `/O1` and 2 or 3
+# to `/O2` on MSVC, so the level tuned below is not identical on Windows).
+# Left at 0, the parse and the YAML round-trip that together dominate the
+# corpus snapshot suites both run roughly twice as slow.
+#
+# Deliberately 1 rather than 2 (#1121). At 2 the root-crate unit suite —
+# thousands of tiny parses, where the extra inlining grows the hot code
+# more than the optimization saves — measured consistently slower, while 1
+# left it within run-to-run noise and still captured the corpus win
+# (`deepspeech_test` roughly halved). Per-config numbers and their spread
+# are recorded in #1121; they are single-machine, so re-measure before
+# changing this rather than trusting a remembered figure.
+#
+# Cost: a one-time full dependency rebuild, and a cold `cargo test
+# --no-run` that is markedly slower than at opt-level 0.
+[profile.dev.package."*"]
+opt-level = 1
+
+# A named override beats the `"*"` glob above, so these two keep the
+# opt-level 3 insta's own docs recommend for snapshot diffing (added in
+# 8ccffeb2).
 [profile.dev.package.insta]
 opt-level = 3
 

Makefile

@@ -59,6 +59,35 @@ FIND_EXCLUDE   := $(foreach dir,$(EXCLUDE_DIRS),! -path './$(dir)/*')
 # warnings on `$(2)`, e.g. $(call find-by-ext,md,).
 find-by-ext = $(if $(FD),$(FD) --extension $(1) $(FD_EXCLUDE) $(2),find . -name "*.$(1)" -type f $(FIND_EXCLUDE))
 
+# Test runner: prefer cargo-nextest, fall back to `cargo test` (#1120).
+# `cargo test` runs the workspace's 72 test binaries strictly one after
+# another — parallelism exists only *within* a binary, so the ten root
+# integration binaries that hold a single `#[test]` each pin the whole
+# machine to one core for their duration. nextest schedules every
+# binary's tests into one global pool instead. CI already runs nextest
+# (the `test` job in .github/workflows/ci.yml); this is the local half.
+#
+# The two paths execute an identical target set: nextest's default
+# selection is exactly `--lib --bins --tests`, verified by diffing
+# `cargo nextest list --workspace --all-features` against the same
+# invocation with those flags spelled out (4,731 tests, plus the 10
+# `#[ignore]`d ones both runners skip).
+#
+# Expect a smaller wall-clock win than the scheduling change implies:
+# one test (`cfg_predicate::tests::rust_attr_test_handles_deeply_nested_
+# cfg_without_overflow`, the O(len^2) blowup tracked in #1105) runs for
+# ~78s and is now the critical path. The other 4,730 finish in ~17s
+# together, so fixing #1105 is what converts this into a ~4x win.
+#
+# Held in a variable rather than probed inline with `command -v`, and used
+# as the command itself, so it overrides the same way `FD` above does:
+# `make test NEXTEST=` exercises the fallback without uninstalling
+# anything, and `make test NEXTEST=/path/to/cargo-nextest` picks a
+# specific build. A cargo subcommand binary takes its own name as the
+# first argument, hence `$(NEXTEST) nextest run`.
+NEXTEST        := $(shell command -v cargo-nextest 2>/dev/null)
+TEST_CMD       = $(if $(NEXTEST),$(NEXTEST) nextest run --workspace --all-features,cargo test --workspace --all-features --lib --bins --tests)
+
 .PHONY: help check-tools build build-release check test test-doc fmt fmt-check markdown-fmt markdown-lint shellcheck sh-fmt sh-fmt-check toml-fmt toml-fmt-check toml-lint makefile-check actionlint snapshot-anchors grammar-marker-sync grammar-marker-sync-test check-versions check-manpage-assets enums-check enums-codegen-drift enums-codegen-drift-test self-scan self-scan-headroom self-scan-write-baseline self-scan-write-baseline-headroom vcs lint clippy udeps insta-review insta-accept clean distclean install install-cli install-web doc doc-open doc-check doc-check-docsrs book book-serve book-pot book-po-update book-ja book-deploy all pre-commit ci release-check verify-changelog pkg-deb-local pkg-rpm-local dev-env-build dev-env-run dev-env-shell dev-env-rm py-bootstrap py-sync py-relock py-clean py-fmt py-fmt-check py-lint py-typecheck py-test py-stubtest smoke smoke-cli smoke-lib bench bench-scaling bench-walk _check-find _pc-fmt _pc-clippy _pc-test _pc-doc-check _pc-udeps _pc-shellcheck _pc-markdown-lint _pc-toml-lint _pc-makefile-check _pc-actionlint _pc-snapshot-anchors _pc-grammar-marker-sync _pc-grammar-marker-sync-test _pc-check-versions _pc-check-versions-test _pc-check-grammar-crate-test _pc-check-manpage-assets _pc-enums-check _pc-enums-codegen-drift _pc-enums-codegen-drift-test _pc-self-scan _pc-self-scan-headroom _pc-py-fmt _pc-py-typecheck _pc-py-test _pc-py-stubtest _ci-fmt-check _ci-clippy _ci-test _ci-doc-check _ci-build _ci-udeps _ci-shellcheck _ci-markdown-lint _ci-toml-lint _ci-makefile-check _ci-actionlint _ci-snapshot-anchors _ci-grammar-marker-sync _ci-grammar-marker-sync-test _ci-check-versions _ci-check-versions-test _ci-check-grammar-crate-test _ci-check-manpage-assets _ci-enums-check _ci-enums-codegen-drift _ci-enums-codegen-drift-test _ci-enums-codegen-drift-test _ci-self-scan _ci-self-scan-headroom _ci-cargo-pipeline _ci-py-fmt-check _ci-py-lint _ci-py-typecheck _ci-py-test _ci-py-stubtest
 
 # Default target
@@ -76,7 +105,7 @@ help:
 	@echo "  check                                Run cargo check"
 	@echo ""
 	@echo "Test targets:"
-	@echo "  test                                 Run unit and integration tests"
+	@echo "  test                                 Run unit and integration tests (nextest if present)"
 	@echo "  test-doc                             Run cargo doc tests"
 	@echo "  insta-review                         Review pending insta snapshot diffs"
 	@echo "  insta-accept                         Accept all pending insta snapshots"
@@ -189,11 +218,22 @@ check:
 # Test
 # ---------------------------------------------------------------------------
 test:
-	cargo test --workspace --all-features --lib --bins --tests
+	@if [ -z "$(NEXTEST)" ]; then \
+	  echo "cargo-nextest not found or disabled; using 'cargo test' (same tests, slower)."; \
+	  echo "Install with: cargo install --locked cargo-nextest"; \
+	fi
+	$(TEST_CMD)
 
+# Doctests stay on `cargo test`: nextest cannot run them at all, so this
+# is a separate target rather than an oversight (see .config/nextest.toml).
 test-doc:
 	cargo test --workspace --all-features --doc
 
+# `cargo insta test` shells out to `cargo test`, not $(TEST_CMD): insta's
+# nextest integration needs `--test-runner nextest`, and under it insta
+# cannot force-pass a failing assertion to collect every pending snapshot
+# in one run — which is exactly what review/accept need. Snapshot review
+# is an interactive, non-hot-loop operation, so it keeps the slower runner.
 insta-review:
 	cargo insta test --review
 
@@ -1021,8 +1061,8 @@ _pc-clippy: _pc-fmt
 	cargo clippy --workspace --all-targets --all-features -- -D warnings
 
 _pc-test: _pc-clippy
-	cargo test --workspace --all-features --lib --bins --tests
-	cargo test --workspace --all-features --doc
+	$(MAKE) test
+	$(MAKE) test-doc
 
 _pc-doc-check: _pc-test
 	$(MAKE) doc-check
@@ -1160,9 +1200,17 @@ _ci-clippy:
 	cargo clippy --workspace --all-targets -- -D warnings
 	cargo clippy --workspace --all-targets --all-features -- -D warnings
 
+# Mirrors the `test` job in .github/workflows/ci.yml (nextest for the
+# lib/bin/integration tests, `cargo test --doc` for doctests). The
+# workflow additionally passes `--profile ci --locked`. Neither changes
+# the executed test set: the `ci` profile differs from `default` only in
+# emitting JUnit XML for the PR annotation step (both disable fail-fast,
+# see .config/nextest.toml), and `--locked` is a lockfile assertion that
+# every `_ci-*` target omits — `make ci` will not catch a `Cargo.lock`
+# that CI's `--locked` rejects.
 _ci-test:
-	cargo test --workspace --all-features --lib --bins --tests
-	cargo test --workspace --all-features --doc
+	$(MAKE) test
+	$(MAKE) test-doc
 
 _ci-doc-check:
 	$(MAKE) doc-check

big-code-analysis-book/src/developers/README.md

@@ -31,16 +31,17 @@ The repository ships a `Makefile` that wraps every common build, test,
 lint, format, and docs task. Run `make help` to see the full list of
 targets, and `make check-tools` to verify which optional tools
 (`taplo`, `rumdl`, `shellcheck`, `shfmt`, `checkmake`,
-`mdbook`, `cargo-insta`, `cargo-udeps`) are present on your machine.
+`mdbook`, `cargo-insta`, `cargo-udeps`, `cargo-nextest`) are present on
+your machine.
 
 The two composite targets you will use most:
 
 - `make pre-commit` — the recommended local gate before committing.
   Runs `cargo fmt --check`, both clippy invocations
-  (default-features and `--all-features`), `cargo test --workspace
-  --all-features` (lib + bin + integration + doc), `cargo +nightly
-  udeps`, and the markdown / TOML / shell / Makefile lint families
-  in one parallel pass.
+  (default-features and `--all-features`), the full test suite via
+  `make test` + `make test-doc` (lib + bin + integration + doc),
+  `cargo +nightly udeps`, and the markdown / TOML / shell / Makefile
+  lint families in one parallel pass.
 - `make ci` — the same checks in the order CI runs them, with no
   auto-fixing. Use this to reproduce a failing CI run locally.
 
@@ -74,10 +75,31 @@ type-checking during iteration.
 To verify that all tests pass:
 
 ```console
-make test       # cargo test --workspace --all-features --lib --bins --tests
+make test       # cargo nextest run --workspace --all-features
 make test-doc   # cargo test --workspace --all-features --doc
 ```
 
+`make test` prefers [cargo-nextest](https://nexte.st/), which is also
+what CI runs. Plain `cargo test` finishes each test binary before
+starting the next, so parallelism exists only within a binary; nextest
+schedules every binary's tests into one global pool. Install it with
+`cargo install --locked cargo-nextest`.
+
+Without it, `make test` falls back to `cargo test --workspace
+--all-features --lib --bins --tests`, which runs the same set of tests.
+The two runners are not quite interchangeable, though: nextest runs each
+test in its own process, so anything sharing state across tests within a
+binary — a `OnceLock` cache, an environment variable, the working
+directory — is isolated under nextest and shared under `cargo test`.
+Preferring nextest locally means local runs match CI on that axis.
+
+Set `NEXTEST=` to force the fallback without uninstalling anything
+(`make test NEXTEST=`), or point it at a specific binary
+(`make test NEXTEST=/path/to/cargo-nextest`).
+
+Doctests are a separate target because nextest cannot run them.
+`make pre-commit` and `make ci` run both.
+
 If you only want to run the cargo command yourself:
 
 ```console