[guluchen]

Objective

Reorganize QDistSAT's benchmark matrices into difficulty tiers data/tier0 … data/tier5 and retire the QT/QT2 directory names in favour of TN, following the PI benchmark-tier decisions of 2026-10-11, while keeping every consumer of data/ working — in particular DistQLDPC's cross-repo CI (compare_distqldpc_binaries.py --data-root "$GITHUB_WORKSPACE/qdistsat/data" --stems LP_136_32_4 BB_108_8_10).

What changed

Data move — 27 codes / 108 files, all git mv (R100: content and stems unchanged). The spec renames (PK_31, xu_30, xu_42, TN_648_10_71) don't apply to any stem here, and codes that exist only in DistQLDPC are not copied.

Tier	Stem (former dir)
tier0	BB_72_12_6 (BB), LP_34_20_2 (LP), LP_136_32_4 (LP), TN_36_8_3 (QT), TN_36_8_4 (QT), TN_54_11_4 (QT), TN_72_14_4 (QT), TN_72_8_8 (QT), TN_108_2_12 (QT), TN_200_10_10 (QT)
tier1	BB_90_8_10 (BB), GB_144_12_8 (BB), BB_108_8_10 (BB), LP_238_44_6 (LP)
tier2	LP_340_56_8 (LP)
tier3	BB_144_12_12 (BB2), GB_144_12_12 (BB), BB_144_14_14 (BB2), LP_442_68_10 (LP2), LP_544_80_12 (LP), TN_144_2_13 (QT2)
tier4 (2000 s)	TN_250_10_15 (QT), BB_288_12_unknown (BB2), LP_714_100_unknown (LP2)
tier5 (21600 s)	BB_360_12_unknown (BB), LP_1768_224_unknown (LP2), TN_360_4_unknown (QT2)

data/BB, BB2, LP, LP2, QT and QT2 are gone. tables/*.tex are kept unchanged because no stem was renamed.

Tools / code

• benchmarks/compare_distqldpc_binaries.py: matrix_prefix now searches recursively under --data-root, so it works with the tier directories, a single tier, or a flat directory. It still requires exactly one match and now lists the matches when that fails. The CLI is unchanged.
• scripts/run_benchmark_six_stemdirs.sh: runs the six tier directories (one process and one log per tier) instead of the six family batches. New --tiers "tier0 tier1 …" runs a subset. New --tier-timeouts applies tier4 = 2000 s and tier5 = 21600 s; other tiers keep --timeout. Default behaviour is unchanged otherwise, and the help text now shows the actual default timeout (7200 s).
• scripts/run_benchmark_stems_dir.sh, benchmarks/benchmark_solver_performance.py, precompute-logicals (--benchmark-dir help), and the distqldpc_runner install hint: tier paths in help text and examples. Stem lookup was already recursive (qecc_sat.io).
• Tests: tests/test_repository_paths.py now checks tier membership, that each stem has all 4 matrices, that every shipped stem is in exactly one tier, that only tier subdirectories exist and that no QT_* stem remains. It also checks that the harness finds LP_136_32_4 (tier0), BB_108_8_10 (tier1) and TN_360_4_unknown (tier5), both from data/ and from the tier directory. The tests/matrix_stem.py fixture now points at data/tier0.
• Docs: README.md, data/README.md (tier table with timeouts, naming rule {family}_{n}_{k}_{d} with family in BB/GB/LP/TN, QT -> TN, previous layout), and docs/REPO_SCOPE.md. In NOTICE all attribution is kept, and a new §2c records the move from each former directory to its tier.

Validation

• python -m pytest tests/ -q (what CI runs; local Python 3.9.6 venv with pip install -e ".[dev]"): 44 passed, 1 skipped (the skip is cashw-coreplus not installed).
• The DistQLDPC cross-repo comparison, run locally with the CI arguments against this branch's data/, using the DistQLDPC main 7eadd54 binary as both baseline and candidate:

  python3 qdistsat/benchmarks/compare_distqldpc_binaries.py --baseline-bin frozen-main7e/distqldpc --candidate-bin frozen-main7e/distqldpc \
    --data-root qdistsat/data --stems LP_136_32_4 BB_108_8_10 --timeout 45 --json-out … --markdown-out …
  

  It exited 0 with scientific_semantics_match: true.
• benchmark_solver_performance.py --stem TN_36_8_4 -d 4 --solvers rc2-glucose42 resolved data/tier0/TN_36_8_4_Hx.txt and gave d=4 (OK).
• precompute-logicals LP_136_32_4 resolved data/tier0/… (and skipped it because the file exists). discover_css_matrix_stems('data') finds 27 stems with the counts 10/4/1/6/3/3.
• run_benchmark_six_stemdirs.sh --dry-run --tier-timeouts builds one command per tier, with --timeout 2000 for tier4 and --timeout 21600 for tier5. --tiers "tier0 tier2" runs only those two.

Scientific impact

• No scientific semantics/assumptions changed (file moves only; matrix bytes and stems identical)
• Scientific semantics/assumptions changed — PI review required

Results / evidence

Stem	Config	Baseline	Candidate	Semantic match
LP_136_32_4	no-card	d=4 (0.07s)	d=4 (0.07s)	YES
LP_136_32_4	card-mto	d=4 (0.07s)	d=4 (0.06s)	YES
BB_108_8_10	no-card	d=10 (0.45s)	d=10 (0.45s)	YES
BB_108_8_10	card-mto	d=10 (0.58s)	d=10 (0.57s)	YES

Discrepancy — reported, not resolved here: QDistSAT ships TN_360_4_unknown (now in data/tier5) under that name. DistQLDPC and the tier spec list the same tier5 code as TN_360_4_24. Per the spec, the QDistSAT name is kept until the distance is cross-checked.

PI action required

• No
• Yes — merge go-ahead. DistQLDPC's qdist-cross-repo-benchmark.yml checks out QDistSAT main, so merging this changes the data layout that CI sees. The harness on this branch resolves the CI stems in the new layout, as shown above. Options: (a) merge now; (b) hold until the matching DistQLDPC tier change lands. Also still open: the TN_360_4_unknown vs TN_360_4_24 name.

🤖 Generated with Claude Code