diff --git a/.github/release-notes/v2.3.1.md b/.github/release-notes/v2.3.1.md index d3d1d42f..3bfad469 100644 --- a/.github/release-notes/v2.3.1.md +++ b/.github/release-notes/v2.3.1.md @@ -1,29 +1,13 @@ -RustyNES **v2.3.1 "Plumb Line"** is a measurement release. It makes the -performance apparatus trustworthy and then uses it — and what it found is that -**none of the ten hot-path candidates it measured yielded a shippable -improvement.** That is a claim about those ten, not about the core as a whole: -two core leads the campaign surfaced (the APU at 18.7% of frame, and `range.rs` -inlined inside `Ppu::tick` at 1.52%) remain **unmeasured** and are carried -forward. - -**No emulation-core changes.** AccuracyCoin holds at **exactly 141/141** and -nestest is 0-diff, verified after every experimental probe was reverted rather -than merely asserted by construction: this release did land and remove real -edits. +RustyNES **v2.3.1 "Plumb Line"** is a measurement release. It makes the performance apparatus trustworthy and then uses it — and what it found is that **none of the ten hot-path candidates it measured yielded a shippable improvement.** That is a claim about those ten, not about the core as a whole: two core leads the campaign surfaced (the APU at 18.7% of frame, and `range.rs` inlined inside `Ppu::tick` at 1.52%) remain **unmeasured** and are carried forward. + +**No emulation-core changes.** AccuracyCoin holds at **exactly 141/141** and nestest is 0-diff, verified after every experimental probe was reverted rather than merely asserted by construction: this release did land and remove real edits. ## Why a measurement release Two failures in the preceding release motivated it. -- **v2.3.0's adopted PPU optimization measured `+2%` on a contended host and - `−5.13%` re-measured quiet** — the same commit, opposite sign. The project's - adopt/reject bar is only as good as the host it runs on, and nothing noticed - the host. -- **The profile the campaign was scoped from does not contain the APU.** - `perf report` shows zero `rustynes_apu::` symbols at any percent limit, because - fat LTO inlines the APU wholesale into `cpu_clock`. The working split - "PPU ~53%, CPU+bus ~39%" had folded roughly a fifth of the frame into the wrong - bucket. +- **v2.3.0's adopted PPU optimization measured `+2%` on a contended host and `−5.13%` re-measured quiet** — the same commit, opposite sign. The project's adopt/reject bar is only as good as the host it runs on, and nothing noticed the host. +- **The profile the campaign was scoped from does not contain the APU.** `perf report` shows zero `rustynes_apu::` symbols at any percent limit, because fat LTO inlines the APU wholesale into `cpu_clock`. The working split "PPU ~53%, CPU+bus ~39%" had folded roughly a fifth of the frame into the wrong bucket. ## New measurement tooling @@ -33,13 +17,9 @@ Two failures in the preceding release motivated it. | `frame_breakdown.sh` — attribution by source file | the **APU is 18.7% of frame time**; `perf report --inline` does *not* recover it | | `ab_check.sh` — adoption A/B with an A/B/A order-bias control | the reference drifts up to **−1.17% from run position alone** | -Corrected subsystem split: **PPU 52.1% · APU 18.7% · CPU 10.1% · bus/scheduler -coupling 9.9% · std inlined at call sites 6.7% · mappers 2.5%.** The CPU proper -is about a third of what the symbol profile implied. +Corrected subsystem split: **PPU 52.1% · APU 18.7% · CPU 10.1% · bus/scheduler coupling 9.9% · std inlined at call sites 6.7% · mappers 2.5%.** The CPU proper is about a third of what the symbol profile implied. -`bench_relative_check.sh` additionally declines to emit a verdict when the host -was too noisy to resolve the effect under test, keyed on a robust MAD-based -coefficient of variation. +`bench_relative_check.sh` additionally declines to emit a verdict when the host was too noisy to resolve the effect under test, keyed on a robust MAD-based coefficient of variation. ## Ten candidates measured, ten rejected @@ -52,36 +32,24 @@ coefficient of variation. | the target is too small to matter | the `bg_split_state` capability gate (0.09% of frame) | | forbidden by the ownership model | hoisting `PpuBusAdapter` (borrow checker, with no `unsafe` permitted) | -Six distinct mechanisms, which is what makes this a finding rather than one bad -assumption repeated: **the per-dot loop has no incidental overhead left to -reclaim.** Its ~3.78 ms is work the accuracy model requires. That corroborates -the existing record, where bounds-check elision and a SIMD blitter both measured -*slower*. +Six distinct mechanisms, which is what makes this a finding rather than one bad assumption repeated: **the per-dot loop has no incidental overhead left to reclaim.** Its ~3.78 ms is work the accuracy model requires. That corroborates the existing record, where bounds-check elision and a SIMD blitter both measured *slower*. ## Two near-misses Worth recording, because each would have shipped on a single reading: -- One candidate produced a textbook **−1.84% … −2.75% at p = 0.00 on all four - workloads** — entirely an order-bias artifact. It measured as exactly zero on - re-run. This is what prompted the A/B/A control. -- Another measured **−0.51% at p = 0.00 on a shipped configuration** with a clean - control, then **+0.01% (p = 0.96)** on re-run. +- One candidate produced a textbook **−1.84% … −2.75% at p = 0.00 on all four workloads** — entirely an order-bias artifact. It measured as exactly zero on re-run. This is what prompted the A/B/A control. +- Another measured **−0.51% at p = 0.00 on a shipped configuration** with a clean control, then **+0.01% (p = 0.96)** on re-run. Both were caught only by requiring an independent second run. ## Also in this release -- The PGO workflow's BOLT probe no longer reports success without BOLT. It ran - `apt-get install bolt` and trusted the exit status — but on Ubuntu that package - is the **Thunderbolt 3 device manager**, so the stage failed on the tool it had - just "confirmed" instead of skipping as its best-effort contract intends. -- Every rejected experiment is recorded in `docs/performance.md` with its - numbers, its order-bias control, and the mechanism behind the null result. +- The PGO workflow's BOLT probe no longer reports success without BOLT. It ran `apt-get install bolt` and trusted the exit status — but on Ubuntu that package is the **Thunderbolt 3 device manager**, so the stage failed on the tool it had just "confirmed" instead of skipping as its best-effort contract intends. +- Every rejected experiment is recorded in `docs/performance.md` with its numbers, its order-bias control, and the mechanism behind the null result. ## Verification -- `cargo test --workspace --features test-roms` green — AccuracyCoin **141/141**, - `visual_regression` 9/9, nestest 0-diff. +- `cargo test --workspace --features test-roms` green — AccuracyCoin **141/141**, `visual_regression` 9/9, nestest 0-diff. - Workspace clippy clean at `-D warnings`; `cargo fmt --all --check` clean. - `shellcheck` clean on every touched script. diff --git a/Cargo.lock b/Cargo.lock index dc4113d9..d89a7b23 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -6231,9 +6231,9 @@ dependencies = [ [[package]] name = "wide" -version = "1.6.0" +version = "1.6.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "be99e8317aa9f08e7d16e13033ca43faab59ab582b1d0feab7b385424c42f8b1" +checksum = "de2aaf408e58689c2096682331b1f42bb2d9f2ed6b11560407d023cd0a6c634e" dependencies = [ "bytemuck", "safe_arch", diff --git a/docs/SALVAGE_MANIFEST.md b/docs/SALVAGE_MANIFEST.md index 7634e092..8e95ca7a 100644 --- a/docs/SALVAGE_MANIFEST.md +++ b/docs/SALVAGE_MANIFEST.md @@ -102,3 +102,36 @@ testing, and a built MkDocs static site — all spent/regenerable, none salvaged. Only `scripts/probes/probe_rev.rs` was promoted from untracked scratch to committed (see the row above) — everything else from the prior pass stays as previously recorded. + +## 2026-08-06 — pre-reboot salvage (manual, curated) + +Run before rebooting for an NVIDIA driver/kernel update. The `tmp-salvage` dry +run offered **305 candidates / ~600 MB**; **3 files** were taken. The rest was +deliberately left to be wiped: + +| left behind | why | +| --- | --- | +| `perf.data` (586 MB), `probe.perf`, `fp.data`, `di.data`, `bolt.log` | perf recordings and CI logs — regenerable with one command | +| ~100 `.md`/`.json` PR-review files | already posted to GitHub PR #348 | +| `fisk-*.png`, `v2p-*.png` (8 MB) | screenshots, possibly commercial-ROM framebuffers — not swept in unexamined | +| `/tmp/accuracycoin-build/` | third-party MIT source + a Windows `nesasm.exe`; vendoring is a deliberate provenance decision, not a salvage side effect | +| `agy-review.sh`, `agy-review-selftest.sh` | duplicates of files already in `scripts/` and `Local_Only-Projects/antigravity-pr-review/` | +| `p3.sh`, `p4.sh`, `probe*.sh`, `mi.rs` | extraction artifacts (YAML pulled out for `shellcheck`) and a temp probe already reverted on purpose | + +Everything built during the session was **already committed** (`frame_probe.rs`, +`ab_check.sh`, `frame_breakdown.sh`, the `perf_log_check.py` p99 gate), so the +scratchpad held no unique source beyond the three below. + +| file | → | note | +| --- | --- | --- | +| `reflow.py` | `scripts/release-automation/` | markdown full-width reflow; fixed the release-notes formatting. Existed nowhere else on disk. | +| `assemble.py` | `scripts/release-automation/` | HTML fragment → document (needs `bs4`). | +| `guardrails_assemble.py` | `scripts/release-automation/` | same, for the provenance-guardrails doc. | + +Each file was SHA-256 verified identical to its `/tmp` source **at the moment of +copy** — that is the recovery evidence, and it covers the recovery only. All +three have been edited since: `reflow.py` was fixed (hard-break and fence +handling) and both `bs4` scripts gained input validation, in response to the +PR #349 review. The files in the tree are therefore **not** byte-identical to the +recovered artifacts, and nothing here claims they are. See +`scripts/release-automation/README-doc-tools.md` for usage. diff --git a/scripts/release-automation/README-doc-tools.md b/scripts/release-automation/README-doc-tools.md new file mode 100644 index 00000000..b53806fd --- /dev/null +++ b/scripts/release-automation/README-doc-tools.md @@ -0,0 +1,72 @@ +# Release-notes / doc rendering helpers + +Three one-off tools rescued from `/tmp` before a reboot (2026-08-06). They were +written during the v2.2.5–v2.3.0 documentation work, existed **nowhere else on +disk**, and would have been lost. Recorded here so they are findable rather than +rediscovered. + +| script | needs | what it does | +| --- | --- | --- | +| `reflow.py` | stdlib only | Unwraps hard-wrapped markdown into single full-width lines. Tests: `test_reflow.py` (16 cases). | +| `assemble.py` | `bs4` | Assembles a rendered HTML fragment into a full document. Tests: `test_assemble.py` (input contract). | +| `guardrails_assemble.py` | `bs4` | Same, for the provenance-guardrails doc: injects a title block and reddens a curated set of hard takeaways. | + +## `reflow.py` — the one you will want again + +This is the tool that fixed the GitHub release-notes formatting complaint: notes +published from v2.2.5 onward had been hard-wrapped at ~80 columns, which GitHub +renders as artificially narrow text instead of using the full width available. + +It unwraps paragraphs and list items to one line each while **preserving** +blank lines, ATX headings, horizontal rules, fenced code blocks, tables, +blockquotes, and raw HTML lines — the things that break if naively joined. + +```bash +python3 scripts/release-automation/reflow.py < in.md > out.md +``` + +Worth running over any hand-wrapped `.github/release-notes/vX.Y.Z.md` before +publishing. + +## The `bs4` pair + +`assemble.py` and `guardrails_assemble.py` turn a rendered HTML fragment into a +standalone document. They need BeautifulSoup, which is not a project dependency +but **is** present on this machine (`bs4` 4.15.0): + +```bash +python3 scripts/release-automation/assemble.py frag.html out.html +# or, on a machine without it: +python3 -m venv /tmp/venv && /tmp/venv/bin/pip install beautifulsoup4 +``` + +Both are specific to the one-time provenance/guardrails PDF build +(`ref-docs/`) and are kept for reproducing those artifacts, not for routine use. + +### `assemble.py` does not accept an arbitrary fragment + +It slices the document around two required top-level elements, in this order: + +1. a ``, and +2. a `

` whose text starts with `NOTE`. + +Missing either — or a NOTE that precedes the table — is now a usage error with an +actionable message rather than a `StopIteration` traceback. `test_assemble.py` +covers each case. + +### On the `ruff` nits + +Both still carry `SIM115` (context managers) and `UP031` (percent-format) from +their `/tmp` originals, and the transform bodies are otherwise unmodified. Only +input validation was added on salvage, because that is additive and cannot change +the success path. The transform itself is still not covered by a test: **no +original rendered fragment survived**, so there is nothing to assert the output +against beyond "it did not crash". Clean up the lint nits the first time you run +one of these for real, with a genuine input to diff against. + +An earlier version of this file claimed `bs4` was not installed here. That was +wrong, and it was half the stated reason for leaving these scripts untouched — +corrected in the PR #349 review rather than left standing. + +`reflow.py` — stdlib-only and fully testable — has real coverage in +`test_reflow.py`. diff --git a/scripts/release-automation/assemble.py b/scripts/release-automation/assemble.py new file mode 100755 index 00000000..e7bd830c --- /dev/null +++ b/scripts/release-automation/assemble.py @@ -0,0 +1,120 @@ +#!/usr/bin/env python3 +import sys + +from bs4 import BeautifulSoup, NavigableString + +# Argument check added on salvage (PR #349 review). Everything BELOW this +# point is the original /tmp source, unmodified: a usage message is additive and +# cannot change the success path, whereas rewriting the parsing logic could — +# and `bs4` is not installed here and no sample fragment survived, so a rewrite +# could not be run to prove it still behaved. +if len(sys.argv) != 3: + raise SystemExit(f"usage: {sys.argv[0]} INPUT_FRAGMENT OUTPUT_HTML") +frag_path, out_path = sys.argv[1], sys.argv[2] +html = open(frag_path, encoding="utf-8").read() +soup = BeautifulSoup(html, "html.parser") + +# --- RED for curated genuine takeaways (exact text contains) --- +KEY = [ + "more serious LLM error", + "did not follow it", + "AI self-attestation of license compliance is not trustworthy", + "single most important piece of evidence", + "Correct.", + "not written purely from hardware documentation", +] +for s in soup.find_all("strong"): + t = s.get_text() + if any(k in t for k in KEY): + cls = s.get("class", []) + s["class"] = cls + ["key"] + +# --- top-level elements, in order --- +els = [c for c in soup.contents if getattr(c, "name", None)] +# This script does not accept an arbitrary fragment: it slices the document +# around a top-level

followed by a top-level

starting with "NOTE". +# Bare `next()` raised StopIteration when either was absent, which reads as an +# interpreter bug rather than "wrong input". Validated on salvage (PR #349 +# review) and exercised by `test_assemble.py`. +table_i = next((i for i, e in enumerate(els) if e.name == "table"), None) +note_i = next( + (i for i, e in enumerate(els) + if e.name == "p" and e.get_text().lstrip().startswith("NOTE")), + None, +) +if table_i is None: + raise SystemExit(f"{frag_path}: no top-level

found") +if note_i is None: + raise SystemExit(f"{frag_path}: no top-level

starting with 'NOTE' found") +if table_i >= note_i: + raise SystemExit( + f"{frag_path}: the NOTE paragraph must come after the table " + f"(table at index {table_i}, NOTE at {note_i})" + ) + +before = "".join(str(e) for e in els[:table_i]) +table = str(els[table_i]) +middle = "".join(str(e) for e in els[table_i + 1:note_i]) + +# --- italicize everything after "): " in the NOTE paragraph (PDF only) --- +note_p = els[note_i] +MARKER = "): " +kids = list(note_p.children) +head, tail_nodes, splitting = [], [], True +for k in kids: + if splitting and isinstance(k, NavigableString) and MARKER in k: + pre, post = str(k).split(MARKER, 1) + head.append(NavigableString(pre + MARKER)) + if post: + tail_nodes.append(NavigableString(post)) + splitting = False + elif splitting: + head.append(k) + else: + tail_nodes.append(k.extract() if hasattr(k, "extract") else k) +if tail_nodes: # only rebuild if the marker was found + note_p.clear() + for h in head: + note_p.append(h) + em = soup.new_tag("em") + for t in tail_nodes: + em.append(t) + # Bold + upright "Fiskbit" inside the italic note body (PDF styling only) + for tnode in list(em.find_all(string=True)): + if "Fiskbit" in tnode: + parts = str(tnode).split("Fiskbit") + repl = [] + for i, seg in enumerate(parts): + if i > 0: + st = soup.new_tag("strong") + st["class"] = ["upright"] + st.string = "Fiskbit" + repl.append(st) + if seg: + repl.append(NavigableString(seg)) + tnode.replace_with(*repl) + break + note_p.append(em) +note = str(note_p) + +TITLE = """

+

RUSTYNES · INCIDENT RECORD · GPL PROVENANCE

+

Provenance Failure Post-Mortem

+

How GPL Emulator Code Was Lifted Despite a Black-Box Instruction

+

Forensic Root-Cause Analysis · 2026-08-04 · RustyNES v2.2.9

+
""" + +body = (TITLE + + '
' + before + '
' + + table + + '
' + middle + '
' + + '
' + note + '
') + +doc = ('' + 'Provenance Failure Post-Mortem' + + body + '') + +open(out_path, "w", encoding="utf-8").write(doc) +print("assembled ->", out_path, + "| before/middle/note split at table_i=%d note_i=%d" % (table_i, note_i), + "| reddened %d strong tags" % len(soup.select("strong.key"))) diff --git a/scripts/release-automation/guardrails_assemble.py b/scripts/release-automation/guardrails_assemble.py new file mode 100755 index 00000000..f24a8ae3 --- /dev/null +++ b/scripts/release-automation/guardrails_assemble.py @@ -0,0 +1,47 @@ +#!/usr/bin/env python3 +"""Assemble the guardrails HTML: inject a title block, redden a curated +set of hard takeaways. Single-column theme handles the rest of the flow.""" +import sys + +from bs4 import BeautifulSoup + +# Argument check added on salvage (PR #349 review). Everything BELOW this +# point is the original /tmp source, unmodified: a usage message is additive and +# cannot change the success path, whereas rewriting the parsing logic could — +# and `bs4` is not installed here and no sample fragment survived, so a rewrite +# could not be run to prove it still behaved. +if len(sys.argv) != 3: + raise SystemExit(f"usage: {sys.argv[0]} INPUT_FRAGMENT OUTPUT_HTML") +frag_path, out_path = sys.argv[1], sys.argv[2] +html = open(frag_path, encoding="utf-8").read() +soup = BeautifulSoup(html, "html.parser") + +# --- RED reserved for a few genuine, full-span takeaways --- +KEY = [ + "capability plus availability plus an accuracy objective", + "C used as if it were A", + "source physically unavailable to the agent", + "Never launder", +] +reddened = 0 +for s in soup.find_all("strong"): + t = s.get_text() + if any(k in t for k in KEY): + s["class"] = s.get("class", []) + ["key"] + reddened += 1 + +body = "".join(str(c) for c in soup.contents) + +TITLE = """
+

COMMUNITY BEST-GUIDANCE · AI-ASSISTED EMULATOR DEVELOPMENT

+

Provenance & License Guardrails

+

A ready-to-ingest ruleset for Claude Code and other agentic / AI-assisted development tools

+

Preventing the copyleft-source-lifting trap · 2026-08-04

+
""" + +doc = ('' + 'Provenance & License Guardrails' + + TITLE + body + '') + +open(out_path, "w", encoding="utf-8").write(doc) +print("assembled ->", out_path, "| reddened %d strong tags" % reddened) diff --git a/scripts/release-automation/reflow.py b/scripts/release-automation/reflow.py new file mode 100755 index 00000000..c4da9ec6 --- /dev/null +++ b/scripts/release-automation/reflow.py @@ -0,0 +1,156 @@ +#!/usr/bin/env python3 +"""Unwrap hard-wrapped markdown paragraphs/bullets into single full-width lines. + +Preserves: blank lines (including runs), ATX headings (#...), horizontal rules +(--- / ***), fenced code blocks, tables (| ...), blockquotes (> ...), raw HTML +lines, and **hard breaks** (a line ending in two or more spaces). +Joins wrapped continuation lines within a paragraph or a single list item. + +# Hard breaks + +A markdown hard break is encoded as two trailing spaces, and it MEANS "end the +line here". Joining across one changes the rendered output, so a hard-break line +terminates its logical line and keeps its two spaces. The original version +`strip()`ed every buffered piece and silently destroyed them. + +# Fences + +A fenced block is closed only by a fence of the SAME character and at least the +same length, per CommonMark -- so ``` does not close a ````-opened block, and a +tilde fence does not close a backtick one. Fences nested inside a blockquote or +list item are NOT recognized; such a block's lines are treated as ordinary text. +That is a known limit, not an oversight: handling container prefixes properly +needs a real block parser, and release notes do not use the construct. + +Requires Python 3.9+ (`str.removeprefix`). CI runs 3.12. +""" +import re +import sys + +STRUCT_PREFIXES = ("#", ">", "|") + +# An opening/closing fence: optional indent, then 3+ backticks or 3+ tildes. +FENCE_RE = re.compile(r"^[ \t]*(`{3,}|~{3,})") + + +def is_list_item(s): + t = s.lstrip() + if t[:2] in ("- ", "* ", "+ "): + return True + # ordered "N. " or "N) " + i = 0 + while i < len(t) and t[i].isdigit(): + i += 1 + return i > 0 and i < len(t) and t[i] in ".)" and t[i + 1 : i + 2] == " " + + +def is_hr(s): + t = s.strip() + return len(t) >= 3 and set(t) <= {"-", "*", "_"} and len(set(t)) == 1 + + +def is_struct(s): + t = s.lstrip() + if not t: + return False + if t[0] in STRUCT_PREFIXES: + return True + if t.startswith("<") and t.rstrip().endswith(">"): + return True # standalone HTML line + return is_hr(s) or is_list_item(s) + + +def reflow(text): + lines = text.split("\n") + out = [] + buf = [] # accumulated logical line pieces (trailing space PRESERVED) + fence = None # (char, run_length) of the open fence, or None + + def flush(): + """Emit the buffered pieces as one logical line, splitting at hard breaks.""" + if not buf: + return + + def indent_of(line): + return line[: len(line) - len(line.lstrip())] + + parts = [] # (indent, text) + cur = [] + cur_indent = indent_of(buf[0]) + for piece in buf: + if not cur: + # Each emitted line keeps the indent of the piece that STARTS + # it, not the first piece of the whole buffer. After a hard + # break inside a list item the continuation must keep the item's + # content indent, or markdown ends the list item there. + cur_indent = indent_of(piece) + cur.append(piece.strip()) + # Two-or-more trailing spaces on a non-empty line = hard break: end + # the logical line here and keep the marker so it still renders. + if piece.strip() and piece.endswith(" "): + parts.append((cur_indent, " ".join(cur) + " ")) + cur = [] + if cur: + parts.append((cur_indent, " ".join(cur))) + for ind, text in parts: + out.append(ind + text) + buf.clear() + + for ln in lines: + stripped = ln.strip() + # Fenced code block. Only a fence of the same character and at least the + # same length closes an open one (CommonMark), so a ``` line inside a + # ````-opened block stays content. + m = FENCE_RE.match(ln) + if m: + marker = m.group(1) + ch, run = marker[0], len(marker) + if fence is None: + flush() + out.append(ln) + fence = (ch, run) + continue + if ch == fence[0] and run >= fence[1]: + out.append(ln) + fence = None + continue + # A non-matching fence inside an open block is just content. + if fence is not None: + out.append(ln) + continue + if stripped == "": + flush() + out.append("") + continue + # Blockquote: join consecutive `>` lines into one full-width `> ...` line. + if stripped.startswith(">"): + content = ln.lstrip()[1:] + content = content.removeprefix(" ") + if buf and buf[0].lstrip().startswith(">"): + buf.append(content) # continuation of the current blockquote + else: + flush() + buf.append(ln) # seed with the full `> ...` line + continue + if is_struct(ln): + # A structural line starts its own logical line. For a list item we + # still want to absorb ITS wrapped continuations, so seed the buffer. + flush() + if is_list_item(ln): + buf.append(ln) + else: + out.append(ln.rstrip()) + continue + # Plain text: continuation of the current paragraph/list item, or a new + # paragraph if the buffer is empty. + buf.append(ln) + flush() + # Blank-line RUNS are preserved. The original collapsed them, which + # contradicted this module's own "preserves blank lines" contract; markdown + # renders one blank the same as three, so collapsing bought nothing and cost + # fidelity to the author's file. + return "\n".join(out).rstrip() + "\n" + + +if __name__ == "__main__": + sys.stdout.write(reflow(sys.stdin.read())) diff --git a/scripts/release-automation/test_assemble.py b/scripts/release-automation/test_assemble.py new file mode 100644 index 00000000..1f9c2e36 --- /dev/null +++ b/scripts/release-automation/test_assemble.py @@ -0,0 +1,60 @@ +#!/usr/bin/env python3 +"""Structure-validation tests for `assemble.py`. + +Needs `bs4` (which IS present here — the salvage README claimed otherwise and +was wrong). Run: + + python3 scripts/release-automation/test_assemble.py + +These cover only the input contract the script enforces, not the full PDF +transform: no original rendered fragment survived the salvage, so the transform +itself is still exercised only by using it for real. +""" + +import subprocess +import sys +import tempfile +from pathlib import Path + +SCRIPT = Path(__file__).resolve().parent / "assemble.py" +GOOD = "

T

x

NOTE (a): b

" +FAILURES = [] + + +def run(fragment, args=None): + with tempfile.TemporaryDirectory() as d: + frag = Path(d) / "frag.html" + frag.write_text(fragment, encoding="utf-8") + out = Path(d) / "out.html" + argv = [sys.executable, str(SCRIPT)] + (args if args is not None else [str(frag), str(out)]) + return subprocess.run(argv, capture_output=True, text=True) + + +def expect_fail(name, fragment, needle, args=None): + r = run(fragment, args) + if r.returncode == 0: + FAILURES.append(f"{name}: expected failure, got exit 0") + elif needle not in (r.stderr + r.stdout): + FAILURES.append(f"{name}: message missing {needle!r}\n got: {(r.stderr + r.stdout).strip()[:200]}") + + +expect_fail("no arguments", GOOD, "usage:", args=[]) +expect_fail("one argument", GOOD, "usage:", args=["only-one"]) +expect_fail("missing table", "

T

NOTE (a): b

", "no top-level ") +expect_fail("missing NOTE", "

T

x
", "no top-level

starting with 'NOTE'") +expect_fail( + "NOTE before table", + "

NOTE (a): b

x
", + "must come after the table", +) + +r = run(GOOD) +if r.returncode != 0: + FAILURES.append(f"well-formed fragment should succeed, got exit {r.returncode}\n {(r.stderr + r.stdout).strip()[:300]}") + +if FAILURES: + print(f"FAIL: {len(FAILURES)} case(s)\n") + for f in FAILURES: + print(f + "\n") + sys.exit(1) +print("assemble: all cases pass") diff --git a/scripts/release-automation/test_reflow.py b/scripts/release-automation/test_reflow.py new file mode 100644 index 00000000..12abe879 --- /dev/null +++ b/scripts/release-automation/test_reflow.py @@ -0,0 +1,127 @@ +#!/usr/bin/env python3 +"""Round-trip tests for `reflow.py`. + +Stdlib only, so this runs anywhere the tool does: + + python3 scripts/release-automation/test_reflow.py + +Every case here exists because the behaviour was wrong once. The hard-break and +fence cases came out of the PR #349 review; the rest pin the preservation +guarantees the module docstring claims, so the doc and the code cannot drift. +""" + +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from reflow import reflow # noqa: E402 + + +FAILURES = [] + + +def check(name, given, want): + got = reflow(given) + if got != want: + FAILURES.append( + f"{name}\n given: {given!r}\n want: {want!r}\n got: {got!r}" + ) + + +# --- the point of the tool ------------------------------------------------- + +check( + "wrapped paragraph joins to one line", + "alpha\nbeta\ngamma\n", + "alpha beta gamma\n", +) + +check( + "a list item absorbs its own continuations", + "- alpha\n beta\n- gamma\n", + "- alpha beta\n- gamma\n", +) + +# --- hard breaks (regression: `strip()` destroyed these) -------------------- + +check( + "a hard break ends the logical line and keeps its marker", + "first \nsecond\n", + "first \nsecond\n", +) + +check( + "text after a hard break still wraps into one line", + "first \nsecond\nthird\n", + "first \nsecond third\n", +) + +check( + "a trailing hard break inside a list item is kept", + "- alpha \n beta\n", + "- alpha \n beta\n", +) + +check( + "ONE trailing space is not a hard break and is dropped", + "first \nsecond\n", + "first second\n", +) + +# --- blank-line runs (regression: collapsed to a single blank) ------------- + +check( + "runs of blank lines are preserved", + "alpha\n\n\n\nbeta\n", + "alpha\n\n\n\nbeta\n", +) + +# --- fences (regression: any fence closed any block) ----------------------- + +check( + "a ``` line does not close a ````-opened block", + "````\n```\nstill code\n````\nafter one\nafter two\n", + "````\n```\nstill code\n````\nafter one after two\n", +) + +check( + "a tilde fence does not close a backtick fence", + "```\n~~~\nstill code\n```\nafter one\nafter two\n", + "```\n~~~\nstill code\n```\nafter one after two\n", +) + +check( + "code inside a fence is never rewrapped", + "```\nlet a = 1;\nlet b = 2;\n```\n", + "```\nlet a = 1;\nlet b = 2;\n```\n", +) + +check( + "a longer closing fence still closes", + "```\ncode\n`````\nafter\n", + "```\ncode\n`````\nafter\n", +) + +# --- the other preservation guarantees ------------------------------------- + +check("headings stand alone", "# Title\nbody\n", "# Title\nbody\n") +check("tables are untouched", "| a | b |\n| - | - |\n", "| a | b |\n| - | - |\n") +check("horizontal rules stand alone", "alpha\n\n---\n\nbeta\n", "alpha\n\n---\n\nbeta\n") +check( + "a blockquote joins into one wide line", + "> alpha\n> beta\n", + "> alpha beta\n", +) +check( + "a standalone HTML line is untouched", + '\nalpha\nbeta\n', + '\nalpha beta\n', +) + +if FAILURES: + print(f"FAIL: {len(FAILURES)} case(s)\n") + for f in FAILURES: + print(f) + print() + sys.exit(1) +print("reflow: all cases pass")