fix(frontmatter): every write silently deletes all YAML comments (JSON round-trip) #491

Problem

Every command that writes a track's frontmatter silently deletes every YAML comment in it. Not occasionally — structurally.

lib/frontmatter.py round-trips frontmatter through JSON:

def read_file(...):
    meta = _yaml_to_dict(match.group(1))      # yq -o=json .   -> JSON

def write_file(path, meta, body):
    yaml_text = _dict_to_yaml(meta)           # yq -P . <- json.dumps(meta)
    p.write_text(f"---\n{yaml_text}---\n{body}")

lib/frontmatter.py:19,37,41-51

JSON has no comment concept, so comments cannot survive the trip. Any writer — refresh-md, reconcile, slot, batch-slot, handoff --set-next, anything calling write_file — drops them all. Silently: no warning, no diff summary, no confirmation naming the loss.

Observed

Ran hygiene --repo=CritForge --yes (steps 1–2 write). One track lost 213 lines, including all of its # TIER 1 — / # TIER 2 — / # TIER 3 — section headers and the per-issue rationale beneath each next_up entry.

The next_up order survived perfectly — all 45 entries, correct sequence — which is what makes this dangerous. A quick sanity check on the ordering passes, and the loss is only visible if you diff the whole file or go looking for prose that is no longer there. The data survives; the reasoning does not.

Measured exposure on one repo right now: 117 comment lines across 2 tracks, 110 of them in a single launch-critical track. That is the entire written justification for a priority ordering — why each issue sits where it does, which premises were investigated and refuted, which placements were a deliberate override.

This has bitten before: an earlier incident logged slot erasing 314 lines of rationale the same way.

Why it matters more than it looks

The next_up list is a bare sequence of issue numbers. Comments are the only place the why can live inline, and the toolkit's own conventions encourage recording it there (a re-rank is worthless six weeks later if nobody can reconstruct the reasoning). So the format invites putting irreplaceable prose in exactly the place a routine weekly command destroys.

Worse, the destroying command is hygiene — the one advertised as safe, routine maintenance you run without thinking.

Options

1. Preserve comments on write. yq can edit YAML in place (yq -i '.key = value') and retains comments on the paths it does not rewrite. Reshaping write_file into targeted yq -i assignments rather than a whole-document re-dump would keep untouched keys — and their comments — intact. Biggest change, correct fix.
2. Warn loudly and require confirmation when the frontmatter being overwritten contains comments, naming the count. Cheap, and turns silent loss into a decision.
3. At minimum, document it — in the README and in write_file's docstring — so nobody records rationale in frontmatter believing it is durable.

(1) is the real fix. (2) is a same-day mitigation that would have prevented today's loss.

Acceptance

• A track carrying frontmatter comments survives refresh-md, reconcile, and hygiene with those comments intact — or, if that is deferred, the user is warned and must confirm before they are dropped.
• A test writes frontmatter containing comments and asserts they are still present afterward.

Related

• feat(hygiene): audit next_up order against GitHub milestones and issue state #489 / feat(hygiene): audit next_up against GitHub state and milestones #490 — milestone-drift, found during the same session. Note the irony: that command exists to detect a next_up list drifting out of sync with reality, while hygiene was quietly deleting the explanation of why the list is ordered as it is.