Skip to content

fix(frontmatter): warn on comment loss, and lift rationale into the body - #492

Open
evemcgivern wants to merge 1 commit into
devfrom
fix/491-rationale-body
Open

evemcgivern wants to merge 1 commit into
devfrom
fix/491-rationale-body

Conversation

@evemcgivern

Copy link
Copy Markdown
Contributor

Frontmatter comments cannot survive a write. This makes the loss loud, and gives rationale a home that survives.

The bug

lib/frontmatter.py round-trips frontmatter through JSON:

meta = _yaml_to_dict(match.group(1))   # yq -o=json .
yaml_text = _dict_to_yaml(meta)        # yq -P .  <- json.dumps(meta)

JSON has no comment concept, so every writer erases every frontmatter comment — refresh-md, reconcile, slot, hygiene, anything reaching write_file. Structurally, not occasionally.

A routine hygiene --yes run deleted 213 lines of ranking rationale from a real track. The next_up order survived perfectly — all 45 entries, correct sequence — which is exactly what made it invisible: a sanity check on the ordering passes, and the loss only appears in a full diff. The data survived; the reasoning did not.

Two changes

1. write_file warns, naming the count. count_frontmatter_comments is fail-soft: an unreadable or frontmatter-less file has nothing to lose, and a warning path must never be able to break a write.

2. lift-rationale moves rationale into a ## Ranking rationale body section. The body is passed through write_file verbatim, so it survives — and there's a test pinning that premise rather than assuming it.

Why separate instead of preserving comments

Preserving them via targeted yq -i writes was the obvious alternative, and it was rejected for a reason beyond effort: frontmatter comments never render as markdown, never reach the VS Code viewer, and never appear in export --json. Rationale kept there was already invisible to every surface anyone actually reads. The fragility just made that visible.

The part most worth reviewing

Association is by indentation, not adjacency.

# TIER 1 — the golden path is broken     <- indent 2, header
- 6374   # first line, inline            <- entry, indent 2
         # continues here                <- indent 9, belongs to 6374
- 6368   # ...

Adjacency cannot distinguish a trailing comment from a section header — both sit between two entries. My first implementation used adjacency and glued TIER 2 banners onto the preceding issue; test_header_is_not_glued_to_the_previous_entry is that bug, pinned.

The inline comment on the entry line also has to be captured: the real format puts the first and most important line of rationale there, so a naive - (\d+)\s*(?:#.*)?$ silently drops the summary of every entry. test_inline_comment_on_the_entry_line_is_captured covers it.

Tests are written against the real file's shape, not an invented one — TestRealFormat is transcribed from the track this was found on.

Evidence

Claim Result
python3 -m unittest discover tests 1474 tests, OK
tests.test_lift_rationale 25 tests, OK
Dry run, --repo=CritForge 170 comment lines / 72 blocks on one track, 11 / 5 on another — correctly grouped
Clean track produces no warning test_write_is_silent_when_there_is_nothing_to_lose
Body survives a write verbatim test_body_survives_a_write_verbatim (incl. # and : in the prose)

The falsification cases matter as much as the positive ones here: a warning that fires on every routine write becomes noise and gets ignored, so a comment-free track must stay silent.

Notes

  • Pure stdlib, offline tests, per CLAUDE.md.
  • README updated: new reference row plus a warning on the weekly-cadence line.
  • Does not change how frontmatter is serialized — that stays a JSON round-trip. This makes the consequence visible and gives rationale somewhere durable, rather than pretending YAML comments are safe.

Refs #491

Frontmatter is read and written by round-tripping through JSON
(_yaml_to_dict -> _dict_to_yaml). JSON has no comment concept, so every writer
— refresh-md, reconcile, slot, hygiene — erases every YAML comment in a track's
frontmatter. Structurally, not occasionally, and silently.

A routine `hygiene --yes` run deleted 213 lines of ranking rationale from a real
track. The next_up ORDER survived perfectly, which is what made it invisible: a
sanity check on the ordering passes and the loss only shows in a full diff.

Two changes, neither of which tries to make YAML comments durable:

1. write_file now WARNS, naming the count, when a write would drop comments.
   count_frontmatter_comments is fail-soft — an unreadable or frontmatter-less
   file has nothing to lose, and a warning path must never break a write.

2. New `lift-rationale` migrates rationale into a `## Ranking rationale` body
   section. The body is passed through write_file verbatim, so it survives;
   there is a test pinning that premise rather than assuming it.

Association is by INDENTATION, not adjacency: a comment indented deeper than
the most recent `- <number>` entry belongs to that entry, otherwise it is
free-standing. Adjacency cannot tell a trailing comment from a section header,
since both sit between two entries — the first implementation got this wrong
and glued TIER banners onto unrelated issues. An inline comment on the entry
line itself starts that entry's block, because the real format puts the first
and most important line of rationale there.

Tests are written against the REAL file's shape (inline first line + indented
continuations), not an invented one, including the falsification cases: a clean
track must produce NO warning, and a header must not bind to the preceding
entry.

Separating rather than preserving is the better fix for a second reason:
frontmatter comments never render as markdown, never reach the VS Code viewer,
and never appear in export --json. Rationale kept there was already invisible
to every surface anyone reads.

Refs #491

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E2t8b5kFLpvbqxsff1tbtY

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant