fix(frontmatter): warn on comment loss, and lift rationale into the body - #492
Open
evemcgivern wants to merge 1 commit into
Open
evemcgivern wants to merge 1 commit into
evemcgivern wants to merge 1 commit into
Conversation
Frontmatter is read and written by round-tripping through JSON (_yaml_to_dict -> _dict_to_yaml). JSON has no comment concept, so every writer — refresh-md, reconcile, slot, hygiene — erases every YAML comment in a track's frontmatter. Structurally, not occasionally, and silently. A routine `hygiene --yes` run deleted 213 lines of ranking rationale from a real track. The next_up ORDER survived perfectly, which is what made it invisible: a sanity check on the ordering passes and the loss only shows in a full diff. Two changes, neither of which tries to make YAML comments durable: 1. write_file now WARNS, naming the count, when a write would drop comments. count_frontmatter_comments is fail-soft — an unreadable or frontmatter-less file has nothing to lose, and a warning path must never break a write. 2. New `lift-rationale` migrates rationale into a `## Ranking rationale` body section. The body is passed through write_file verbatim, so it survives; there is a test pinning that premise rather than assuming it. Association is by INDENTATION, not adjacency: a comment indented deeper than the most recent `- <number>` entry belongs to that entry, otherwise it is free-standing. Adjacency cannot tell a trailing comment from a section header, since both sit between two entries — the first implementation got this wrong and glued TIER banners onto unrelated issues. An inline comment on the entry line itself starts that entry's block, because the real format puts the first and most important line of rationale there. Tests are written against the REAL file's shape (inline first line + indented continuations), not an invented one, including the falsification cases: a clean track must produce NO warning, and a header must not bind to the preceding entry. Separating rather than preserving is the better fix for a second reason: frontmatter comments never render as markdown, never reach the VS Code viewer, and never appear in export --json. Rationale kept there was already invisible to every surface anyone reads. Refs #491 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E2t8b5kFLpvbqxsff1tbtY
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Frontmatter comments cannot survive a write. This makes the loss loud, and gives rationale a home that survives.
The bug
lib/frontmatter.pyround-trips frontmatter through JSON:JSON has no comment concept, so every writer erases every frontmatter comment —
refresh-md,reconcile,slot,hygiene, anything reachingwrite_file. Structurally, not occasionally.A routine
hygiene --yesrun deleted 213 lines of ranking rationale from a real track. Thenext_uporder survived perfectly — all 45 entries, correct sequence — which is exactly what made it invisible: a sanity check on the ordering passes, and the loss only appears in a full diff. The data survived; the reasoning did not.Two changes
1.
write_filewarns, naming the count.count_frontmatter_commentsis fail-soft: an unreadable or frontmatter-less file has nothing to lose, and a warning path must never be able to break a write.2.
lift-rationalemoves rationale into a## Ranking rationalebody section. The body is passed throughwrite_fileverbatim, so it survives — and there's a test pinning that premise rather than assuming it.Why separate instead of preserving comments
Preserving them via targeted
yq -iwrites was the obvious alternative, and it was rejected for a reason beyond effort: frontmatter comments never render as markdown, never reach the VS Code viewer, and never appear inexport --json. Rationale kept there was already invisible to every surface anyone actually reads. The fragility just made that visible.The part most worth reviewing
Association is by indentation, not adjacency.
Adjacency cannot distinguish a trailing comment from a section header — both sit between two entries. My first implementation used adjacency and glued
TIER 2banners onto the preceding issue;test_header_is_not_glued_to_the_previous_entryis that bug, pinned.The inline comment on the entry line also has to be captured: the real format puts the first and most important line of rationale there, so a naive
- (\d+)\s*(?:#.*)?$silently drops the summary of every entry.test_inline_comment_on_the_entry_line_is_capturedcovers it.Tests are written against the real file's shape, not an invented one —
TestRealFormatis transcribed from the track this was found on.Evidence
python3 -m unittest discover teststests.test_lift_rationale--repo=CritForgetest_write_is_silent_when_there_is_nothing_to_losetest_body_survives_a_write_verbatim(incl.#and:in the prose)The falsification cases matter as much as the positive ones here: a warning that fires on every routine write becomes noise and gets ignored, so a comment-free track must stay silent.
Notes
Refs #491