feat(hygiene): audit next_up against GitHub state and milestones - #490
Open
evemcgivern wants to merge 1 commit into
Open
evemcgivern wants to merge 1 commit into
evemcgivern wants to merge 1 commit into
Conversation
A next_up list is hand-curated and nothing keeps it honest, so it rots in ways
that are invisible from inside the file — the list still reads like a plausible
priority order. Found on a real track whose queue HEAD was four issues, all
closed weeks earlier.
Adds `milestone-drift`, report-only, and runs it as hygiene step 4 of 5:
- RANKED BUT CLOSED — finished work still occupying the queue.
- MILESTONE INVERSION — an issue ranked above another while carrying a LATER
milestone, so the ranking and the release plan disagree. This is the check
worth having: it derives from next_up order plus milestones alone, with no
dependency on how a track writes its tiers (prose comments, headers, or
nothing — a generic checker cannot key off them and does not need to).
- RANKED WITH NO MILESTONE.
`--unranked` adds open track issues absent from next_up. Off by default on
measured evidence: a real 33-track repo produced ~1000 findings with it and 30
without, because next_up is a deliberate shortlist for most tracks. Default-on
would have buried the signal the command exists to surface.
Report-only always, never a rewrite: either side of an inversion can be the
wrong one — a high-ranked issue with a late milestone might mean the rank is
too high OR the milestone is too late. That is a judgement call.
Milestone order comes from due_on, falling back to title. When a repo's
milestones cannot be read, the inversion check is SKIPPED rather than guessed —
milestone_rank returns {} and callers treat that as "cannot compare", matching
the fail-soft contract fetch_open_issues already uses.
Closes #489
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E2t8b5kFLpvbqxsff1tbtY
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
milestone-drift— a report-only audit of each track'snext_upqueue against live GitHub — and wires it intohygieneas step 4 of 5.Why
A
next_uplist is hand-curated and nothing keeps it honest. It rots in ways that are invisible from inside the file, because the list still reads like a plausible priority order. Found while re-ranking a real track whose queue HEAD was four issues, all closed weeks earlier — the top of the priority list pointed entirely at finished work.What it reports
--unrankedadditionally lists open track issues absent fromnext_up.The inversion check is the one worth building for: it derives from
next_uporder plus each issue's milestone alone, with no dependency on how a track writes its tiers. Tracks express tiers as prose comments, section headers, or not at all, so a generic checker cannot key off them — and does not need to.Two decisions worth reviewing
--unrankedis off by default, on measured evidence. First real run against a 33-track repo produced 1043 findings; with the unranked check gated it produced 30.next_upis a deliberate shortlist for most tracks, so reporting every unranked issue buries the signal the command exists to surface. The 30 were all genuine — real closed issues sitting in real queues.Report-only always, not just inside hygiene. Either side of an inversion can be the wrong one: a high-ranked issue with a late milestone might mean the rank is too high, or the milestone is too late. Auto-"fixing" would guess. Same reasoning as
dedupe-tiersrefusing to delete during hygiene.Fail-soft on unknown milestones.
milestone_rankreturns{}when milestones can't be read, and callers treat that as "cannot compare" — the inversion check is skipped, not passed. This matches the contractfetch_open_issuesalready establishes (None means "we don't know", distinct from "confirmed empty"). An inversion report built on a guessed milestone order would be worse than no report.Evidence
python3 -m unittest discover teststests.test_milestone_drift--repo=CritForgehygiene --repo=CritForge./install.sh+--helpThe test suite includes the falsification cases, not just the positive ones: correctly-ordered milestones must stay silent (
test_correct_order_has_no_inversion,test_ascending_milestones_stay_silent_across_a_long_queue), equal milestones are not an inversion, a closed issue's milestone never triggers one, and an unresolvable issue number is skipped rather than guessed at. The inversion logic was in fact backwards on first write — it tracked the earliest milestone seen instead of the latest — and these tests are what caught it.Notes
gh, every test offline (per CLAUDE.md).fetch_milestones/milestone_rankinlib/github_state.py._ranked_orderdeliberately does not useresolve_next_up_order— that reads thenext_up_ordermapping and returns sort-criteria names, not issue numbers. Using it here would have silently produced an empty audit; there's a regression test pinning that.Closes #489