Skip to content

feat(hygiene): audit next_up against GitHub state and milestones - #490

Open
evemcgivern wants to merge 1 commit into
devfrom
feat/489-milestone-drift
Open

evemcgivern wants to merge 1 commit into
devfrom
feat/489-milestone-drift

Conversation

@evemcgivern

Copy link
Copy Markdown
Contributor

Adds milestone-drift — a report-only audit of each track's next_up queue against live GitHub — and wires it into hygiene as step 4 of 5.

Why

A next_up list is hand-curated and nothing keeps it honest. It rots in ways that are invisible from inside the file, because the list still reads like a plausible priority order. Found while re-ranking a real track whose queue HEAD was four issues, all closed weeks earlier — the top of the priority list pointed entirely at finished work.

What it reports

  • RANKED BUT CLOSED — finished work still occupying the queue.
  • MILESTONE INVERSION — an issue ranked above another while carrying a later milestone. The ranking and the release plan disagree and neither side knows it.
  • RANKED WITH NO MILESTONE.
  • --unranked additionally lists open track issues absent from next_up.

The inversion check is the one worth building for: it derives from next_up order plus each issue's milestone alone, with no dependency on how a track writes its tiers. Tracks express tiers as prose comments, section headers, or not at all, so a generic checker cannot key off them — and does not need to.

Two decisions worth reviewing

--unranked is off by default, on measured evidence. First real run against a 33-track repo produced 1043 findings; with the unranked check gated it produced 30. next_up is a deliberate shortlist for most tracks, so reporting every unranked issue buries the signal the command exists to surface. The 30 were all genuine — real closed issues sitting in real queues.

Report-only always, not just inside hygiene. Either side of an inversion can be the wrong one: a high-ranked issue with a late milestone might mean the rank is too high, or the milestone is too late. Auto-"fixing" would guess. Same reasoning as dedupe-tiers refusing to delete during hygiene.

Fail-soft on unknown milestones. milestone_rank returns {} when milestones can't be read, and callers treat that as "cannot compare" — the inversion check is skipped, not passed. This matches the contract fetch_open_issues already establishes (None means "we don't know", distinct from "confirmed empty"). An inversion report built on a guessed milestone order would be worse than no report.

Evidence

Check Result
python3 -m unittest discover tests 1469 tests, OK
tests.test_milestone_drift 20 tests, OK
Real run, --repo=CritForge 30 findings across 33 tracks
Real hygiene --repo=CritForge step 4 of 5 runs in 7.2s
./install.sh + --help command and flags render correctly

The test suite includes the falsification cases, not just the positive ones: correctly-ordered milestones must stay silent (test_correct_order_has_no_inversion, test_ascending_milestones_stay_silent_across_a_long_queue), equal milestones are not an inversion, a closed issue's milestone never triggers one, and an unresolvable issue number is skipped rather than guessed at. The inversion logic was in fact backwards on first write — it tracked the earliest milestone seen instead of the latest — and these tests are what caught it.

Notes

  • Pure stdlib, all GitHub state via gh, every test offline (per CLAUDE.md).
  • New helpers fetch_milestones / milestone_rank in lib/github_state.py.
  • README updated in all four places the convention requires: quick-start row, the weekly-cadence line, the hygiene reference row, and a new reference row.
  • _ranked_order deliberately does not use resolve_next_up_order — that reads the next_up_order mapping and returns sort-criteria names, not issue numbers. Using it here would have silently produced an empty audit; there's a regression test pinning that.

Closes #489

A next_up list is hand-curated and nothing keeps it honest, so it rots in ways
that are invisible from inside the file — the list still reads like a plausible
priority order. Found on a real track whose queue HEAD was four issues, all
closed weeks earlier.

Adds `milestone-drift`, report-only, and runs it as hygiene step 4 of 5:

- RANKED BUT CLOSED — finished work still occupying the queue.
- MILESTONE INVERSION — an issue ranked above another while carrying a LATER
  milestone, so the ranking and the release plan disagree. This is the check
  worth having: it derives from next_up order plus milestones alone, with no
  dependency on how a track writes its tiers (prose comments, headers, or
  nothing — a generic checker cannot key off them and does not need to).
- RANKED WITH NO MILESTONE.

`--unranked` adds open track issues absent from next_up. Off by default on
measured evidence: a real 33-track repo produced ~1000 findings with it and 30
without, because next_up is a deliberate shortlist for most tracks. Default-on
would have buried the signal the command exists to surface.

Report-only always, never a rewrite: either side of an inversion can be the
wrong one — a high-ranked issue with a late milestone might mean the rank is
too high OR the milestone is too late. That is a judgement call.

Milestone order comes from due_on, falling back to title. When a repo's
milestones cannot be read, the inversion check is SKIPPED rather than guessed —
milestone_rank returns {} and callers treat that as "cannot compare", matching
the fail-soft contract fetch_open_issues already uses.

Closes #489

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E2t8b5kFLpvbqxsff1tbtY

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant