[Bortlesboat]

Fixes #44.

Run ids are fixed labels, so a task that is re-run (a partial task on resume, for example) writes its new rollout under the same run id. _merge in cli/judge.py treated "same run id" as "same rollout", so when the next judge call failed it kept the grade of the rollout that had been replaced, and the report showed a score for a deliverable that no longer exists.

This stamps each selection record with selected_at when _run_one_mode writes it, carries the stamp on Scored, and makes _merge keep the old graded row only when the run id and the stamp both match. A transient judge failure on an unchanged task still keeps its score. A re-run task whose new judge call fails is left ungraded. Older judged.json rows and summaries have no stamp, so they compare equal and behave exactly as before.

I went with a stamp on the selection record rather than hashing the run dir at judge time because the LAB adapter writes scores.json and judge.log into the run dir when it grades, so a content hash would change after the first successful judge.

Tested with the repro script from #44 (judge 2 now prints None/None graded=False) and python3 -m unittest tests.test_workflow on Python 3.12, which adds a regression test covering both the transient-failure and re-run cases.

[jacob-adm]

@Bortlesboat DTWD

I reviewed this because my #36 makes tasks re-run on resume when their selection or experience prompts change, and each of those re-runs could hit the stale-grade problem this PR fixes. The approach looks right to me:

• selected_at is written only when a mode actually runs. A task that resumes keeps its record and its stamp, so a failed re-judge still keeps the old grade.
• Any task that re-runs gets a new stamp, whether from a partial resume, a changed harness or library, or a prompt change under fix(schedule): re-run tasks on resume when prompts or library change #36. A failed judge then leaves it ungraded instead of reporting a score for a rollout that has been replaced.
• Old judged.json rows and selection records without the field both default to "", so existing runs behave as before.

Agreed on not hashing the run dir, since LAB writes into it while grading.

Receipts, offline, calling _merge directly on this PR at 157ba5e:

_merge(old graded row, new failed re-judge):
legacy row + legacy selection, re-judge fails -> kept graded=True score=3/3
same stamp, re-judge fails -> kept graded=True score=3/3
task re-ran (new stamp), re-judge fails -> kept graded=False score=None/None
legacy row, task re-ran after upgrade -> kept graded=False score=None/None

tests.test_workflow -k failed_rejudge Ran 1 test ... OK

Your test already covers the resume-vs-re-run stamps. The first and last rows above cover upgrading: existing scores survive, and a task re-run after the upgrade correctly drops its stale score.

For the maintainers: this conflicts with #32, #36, #37 and #6 only in tests/test_workflow.py, where each PR adds its own tests and both sides can be kept. Whichever merges second needs a quick rebase.

EDIT: #32, #36 and #37 now merge cleanly with this PR; their tests moved into separate files.