[FEAT]: Serialize final evaluation metadata - #147
Closed
Spencer Schoenberg (spencrr) wants to merge 7 commits into
Closed
Spencer Schoenberg (spencrr) wants to merge 7 commits into
Spencer Schoenberg (spencrr) wants to merge 7 commits into
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
3 tasks done
Spencer Schoenberg (spencrr)
force-pushed
the
dev/spencrr/trace-serialization
branch
from
August 8, 2026 02:31
6f48cd2 to
a7fda0f
Compare
Spencer Schoenberg (spencrr)
force-pushed
the
dev/spencrr/trace-serialization
branch
from
August 24, 2026 23:15
a7fda0f to
42fded4
Compare
Spencer Schoenberg (spencrr)
force-pushed
the
dev/spencrr/trace-serialization
branch
from
August 27, 2026 17:30
42fded4 to
60f68cb
Compare
3 tasks done
Spencer Schoenberg (spencrr)
deleted the
dev/spencrr/trace-serialization
branch
September 8, 2026 20:42
Spencer Schoenberg (spencrr)
added a commit
that referenced
this pull request
Sep 30, 2026
## Description Adds the per-execution provenance needed before final-trace verdict cadence changes. `Result.terminal_evaluation` stores the evaluator output for the terminal trace, `Result.trace_end_reason` records why trace production ended, and `Turn.eval_purpose` identifies online stop checks. `Result.turn_evaluations` makes the online evidence boundary explicit while `eval_results` remains a compatibility view. The direct attack and probe resolvers require one evaluation and reject unknown runtime outcomes instead of falling through. Population provenance now shares validation across `PopulationRef`, `PopulationResult`, and `execute_trials_async`, with invalid thresholds rejected before an execution factory runs. This PR also includes the persistence work previously split into #147 so the contract cannot land without transport support. The current xdist v2 envelope and JSON report carry terminal evaluation, trace-end reason, turn purpose, and population provenance together. Malformed worker data fails closed, including overflowing confidence values, and oversized results produce bounded incomplete markers while retaining population provenance when it fits. ## Breaking changes None for valid callers. Invalid population provenance and malformed evaluator outcomes now fail early instead of being accepted or falling through. ## Checklist - [x] `pre-commit run --all-files` passes - [x] Tests cover contract fields, compatibility views, resolvers, population validation, xdist transport, truncation, JSON output, malformed confidence, and real worker/controller persistence - [x] Documentation updated Validation: 1,122 unit tests pass. Strict documentation build and all pre-commit checks pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Superseded by #146. The persistence changes now live with the result contract so the public model and full-fidelity transport land atomically.