fix: compute run-level extra scores when merging result files

[abhinav-pola]

TL;DR

mergeResultFilesToParquet can now fill extra_scores in the merged file by running the benchmark's runLevelScores over every merged sample. Before this change it always wrote null.

What changed?

• MergeResultFilesInput has a new optional runLevelScores?: (result: RunResult) => readonly ExtraScore[]. Callers pass benchmark.runLevelScores.
• When runLevelScores is set, the first merge pass keeps each sample's answer, explanation, scorer_trajectory and metadata, which are the fields the existing run-level scorers read. It then calls the scorer once on a RunResult built from all files, and every merged row gets that JSON in extra_scores.
• If runLevelScores is not set, the merge works as before and keeps only the id, epoch and score of each sample.
• A malformed scorer_trajectory or metadata column fails the merge. It is not dropped.

Why?

The per-chunk extra_scores in openrouter-web's Temporal chunk parquets are computed from one chunk's samples only, so they can't be copied into the merged file or averaged. openrouter-web #47097 makes the merged file the only results path for multi-chunk runs, and without this change those runs lose their run-level scores.

How to test

bun test src/results/parquet.test.ts -t "recomputes run-level extra scores" merges three single-sample files with chunk-only extra scores. It checks that the scorer sees every sample's answer, explanation, trajectory and metadata, and that each merged row holds the recomputed score rather than the chunk values.

Reviewer focus

• Memory. With runLevelScores set, the first pass keeps explanation, scorer trajectory and metadata for every sample, but still not messages, request bodies or ATIF trajectories. For judge-based benchmarks this is small. For a verifier_log trajectory it grows with the log size.

Checklist

• Tests cover changed behavior
• Public API or configuration changes are backward compatible, or the break is documented
• No credentials, private results, or restricted dataset contents are included

Link to Devin session: https://openrouter.devinenterprise.com/sessions/cff9171c8dd8432ebb515e2a54374727
Open in Devin Desktop: https://openrouter.devinenterprise.com/desktop/session/cff9171c8dd8432ebb515e2a54374727?variant=devin
Requested by: @abhinav-pola

---

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access that start with 'DevinAI' or '@devin'.

• Disable automatic comment, CI, and merge conflict monitoring

---

Original prompt from Abhinav

SYSTEM:
<latest_message>
Abhinav Pola (U090K0G7JF3) [ts=1790356866.277609]: @Devin read this thread <https://openrouter.slack.com/archives/C0BMHG5CG1E/p1789516956636439>
</latest_message>

=== BEGIN THREAD HISTORY (in #agents-benchmarks) ===
Abhinav Pola (U090K0G7JF3) [ts=1790356866.277609]: @Devin read this thread <https://openrouter.slack.com/archives/C0BMHG5CG1E/p1789516956636439>
=== END THREAD HISTORY ===
Channel ID: C0BAKP8P5C3
Thread URL: https://openrouter.slack.com/archives/C0BAKP8P5C3/p1790356866277609?thread_ts=1790356866.277609&amp;cid=C0BAKP8P5C3

The <latest_message> is the message that you should use to guide your goals + task for this session, and you should use the rest of the slack thread as context.
A [ts=...] marker on a Slack message is that message's timestamp. To act on a specific message with the slack tool (e.g. adding an emoji reaction via the reaction command), pass that value as timestamp along with the Channel ID — no extra lookup call is needed.

[devin-ai-integration]

Devin Review found 1 potential issue.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)