Skip to content

rlf-codex-rl 5.8:gate 磁带收集器同时接受 rl_trajectory_reward 事件(ports 引擎关键修复) - #235

Merged
michaellchung merged 1 commit into
rlf-pr5-tape-outcomesfrom
rlf-pr6-tape-reward-records
Oct 11, 2026
Merged

michaellchung merged 1 commit into
rlf-pr5-tape-outcomesfrom
rlf-pr6-tape-reward-records

Conversation

@michaellchung

Copy link
Copy Markdown
Contributor

范围与动机

-9 gate 跑的关键修复。-8 跑发现:ports launcher 把 --rollout-all-samples-process-path 绑定到 rollout_meta_hook.extract_rollout_metadata,导致 #234 中 legacy queue_completed_groups 的 _emit_sample_outcomes hook 在 ports 下从不执行,磁带上零个 rl_sample_outcome 事件,gate 拿不到样本级证据。

而 meta hook 本身已上报 per-sample 的 task_id/reward/success,driver 侧在 --rl-observe-timeline 下会记为 rl_trajectory_reward 磁带事件。本 PR 教 gate 收集器同时接受两种事件形态(rl_trajectory_reward 的 sample_index 填充 index),fail-closed 规则不变(缺数值 reward 跳过、磁带空洞拒绝)。

对应 openspec change:rlf-codex-rl(任务 5.8,attempt 8 教训)。

测试证据

tests/test_rlf_rl_gate.py: 24 passed in 1.73s

新增 2 个用例:rl_trajectory_reward 事件被收集(sample_index 填位)、与 rl_sample_outcome 混收判定。

依赖关系

Made with Cursor

rlf-codex-rl 5.8 attempt 8 lesson: the ports launcher binds
--rollout-all-samples-process-path to rollout_meta_hook.extract_rollout_metadata,
so the legacy queue_completed_groups hook (054c20d5) never runs there and the
tape carried zero rl_sample_outcome events. The meta hook already reports
per-sample task_id/reward/success which the driver tapes as
rl_trajectory_reward behind --rl-observe-timeline; teach the gate collector
to accept both event shapes (sample_index fills index), same fail-closed rules.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants