Repository navigation
rlf-codex-rl：5.8 gate 判定器 + Modal 沙箱修复 + 训练 flag + outcome 磁带 + 5.2/5.8 通过证据（原 #231–#236 恢复合并） - #237
Merged
Merged
Conversation
…vidence + lessons
Gate attempt 5: a CTF VM was reclaimed mid-episode by Modal's 300s idle kill while the model was in a long generation turn (no exec running), the next exec raised NotFoundError, the episode aborted without trusted finish and the fail-closed preflight discarded the group. Default idle_timeout now follows ttl_s (hard cap still bounds leaks); overridable via the new RlfModalVmConfig.idle_timeout_s field. Co-authored-by: Cursor <cursoragent@cursor.com>
…e-surviving gate evidence
- --rl-training-attention-backend {auto,flash,fused,unfused}: recipe default
overridable (flash-only recipes refuse); gate run uses unfused after the
L40S flash-attn cute JIT crd2idx compile failure (failure class
trainer_kernel_compile, attention fwd, LoRA-independent)
- rlf_rl_gate: policy-update proof from the island event tape rl_publication
payload hashes (代拍板 v2) — survives island teardown via --modal-tape-volume;
adapter-file path kept; policy_update_source recorded in evidence
- CPU tests: 4 new backend-override + 5 new tape-collection tests
Co-authored-by: Cursor <cursoragent@cursor.com>
gate attempt 2 (rlf-gate-20261011-4) passed the attention-backend override (AttnBackend.unfused on the island) but died on a session-server import storm: Miles defaults --session-server-workers to 32, and 32 parallel sglang imports on the 4-core L40S island missed the 300 s ready window (same failure class as compaction G3 run c). Yeti had no flag for the worker count; plumb it cli -> launcher -> island -> AgentConfig -> Miles argv, launch-side >=1 validation, translation allowlist. The gate run drops to 8 workers (rollout is 4 groups x 4 samples). Silver lining: --modal-tape-volume + completed-groups in ~/yeto-output worked — the island tape was pulled despite the failed run. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Run rlf-gate-20261011-7 SUCCEEDED (2 rounds trained, reward 0.125) but the gate could not judge: on the ports engine the island checkpoint only ever holds an empty completed-group queue, so per-sample reward/success died with the island. queue_completed_groups now also emits one compact rl_sample_outcome event per sample (fail-closed: skips non-numeric reward, success=None is rejected by the judge), and the gate collects samples from --tape-events when --completed-groups is absent. 3 new tests. Co-authored-by: Cursor <cursoragent@cursor.com>
rlf-codex-rl 5.8 attempt 8 lesson: the ports launcher binds --rollout-all-samples-process-path to rollout_meta_hook.extract_rollout_metadata, so the legacy queue_completed_groups hook (054c20d5) never runs there and the tape carried zero rl_sample_outcome events. The meta hook already reports per-sample task_id/reward/success which the driver tapes as rl_trajectory_reward behind --rl-observe-timeline; teach the gate collector to accept both event shapes (sample_index fills index), same fail-closed rules. Co-authored-by: Cursor <cursoragent@cursor.com>
…011-9) Five criteria on the ports engine tape (32 rl_trajectory_reward records, 2 rounds x 16 samples): reward_sum=6.0, reward_std=0.390, success_count=6, advantage_abs_sum=7.0, policy_update payload hashes differ (v0/v1), collector=miles_island. Decision log: -6/-7/-8 root causes (cuDNN fused refusal, ports empty completed-group queue, all-samples hook occupied by extract_rollout_metadata). Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
栈式合并事故恢复：#233/#235 被误合进各自 base 分支而非 main，导致 #231/#232/#234/#236 随分支删除被关闭。本 PR 以 cherry-pick 方式恢复全部 10 个提交（b27c4ff4..64632b70），内容与原 6 个 PR 并集完全一致：
openspec change: rlf-codex-rl（5.2/5.8 已通过，本 PR 含验收证据）。
测试：67 passed（test_rlf_rl_gate / test_rl_training_attention_backend / test_rlf_env_lease / test_rlf_baseline / test_rlf_bridge）。
注：仓库 main 的 python CI 自 10-10 起红（transformers 缺 qwen3_5_moe，test_rl_fn_boot_only），与本 PR 无关。