Skip to content

fix(benchmark): pin UTF-8 for the native process start identity read - #7745

Merged
huangruiteng merged 1 commit into
loopx-project:mainfrom
xiaoan17:codex/pin-utf8-in-native-codex-process
Nov 1, 2026
Merged

huangruiteng merged 1 commit into
loopx-project:mainfrom
xiaoan17:codex/pin-utf8-in-native-codex-process

Conversation

@xiaoan17

Copy link
Copy Markdown
Contributor

Goal and source

main is red on the shared test-shard lane for a reason that has nothing to
do with the runtime path: the repository's own text-I/O guard fails
deterministically.

FAILED tests/test_loopx_text_io_utf8.py::test_shipped_runtime_pins_utf8_for_every_text_file_read_and_write
E       AssertionError: assert ['loopx/capab...rocess.py:71'] == []
E         Left contains one more item: 'loopx/capabilities/benchmark_toolkit/native_codex_process.py:71'

The guard is the acceptance for the #4941 locale bug: every text-mode read or
write in the shipped runtime has to pin UTF-8 explicitly, because an implicit
locale codec reopens that failure. The offending line arrived with the native
process evidence work (#7707, merged as 04c5799b8):

stat = Path(f"/proc/{process.pid}/stat").read_text()

Path.read_text() without a codec uses the process locale, so this read is
exactly the shape the guard exists to reject. The same shard lane reports it
on every branch, including main, which is why the repository's shard runs
are red for everyone at the moment.

The change

One line, pinning the codec the repository already uses everywhere else:

stat = Path(f"/proc/{process.pid}/stat").read_text(encoding="utf-8")

The failure semantics do not change: a process name that is not valid UTF-8
raises UnicodeDecodeError, which is a ValueError, and the existing
except (OSError, ValueError, IndexError) already swallows it and leaves the
start identity unknown — the same outcome the implicit locale codec produced
for a non-UTF-8 locale.

Validation

uv run --extra test python -m pytest tests/test_loopx_text_io_utf8.py -q
    2 passed          (1 failed on the base revision, at the same line)

The guard is the acceptance; no new test is needed, and no other file changes.

Boundary

This changes one call in one file and no behaviour, budget, protocol or state.
It is a repair of the acceptance that main currently fails, so every
branch's shard lane stops carrying that failure. It does not touch the two
other failures observed in the same shard runs
(test_dashboard_launcher_selects_* and
test_native_restart_uses_real_accepted_work_fence_before_app_quit), which
are environment and native-host failures in their own lanes.

The shipped text-I/O guard fails on main for this line, and the shard lane
reports it on every branch:

    tests/test_loopx_text_io_utf8.py::test_shipped_runtime_pins_utf8_for_every_text_file_read_and_write
    E   Left contains one more item: 'loopx/capabilities/benchmark_toolkit/native_codex_process.py:71'

Path.read_text() without a codec uses the process locale, which is exactly the
shape the guard exists to reject. Pin the codec the runtime already uses
everywhere else.

The failure semantics do not change: a process name that is not valid UTF-8
raises UnicodeDecodeError, which is a ValueError, and the existing
except (OSError, ValueError, IndexError) already leaves the start identity
unknown.

Signed-off-by: xiaoan17 <anbc1224@gmail.com>
@xiaoan17

Copy link
Copy Markdown
Contributor Author

The one red check is a main-side flake, not this change

chat-bundle-browser (1) failed on the delegation-search smoke, and
checks failed with it:

page.waitForFunction: Timeout 30000ms exceeded.
    at rows (apps/presentation/dashboard/smoke/delegation-search-browser-smoke.mjs:56:32)

That is not this branch's code. The same smoke fails the same way on main's
own run (41146897061, job 124374451899), and it passes locally on this
branch's base:

node smoke/delegation-search-browser-smoke.mjs
PASS: packaged search, pre-pagination match, count, original conversation,
clear/paging, failure/retry, returned history, stale response, Escape/focus,
workspace isolation, narrow layout, English, source refusal/recovery,
changed-snapshot More and read-only receipts.

Other branches' runs of the same lane succeed, so the smoke is intermittent in
CI — a 30 s wait for the delegation rows on a runner that is carrying the
shared shard lane at the same time.

The change in this branch is a single codec argument on a /proc/<pid>/stat
read inside the native process observation, and it is not on the delegation
search path at all. The lane is left to the maintainers' re-run; no content
change is made for a flake.

@songoow songoow left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent; GPT-6; OpenAI; self_reported.

动机 / Motivation

受影响的是运行原生Codex进程观察和仓库共享测试分片的开发者。新进程起始身份读取漏掉显式codec,导致UTF-8边界守卫在主干稳定失败;本项补齐这一处读取。同一现有守卫在固定base为1失败1通过,在精确head为2通过,失败位置与修改行完全对应。本项不修复其他浏览器或native host失败,不更改进程生命周期及退出判定。

这次判断绑定PR #7745的完整head e7692c9。评价对象是已经存在的实际调用任务与局部可观察结果;重复调用带来的成本或错误恢复路线是本项动机,不能把较小响应或更少导入直接等同于长期任务质量。

改动思路 / Approach

在现有 /proc stat read_text调用显式传入UTF-8;读不到或无法解码时仍由已有异常分支保留未知身份。本 PR 的合理边界是一处文本读取的编码约束修复;无需新增测试框架或改写进程观察所有者。

沿入口跟踪到原所有者、返回值和失败恢复后,这个实现仍在现有边界上处理问题。无改动会保留原成本或错误指引;另起一套业务规则、持久化状态或通用缓存会增加同步义务,此处没有这类必要性。新增说明和投影应服务原调用方完成任务,不能改变原来的准入、权限、结算或来源完整性要求。

具体改动 / Changes

关键代码讲解

  • started:读取kernel进程开始tick时固定编码,避免依赖进程locale。 输入与拒绝分支沿用现有调用路径;这一处的返回内容或依赖时机改变,不把局部投影提升为新的业务判定。
  • _process_status:沿用原poll结果说明进程仍在运行或已退出,不把未知起始身份伪造为父时间戳。 输入与拒绝分支沿用现有调用路径;这一处的返回内容或依赖时机改变,不把局部投影提升为新的业务判定。

文本编码从locale默认改为显式UTF-8,这是有意约束;标准UTF-8主机读取结果不变,解码失败仍归于已有未知身份路径。正向场景是:既有静态守卫针对所有shipped文本读写检查,head完全通过。负向场景是:不存在 /proc、进程提前退出或无效UTF-8继续落入原OSError/ValueError/IndexError fallback,身份保持未知。这些是依据实际源码与已有调用测试核对的行为边界;失败和恢复继续由原owner负责,没有以返回字段存在代替真实状态读取。

规范依据为原公开调用行为、既有所有者和相应回归;已检索改动路径与周边文档,没有为这次局部性能/适配修正找到另一个已接受专项RFC。作者的量测叙述不充当独立功能合同。

实际验证 / Validation

固定base04c5799b守卫1 FAIL/1 PASS,精确head2 PASS;Python编译与 diff hygiene PASS。

执行命令:python -m pytest -q tests/test_loopx_text_io_utf8.py。同一既有UTF-8守卫在固定base04c5799b88f6c4cf12a224b7fa679f272e1e794e为1失败1通过,失败精确落在此行;head变为2通过。这是本项修复的独立对照,不能顺带归因其他分片。

对主干的风险 / Risks

文本编码从locale默认改为显式UTF-8,这是有意约束;标准UTF-8主机读取结果不变,解码失败仍归于已有未知身份路径。最需要防止的情况是:不存在 /proc、进程提前退出或无效UTF-8继续落入原OSError/ValueError/IndexError fallback,身份保持未知。已核对相关调用方和已有review,未重复发布旧head意见;本轮对新增head的源代码与局部回归单独作判断。原持久化状态和业务所有者保持,仍然不能由小范围通过推断整仓或所有部署入口已经合格。

当前精确head失败证据:test-shard (3)、test-shard (6)、chat-bundle-browser (1)。本项虽修复一类确定的main失败,当前head另有test-shard和delegation-search失败;这些当前失败尚不能全部由本次guard对照归因。 未取得的分片日志明确留作未知;同名错误或改动文件不相交都不足以建立pre_existing归因。

最小补齐是针对当前实际失败取得日志,并提供不可变base/head同命令同条件归因,或能独立覆盖该失败不变量的当前真实入口证据。已完成的聚焦回归不应无条件重做,也不要求无关功能重写;如果归因确认是回归,再修相应最小代码路径。未完成的检查不是单凭pending就要求改代码的理由。

我的整体评价 / Verdict

REQUEST_CHANGES:当前阻断是具体的验证证据缺口。聚焦审查尚未发现可复现的阻断性生产代码缺陷,不能把未归因红灯写成代码finding。 审查仅适用于exact head e7692c9。

本 PR 的合理边界是一处文本读取的编码约束修复;无需新增测试框架或改写进程观察所有者。 同一现有守卫在固定base为1失败1通过,在精确head为2通过,失败位置与修改行完全对应。 已检查 loopx/capabilities/benchmark_toolkit/native_codex_process.py:started;loopx/capabilities/benchmark_toolkit/native_codex_process.py:_process_status;变更是CLI/内部读取或文本协议,没有修改浏览器页面布局,视觉截图不适用。

本项虽修复一类确定的main失败,当前head另有test-shard和delegation-search失败;这些当前失败尚不能全部由本次guard对照归因。 本项不修复其他浏览器或native host失败,不更改进程生命周期及退出判定。所有本轮探针使用合成fixture,没有运行真实模型任务或发起业务作业;本review陈述审查与验证事实,不声称已经发布部署或完成合并。

English verdict: REQUEST_CHANGES — exact head e7692c9. Focused code review and the listed local regressions found no confirmed blocking implementation defect. Approval is held for the named current validation gaps; resolve their source-bound evidence rather than changing unrelated code.

@huangruiteng
huangruiteng merged commit 0a44fe1 into loopx-project:main Nov 1, 2026
32 of 42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants