Repository navigation
[Bug]: management-triggered catalog convergence reports reason:"disk" with no disk failure and discards the original error #1784
Description
Client or integration
Direct HTTP/API client
Area
Catalog / models
Summary
Catalog convergence triggered from the management API fails deterministically and reports reason: "disk" when there is no disk problem. The underlying exception is discarded, so the real cause cannot be recovered from outside — and, as far as I can tell, from inside either.
PUT /api/subagent-models returns 200 ok with:
{"status":"failed","reason":"disk","phase":"gather","retryable":false,"partialWrite":false}There is no disk failure. The Codex catalog is 871 KB, parses, holds 37 entries, and the volume has 47 GB free. partialWrite:false says nothing was written, and ocx ensure writes the same catalog successfully seconds before and after.
Three things compound here, and I think the third is the actual bug:
-
reason: "disk"is a hard-coded label, not an observation.unexpectedCatalogFailure()insrc/codex/management-convergence.tsreturnsreason: "disk"unconditionally, and the wrapper insrc/server/management-api.tsdoes the same in its owncatch. Nothing in either path checked a disk. -
retryable: falseis asserted, and is wrong at least sometimes. The identical code path, driven out-of-process against the same config, returns{"status":"committed","changed":true,"degraded":true,"notices":["fallback","provider-auth"]}. An operation that succeeds when invoked differently is not non-retryable. -
Three unrelated causes collapse into one indistinguishable disposition, and the original error is destroyed. All of these produce byte-identical
{failed, disk, gather, false}:- the request-shape guard in
createManagementConvergeCodex(request.scope !== "catalog"), - the
admissionFailure()fallthrough, when a thrown message matches neither"config generation is busy"nor"config generation is database", - the
catchinmanagement-api.tsbeforeconvergenceInvokedis set.
None of the three logs anything.
service.logcontains zero lines about these failures.admissionFailure()classifying by substring match onerror.messagemeans any new or reworded throw upstream of it silently becomes a non-retryable disk failure. - the request-shape guard in
I could not determine which of the three fires here, or what the underlying exception is, despite reading the source and probing the path directly. That is the point of the report: with full source access and a deliberate effort, the diagnostic is unrecoverable. An operator sees "disk failure", checks a healthy disk, and has nowhere to go.
Suggested direction: preserve the original error (log it, or carry a cause/detail field on the disposition), and distinguish the three paths — a request-shape mismatch is a programming error, an admission refusal is contention, and a write failure is disk. Only the last one is "disk".
Reproduction
Verified on 2.20.0, deterministic, 3/3.
- Start the proxy normally:
ocx restart && ocx ensure— note thatensurereports the catalog written successfully. - Read the current subagent roster and write the same list back, so the request is a no-op in content:
node -e "const fs=require('fs'),h=require('http');const T=fs.readFileSync(process.env.USERPROFILE+'/.opencodex/admin-api-token','utf8').trim();const o={host:'127.0.0.1',port:10100,headers:{authorization:'Bearer '+T}};h.get({...o,path:'/api/subagent-models'},r=>{let d='';r.on('data',c=>d+=c);r.on('end',()=>{const b=JSON.stringify({models:JSON.parse(d).chosen});const q=h.request({...o,path:'/api/subagent-models',method:'PUT',headers:{...o.headers,'content-type':'application/json','content-length':Buffer.byteLength(b)}},x=>{let e='';x.on('data',c=>e+=c);x.on('end',()=>console.log(x.statusCode,e))});q.write(b);q.end()})})"- Observe
catalogRefresh: {"status":"failed","reason":"disk","phase":"gather","retryable":false,"partialWrite":false}. Repeating the call reproduces it every time.
The control that makes this a defect rather than an environment problem — the same path, out-of-process, succeeds:
const { loadConfig } = await import("<pkg>/src/config.ts");
const { createCatalogConvergeRequest } = await import("<pkg>/src/codex/catalog-admission.ts");
const { createManagementConvergeCodex } = await import("<pkg>/src/codex/management-convergence.ts");
const config = loadConfig();
const outcome = await createManagementConvergeCodex(config)(createCatalogConvergeRequest({ deadlineMs: 1_000 }));
console.log(outcome.kind, JSON.stringify(outcome.catalogRefresh));Output:
catalog-only {"status":"committed","changed":true,"degraded":true,"notices":["fallback","provider-auth"]}
Same machine, same config, same second. In-process it is failed/disk/gather 3/3; out-of-process it commits. Whatever differs is process state — which is exactly what the discarded exception would have named.
Expected: the response identifies the real cause, or at minimum the original error reaches the log so it can be identified.
Version
2.20.0
Operating system
Windows 11 Home Single Language 26200
Provider and model
Not provider-specific — the failure is in catalog convergence, before any provider is contacted.
Logs or error output
# three consecutive PUTs with identical bodies
tentativa 1: HTTP 200 catalogRefresh={"status":"failed","reason":"disk","phase":"gather","retryable":false,"partialWrite":false}
tentativa 2: HTTP 200 catalogRefresh={"status":"failed","reason":"disk","phase":"gather","retryable":false,"partialWrite":false}
tentativa 3: HTTP 200 catalogRefresh={"status":"failed","reason":"disk","phase":"gather","retryable":false,"partialWrite":false}
# the disk is fine
catalog: C:\Users\<user>\.codex\opencodex-catalog.json 871413 bytes, parses, 37 entries
volume: 465G total, 47G available
# service.log contains no line about any of these failures
$ grep -c "converge\|catalog.*fail" service.log
0Redacted configuration
{
"subagentModels": [
"PROVIDER-A/model-a",
"PROVIDER-B/model-b"
]
}Checks
- I searched existing issues and documentation.
- I removed secrets, tokens, account details, request credentials, and personal data.
Activity
coderabbitai commentedon Aug 15, 2026
🔗 Related PRs
#354 - fix(models): surface discovery failure status when /v1/models returns 401 (
#329) [open]#391 - feat: quota-aware subagent model fallback chain (
#374) [merged]#1212 - feat(codex): converge account picker catalogs [merged]
#1331 - fix(catalog): keep failover combos when live discovery omits members [merged]
#1460 - fix(catalog): skip byte-identical catalog writes so a resync cannot mark Codex stale [open]
📝 Issue Planner
Check the box below or use the
@coderabbitai plancommand to generate an implementation plan and prompts that you can use with your favorite coding assistant.🧪 Issue enrichment is currently in open beta.
To disable automatic issue enrichment, add the following to your
.coderabbit.yaml:💬 Have feedback or questions? Drop into our discord!
fix(codex): stop reporting every convergence failure as a disk failure
lidge-jun commentedon Aug 16, 2026
Fixed on
devby #1848 (merged as73440d4bb).reason: "disk"was manufactured for any unrecognized failure in two places — the management adapter's catch and the management API route — so a malformed request and a genuinely full filesystem were indistinguishable, and both were reported non-retryable.CatalogDispositiongainsrequest-invalid,admissionandinternalalongside the existing reasons, narrowingdiskto real IO.admissionis the one class marked retryable, since contention is the failure worth retrying unchanged.The new
causefield is deliberately not an error summary. Both fields are closed vocabularies:kindmaps onto a fixed set rather than echoingError.constructor.name(any thrown custom class names itself), andcodeis emitted only for a recognized errno token. AnError.messageroutinely carries paths, home directories and account identifiers, andredactSecretStringmasks token shapes but none of those.normalizeCatalogDisposition— the allowlist that rebuilds the management response — was updated in the same change. Without that the new reasons and cause would have been silently dropped and the fix would have looked implemented while changing nothing.Regressions: an escaping factory error is
internal, aTypeErrorisrequest-invalid, and an error whose message embeds both a home path and a token-shaped string produces a response containing neither.Closing as completed. The change is on
devand ships with the next release.fix(codex): stop reporting every convergence failure as a disk failure