Repository navigation
Conversation
Deploying control-layer with
|
| Latest commit: |
97cfae9
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://10c8590e.control-layer.pages.dev |
| Branch Preview URL: | https://feat-dw-992-file-expiry-api.control-layer.pages.dev |
|
Running ultrareview automatically — this PR appears to change a public API contract, where a missed bug can break the clients and integrations that depend on it, so a deeper multi-pass review is worth running. I'll post findings when complete. |
There was a problem hiding this comment.
Ultrareview ran automatically — this PR appears to change a public API contract, where a missed bug can break the clients and integrations that depend on it, so a deeper multi-pass review is worth running.
All reported issues were addressed across 10 files
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
There was a problem hiding this comment.
0 issues found across 2 files (changes from recent commits).
Requires human review: Auto-approval blocked by 1 unresolved issue from previous reviews.
View guided diff | Turn on auto-fix | Re-trigger cubic
The API shows an expires_at on every input file, but nothing enforced it: content stayed downloadable past the date, output and error files never had one, and an upload could ask for any expires_after at all, recording a date that content retention would not honour. Batch creation then turned an expired input into a 500 from the repository's check. One setting now governs the three: batches.files.default_expiry_seconds (30 days, what uploads already received) is the lifetime given to an upload without expires_after and to a batch's results files from the batch's creation; max_expiry_seconds caps expires_after. Results files created before this read as created_at + default rather than being backfilled. Downloads past expires_at answer 410, batch creation refuses an input that would expire inside the completion window with 400, and an out-of-range expires_after is a 400 before any bytes are read. Groundwork for retiring content weeks on a single creation-anchored clock: once the API cannot show a date content will not reach, the retirement rules no longer need per-file state.
…e (DW-992)
/batches/{id}/results kept streaming bodies after the output and error
files had passed expires_at, so expiry was only enforced on
/files/{id}/content. It now reads both results files through the
repository, which supplies the date older results files are given, and
answers 410 once either has expired.
The request manager is built from the expiry keys on every instance and
refuses a misordered or non-positive expiry, but config validation of
those keys only ran with batches enabled. With batches disabled a bad
value panicked at startup; it is now a config error.
Drop the create_batch check that refused an input expiring inside the
completion window. OpenAI does not make it, and the repository already
refuses deleted, expired and unprocessed sources, now mapped to 400.
Tests build the app with a non-default default_expiry_seconds so they
show the setting reaches fusillade, rather than matching its fallback.
The configuration reference describes the 30-day default, the upload
range, the 400 and 410 responses, and that results files expire from
batch creation rather than completion.
File metadata has no status field, so it cannot report 'expired'; invalid_expiry is a metric label, not an API error code; and the default expiry is not the longest content lifetime, since uploads may request up to max_expiry_seconds.
0df3bcb to
97cfae9
Compare
Bottom layer of the DW-992 stack (single-clock content retention). Stands on its own; the retirement-rule layer follows.
Why
The API shows an
expires_aton every input file, but nothing enforced it:expiredas a side effect of being read;expires_atat all, and kept reportingprocessedafter their content had been dropped with a retired archive week;expires_after[seconds]on upload was uncapped (and silently ignored when it didn't parse), so a file could record a date content retention would never reach;Content retention is moving to one creation-anchored clock for prompts and completions. That only works if the API cannot show a date the content will not reach.
What
One setting governs all three files of a batch.
batches.files.default_expiry_seconds(previously an unused key defaulting to 24 h; now 30 days, which is what uploads already received) is the lifetime given to an upload withoutexpires_after, and to a batch's output and error files measured from the batch's creation.max_expiry_secondscaps what an upload may request. Both reach the repository through a newFileExpirysetting onPostgresRequestManager.expires_after[seconds]outside[min, max](1 h to 30 d by default) or unparseable → 400, and nothing is stored (counted indwctl_upload_rejections_totalasinvalid_expiry). Sent before thefilepart, it is refused before the file is read; sent after, the file is read first and the upload is then aborted, as withpurpose. Same range as OpenAI.expires_at. Ones created before this read ascreated_at + default, derived on read, with no backfill of existing rows.GET /files/{id}/contentpastexpires_at(or statusexpired) → 410, matching how deleted results files already answer.GET /batches/{id}/results→ 410 once the output or error file is pastexpires_at, alongside the existing 410 for a deleted results file. It reads both files through the repository, so older results files get the same derived date.POST /batches: an input that is notprocessed→ 400. The repository's ownValidationErroron a deleted, expired or unprocessed source now maps to 400 instead of 500. There is no separate check that the input outlives the completion window; OpenAI does not make one either.min/default/maxpositive and ordered) is validated whether or notbatches.enabledis set. The request manager is built from these keys on every instance, so a misordered config is now a config error instead of a panic at startup.config.yamlanddocs/src/reference/configuration.mddocument the three keys: the 30-day default, the upload range, the 400 and 410 responses, and when results files expire.Output and error files expire
default_expiry_secondsafter the batch is created. OpenAI counts from when the batch completes, so ours can expire up to one completion window earlier. The docs say so.Not changed:
files.statusis still only written on read (check_and_mark_expired); nothing deletes content here. That is the next layer.Tests
Ten new tests, all against Postgres:
maxis refused and leaves no row.default_expiry_secondsso the test shows the setting reaches fusillade; range rejection, including the bounds.expires_at. 410 from/batches/{id}/resultswhen either results file has a past date, and when the derived date has passed. Both paginated and streaming paths are covered.created_at + defaultunder a non-default config. An expired input gets 400. That test runs the repository'sValidationError→ 400 mapping.Full
just test rust: 4,296 passed.