fix(documents): preserve retry metadata until cleanup succeeds - #168
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a04670e51e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if document and document.processing_attempt_id: | ||
| raise DocumentDeletionConflict("Document is being processed; retry deletion later") |
There was a problem hiding this comment.
When a worker terminates after claiming a document, processing_attempt_id remains populated even though no processing is active. The stale-document recovery path in src/workers/recovery.py changes the document status and commits without clearing that field, so this unconditional check makes every subsequent delete return 409 forever; previously deletion was the user's escape hatch for such stuck documents. Clear stale ownership during recovery or only report a conflict when the processing lease is known to be live.
Useful? React with 👍 / 👎.
1 participant
Document and folder deletion can currently report success after external cleanup fails, permanently losing the metadata needed to retry. This change preserves the PostgreSQL row until graph, vector and all generation source objects are removed, propagates cleanup errors, and prevents deletion while ingestion owns the document.
Graph cleanup and community stale marking run in one Neo4j transaction. Folder deletion stops on a failed document instead of silently deleting the folder.
Validation: 12 focused unit tests and 2 checks against disposable local Neo4j 5 pass; full-source Ruff lint, changed-file mypy and all five import contracts pass. The full unit suite has 1,052 passes, 4 skips and 2 environment failures reproduced on unmodified main (HEIF decoder and local Starlette version). No production deployment or cleanup is included.