Phase 167: /batch-export — one zip for a screened list - #211
Merged
Conversation
The third piece of batch screening: the combined export. GET /batch-export runs the same loop as /batch-stream to completion (free inside the 15-minute replay window, which is the case when the table has just been shown) and returns opencheck-batch-<stamp>.zip with bundle.json (every row's BODS statements, de-duplicated by statementId — two subsidiaries of one group share the parent's entity statement and it appears once), rows.csv (the same columns as the page's CSV button, failed rows included with their reason), manifest.json (totals, the failed list, contributing sources, the licence verdict) and LICENSES.md computed over the union of sources, so one CC-BY-NC source in one row makes the whole bundle non-commercial and the notes say so. - opencheck/batch.py: iter_batch yields the LookupResponse beside each row (run_batch drops it, unchanged for its callers); collect_batch runs to completion in paste order and merges; BatchResult; rows_csv with CSV_COLUMNS mirroring frontend/src/lib/batch.ts. - routers/batch.py: build_batch_zip reuses export._build_licenses_md and licensing.assess; the route is on the heavy tier, behind the same bot gate (message names /batch-export), robots-disallowed, and sets X-OpenCheck-Batch-Failed so a script can notice a partial batch without opening the archive. - Frontend: fetchBatchExport (server's own words on 4xx, filename from Content-Disposition) and a "Download BODS bundle (zip)" button on /batch; /api page + docs/mcp-server.md updated. - 6 tests: merge + dedupe + a failed row contributes nothing, rows.csv shape, union licence = strictest member, the route's zip + header, the gate/422/robots, the tier. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016TF8feptbqf5yz3KoeDFQq
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016TF8feptbqf5yz3KoeDFQq
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third and last piece of batch / portfolio screening (Notion ticket, Phase C).
GET /batch-export?leis=: runs the Phase 164 loop to completion (free inside the replay window) and returnsopencheck-batch-<date>.zip—bundle.json(every row's BODS statements, de-duplicated bystatementId),rows.csv(same 16 columns as the page's CSV, failed rows marked not checked),manifest.json,LICENSES.mdover the union of sources (strictest member wins — pinned by a test).X-OpenCheck-Batch-Failedheader. Heavy tier, bot gate, robots disallow.opencheck/batch.py:iter_batchhands theLookupResponseback beside each row;collect_batch+BatchResult+rows_csv.run_batchand the MCP tool are unchanged./batchpage: Download BODS bundle (zip) beside the CSV button,fetch-based so a 429 is worded./apipage +docs/mcp-server.mdupdated.🤖 Generated with Claude Code
https://claude.ai/code/session_016TF8feptbqf5yz3KoeDFQq