Add governed institutional research workflow

.gitignore

@@ -4,6 +4,7 @@ dist/
 build/
 .omx/
 .longtable/team/
+.worktrees/
 .codex/
 .claude/
 .env

README.md

@@ -147,6 +147,20 @@ longtable search --query "<topic>"
 longtable panel --prompt "review this measurement plan" --json
 ```
 
+## Institutional Research
+
+`longtable research` is the single surface for reproducible institutional-database work. It provides pilot and protocol checkpoints, immutable academic-database exports, append-only screening decisions, external Research PDF Vault intake, audited reports, and APA 7 or journal-profile Word packages.
+
+```bash
+longtable research init --cwd "<project-path>"
+longtable research pilot --run-id pilot-1 --protocol-reference draft-1 --cwd "<project-path>"
+longtable research freeze --protocol-file protocol.json --cwd "<project-path>"
+longtable research ingest-export --file export.ris --database scopus --format ris --cwd "<project-path>"
+longtable research status --cwd "<project-path>"
+```
+
+MCP is the preferred checkpoint transport and numbered text is the recorded fallback. Authentication, MFA, CAPTCHA, terms decisions, and ambiguous methods remain researcher-owned. Bulk production is unavailable until an approved institution profile passes a 5-20 record live smoke gate. See [the institutional research workflow](docs/INSTITUTIONAL-RESEARCH-WORKFLOW.md) for artifact locations, resume rules, reporting-to-manuscript mapping, PDF boundaries, and all commands.
+
 Provider skill installation is explicit:
 
 ```bash

Spec.md

@@ -10,6 +10,10 @@ Repository framing and migration boundary are defined in `README.md`, `RELATIONS
 
 Active supporting docs are:
 
+- `docs/superpowers/specs/2026-08-06-longtable-research-institutional-workflow-design.md`
+- `docs/superpowers/plans/2026-08-06-longtable-research-institutional-workflow.md`
+- `docs/INSTITUTIONAL-RESEARCH-WORKFLOW.md`
+- `docs/INSTITUTIONAL-RESEARCH-ACCEPTANCE-AUDIT.md`
 - `docs/ARCHITECTURE.md`
 - `docs/ACTIVE-DECISIONS.md`
 - `docs/CHECKPOINTING.md`
@@ -94,6 +98,8 @@ Current code-level scaffolds now cover:
 - question runtime contracts now separate shared checkpoint judgment from Claude/Codex-specific presentation surfaces
 - panel orchestration is now scoped as provider-neutral `InvocationIntent` and `PanelPlan` semantics with provider-specific execution surfaces and sequential fallback
 - `@longtable/mcp` now exposes `longtable-state` as optional structured transport over `.longtable/` state, not as a replacement source of truth
+- `@longtable/scholar-research` now exposes one receipt-backed institutional workflow for protocol freezing, export intake, screening, Research PDF Vault manifests, audited reporting, and declarative Word render profiles
+- institutional production eligibility requires profile approval, deterministic replay, a researcher-approved 5-20 record live smoke run, and one or two permitted PDF intakes
 
 ## Core Domain Objects
 

docs/INSTITUTIONAL-RESEARCH-ACCEPTANCE-AUDIT.md

@@ -0,0 +1,30 @@
+# Institutional Research Acceptance Audit
+
+This matrix maps every criterion in the approved institutional-workflow design to a fresh automated assertion and a canonical artifact contract. Test names refer to scripts included in `npm test`; artifact paths are relative to a generated research project unless they point to a source contract.
+
+| Acceptance criterion | Automated assertion | Canonical artifact or contract |
+|---|---|---|
+| One LongTable Research surface | `e2e-institutional-review.mjs` asserts the exact command contract; CLI build and pilot smoke exercise `longtable research` | `packages/longtable-scholar-research/src/coordinator.ts`; CLI `research` route |
+| MCP preferred for required checkpoints | `smoke-institutional-checkpoints.mjs` checks every hard stop has `mcp_elicitation` then `numbered`; E2E records MCP decline and numbered fallback | `audit/checkpoints.jsonl`; checkpoint delivery record |
+| No hard-stop resume without DecisionRecord | `smoke-institutional-checkpoints.mjs` loops over every declared hard-stop code, rejects empty decisions, and resumes only with linked IDs | blocked `ResearchRun`, `QuestionRecord`, linked `DecisionRecord` |
+| Research variables are extracted independently and frozen before analysis | `smoke-data-extraction.mjs` tests profile hashing, required evidence, double extraction, conflict detection, adjudication, reliability, idempotent dataset hashing, and freeze rejection | `protocol/extraction-profiles/`, extraction ledgers, `audit/extracted-data-freezes.jsonl`, `data/08_analysis-ready/` |
+| Approved protocol revisions are immutable | `smoke-institutional-workflow.mjs` rejects different content at an existing revision; E2E freezes revisions 1 and 2 with different hashes | `protocol/amendments/protocol-revision-NNNN.json` |
+| Mechanical stages use zero model calls | `smoke-research-failures.mjs` loops over `ZERO_MODEL_CALL_STAGES` and rejects one observed call | model-budget record and deterministic stage receipt |
+| Interrupted runs resume safely without duplicate artifacts | E2E compares repeated resume results and replays export, screening, PDF intake, and Word rendering | `latestSafeCursor`, SHA raw export, screening ledger, PDF manifest, content hash |
+| Count invariants pass before rendering | `smoke-research-reporting.mjs` renders balanced counts and rejects a modified final-included count | `reports/review-flow-counts.md`, `reports/review-flow.svg` |
+| Material records, exclusions, acquisitions, and manuscript counts have provenance | corpus, screening, PDF, reporting, and manuscript smokes check source links, evidence IDs, SHA/access basis, artifact provenance, and every rendered Word object ID | `corpus/papers.jsonl`, screening ledger, PDF manifest, report provenance, manuscript provenance sidecar |
+| No credentials, sessions, secrets, or licensed PDFs enter state or Git | `smoke-research-failures.mjs` seeds six violation classes and verifies redacted findings; repository audit scans canonical research artifact roots | `scripts/audit-research-artifacts.mjs` report |
+| Research PDF Vault is the only canonical PDF store | `smoke-pdf-vault.mjs` rejects a vault inside the project and verifies one SHA-addressed copy with manifest-only project state | external Research PDF Vault; `corpus/pdf-manifest.jsonl` |
+| Final reports and Word artifacts derive from canonical state | reporting and manuscript smokes reject invariant, database-execution, full-text, and analysis-approval failures | `reports/`, `manuscript/*.docx`, provenance sidecars |
+| Same approved state rerenders identically | reporting smoke deep-compares all outputs; manuscript smoke and E2E compare content hashes | deterministic report artifacts and DOCX content hash |
+| New institutions require calibration, tests, live smoke, and approval | `smoke-institution-adapters.mjs` tests each missing gate and complete eligibility; E2E enforces 5-20 records, one or two PDFs, and four approval flags | institution calibration checklist and live-smoke `StageReceipt` |
+
+## Completion-specific cross-checks
+
+- Every declared hard-stop code traverses creation, MCP preference, numbered fallback, decision rejection, and linked resume in one parameterized test.
+- An eligible corpus is not called analysis-ready until its extraction profile, independent extraction records, reliability result, adjudications, and extracted-data freeze all pass.
+- All stages declared in `ZERO_MODEL_CALL_STAGES` reject any observed model call.
+- Every Word paragraph role generated by the renderer has a provenance object: title, author lines, abstract heading/body, keywords, every section heading/body, readiness table, references heading, and every reference.
+- Raw exports are content-addressed and immutable after intake; identical replay is idempotent.
+- PDF bytes remain outside the project and are referenced only by a verified manifest.
+- The included institution fixture remains non-production until login verification, resolver discovery, per-database calibration, fixture replay, limited live smoke, and the profile DecisionRecord are all present.

docs/INSTITUTIONAL-RESEARCH-WORKFLOW.md

@@ -0,0 +1,162 @@
+# Institutional Research Workflow
+
+This is the researcher-facing operating contract for the integrated `longtable research` surface. It covers reproducible academic-database retrieval, document screening, lawful full-text intake, reporting, and manuscript rendering. It does not automate authentication, decide an ambiguous inclusion rule, or convert a pilot into production without a recorded researcher decision.
+
+## Method boundary
+
+Use two related but distinct evidence streams for a demand-supply mismatch study:
+
+1. **Demand corpus:** procurement notices, commissioned-research topics, policy documents, and employer demand documents. Treat this as a document-analysis population with an explicit sampling frame, inclusion criteria, extraction matrix, pilot coding, agreement/adjudication, and missing-document audit. Do not call this corpus a PRISMA literature search.
+2. **Supply and scholarly corpus:** training-course metadata and scholarly records from selected academic databases. The academic search can use the workflow below and can produce review-flow artifacts when its count invariants pass.
+
+The two streams meet only after each has its own stable unit of analysis and codebook. A conceptual framework may then map demanded competencies and problem types to supplied curricula, learning protocols, and measured competency outcomes. This separation prevents procurement demand from being mistaken for scholarly evidence and prevents course availability from being mistaken for effectiveness.
+
+## One research surface
+
+Initialize the canonical folders before the pilot:
+
+```bash
+longtable research init --cwd "<project>"
+longtable research pilot --run-id <run-id> --protocol-reference draft-1 --cwd "<project>"
+```
+
+The pilot creates one compact search-strategy checkpoint. MCP elicitation is attempted first. Decline, timeout, unsupported transport, or transport error records the failed attempt and renders the same checkpoint as numbered text; it does not infer an answer.
+
+After the researcher approves a protocol revision:
+
+```bash
+longtable research freeze --protocol-file protocol-revision.json --cwd "<project>"
+longtable research ingest-export --file export.csv --database web-of-science --format csv --cwd "<project>"
+longtable research screen --decisions-file screening-decisions.json --cwd "<project>"
+longtable research acquire --candidate-path paper.pdf --vault-root "<external-vault>" --paper-id <id> \
+  --acquisition-method researcher_manual_upload --access-basis researcher_provided \
+  --version accepted-manuscript --project-inclusion included --screening-state fulltext_included \
+  --cwd "<project>"
+longtable research freeze-extraction --profile-file extraction-profile.json --cwd "<project>"
+longtable research extract --profile-file extraction-profile.json --records-file extraction-records.json --cwd "<project>"
+longtable research adjudicate-extraction --profile-file extraction-profile.json \
+  --adjudications-file adjudications.json --cwd "<project>"
+longtable research freeze-data --profile-file extraction-profile.json --freeze-file data-freeze.json --cwd "<project>"
+longtable research report --input-file report-input.json --cwd "<project>"
+longtable research package --input-file manuscript-input.json --profile-file apa7.json \
+  --output-path manuscript/review.docx --cwd "<project>"
+```
+
+`freeze` writes immutable, numbered revisions. The same revision and content is idempotent; different content cannot overwrite an existing revision number. `ingest-export` stores the original export under its SHA-256, then produces normalized and deduplicated derivatives. Re-ingesting identical bytes does not create a second raw artifact. `screen` appends versioned decisions and ignores an identical decision ID on replay.
+
+Metadata identifies research units; it is not the coded research dataset. `freeze-extraction` locks the unit of analysis, variables, value types, evidence requirements, double-extraction rule, and reliability threshold. `extract` validates independent records and detects field-level conflicts. `adjudicate-extraction` records a human resolution with evidence. `freeze-data` refuses unresolved conflict, incomplete double extraction, below-threshold reliability, or missing required evidence, then writes a deterministic analysis-ready dataset hash.
+
+## Mandatory checkpoints and resume rule
+
+| Event | Immediate action | Resume evidence |
+|---|---|---|
+| Login, MFA, CAPTCHA, or expired session | Stop at the last safe cursor | Researcher restores access and records a linked DecisionRecord |
+| Terms or access basis is unclear | Stop before retrieval | Researcher records the permitted route or excludes the route |
+| Query drift, result cap, export mismatch, or changed download pattern | Stop production | Researcher approves a protocol amendment or corrective action |
+| Ambiguous screening rule or human-AI disagreement | Preserve both decisions; do not collapse | Adjudication decision linked to the QuestionRecord |
+| Missing full-text threshold exceeded | Stop full-text screening | Revised acquisition plan or approved limitation |
+| Analysis method changes | Stop synthesis | Approved analysis-method DecisionRecord |
+| Corpus freeze is absent | Refuse analysis | Frozen corpus receipt |
+| Extraction profile is unfrozen or a coding rule is ambiguous | Stop extraction | Approved profile or rule revision linked to a DecisionRecord |
+| Pilot reliability is below threshold | Stop production extraction | Revised pilot/codebook or approved threshold decision |
+| Double extraction conflicts remain | Preserve both records | Evidence-backed adjudication |
+| Extracted-data freeze is absent | Refuse analysis and manuscript rendering | Deterministic dataset freeze linked to a DecisionRecord |
+
+Every stage advances only with a `StageReceipt`. A blocked run resumes only when the DecisionRecord references both the blocking checkpoint key and QuestionRecord ID. Replaying resume from the same blocked state with the same timestamp and decision produces the same state.
+
+## Canonical project artifacts
+
+```text
+<project>/
+  protocol/
+    research-question.yaml
+    search-protocol.yaml
+    inclusion-exclusion.yaml
+    database-profiles/
+    extraction-profiles/
+    amendments/protocol-revision-0001.json
+  data/
+    00_raw-exports/
+    01_normalized/
+    02_deduplicated/
+    03_title-abstract-screening/
+    04_fulltext-screening/
+    05_extraction-pilot/
+    06_extracted/
+    07_adjudicated/
+    08_analysis-ready/
+  corpus/
+    papers.jsonl
+    acquisition-ledger.jsonl
+    pdf-manifest.jsonl
+    screening-decisions.jsonl
+    extraction-records.jsonl
+    extraction-conflicts.jsonl
+    extraction-adjudications.jsonl
+    manual-action-queue.csv
+  audit/
+    stage-receipts.jsonl
+    checkpoints.jsonl
+    decisions.jsonl
+    exclusions.jsonl
+    deviations.jsonl
+    failures.jsonl
+    extracted-data-freezes.jsonl
+  reports/
+  manuscript/
+  analysis/
+  references/
+```
+
+The Research PDF Vault must be outside the project. The project retains only a Vault-relative manifest locator, SHA-256, access basis, acquisition method, version, and screening state; the approved institution storage profile resolves the external Vault root at runtime. Placeholder, partial, corrupt, symbolic-link, absolute-path, or token-bearing PDF intake is rejected.
+
+## Final researcher report and paper mapping
+
+The report command generates a researcher audit report, count table, deterministic review-flow SVG, systematic-review Methods/Results text, and provenance records.
+
+| Research artifact | Direct manuscript use |
+|---|---|
+| Frozen protocol and exact database queries | Methods: information sources, search strategy, eligibility criteria |
+| Database yields and raw-export hashes | Methods/Results: search dates, yields, export reconciliation |
+| Screening ledger and exclusion reasons | Methods: selection process; Results: study selection and exclusions |
+| PDF manifest and missing-text audit | Methods: retrieval routes; Results/Limitations: reports not retrieved |
+| Count-invariant report and review-flow SVG | Results table and study-flow figure |
+| Deviation and DecisionRecord ledger | Protocol amendments and deviations |
+| Analysis-readiness table | Analysis plan, missing statistics, dependency and author-contact needs |
+| Extraction profile, reliability, conflict/adjudication, and data-freeze ledger | Methods: data items and coding; Results: reliability, missingness, and final analytic dataset |
+| Paragraph/table provenance map | Internal audit before submission |
+
+Planned work renders future-tense protocol language. Completed work renders past-tense Methods and observed Results. Rendering fails when count equations do not balance or when database execution, full-text verification, extracted-data freeze, or analysis approval is absent.
+
+The review-flow output is an internal deterministic representation, not a claim that every document corpus is a systematic review. Final reporting should be checked against the [PRISMA 2020 checklist and flow materials](https://www.prisma-statement.org/prisma-2020) and, for literature-search reporting, the [PRISMA-S extension](https://doi.org/10.1186/s13643-020-01542-z).
+
+## Word profiles
+
+The included `apa7-memory` profile fixes a working-manuscript layout: US Letter, one-inch margins, Times New Roman 12 pt, double spacing, top-right page numbers, title page, 0.5-inch first-line indents, and 0.5-inch hanging reference indents. It is a declared working profile, not a substitute for a named journal's current author instructions.
+
+The Korean profile is explicitly `provisional_until_journal_rules_verified`. A named target-journal profile should be approved only after recording its aims and scope, author instructions, article type, recent reference-paper pattern, table/figure conventions, and template hash. User templates are accepted only with a matching SHA-256.
+
+## Live institution smoke gate
+
+Production eligibility requires a researcher-owned login, an approved institution profile, deterministic recipe replay, and a live smoke run:
+
+```bash
+longtable research live-smoke --run-id <id> --record-count 5 --pdf-count 1 \
+  --researcher-approved --profile-approved --calibrated --replay-passed --cwd "<project>"
+```
+
+The gate accepts 5-20 metadata records and one or two permitted PDFs. Anything larger is not a smoke run. The receipt records the counts and eligibility decision; it never stores credentials, cookies, session tokens, MFA material, or authentication headers.
+
+## Researcher closeout
+
+Before manuscript submission, the researcher receives and reviews:
+
+1. frozen protocol revision and amendments;
+2. database-by-database yield and export reconciliation;
+3. unresolved issues, recovered failures, exclusions, and deviations;
+4. corpus and PDF-manifest invariants;
+5. analysis-readiness and required-action list;
+6. manuscript profile status and unresolved citation fields;
+7. artifact provenance map and security-audit result.
+
+This closeout is the final checkpoint. The automation prepares the evidence package; the researcher remains responsible for methodological interpretation, journal fit, and submission.

docs/LONGTABLE-COMMAND-SURFACE.md

@@ -180,6 +180,24 @@ Tmux is not a LongTable core requirement. If an OMX-style Codex popup transport
 is added, it must be documented as optional, attached-tmux-only, and backed by
 the standard fallback path.
 
+## LongTable Research Surface
+
+`longtable research` is the single public command surface for the integrated
+institutional-database workflow. It owns project initialization, readiness,
+protocol checkpoints, corpus processing, Research PDF Vault references,
+screening, reports, and manuscript packaging behind one capability tree.
+
+```bash
+longtable research doctor --json
+longtable research init --cwd "<review-project>" --json
+longtable research status --cwd "<review-project>" --json
+```
+
+`longtable scholar-research` is retained only as a deprecated compatibility
+alias. Required decisions use MCP elicitation first and numbered fallback
+second. A fallback reuses the same pending QuestionRecord by checkpoint key;
+it never creates an implicit answer.
+
 ## Supporting Commands
 
 ```bash