feat: ship ingest review UI with settings, Docling preview, and chunk search

[Wallgau]

Summary

Add the ingest review dialog: Docling document view (or DocPage skeleton while loading), index pipeline steps, file-scoped searchable chunks, expand fill layout, and header search / auto-open footer.

Add Ingest preview settings (auto-open Every / First run / Never, chunk/pipeline toggles, sample ingest) gated by OSS/SaaS run mode, with prefs shared via useIngestPreviewSettings.

Reuse FileChunksPanel + controlled search on /knowledge/chunks and in review so paste search filters locally after a file-scoped load.
Wire preview=true uploads from knowledge; extend preview backend for Docling page images / index-proof and related tests.

Test plan

1. OPENRAG_INGEST_PREVIEW_ENABLED=true + OPENRAG_RUN_MODE=oss (or saas);
   --follow onboarding steps, upload a doc--
   and/or
2. upload a PDF and confirm review auto-opens per setting
3. Confirm Docling skeleton → annotated pages, index steps complete, chunk click highlights page
4. Paste chunk text into header search and confirm in-panel filter (not empty “No knowledge”)
5. Expand/collapse: body grows; title/toolbar/footer stay compact
6. Folder upload: file dropdown switches files; search resets on file change
7. Settings → Ingest preview: change auto-open / toggles; run sample ingest
8. /knowledge/chunks?filename=…: same chunk search UI and filter behavior
9. Auto-open Never / First run only / Every upload behave as expected after close + re-upload

Summary by CodeRabbit

• New Features
  • Added ingest preview/review dialog with Docling parsed previews (including page/image rendering when available) and indexing pipeline progress.
  • Added ingest-preview settings (new tab, auto-open policy, and “Run a sample ingest”).
  • Enhanced chunk browsing with file-scoped chunk loading and optional highlighting/search.
• Bug Fixes
  • Improved preview retrieval behavior for missing/expired data and disabled preview mode; safer polling completion/error handling.
• Tests
  • Added unit coverage for parse-preview endpoint, Docling preview caching, layout summary counts, and per-task cache limits.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

Walkthrough

Adds an ingest-preview flow spanning upload handling, Docling preview caching, polling APIs, settings, and a review dialog. It also adds Docling renderers, file-scoped chunk browsing, configurable search behavior, and backend/frontend tests.

Changes

Ingest preview backend

Layer / File(s)	Summary
Backend preview pipeline src/services/..., src/models/processors.py, src/api/ingest_preview.py, src/app/..., tests/unit/...	Preview-mode uploads pass through ingestion and Docling, completed documents are cached with TTL and per-task limits, and a new parse-preview endpoint retrieves cached results with API and service tests.

Preview data and settings

Layer / File(s)	Summary
Preview data and persisted settings frontend/app/api/queries/..., frontend/hooks/..., frontend/lib/..., frontend/lib/upload-utils.ts, frontend/package.json, frontend/types/...	Adds polling hooks, upload preview response handling, file-scoped chunk queries, page-reference helpers, runtime gating, persisted preview settings, and the Docling component declaration.

Chunk and Docling presentation

Layer / File(s)	Summary
Chunk and Docling presentation frontend/components/docling-preview.tsx, frontend/components/file-chunks-panel.tsx, frontend/components/knowledge-search-input.tsx, frontend/app/knowledge/chunks/page.tsx	Adds page-image and structured Docling renderers, reusable chunk browsing and filtering, configurable search input behavior, and file-scoped chunk page rendering.

Review dialog and upload integration

Layer / File(s)	Summary
Review dialog and upload integration frontend/components/ingest-review.tsx, frontend/components/knowledge-dropdown.tsx, frontend/components/ingest-preview-auto-open-control.tsx, frontend/app/settings/..., frontend/app/onboarding/...	Adds the ingestion review dialog, connects single-file, folder, and onboarding uploads to preview task IDs, adds auto-open controls and settings UI, and gates the settings tab by run mode.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant KnowledgeDropdown
  participant UploadAPI
  participant LangflowFileService
  participant IngestReviewDialog
  participant PreviewQueries
  User->>KnowledgeDropdown: upload file or folder
  KnowledgeDropdown->>UploadAPI: submit with preview enabled
  UploadAPI-->>KnowledgeDropdown: task IDs and preview mode
  KnowledgeDropdown->>IngestReviewDialog: open with task IDs and files
  IngestReviewDialog->>PreviewQueries: poll Docling and index-proof endpoints
  PreviewQueries-->>IngestReviewDialog: document, pipeline status, and chunks
  IngestReviewDialog-->>User: render preview and chunk highlights

Loading

Possibly related PRs

• langflow-ai/openrag#1572: Extends the two-phase Docling polling flow used by preview caching.
• langflow-ai/openrag#1695: Changes the processor-to-Langflow ingestion path used by preview metadata forwarding.
• langflow-ai/openrag#2051: Provides the index-proof preview endpoint consumed by the frontend review dialog.

Suggested reviewers: lucaseduoli, mfortman11, mpawlow

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 41.11% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: ingest review UI with settings, Docling preview, and chunk search.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch visual-parser-frontend

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

React Doctor found no new issues. 🎉

_{Reviewed by React Doctor for commit 212653c.}

[coderabbitai]

Actionable comments posted: 11

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

frontend/app/settings/[tab]/page.tsx (1)

18-25: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Apply ingest-preview run-mode gating at the route boundary.

The navigation hides this tab outside OSS/SaaS, but VALID_TABS accepts /settings/ingest-preview unconditionally, so direct navigation still renders the settings and sample-upload UI. Apply the same isIngestPreviewEnabled(runMode) check server-side and redirect unsupported modes.

Also applies to: 142-154

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@frontend/app/settings/`[tab]/page.tsx around lines 18 - 25, Update the
settings route validation around VALID_TABS to gate “ingest-preview” with
isIngestPreviewEnabled(runMode). When the tab is requested in an unsupported run
mode, redirect before rendering the settings or sample-upload UI; preserve
existing behavior for all other tabs and supported modes.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@frontend/components/docling-preview.tsx`:
- Around line 41-51: Update the useEffect around loadDoclingComponents to catch
dynamic-import failures, clear the cached doclingComponentsLoaded promise so
later viewers can retry, and set state for rendering an error instead of leaving
the preview blank. Preserve cancellation handling so unmounted components are
not updated, and add the corresponding error-state rendering in the preview
component.

In `@frontend/components/file-chunks-panel.tsx`:
- Around line 178-188: Update handleCopy to await navigator.clipboard.writeText
before setting copiedIndex, and only update the success state and timer after
the write resolves successfully. Stop removing newlines, carriage returns, and
tabs; preserve the original text or only apply non-destructive trimming as
appropriate.
- Around line 146-149: Update the allChunks mapping in the file-scoped chunks
flow to preserve each chunk’s API-provided index instead of overwriting it with
the array position. Keep the existing chunk data and fallback to an empty array
unchanged.

In `@frontend/components/ingest-review.tsx`:
- Around line 873-877: Update the useEffect that calls markIngestPreviewSeen so
it only consumes first-run auto-open after preview mode is confirmed, using the
available confirmed task ID or successful preview response state. Keep the
dialog’s open behavior unchanged, and do not mark the preview seen when the
upload fails or previewMode is false.
- Around line 520-555: Update the file status rendering in the files map to call
isFileEntryFailed(file.entry) separately from isPreviewReady(file.entry).
Display a distinct “failed” status and appropriate styling for failed entries;
keep “ready” for preview-ready files and “processing” only for entries that are
neither ready nor failed.
- Line 819: Update the awaitingChunks prop passed to FileChunksPanel so chunk
display is enabled whenever chunks exist, including during embedding and
OpenSearch storage; use chunkCount rather than requiring ready, while still
handling failed or unavailable states appropriately.

In `@frontend/components/knowledge-dropdown.tsx`:
- Around line 465-480: Update the folder upload flow around uploadFiles and
result.previewMode so a false previewMode closes the pre-opened preview dialog
and clears its loading state instead of leaving it without task IDs. Preserve
the existing task ID assignment and refetchTasks behavior when previewMode is
enabled, matching the single-file path’s rejection handling.
- Around line 461-497: Update the folder upload flow around the batches.map
Promise.all block to process batches with a bounded concurrency limit instead of
starting every upload simultaneously. Track failed batch indices or errors while
preserving successful results, then propagate an aggregate failure after
processing completes so callers do not treat partially failed uploads as fully
successful; retain the existing tracking, logging, and toast behavior for each
failure.

In `@frontend/components/knowledge-search-input.tsx`:
- Around line 76-86: Update showClear in the knowledge search input to derive
visibility from inputValue in both controlled and uncontrolled modes, so the
clear button reflects the current typed value before submission. Keep
handleClear’s existing mode-specific clearing behavior unchanged.

In `@frontend/hooks/use-ingest-preview-settings.ts`:
- Around line 50-54: Update the localStorage parsing logic in the ingest preview
settings hook to validate the result of JSON.parse before accessing properties
such as parsed.autoOpen. Treat null and all non-object values as invalid and
return DEFAULT_INGEST_PREVIEW_SETTINGS, while preserving the existing property
extraction for valid objects.

In `@src/services/ingest_preview_service.py`:
- Around line 89-134: Update the preview cache used by store_docling_preview to
automatically evict expired entries and enforce a bounded maximum size, using a
TTL cache or an equivalent periodic eviction mechanism. Replace the global
self._entries scan used to calculate existing with an approach that avoids O(N)
insertion cost while preserving the per-task preview cap and existing retrieval
behavior.

---

Outside diff comments:
In `@frontend/app/settings/`[tab]/page.tsx:
- Around line 18-25: Update the settings route validation around VALID_TABS to
gate “ingest-preview” with isIngestPreviewEnabled(runMode). When the tab is
requested in an unsupported run mode, redirect before rendering the settings or
sample-upload UI; preserve existing behavior for all other tabs and supported
modes.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 678c9e1a-3619-4f9c-90c0-226e1359bad0

📥 Commits

Reviewing files that changed from the base of the PR and between c6d2e92 and a2ad306.

⛔ Files ignored due to path filters (1)

• frontend/package-lock.json is excluded by !**/package-lock.json

📒 Files selected for processing (28)

• frontend/app/api/queries/useFileScopedChunksQuery.ts
• frontend/app/api/queries/useIngestPreviewQuery.ts
• frontend/app/knowledge/chunks/page.tsx
• frontend/app/settings/[tab]/page.tsx
• frontend/app/settings/_components/ingest-preview-settings-section.tsx
• frontend/app/settings/_components/settings-nav.tsx
• frontend/components/docling-preview.tsx
• frontend/components/file-chunks-panel.tsx
• frontend/components/ingest-preview-auto-open-control.tsx
• frontend/components/ingest-review.tsx
• frontend/components/knowledge-dropdown.tsx
• frontend/components/knowledge-search-input.tsx
• frontend/hooks/use-ingest-preview-settings.ts
• frontend/lib/file-chunks.ts
• frontend/lib/ingest-preview.ts
• frontend/lib/upload-utils.ts
• frontend/package.json
• frontend/types/docling-components.d.ts
• src/api/ingest_preview.py
• src/app/container.py
• src/app/routes/internal.py
• src/models/processors.py
• src/services/docling_service.py
• src/services/ingest_preview_service.py
• src/services/langflow_file_service.py
• src/services/task_service.py
• tests/unit/api/test_ingest_preview.py
• tests/unit/services/test_ingest_preview_service.py

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@frontend/components/docling-preview.tsx`:
- Around line 351-382: Reset fallbackIndex to zero before each of the texts,
tables, and pictures fallback extraction loops in the relevant document
extraction logic. Keep each generated id indexed from zero within its own
collection, while preserving the existing limit checks and result accumulation.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e6ba4821-a5bc-4de4-8070-25edb9a3a6dc

📥 Commits

Reviewing files that changed from the base of the PR and between a2ad306 and 43b5e45.

📒 Files selected for processing (3)

• frontend/components/docling-preview.tsx
• frontend/components/ingest-review.tsx
• frontend/lib/ingest-preview.ts

🚧 Files skipped from review as they are similar to previous changes (2)

• frontend/lib/ingest-preview.ts
• frontend/components/ingest-review.tsx