Skip to content

Minimal workspace UI: one canvas over an ingest run - #142

Draft
kstonekuan wants to merge 1 commit into
mainfrom
ui-canvas
Draft

Minimal workspace UI: one canvas over an ingest run#142
kstonekuan wants to merge 1 commit into
mainfrom
ui-canvas

Conversation

@kstonekuan

Copy link
Copy Markdown
Contributor

A second attempt at the workspace UI, built to test one question: is a single drill-down canvas more useful than the multi-page app in #87?

It looks like a workflow builder because that is what people already know how to read: a left-to-right node graph, a breadcrumb, an inspector on the right, and one thing to learn — a node with a > opens.

The shape

Two branches leave the run, because a run has two things worth following:

run  ->  stage  ->  steps          the orchestration, and the code inside it
run  ->  episodes  ->  episode     the data it produced
level what it draws
run the ingest DAG as a chain: resolve the profile, then each stage
stage one stage's sub-DAG: plan, the process_batch fan-out, the budget gate
steps what one batch does per episode: the pipeline's checks and enrichments, the tier barrier, the quarantine gate, the catalog append
episodes the episodes whose current catalog row came out of this run
episode every recorded check with its verdict, its gate, and the measurement it was judged on

The last two are the drill-down that episodes.orchestrator_run_id (#140) unblocked.

Where the decisions live

src/canvas/buildGraph.ts is the whole thing: focus + server payloads -> nodes and edges, pure, and tested without a browser. Its rule is an edge means a real dependency:

  • The run level merges each stage's enabled_<stage> and trigger_<stage> into one node and drops the shortcuts the stage chain already implies. That turns a nine-rank ribbon — which only fit on screen at a zoom nobody could read — into a five-node line. Both task ids, both states and both durations stay on the node, so the summary hides nothing.
  • The stage level expands the mapped process_batch into one node per batch while that is readable, and into one stacked node carrying mapped_summary.by_state once it is not. The stacked node takes the worst state in the fan, so one failure among a hundred successes is still what you see.
  • The steps level draws a tier as a column and arrows only at the real boundaries, because hflow_server._graph is explicit that registered steps have no dependency edges on each other.
  • The episodes branch hangs off the run, not off a stage, and filters on every stage run id at once. Measured rather than assumed: the catalog's episodes view is one row per episode (latest append wins), so asking per stage returns total: 0 for sync, meta and labels in a full ingest and 6 only for media. The union is the honest question.

Verification

  • 37 vitest tests over buildGraph and layoutGraph, plus tsc --noEmit and biome check.
  • Non-vacuous: each load-bearing invariant was broken on purpose and the suite caught it (fan threshold, same-tier edges, mapped rewiring, shared-position layout bug).
  • All five levels walked in a real browser against a real hflow serve — real bundle, real pipeline, real catalog, with only the Airflow HTTP calls stubbed. Light and dark.
  • src/apiSchema.ts is generated from the server's own OpenAPI declaration; the new CI job fails if it goes stale, so nothing here hand-copies a payload field.
  • Repo suite unchanged: 802 passed, 6 skipped; ruff and ty clean.

Size

7,440 additions across 24 files, of which 2,245 are the generated schema and ~1,400 the lockfile — about 3,400 lines hand-written including tests and docs. #87 was 19,304 additions across 52 files.

Notes

  • TypeScript is pinned to 5.x on purpose: openapi-typescript drives the compiler API through ts.factory, which TypeScript 7's native port does not expose, so pnpm gen:api dies on 7. Recorded in ui/README.md.
  • feat(ui): a React workspace UI over the hflow-server API #87 stays open as reference until this can be compared against it in practice.

🤖 Generated with Claude Code

The workspace UI as a single screen people already know how to read: a
left-to-right node graph with a breadcrumb, an inspector, and one thing to
learn -- a node with a chevron opens.

Two branches leave the run, because a run has two things worth following:

  run -> stage -> steps          the orchestration, and the code inside it
  run -> episodes -> episode     the data it produced

Every judgement about what is honest to draw lives in one pure function,
src/canvas/buildGraph.ts, tested without a browser. The rule it keeps is that
an edge means a real dependency:

- The run level merges each stage's gate and trigger into one node and drops
  the shortcuts the stage chain already implies, turning a nine-rank ribbon
  into a five-node line. Both task ids, states and durations stay on the node,
  so the summary hides nothing.
- The stage level expands the mapped process_batch into one node per batch
  while that is readable, and into one stacked node carrying the state split
  once it is not. The stacked node takes the WORST state in the fan, so one
  failure among a hundred successes is still what you see.
- The steps level draws a tier as a column and arrows only at the boundaries
  that are real, because the server is explicit that registered steps have no
  dependency edges on each other.
- The episodes branch hangs off the RUN, not off a stage, and filters on every
  stage run id at once. Measured, not assumed: the catalog's episodes view is
  one row per episode, so a per-stage question answers 0 for every stage the
  same ingest later superseded.

src/apiSchema.ts is generated from the server's own OpenAPI declaration and CI
fails if it goes stale, so nothing in the client hand-copies a payload field.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant