Skip to content

lab(drf-w4): hand flushed row images to the real CycleWriter (W4-W7) - #1767

Merged
AdaWorldAPI merged 2 commits into
mainfrom
lab/drf-w4-handoff-20261025
Oct 25, 2026
Merged

AdaWorldAPI merged 2 commits into
mainfrom
lab/drf-w4-handoff-20261025

Conversation

@AdaWorldAPI

Copy link
Copy Markdown
Owner

This connects the drf-focus-probe writeback to the production CycleWriter in lance-graph-supervisor and pins W4, W6 and W7 from .claude/plans/d-dr64-focus-cache-falsifiers-v1.md §4.

Change

  • New feature handoff, which adds RowImage::into_slot and into_slots. They turn flush output into owned SweepSlot casts: one cast per distinct row, with the row bytes moved rather than copied. Stream positions continue across cycles, because the writer's recovery watermark depends on that.
  • RowImage::row is now Vec<u8> (it was [u8; 512]), so moving it into the cast's payload is not a second copy.
  • New dev-dependencies: lance-graph-supervisor (feature cycle-driver), lance-graph-planner, lance-graph-contract and tokio.

Tests (tests/integration/handoff.rs): the real cycle_writer runs over a fake WAL that reconciles before appending.

  • W4: 64 writes over 37 rows become 37 casts. The source changes after flush while the WAL is still holding the commit. Nothing is durable until the WAL releases it, and what lands is the flushed snapshot.
  • W2, through the writer: two fields of one row both land. Contrast arm: one cast per raw write loses the first field, because the writer keeps only the later payload for a row, whole. That is why the handoff takes flush output and nothing else.
  • W6: after a lost acknowledgement the writer says "resubmit the frozen batch" (ResubmitFrozen), and the casts it hands back are byte-identical to the frozen batch. Re-submitting them is reconciled with a single append. Regenerating the cycle from the changed source instead is refused (HashConflict, then Escalate). A full queue hands the casts back unchanged.
  • W7: a cycle with no dirty row returns NoChange without calling the WAL. The next dirty cycle publishes exactly one version.

Gates: 22 tests pass with --features handoff, 17 without, and 22 on x86-64-v3; clippy -D warnings is clean with and without the feature. No CI job runs this lab crate, so these gates are local runs only.

Disable runs (each turns tests red)

  • One image per write instead of per row: the W1–W3 tests and the through-writer W2 test fail.
  • into_slots not advancing the stream position: W4 fails.

Not covered

  • W5 (a descriptor-only cast pointing into live rows) needs a payload type the writer does not have.
  • W8 (eviction under memory pressure) is not run.
  • The fake WAL stands in for Lance.

Board: .claude/board/entries/2026-10-25-drf-w4-handoff.md.


Generated by Claude Code

Feature handoff: RowImage::into_slot / into_slots turn flush output into
owned SweepSlots (row bytes moved, one cast per distinct row, stream
positions continued across cycles). RowImage::row is now Vec<u8> so the
move into the payload is not a second copy.

Tests drive lance-graph-supervisor's cycle_writer over a reconciliation-
first fake WAL: W4 the source changes after flush and the landed image is
the snapshot, durability waits for the WAL; W2 through the writer (raw
per-patch casts lose a facet to the freeze's whole-payload last-wins);
W6 lost ack -> ResubmitFrozen with the same casts -> Reconciled, while
regenerating from the changed source -> HashConflict/Escalate; backpressure
returns casts; W7 an empty cycle never calls the WAL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 45f2af3c-5cc1-47bc-bd8f-051b4beb4018
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@AdaWorldAPI
AdaWorldAPI marked this pull request as ready for review October 25, 2026 00:50
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-25T00:51:20.438266Z 44794a4 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@AdaWorldAPI
AdaWorldAPI merged commit 79a2c5b into main Oct 25, 2026
12 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 44794a4567

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/drf-focus-probe/tests/integration/handoff.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants