Repository navigation
osm lance: wire the tile-serving read path onto the Lance-owned bytes - #123
Conversation
Closes the gap the previous commit deliberately left open: the map is now actually SERVED from lance-graph storage, not just converted into it. osm_lance::locate_row_column() productionizes the soa_to_lance "P-CACHE-4" mechanism into a boot-time resolver: it searches the first few megabytes of each Lance data file for the slab's own row-0 bytes (row 0 is never gigabytes in), and reports the exact (offset, length) of the verbatim row column within it — bounded, fail-closed, never reads a multi-GiB file whole just to find an anchor that sits near the front. osm_features::open_slab() now prefers OSM_SLAB_ROW_FILE (set at boot once the resolver succeeds) over OSM_SLAB_PATH for the row-byte mmap, bounded to exactly OSM_SLAB_ROW_OFFSET..+OSM_SLAB_ROW_LEN — required because the Lance data file carries its own header/footer bytes around the row run, which RowSlab::new would otherwise reject outright (it demands an exact multiple of NODE_ROW_STRIDE, not a prefix of one). OSM_SLAB_PATH itself is never repointed: it still anchors the .books/.chains sidecar paths, which live beside the raw bake, not inside the Lance dataset's data/ directory. Every step stays fail-closed to the pre-existing behavior: a missing dataset, an unlocatable row column, or any doubt at all leaves OSM_SLAB_ROW_FILE unset and the raw .soa file keeps serving the map exactly as before. Note: this branch was reset onto origin/main after the prior PR (#122) merged mid-session, per this repo's merged-branch policy — the two prior commits are already in main; only this one is new. Verified: cargo check -p cockpit-server clean; cargo clippy -p cockpit-server --bins produces zero warnings in the new code. The 4 new unit tests for locate_row_column/search_head_for_probe could not be EXECUTED this session — cargo test's monolithic test-binary link repeatedly crashed the linker (SIGBUS) under this environment's disk pressure, even after aggressive cleanup. cargo check confirms the code type-checks correctly including the async lance API usage; the tests themselves are straightforward tempfile-based fixtures with no exotic dependencies, but they have not been run. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NMeiLmtDKhomJNSo2ecbJw
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_8c2708b1-74cd-48d3-8040-5dc13a6b7c77) |
Summary
Follow-up to #122 (merged) — closes the gap that PR deliberately left open: the map is now actually served from lance-graph storage, not just converted into it.
osm_lance::locate_row_column()productionizes thesoa_to_lance"P-CACHE-4" mechanism into a boot-time resolver: searches the first few megabytes of each Lance data file for the slab's own row-0 bytes (row 0 is never gigabytes in — never reads a multi-GiB file whole), and reports the exact(offset, length)of the verbatim row column within it.osm_features::open_slab()now prefersOSM_SLAB_ROW_FILE(set at boot once the resolver succeeds) overOSM_SLAB_PATHfor the row-byte mmap, bounded to exactlyOSM_SLAB_ROW_OFFSET..+OSM_SLAB_ROW_LEN— required because the Lance data file carries its own header/footer bytes around the row run, whichRowSlab::newwould otherwise reject outright (it demands an exact multiple ofNODE_ROW_STRIDE).OSM_SLAB_PATHis never repointed — it still anchors the.books/.chainssidecar paths, which live beside the raw bake, not inside the Lance dataset'sdata/directory.OSM_SLAB_ROW_FILEunset and the raw.soafile keeps serving the map exactly as before.Branch note: this session's branch was reset onto fresh
mainafter #122 merged mid-session (per this repo's merged-branch policy) — only this one commit is new.Test plan
cargo check -p cockpit-server— cleancargo clippy -p cockpit-server --bins— zero warnings in the new code (same pre-existing warnings as before, unrelated)cargo test -p cockpit-server --bin q2-cockpit osm_lance— could not be executed this session. The monolithic test-binary link repeatedly crashed the linker (SIGBUS,ld terminated with signal 7) under this environment's disk pressure, even after aggressive cleanup (freed target/debug/incremental multiple times).cargo checkconfirms the new code type-checks correctly, including the async lance API usage; the 4 new tests (search_head_for_probe_finds_the_exact_offset,search_head_for_probe_declines_when_the_probe_is_absent,locate_row_column_finds_the_fragment_and_reports_the_slabs_own_length,locate_row_column_declines_when_no_fragment_matches) are straightforward tempfile-based fixtures with no exotic dependencies, but they have genuinely not been run — flagging honestly rather than claiming a pass I didn't observe.Generated by Claude Code