Repository navigation
osm cockpit: WebGL2 rendering + load the bake into lance-graph as a Lance dataset - #122
Merged
Merged
Conversation
Client-side rendering was ~57,690-70,000 DOM <div> elements (one per feature dot), which is the actual source of the reported lag — not the Canvas2D basemap draw calls, which stayed under ~150/frame throughout. Adds an additive WebGL2 layer to the single embedded cockpit page: - initGL() creates a WebGL2 context alongside the existing canvas; automatic fallback to the existing Canvas2D path if it fails or is unavailable (gl stays null and every draw/pick site branches on it). - parseTileBinGL() decodes the same OSM1 wire format into interleaved (x, y, class) vertex buffers instead of Path2D — triangles for fills (pure-JS ear-clipping triangulation), line segments for strokes. - drawBaseGL()/drawDotsGL() replace per-feature DOM nodes with ONE gl.POINTS draw call per tile from a lazily-built Float32Array — the actual fix for the DOM-node lag. - pickFeature() replaces the DOM click-target lookup with a nearest-point scan over the same vertex buffers. - Per-class fill/stroke colors and the meadow/park texture patterns (stipple, hatch) are reproduced in the fragment shader from the same CLASS_STYLE table the Canvas2D path already used. Verified end-to-end with a real GPU (headless Chromium via Playwright, --use-gl=swiftshader), desktop and phone viewports: real triangle/point data painted (570,006 / 137,421 px respectively, glError:0), zero DOM dots, pan-repaint confirmed. A same-synchronous-block draw+readPixels diagnostic was needed to rule out a preserveDrawingBuffer:false compositing false-negative before trusting that measurement. wgpu/wasm is explicitly deferred — this is the WebGL2-now half only. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NMeiLmtDKhomJNSo2ecbJw
Converts the hydrated .soa slab into a Lance dataset on the same persistent volume, following medcare-rs's own hydrate-into-Lance shape (bake_hydrate::hydrate_crystal's warm-skip gate; LanceStore's volume-first-not-S3-backed default) and lance-graph's soa_to_lance example recipe: zero-copy Buffer::from_vec import, the 512-byte SoA contract (NODE_ROW_STRIDE / ENVELOPE_LAYOUT_VERSION, imported from lance_graph_contract rather than restated) persisted once as Arrow schema metadata. Runs at boot right after osm_slab_hydrate::ensure_slab_local() lands the verified .soa/.books/.chains bake, converting into <region>.lance in the same directory. Best-effort: a conversion failure never blocks boot or takes the map down, since osm_features::open_slab still mmaps the raw .soa file unchanged either way. Scope of this change: establishes Lance as the volume's canonical persisted form. It does NOT yet rewire the tile-serving hot path onto the Lance-owned bytes — that's the soa_to_lance "P-CACHE-4" fragment-mmap trick, a diagnostic probe rather than a stable API, and a separately scoped follow-up. classid recorded as the canon's own dormant-default 0x00000000 (see lance-graph PR quarto-dev#902) rather than an invented mint — the OSM bake does not yet participate in OGAR's classid/IdentityQuad federation model. lance/arrow declared as direct deps (both already locked transitively via lance-graph -> lance -> arrow at these exact versions; Cargo.lock diff is 3 lines, no new resolution) — the same move lance-graph PR quarto-dev#931 made for object_store's aws feature. Verified: cargo check -p cockpit-server clean; cargo clippy -p cockpit-server --bins produces zero warnings in the new code (all existing warnings are pre-existing, unrelated to this change). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NMeiLmtDKhomJNSo2ecbJw
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_d8630334-0439-4671-b657-4fa1cbbe2384) |
AdaWorldAPI
marked this pull request as ready for review
August 12, 2026 12:35
AdaWorldAPI
pushed a commit
that referenced
this pull request
Aug 12, 2026
Closes the gap the previous commit deliberately left open: the map is now actually SERVED from lance-graph storage, not just converted into it. osm_lance::locate_row_column() productionizes the soa_to_lance "P-CACHE-4" mechanism into a boot-time resolver: it searches the first few megabytes of each Lance data file for the slab's own row-0 bytes (row 0 is never gigabytes in), and reports the exact (offset, length) of the verbatim row column within it — bounded, fail-closed, never reads a multi-GiB file whole just to find an anchor that sits near the front. osm_features::open_slab() now prefers OSM_SLAB_ROW_FILE (set at boot once the resolver succeeds) over OSM_SLAB_PATH for the row-byte mmap, bounded to exactly OSM_SLAB_ROW_OFFSET..+OSM_SLAB_ROW_LEN — required because the Lance data file carries its own header/footer bytes around the row run, which RowSlab::new would otherwise reject outright (it demands an exact multiple of NODE_ROW_STRIDE, not a prefix of one). OSM_SLAB_PATH itself is never repointed: it still anchors the .books/.chains sidecar paths, which live beside the raw bake, not inside the Lance dataset's data/ directory. Every step stays fail-closed to the pre-existing behavior: a missing dataset, an unlocatable row column, or any doubt at all leaves OSM_SLAB_ROW_FILE unset and the raw .soa file keeps serving the map exactly as before. Note: this branch was reset onto origin/main after the prior PR (#122) merged mid-session, per this repo's merged-branch policy — the two prior commits are already in main; only this one is new. Verified: cargo check -p cockpit-server clean; cargo clippy -p cockpit-server --bins produces zero warnings in the new code. The 4 new unit tests for locate_row_column/search_head_for_probe could not be EXECUTED this session — cargo test's monolithic test-binary link repeatedly crashed the linker (SIGBUS) under this environment's disk pressure, even after aggressive cleanup. cargo check confirms the code type-checks correctly including the async lance API usage; the tests themselves are straightforward tempfile-based fixtures with no exotic dependencies, but they have not been run. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NMeiLmtDKhomJNSo2ecbJw
2 of 3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two commits, both additive/best-effort — neither changes existing serving behavior if it fails.
1. WebGL2 hardware-accelerated basemap rendering (
osm.rs)<div>elements (one per feature dot), not the Canvas2D basemap draw calls (stayed under ~150/frame throughout).glstaysnull, every draw/pick site branches on it):parseTileBinGL()decodes the OSM1 wire format into vertex buffers instead ofPath2D(ear-clipped triangles for fills, line segments for strokes);drawBaseGL()/drawDotsGL()replace per-feature DOM nodes with onegl.POINTSdraw call per tile;pickFeature()replaces DOM click-target lookup with a nearest-point scan over the same buffers.--use-gl=swiftshader), desktop + phone viewports: real triangle/point data painted, zero DOM dots, pan-repaint confirmed.2. Load the hydrated bake into lance-graph as a Lance dataset (
osm_lance.rs)osm_slab_hydrate::ensure_slab_local()lands the verified.soa/.books/.chainsbake on the persistent volume, converts the.soaslab into a Lance dataset (<region>.lance) in the same directory — medcare-rs's own hydrate-into-Lance shape (bake_hydrate::hydrate_crystal's warm-skip gate;LanceStore's volume-first-not-S3-backed default) and lance-graph'ssoa_to_lanceexample recipe (zero-copy import, the 512-byte SoA contract persisted once as Arrow schema metadata, imported fromlance_graph_contractnot restated).osm_features::open_slabstill mmaps the raw.soafile unchanged — a conversion failure never blocks boot or takes the map down.classidrecorded as the canon's dormant-default0x00000000(lance-graph PR Preview: render the skip-to-content link in hub-client preview (bd-21n09dwj) quarto-dev/q2#902) — the OSM bake doesn't yet mint into OGAR's classid/IdentityQuad federation model, so this isn't invented.lance/arrowdeclared as direct deps ofcockpit-server(both already locked transitively vialance-graphat these exact versions —Cargo.lockdiff is 3 lines, no new resolution), the same move lance-graph PR Move derived state out of effects (bd-cp9wzu6o) quarto-dev/q2#931 made forobject_store'sawsfeature.Test plan
cargo check -p cockpit-server— cleancargo clippy -p cockpit-server --bins— zero new warnings (all pre-existing warnings are unrelated to this change)cargo test -p cockpit-server --bin q2-cockpit osm_lance— in progress (large test-profile recompile of the datafusion/lance graph); will report results as a follow-up commentcargo build --workspace/cargo nextest run --workspace— not run this pass (disk-constrained); scoped verification above covers the changed crateGenerated by Claude Code