Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

49 changes: 37 additions & 12 deletions claude-notes/plans/2026-08-15-osm-lance-lifecycle.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
# OSM/Lance lifecycle — three clocks, one lineage per region

**Status:** Phase A implemented, corrected once, tests re-run. Phases B–F not
started (Phase B observational surface not begun this session either).
**Status:** Phase A merged (PR #133). Phase B observational surface landed
(`OsmArtifactManager`, `/api/osm/status`, cgroup memory) — the `OnceLock`
serving path in `osm_features.rs` is untouched; that swap is Phase C. Phases
C–F not started.
**Operator directive:** 2026-08-15, this session.
**Upstream contracts:** `AdaWorldAPI/lance-graph` PRs **#912** (baseline) and
**#913** (the five semantic gates). #911 is historical context and is
Expand Down Expand Up @@ -294,10 +296,12 @@ these paths.
## Phases (PR-sized)

- **A.** Pure lifecycle types, state machine, manifests, storage resolution,
falsifiers. ← *this session*
falsifiers. ← *merged, PR #133*
- **B.** `OsmArtifactManager`, fast listener startup, status endpoint, memory
telemetry. ← *observational parts only, if cleanly separable*
- **C.** Versioned local import with origin/import seals and reconciliation.
telemetry. ← *observational surface landed this session; does not replace
the `OnceLock` serving path*
- **C.** Versioned local import with origin/import seals and reconciliation,
AND the `osm_features.rs` OnceLock→`OsmArtifactManager` migration.
- **D.** Incremental S3 lineage mirroring, publication pointer, recovery,
retention.
- **E.** Native Lance column layout + Morton-shard updates; removes the single
Expand Down Expand Up @@ -371,13 +375,34 @@ actually observe the real system lands it.
- [x] **correction pass** (2026-08-15, same session) — see § Correction pass;
re-ran the full `osm_lifecycle` test module after

### Phase B (observational only this session — not started)
- [ ] `OsmArtifactManager` with atomically replaceable `Arc`
- [ ] `/api/osm/status`
- [ ] cgroup memory split + phase deltas
- [ ] listener/`/health` independent of hydration
- [ ] a real `MaintenanceProvenance::LanceHistoryVerified` resolver, OR name it
as still deferred to Phase C at the status-endpoint level
### Phase B (observational only — landed this session)
- [x] `OsmArtifactManager` with atomically replaceable `Arc`
(`crates/cockpit-server/src/osm_artifact_manager.rs`) — two independent
swaps: `ArcSwap<Lifecycle>` (real today) and `ArcSwapOption<ActiveArtifact>`
(deliberately an empty marker; Phase C fills it). **Purely additive**:
wired into `AppState` and constructed `absent()` at startup (no I/O), but
does NOT replace the four `OnceLock`s in `osm_features.rs`
(`SLAB_MMAP`/`BOOKS`/`CHAINS`/`SLAB_DIGEST`) — that swap needs real
hydration content to publish and is Phase C's job. Falsifier proven: a
reader holding an `Arc` from `current()`/`artifact()` before a
`publish_*` call keeps observing the old value after the call.
- [x] `/api/osm/status` — new route, reports the manager's lifecycle/artifact
snapshot + cgroup memory as JSON. Reads no map data; independent of the
existing `/api/osm/health` (which still reports the OnceLock state).
- [x] cgroup memory split — `read_cgroup_memory()` (Linux-gated per
`.claude/rules/cross-platform.md`, honest `None` fallback elsewhere),
pure `parse_cgroup_current`/`parse_cgroup_max` helpers.
- [ ] **phase deltas** — deferred, not attempted. Tracking a memory delta
across a real phase transition needs a real transition to measure
across; there isn't one until Phase C's hydration exists. Recorded as
a named gap, not silently dropped.
- [x] listener/`/health` independent of hydration — **already true**, verified
rather than fixed: `health_handler` is static JSON with no OSM
reference, and OSM's OnceLocks are lazy (first-request-triggered), so
nothing on the startup/listener path touches map data either way.
- [ ] a real `MaintenanceProvenance::LanceHistoryVerified` resolver — still
deferred to Phase C, as originally planned; nothing in this session
claims otherwise.

### Deferred, named
- [ ] braid epic — **the `braid` CLI is not installed in this container**; the
Expand Down
5 changes: 5 additions & 0 deletions crates/cockpit-server/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,11 @@ object_store = { version = "0.13.2", features = ["aws"] }
sha2 = "0.10"
hex = "0.4"
futures = "0.3"
# `osm_artifact_manager`: the atomically-replaceable-Arc serving mechanism
# (Phase B of the OSM/Lance lifecycle plan). Already in the lockfile
# transitively at 1.9 via lance-namespace/goosefs, so this adds no new
# dependency tree — it only promotes an existing version to a direct dep.
arc-swap = "1.9"
# `osm_lance`: convert the hydrated `.soa` slab into a Lance dataset on the
# same persistent volume (the `soa_to_lance` recipe, see that module's doc).
# Both are ALREADY locked transitively at these exact versions via
Expand Down
89 changes: 88 additions & 1 deletion crates/cockpit-server/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ mod mock_driver;
mod osint_classview;
mod osm_tiles;
mod osm;
mod osm_artifact_manager;
mod osm_features;
mod osm_lance;
mod osm_lifecycle;
Expand Down Expand Up @@ -119,6 +120,12 @@ struct AppState {
tx: broadcast::Sender<SseEvent>,
/// Shared scene state for /v1/shader/stream + /v1/shader/status.
scene_state: shader_stream::SharedSceneState,
/// The OSM/Lance lifecycle serving state (Phase B). Constructed
/// `absent()` at startup — no I/O, so it never delays the listener —
/// and stays that way until Phase C wires real hydration through
/// `publish_lifecycle`/`publish_artifact`. `/api/osm/status` reads it;
/// nothing on the existing OnceLock-backed OSM serving path does yet.
osm_manager: Arc<osm_artifact_manager::OsmArtifactManager>,
}

// Shader handlers extract `State<Arc<AppState>>` and read `state.scene_state`
Expand Down Expand Up @@ -240,7 +247,12 @@ async fn main() {

let (tx, _rx) = broadcast::channel::<SseEvent>(256);
let scene_state = shader_stream::new_scene_state();
let state = Arc::new(AppState { tx, scene_state });
let osm_manager = Arc::new(osm_artifact_manager::OsmArtifactManager::absent());
let state = Arc::new(AppState {
tx,
scene_state,
osm_manager,
});

// OpenAI-compatible model API state
let openai_state = Arc::new(tokio::sync::Mutex::new(openai::OpenAiState::new()));
Expand Down Expand Up @@ -316,6 +328,11 @@ async fn main() {
// working map on every other signal (200s, full tiles, correct
// geometry) while drawing grey and untagged.
.route("/api/osm/health", get(osm_features::osm_health_handler))
// Phase B of the OSM/Lance lifecycle plan: the OsmArtifactManager's
// lifecycle/artifact snapshot + cgroup memory, observational only —
// does not read map data and does not touch the OnceLock-backed
// serving path /api/osm/health reports on.
.route("/api/osm/status", get(osm_status_handler))
.route("/api/osm/locate", get(osm_tiles::osm_locate_handler))
.route("/api/osm/tile/:z/:x/:y", get(osm_tiles::osm_tile_meta_handler))
.route("/api/osm/features/:z/:x/:y", get(osm_features::osm_features_handler))
Expand Down Expand Up @@ -1406,6 +1423,76 @@ async fn health_handler() -> Json<serde_json::Value> {
}))
}

// ── OSM/Lance lifecycle status (Phase B) ───────────────────────────────────────

fn import_failure_json(error: &osm_lifecycle::ImportFailure) -> serde_json::Value {
use osm_lifecycle::ImportFailure::{
IdentityConflict, S3Publication, Source, Validation, Write,
};
match error {
Source(message) => serde_json::json!({ "kind": "source", "message": message }),
Write(message) => serde_json::json!({ "kind": "write", "message": message }),
Validation(message) => serde_json::json!({ "kind": "validation", "message": message }),
S3Publication(message) => {
serde_json::json!({ "kind": "s3_publication", "message": message })
}
IdentityConflict => serde_json::json!({ "kind": "identity_conflict" }),
}
}

fn lifecycle_status_json(lifecycle: &osm_lifecycle::Lifecycle) -> serde_json::Value {
let serving = lifecycle.serving.map(|s| {
serde_json::json!({
"active": s.active.to_key(),
"previous": s.previous.map(|p| p.to_key()),
})
});
let operation = match &lifecycle.operation {
osm_lifecycle::Operation::Idle => serde_json::json!({ "state": "idle" }),
osm_lifecycle::Operation::Importing {
candidate,
generation,
} => serde_json::json!({
"state": "importing",
"candidate": candidate.to_key(),
"generation": generation.0,
}),
osm_lifecycle::Operation::Failed {
candidate,
generation,
error,
} => serde_json::json!({
"state": "failed",
"candidate": candidate.to_key(),
"generation": generation.0,
"error": import_failure_json(error),
}),
};
serde_json::json!({ "serving": serving, "operation": operation })
}

/// Phase B observational endpoint: the [`osm_artifact_manager::OsmArtifactManager`]'s
/// current lifecycle/artifact snapshot plus cgroup memory. Deliberately does
/// NOT read any map data (slab/books/chains) — that stays on the existing
/// `/api/osm/health` path, unaffected by this endpoint's presence.
async fn osm_status_handler(State(state): State<Arc<AppState>>) -> Json<serde_json::Value> {
let lifecycle = state.osm_manager.current();
let mem = osm_artifact_manager::read_cgroup_memory();
Json(serde_json::json!({
"lifecycle": lifecycle_status_json(&lifecycle),
"artifact_present": state.osm_manager.artifact().is_some(),
"cgroup_memory": {
"current_bytes": mem.current_bytes,
"max_bytes": mem.max_bytes,
},
"note": "Phase B observational endpoint. Reports the OsmArtifactManager's \
lifecycle/artifact snapshot and cgroup memory; nothing publishes \
real content into it yet — Phase C wires actual hydration through \
publish_lifecycle/publish_artifact. The existing OnceLock-backed \
slab/books/chains serving path (see /api/osm/health) is untouched.",
}))
}

#[cfg(test)]
mod garmin_scene_tests {
use super::{
Expand Down
Loading