Repository navigation
OSINT/Gotham PoC — enriched CAM SoA rendered in 3D from raw bytes - #39
Merged
Merged
Conversation
… upstream q2 AdaWorldAPI/q2 and upstream quarto-dev/q2 have NO shared git history, so the 968-commit upstream maturation can't be merged — it is re-applied. This lands our additive customizations onto the fresh upstream base (origin/main = 61e2d22) on branch claude/q2-upstream-replant: - bring our ours-only crates (zero upstream overlap, conflict-free): cockpit/ (Gotham GUI), crates/cockpit-server, crates/stubs/* (notebook-query, graph-flow, q2-ndarray), crates/aiwar-ingest, crates/neural-debug, crates/quarto/src/notebook_server.rs. - reconcile root Cargo.toml onto upstream's: crates/stubs/* members + the ours-only [workspace.dependencies] — the REAL ../lance-graph/crates/* path deps (per .claude/rules/architectural-compliance.md), axum/tower-http/ async-stream/futures-core, deno_core/serde_v8. - DROP the ~300 stale-core ours-only files (pampa tests, tree-sitter-qmd, hub-client, docs); upstream's matured versions are taken instead. cargo metadata --no-deps resolves (53 workspace members). Full cargo xtask verify build is the remaining gate (not yet run; disk-constrained). Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TzqvDqbFRzyx17EkLKBoZF
…0.404) crates.io deno_core 0.376 is archived, and its stale temporal_rs 0.1.2 fails to compile against the current icu_calendar. deno_core was a deliberate AdaWorldAPI choice (live JS/TS cell execution — the "better than neo4j" cockpit), so per .claude/rules/architectural-compliance.md the fix is the fork, NOT a substitute JS engine. - workspace.dependencies deno_core + serde_v8 -> git AdaWorldAPI/deno#main (deno_core 0.404 at libs/core, current V8). - lock regen bumps the deno subtree: temporal_rs 0.1.2->0.2.3 (resolves the icu_calendar break), v8 142->149, timezone_provider/zoneinfo64 0.2->0.3. Full `cargo metadata` resolves (deno_core 0.404 from the fork, zero conflicts). The heavy V8 compile + any 0.376->0.404 cockpit-server API updates are deferred to a disk-unconstrained build (CI/Railway); this container is at ~9.6G free. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TzqvDqbFRzyx17EkLKBoZF
…o serialization) Normalize the aiwar Gotham/neo4j corpus (aiwar_graph.json + 30 cypher enrichments, ~221 entities + airo/vair schema dimension) into ONE codebook test fixture in the CANON contract::aiwar model — instead of re-serializing the 174 KB of JSON properties: - classid OSINT 0x0700; each node is HEAD-only (family | u16 identity | edge adapters), zero value-slab decode. - codebook label inheritance: 7 family CLASSES (System/Stakeholder/Person/ CivicSystem/HistoricalSystem/SchemaAxis/SchemaValue); entities inherit a family by REFERENCE (mixin f=XX), never a copy. - 4-nibble (u16) identity per node; label CAM interns every label content- addressably (id <-> name, deduped). - edges resolve to out-of-family adapters = target FAMILIES (E-FAMILY-ADAPTER render-stable), not individual members. 674 nodes / 356 edges / 7 families. rho = 0.031 (~32x smaller than the serialized JSON). codebook_normalize.py regenerates it from the harvest corpus (AIWAR_HARVEST env, default ../aiwar-neo4j-harvest). Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TzqvDqbFRzyx17EkLKBoZF
…in model Integration test parsing tests/fixtures/aiwar.codebook and verifying: - 7 family classes; every node inherits a DEFINED family by reference (mixin), never a copy; - u16 (4-nibble) identities are unique discriminators; - label CAM resolves for every node label; - out-of-family edge adapters resolve to defined families (render-stable); - edges resolve to known CAM identities; - head-only: no serialized JSON properties in the fixture. Follows .claude/rules/integration-tests.md. Green with aiwar-ingest's 15 unit tests. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TzqvDqbFRzyx17EkLKBoZF
Post-commit formatter normalization (line wrapping only); no logic change. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TzqvDqbFRzyx17EkLKBoZF
… ClassView Wire the aiwar OSINT harvest into q2's cockpit as a CANON family-basin graph displayed through the OGAR Active-Record ClassView. Corrects the earlier category-as-family draft on three axes the operator flagged: - ClassView is the OGAR AR value, not contract::aiwar. Display is resolved through `&dyn lance_graph_contract::ClassView = &OgarClassView::new()` — the quarto-dev#585 SoC split (contract owns the trait, OGAR owns the value). Wires lance-graph-ogar (light, no surrealdb tree) + the [patch] that folds ogar-class-view's transitive git contract onto q2's path copy. - family(u16) is a BASIN, not a category and not a Louvain cluster. Location is permanent/deterministic via the HHTL; the basin is an interface a node adapts to. Two-tier (round-theme high nibble, anchor low nibble) = up to 256 basins, each addressable by one of a node's 16x8-bit family adapters. - GUID-v2 tail (leaf|family|identity, all u16) via NodeGuid::new_v2; enables the guid-v2-tail feature on the contract. The contract's v1 project_snapshot reads family() (u24), so the v2-aware Gotham projection lives here in q2. cockpit-server: new osint_gotham module (basin plan + v2 rows + v2 projection + OgarClassView display wiring), GET /api/graph/osint route, startup hydrate. Compiles clean (cargo check -p cockpit-server, OGAR + guid-v2-tail linked). Unit tests (theme-stem, v2 basin tail, adapter mask, OGAR ClassView SoC, Gotham hubs/edges) run pending in this session; logic verification to follow. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TzqvDqbFRzyx17EkLKBoZF
… chart The enrichment had never actually run. The harvest aiwar_graph.json carries 1660 bare `NaN` tokens (valid JS, illegal JSON), so serde_json rejected the only copy with a cypher/ sibling and the graph silently fell back to the un-enriched 221-node base. The cypher parser then emitted unbound variables (a/b/v) as phantom edge endpoints (~60% of endpoints). aiwar-ingest: - load_from_str: sanitize NaN/Infinity -> null before parse. - encounter_round::parse_cypher: resolve variables per `;`-statement (bind by id->value->name; handle WITH / multi-MATCH / inline-declared endpoints), skip edges whose endpoints don't resolve, drop the pandas `nan`/empty missing-data marker. Kills phantom a/b/v endpoints. cockpit-server/osint_gotham: - emit the typed entity->entity link chart (CONNECTED_TO / DEVELOPED_BY / PERSON_LINK / VALID_FOR / ...) over the basin overlay. - complete OSINT_SCHEMA to 7 labels (add SchemaValue/SchemaAxis) so every item has a label<->order(identity) pairing (0 missing). - value tenant carries the class-label order; display via OgarClassView. - eval_650 (ignored) richness harness over the real harvest. Measured on the full 30-round set: 596->734 real nodes; phantom hubs gone (top hubs now United States/Palantir/Israel/Epstein); isolated 195->29; edges 3563(60% phantom)->2065(resolved); link chart 0->2032/2065 in view. Tests: 19 aiwar-ingest + 7 cockpit osint_gotham green. Design convergence (OSINT/CAD/splat on one 128-bit node) captured in claude-notes/research/2026-06-22-soa-node-substrate-osint-cad-splat.md. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TzqvDqbFRzyx17EkLKBoZF
…D scene position(&NodeGuid) -> [f32;3]: family picks the basin region, identity places the node on a Vogel/phi-spiral disc (the helix sub-position), HEEL lifts y. No layout pass — the address IS the coordinate. /api/graph/osint now emits x/y/z on every node (members + basin hubs); edges become 3D lines and family mixins are spatial neighbours. The substrate point made concrete: OSINT graph, CAD model, and Gaussian splat are the same 3D scene at the same addresses — only the value-tenant payload at each coordinate differs. Test: guid_decodes_to_a_3d_position (deterministic decode, distinct points, distinct basin regions, x/y/z emitted on every node). 7 cockpit osint_gotham tests green. Design note updated: 3D-coordinate decode + edge-storage convergence (typed_dag = OpenProject = classid 0x0100; HHTL prefix replaces closure tables). Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TzqvDqbFRzyx17EkLKBoZF
The ~700-node enriched scene is baked ONCE to a 32 KB binary SoA buffer (osint_gotham::osint_soa_bytes -> assets/osint_scene.soa: per node a 16-byte GUID + class byte; then edges as u16 index pairs) and embedded via include_bytes!, served at /osint.soa. No cypher at runtime, no JSON. The /osint3d React/three.js view (cockpit) fetches the buffer and decodes each GUID -> xyz IN THE BROWSER (the position() logic ported to JS: family->basin, identity->Vogel spiral, heel->height). The address IS the coordinate; no force-layout. Class (the value-tenant order, resolved through the OGAR AR ClassView) drives node colour. 920 nodes (734 members + 186 family hubs) / 3344 edges. - osint_gotham::osint_soa_bytes + bake_osint_soa (bakes the .soa offline using the harvest cypher; cypher is NOT shipped) + soa_bytes_have_a_parseable_header. - cockpit-server: OSINT_SOA include_bytes! + /osint.soa octet-stream route. - cockpit/src/OsintScene3D.tsx (binary decode + render) + /osint3d route. Verified: cargo build -p cockpit-server and npm run build (cockpit) both green; bake produced 920 nodes / 3344 edges / 32372 bytes; soa header test + 7 osint_gotham tests pass. NOT verified: the live GET /osint.soa (cargo run hit a full disk before binding) -- route/handler/include_bytes! all compile and the .soa is valid, but the running response was not exercised this session. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TzqvDqbFRzyx17EkLKBoZF
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Important Review skippedToo many files! This PR contains 192 files, which is 42 over the limit of 150. To get a review, narrow the scope: Upgrade to a paid plan to raise the limit. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (3)
📒 Files selected for processing (192)
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A working proof-of-concept for the OSINT/Gotham view: the AIWAR graph (Sarah Ciston's AI War Cloud) loaded as the CANON SoA node substrate (classid
0x0700) and rendered as a 3D scene decoded from raw bytes — no JSON, no runtime cypher.The arc
NaN(illegal JSON → serde rejected the only enrichable copy → silent fallback to base 221). Fixed the loader (NaN/Infinity→null) and rewrote the cypher parser to resolve variables per-statement (bind byid→value→name;WITH/ multi-MATCH/ inline endpoints; skip unresolved; drop the pandasnanmarker). Phantoma/b/vendpoints gone; isolated 195→29; 221 → 734 real nodes, 2065 resolved edges.OSINT_SCHEMAcompleted to 7 labels so every item has a label↔order pairing (0 missing).position()decodes a node's 128-bit GUID to[x,y,z](family→basin region, identity→φ-spiral, HEEL→height). The address is the coordinate; no force-layout pass.assets/osint_scene.soa: per node a 16-byte GUID + class byte; edges as u16 index pairs), embedded viainclude_bytes!, served raw at/osint.soa. The/osint3dReact/three.js view fetches the buffer and decodes each GUID→xyz in the browser, rendering spheres + edges. Class resolves through the OGAR ARClassView. 920 nodes (734 members + 186 family hubs) / 3344 edges.Run it
Deploy this branch; open
/osint3d.Verification
cargo build -p cockpit-server+npm run build(cockpit): green.soa_bytes_have_a_parseable_header+ 7osint_gothamtests pass.GET /osint.soa→200, 32372 bytes, magicOSO1, header 920/3344.cargo xtask verifynot yet run locally (disk) — CI covers it; this session is watching the PR and will autofix to green.Design note:
claude-notes/research/2026-06-22-soa-node-substrate-osint-cad-splat.md.🤖 Generated with Claude Code
https://claude.ai/code/session_01TzqvDqbFRzyx17EkLKBoZF
Generated by Claude Code