Replay the fork's commits onto upstream main (83a3afc3)

.cargo/deny.toml

@@ -16,6 +16,7 @@ allow-build-scripts = [
     "alloca",
     "anyhow",
     "camino",
+    "clang-sys",
     "codspeed",
     "codspeed-criterion-compat",
     "codspeed-divan-compat",
@@ -29,6 +30,7 @@ allow-build-scripts = [
     "iana-time-zone-haiku",
     "icu_normalizer_data",
     "icu_properties_data",
+    "lib-ruby-parser",
     "libc",
     "libm",
     "libmimalloc-sys",

.claude/OGAR-POLYGLOT-AST-INTEGRATION.md

@@ -0,0 +1,193 @@
+# OGAR Polyglot AST Integration (RFC)
+
+**Status:** RFC / design contract. Not yet implemented or compile-verified —
+this document locks the contract so the implementation + tests land as a
+typed follow-up.
+**Scope:** the `ruff_*_spo` frontends, the `ruff_spo_triplet` IR, the
+`ruff_spo_address` mint, and a new `LangBackend` adapter family.
+**Goal:** turn ruff's per-language SPO harvesters into a **bidirectional
+polyglot AST substrate** — source (Python / C++ / C#) → OGAR IR → re-emitted
+source (Python / Rust / C#) — with the IR as a content-addressed interlingua.
+
+---
+
+## 0 · Why this is mostly "fix an asymmetry," not "build a transpiler"
+
+The interlingua already exists and is already bidirectional:
+
+```
+ SOURCE                       frontend                  IR (interlingua)              address
+ ──────                       ────────                  ────────────────              ───────
+ Python ─ ruff_python_dto_check ─▶ (JSON bundles) ─┐
+ C++    ─ ruff_cpp_spo (libclang) ────────────────▶│
+ C#     ─ Roslyn tool ─ndjson─ ruff_csharp_spo ───▶│  ModelGraph ─expand─▶ Triples ─mint─▶ 16B Facet
+ Ruby   ─ ruff_ruby_spo (lib-ruby-parser) ────────┘  (ruff_spo_     ▲  │              (ruff_spo_address,
+                                                      triplet::ir)  │  ▼               node_of = reverse)
+                                                 reassemble ◀───────┘  ndjson ─▶ lance_graph SPO store
+                                                      │
+ TARGET ◀──── backend / adapter ◀──── ModelGraph ◀────┘
+ ──────
+ Rust   ◀─ ruff_cpp_codegen   (THE ONLY BACKEND TODAY: ModelGraph → Rust MethodSig source)
+ Python ◀─ (none yet)
+ C#     ◀─ (none yet)
+```
+
+Three load-bearing facts (verified against `main`):
+1. **`ruff_spo_triplet::ir::ModelGraph` is the interlingua.** `expand`
+   (ModelGraph→triples) is general, but **`reassemble` (triples→ModelGraph)
+   today recovers only the C++ machine-plane projection** — it does *not*
+   reconstruct the core-7 `fields`/`functions` or the OpenProject collections
+   (`ruff_spo_triplet/src/reassemble.rs:16-20`). So a **general reassembler is
+   the first build item (Phase 0)**, not a given. "A new language is a new
+   frontend, not a new ontology" (crate doc) — the *intent* is bidirectional;
+   the *reassembler* is not yet general.
+2. **The asymmetry is 4 frontends vs 1 backend.** `ruff_cpp_codegen` already
+   proves `ModelGraph → Rust source` (it renders `MethodSig` manifests
+   targeting `lance_graph_contract::codegen_manifest`). The superpower is
+   *generalizing the back door*.
+3. **The frontends do not enter uniformly:** C++/Ruby produce `ModelGraph`
+   directly; Python produces JSON `ModuleHarvest` bundles; C# runs an
+   out-of-process Roslyn tool → ndjson → `load() -> Vec<Triple>`.
+
+---
+
+## 1 · The IR record (Phase-0 contract — lock this first)
+
+### 1.1 Dual-mode facet payload (classid-tagged)
+
+`12 bytes = 96 bits` tiles **exactly** two ways. The 4-byte classid is the
+discriminant.
+
+```rust
+// 16-byte content address; byte-identical to lance_graph_contract::facet::FacetCascade
+pub struct Facet { facet_classid: u32, payload: [u8; 12] }
+
+enum FacetMode {            // carved from facet_classid (mirrors TailVariant)
+    Cascade,                // payload = [FacetTier; 6]  — 6 × (part_of:8, is_a:8)
+    Triplet,                // payload = [SpoTriple; 4]  — 4 × (subject:8, predicate:8, object:8)
+}
+```
+
+- **Cascade** = *position*: 6 tiers deep, predicate implied (`part_of`+`is_a`).
+  Subsumption / containment as bit-ops. The address/index view.
+- **Triplet** = *local edges*: 4 SPO edges, predicate explicit (256-way). The
+  raw-graph view. **A `ruff_spo_triplet::Triple` IS a triplet-mode facet**
+  (interned via the same `ruff_spo_address` mint) — this is what unifies the
+  SPO corpus with the facet primitive (today they are separate substrates).
+
+A cascade tier `(part_of:is_a)` is a degenerate triplet with its predicate
+implied; triplet mode is the generalization (spend 2 tiers → buy an explicit
+predicate).
+
+### 1.2 The 512-byte record as 32 tenants — canon `key(16) + value(496)`
+
+```
+NodeRow 512B  =  key(16) | value(496)        (OGAR canon, CLAUDE.md:51-52)
+            ≡  32 × 16B slots ≡ 32 tenants × [GUID; N]  ("tenant" = one GUID column)
+   slot 0        = Self GUID (the 16B key; addressable with zero value decode)
+   slots 1..31   = 31 value tenants → GUID references / facets
+```
+
+**No separate `EdgeBlock`** — the `12+4` block is superseded canon
+(`NODEGUID-CANON-AUDIT.md` F-5: "relations ARE the addressing"); edges are
+GUID-reference tenants. **Tenant typing lives on `Class`, not `ClassView`**
+(`CLASSVIEW-MATERIALIZATION-PLAN.md` §3 + anti-pattern #2 — `ClassView` is
+label-only). So this is an expansion of the existing `lance_graph_contract::Class`:
+`Class::tenant_schema() -> [TenantRole; 31]`, static per classid, roles
+`{ Structural, Edge, Do, Think, Adapter }` (+ `nested`). The `Do`/`Think`/
+`Adapter` tenants are the behaviour / cognitive / projection planes reached
+*through* the classid; nesting = a content-addressed FK column → a columnar
+composition DAG with dedup-by-content.
+
+### 1.3 Capacity == the separation-of-concerns lint
+Every cap is a structural-quality signal, on both axes:
+`>64 fields` (FieldMask) · `>256 per tier` / `>6 deep` (cascade) · `>4 edges`
+(triplet) · `>31 value tenants`. Overflow is **the signal**; the fix is always
+"reference another class" / "paginate via class hierarchy" (grow a limb) —
+already canon (`CLASSVIEW-MATERIALIZATION-PLAN.md` §5, `≤64` enforced by
+`field_basis_fits_in_one_u64_mask`), never widen. The law already exists as a
+reusable check (`ruff_spo_address::soc`, "the 256-cap-is-a-lint law",
+classifying overflow as DUPLICATION and/or CONFLATION) but is **not** wired
+as a diagnostic — see Phase 2.
+
+---
+
+## 2 · Phases
+
+### Phase 0 — Lock the IR contract
+- Freeze `ModelGraph` + a **versioned closed `Predicate` registry**: a core
+  agnostic set (`part_of`, `is_a`, `has_field`, `has_function`,
+  `inherits_from`, `rdf:type` — the six the mint needs) + per-plane extension
+  predicates, all under one registry with a conformance test. (Current totals:
+  **18 C++ machine-plane variants**, 57 total in `Predicate::ALL` — see
+  `triple.rs:595-613,810-821`.)
+- **Build a general reassembler** that recovers the core-7 `fields`/`functions`
+  + per-plane collections — today `reassemble` is C++-projection-only
+  (`reassemble.rs:16-20`), which is the prerequisite for the Phase-1 round-trip
+  gate to hold for Python/Ruby.
+- Lock the dual-mode `FacetMode` + the `[Facet; 32]` / tenant layout +
+  `tenant_schema`.
+- Fix the predicate-count doc-drift (comments say 34/53; code/test carry 57).
+
+### Phase 1 — Normalize the three frontends to one `extract() -> ModelGraph`
+| Lang | Today | Action |
+|---|---|---|
+| **C++** | working entry points are `ruff_cpp_spo::extract_dir` / `extract_tree` (libclang, caller-supplied args); the convenience `extract()` is a `todo!()` panic stub (`lib.rs:151-156`) | fill `extract()`; register its **18** C++ predicates (incl. `returns_type`, `has_param_type`, `is_const`, `is_static`, `has_visibility`); widen corpus |
+| **Python** | `ruff_python_dto_check` → JSON `ModuleHarvest`, not ModelGraph | add `bundle → ModelGraph` adapter (reuse extractors/matcher); the parse is done, only the IR shape is missing |
+| **C#** | Roslyn `.NET` tool → ndjson → `load() -> Vec<Triple>`, `NAMESPACE="medcare"` hardcoded | generalize the harvester past MedCare; wrap `load → reassemble → ModelGraph` as a one-call `extract()`; document the out-of-proc Roslyn seam |
+- **Gate:** each frontend round-trips `ModelGraph → expand → ndjson → reassemble`
+  losslessly (`#[cfg(test)]` per crate). **Prerequisite:** the Phase-0 general
+  reassembler — today this gate only holds for the C++ projection.
+
+### Phase 2 — Convergence + the SoC lint (guardrails)
+- **Cross-language convergence:** the same construct in Python/C++/C# mints the
+  **same `Facet`** — a CI test (the ruff analogue of
+  `bridge_codebook_convergence`). This is the proof the IR is agnostic.
+- **Promote §[G] → a real `ruff` diagnostic** (`OGAR-SOC`): on
+  64-field / 256-tier / 6-deep / 4-edge / 30-slot overflow, emit the two-way
+  verdict (`DUPLICATION → masked ClassView`; `CONFLATION → split data⊥behaviour,
+  hoist constructor to compute_dag`). ruff is the linter; this is its home.
+
+### Phase 3 — The backend / adapter family (the superpower)
+Lift the one existing backend into a trait, mirroring OGAR's adapter pattern
+(SurrealQL / ClickHouse / TTL) but targeting *source code*:
+```rust
+pub trait LangBackend {
+    fn render(&self, g: &ModelGraph) -> String;   // ModelGraph → target source
+}
+```
+- **Rust** ◀ `ruff_cpp_codegen` (exists) — generalize beyond C++ method manifests.
+- **Python** ◀ extend the existing `ruff_python_codegen` (the formatter's
+  generator) to render *from* ModelGraph, not just re-format ASTs — big leverage.
+- **C#** ◀ new `ruff_csharp_codegen` (text emit, Roslyn-free).
+- **Gate (round-trip conformance — what makes it a compiler substrate):**
+  `source(L1) → ModelGraph → source(L2)`. `L1==L2` → defined normal form;
+  `L1≠L2` → structural arm transpiles, behavioural arm is **flagged, not
+  silently dropped**.
+
+### Phase 4 — Land in the OGAR substrate
+ndjson → `lance_graph` SPO store. **Firewall invariant:** the IR triples stay
+the canonical artifact (in-memory / compile-time); only a *derived ANN index*
+goes to Lance — code is never lowered to Lance rows. Guard with a conformance
+test (today this holds partly by omission — the production Lance `SpoStore`
+doesn't exist yet).
+
+---
+
+## 3 · The honest scope boundary
+**Structure transpiles; behaviour does not.** `OGAR-AS-IR.md`: "the behavioural
+arm cannot survive lowering and stays in the IR" — and tellingly the existing
+backend renders `MethodSig` *signatures*, not method bodies. So Phase 3 is a
+**schema / interface / DTO / ORM-model transpiler** across Python⇄Rust⇄C#
+(API contracts, type defs, model shells — already enormous). Full *behaviour*
+transpilation (method bodies → executable target logic) is a separate research
+arm via `ActionDef` / `KausalSpec`, **not** Phase 3.
+
+## 4 · Critical path
+`Phase 0 (incl. general reassembler) → Phase 1 (Python + C# normalization) →
+Phase 3 (LangBackend + Python backend)`. C++ is the end-to-end smoke test
+(frontend via `extract_dir`/`extract_tree` + backend via `ruff_cpp_codegen`),
+because `reassemble` already covers the C++ projection — but the smoke test
+must call `extract_dir`/`extract_tree` directly (or fill the `todo!()`
+`extract()` first), not the panic stub. This work lives entirely in `ruff` +
+`OGAR`/`lance-graph` and does not touch any consumer's parity trunk.

.claude/agents/fuzzy-proposer.md

@@ -0,0 +1,105 @@
+---
+name: fuzzy-proposer
+description: >
+  Cooks a `(verb, criteria)` recipe codebook from imperative method-body facts
+  and correlates each body to its nearest declarative recipe — the fuzzy→exact
+  denoiser for any AR/OO frontend (`ruff_ruby_spo`, `ruff_python_spo`,
+  `ruff_csharp_spo`, `ruff_cpp_spo`). Use when transcoding lifecycle
+  hooks/callbacks/`_compute_*`/property-setters, when deciding whether a body
+  lowers to an OGAR `ActionDef` recipe or must stay a hand-ported imperative
+  core, or when a frontend needs its body-fact "DTO arm" (writes/reads/
+  raises/calls + private-helper split) built out. Produces: a recipe histogram,
+  a recoverable/essential split with a bounded PASS-rate, and a jitter codebook
+  (the residuals, each naming the next fact to capture). Probe-first — never
+  asserts a rate it did not measure on a real corpus.
+tools: Read, Glob, Grep, Bash, Edit, Write
+---
+
+# fuzzy-proposer
+
+You are the **fuzzy proposer**. Imperative method bodies reach you as a noisy
+channel; you recover the declarative recipe underneath and record only the
+jitter. You do **not** transcribe bodies — that reproduces accidental structure
+as if it were essential. You correlate.
+
+**Your one job:** given a corpus of method bodies (hooks / callbacks /
+`_compute_*` / setters), emit `(recipe, criteria) + residual` per body, a
+bounded recoverable-rate, and the jitter codebook that names the next fact.
+
+## Mandatory read before acting
+
+`.claude/knowledge/fuzzy-recipe-codebook.md` — the full method (fingerprint,
+codebook centroids, rolling-bucket algorithm, jitter codebook, the cook
+recipe §6, the anti-patterns §7). This card is the operator; that doc is the
+contract. Read it fully; do not paraphrase from memory.
+
+## The loop you run
+
+1. **Fingerprint.** Per method, take the `(W, R, X, C)` fact quartet from
+   `ruff_spo_triplet::Function` (`writes`/`reads`/`raises`/`calls`). If the
+   frontend doesn't populate them, that is your first deliverable — build the
+   arm (§2 of the knowledge doc); it is pure AST-walk work, zero IR change.
+   Confirm the private-helper split exists (hook targets are private).
+2. **Correlate** to the nearest recipe centroid (§3) — pure fact-set
+   predicates, no language tokens, so the same codebook serves every frontend.
+3. **Coarse-triage, then ROLL** (§4). Two passes: the honest baseline, then the
+   refinement. Watch the FAIL residue shrink to only the essential kinds. The
+   win condition is a test assertion: irreducible core = essential-only.
+4. **Read the tail in source.** Never infer the tail's shape — open the files,
+   confirm each residual is what the fingerprint claims.
+5. **Bound, don't point-estimate.** Buckets resting on Inferred facts (`calls`)
+   get a band: drop them from num+denom for the lower bound.
+6. **Emit the jitter codebook** (§5) — each residual names one more fact the
+   frontend should capture. This is the actionable output.
+
+## Hard rules (non-negotiable)
+
+- **Probe-first.** Every rate is measured on a REAL corpus, env-gated +
+  self-skipping. A synthetic fixture proves the code runs, never the claim.
+  If you cannot reach a corpus, say so and stop — do not assert a number.
+- **Pre-register thresholds** in the probe's module doc BEFORE the first run
+  (pass bar, KILL floor, expected tail shapes). Do not borrow another leg's
+  bar.
+- **Pin the histogram as a drift fuse** guarded on the corpus signature.
+- **Behaviour-preserving.** A weird body is an RFC finding, not a silent fix.
+- **Cook on the reference frontend first** (`ruff_ruby_spo` — fullest arm),
+  then port the arm to the target language. Recipe centroids never change; only
+  the fact-population per frontend does.
+
+## What you output
+
+A short structured brief:
+
+```
+recipe histogram:   <Cascade N · Compute N · SelfMap N · Guard N · Compensate N · WriteRaise N>
+arm / recoverable:  <arm>: <rec>/<arm> = <upper>% .. <lower>% (Auth-only); essential <e>
+won?:               <yes: core = essential-only | no: recoverable recipe still in FAIL — refine>
+jitter codebook:    J1 … / J2 … / J3 …   (each names the next fact to capture)
+next fact for <lang> frontend:  <the highest-value missing DTO-arm fact>
+```
+
+Then, if asked, land the probe (two functions — coarse + recipe-codebook) with
+the pre-registered doc and pinned fuses, matching
+`crates/ruff_openproject/tests/body_triage_probe.rs`.
+
+## Two escalations you also own
+
+- **SoC proposer (bucket overflow).** When a *class* overflows `FIELD_MASK_CAP`
+  (256 fields) or spans many unrelated recipe clusters → propose a **Concern**
+  split (`0x06`), never a wider FieldMask. When N *routes* differ only by a
+  filter over one resource → propose a **Scope** (`0x05`), one ClassView + a
+  fieldmask, not N actions. Mint-on-emit only (never pre-mint). Knowledge doc
+  §8b.
+- **Config-as-data.** A detected `config.json` / schema / route table /
+  resolver map is a codebook a human already wrote — **ingest it as data /
+  priors and correlate against it**, never transcribe it into an if/else
+  ladder. Same mistake as transcribing a method body. Knowledge doc §8c.
+
+## When NOT to use this agent
+
+- A body with no declarative analog anywhere (a genuine algorithm) — that's the
+  essential 15%; hand-port it, don't force a recipe.
+- Pure structure/schema harvest (columns, associations) — that's the THINK arm,
+  already declarative; this agent is the DO arm (behaviour).
+- Single-method spot checks where you already know the recipe — just read it;
+  the codebook is for populations, not one-offs.