From 6f5f82fe296fe59023fb703017fb08385a05c73c Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Thu, 29 Oct 2026 09:37:27 -0700 Subject: [PATCH 01/13] docs(rfc): add the GQ 3 grammar RFC One incompatible change to the GQ grammar, edition 3, after which every later capability is a registry row, a plan node behind existing syntax or the filling of a reserved slot. The RFC carries the freeze list (mutation keyword, arrow traversals, a retrieve clause binding score columns, generic calls over a function registry, the closed reserved set, limit as a parameter, generic annotations, and the reserved declaration kinds, blocks, clauses, meta-fields, catalogue names and namespaced type references), the rewriter that migrates edition-2 sources with an IR-equality proof, and the one-release window in which edition 2 is rewritten at boot instead of quarantined. --- docs/rfcs/2026-10-28-gq-3-grammar.md | 632 +++++++++++++++++++++++++++ docs/rfcs/README.md | 1 + 2 files changed, 633 insertions(+) create mode 100644 docs/rfcs/2026-10-28-gq-3-grammar.md diff --git a/docs/rfcs/2026-10-28-gq-3-grammar.md b/docs/rfcs/2026-10-28-gq-3-grammar.md new file mode 100644 index 000000000..315272d89 --- /dev/null +++ b/docs/rfcs/2026-10-28-gq-3-grammar.md @@ -0,0 +1,632 @@ +--- +rfc: "2026-10-28-gq-3-grammar" +title: "GQ 3 grammar: the one break and the slots it freezes" +track: maintainer +status: draft +implementation: not-started +authors: + - Ragnor Comerford (@ragnorc) +created: 2026-10-28 +updated: 2026-10-29 +discussion: null +supersedes: [] +superseded_by: [] +blocked_on: + - "The in-context competence instrument run on the edition-3 spelling through a parser-only build: first-try validity at or above the current grammar's 37 of 45, and 45 of 45 after one repair" + - "The rewriter turning every `.gqt` case under `crates/omnigraph-gqt/cases/` into edition 3 with no hand edit, the rewritten corpus green, and IR equality on every case" +--- + +# RFC: GQ 3 grammar: the one break and the slots it freezes + +> A term set in ***bold italics*** is defined at that spot; it is plain text +> afterwards. Code references are at `main` `ea895066`. + +## Summary + +GQ makes one incompatible change to its grammar, ***edition 3***, and then +stops making incompatible changes. The release that ships edition 3 lands +every syntactic decision that would otherwise force a later break, including +the ones whose semantics arrive in later RFCs: those land as ***slots***, forms +the parser accepts and the type checker refuses with a stable code naming the +RFC that will serve them. After this release a new capability is a registry +row, a plan node behind existing syntax, or the filling of a slot, and none of +those breaks a stored query. + +The break is defined by a ***freeze list***. It renames the write declaration +`mutation`; spells a traversal with a visible arrow (`$p -worksAt-> $c`); gives +retrieval its own clause (`retrieve { }`) that binds typed score columns, +leaving `order` to sort values; makes every function a generic call resolved +against a registry, so no function remains in the grammar; closes the +reserved-word set at the six logic words; lets `limit` take a parameter; admits +a generic `@name(args)` annotation on any declaration; reserves the +declaration kinds `view`, `contract` and `batch`, the block `optional { }`, the +clauses `having { }` and `returning { }`, the meta-fields `@type_id`, +`@distance` and `@score`, the catalogue type names the schema catalogue will +use, and the namespaced type reference `a::b::C`; reserves the binding form `$x: Name($arg)` and a block aggregate in +operand position, so that derived relations, optional patterns and block +aggregates as values arrive without a grammar change; and adds an optional +`gq 3;` edition line. Schema declarations and query declarations share one +grammar; a `.pg` file is a GQ file holding only schema declarations. + +A ***rewriter***, `omnigraph queries migrate`, turns every edition-2 source +into edition 3 by byte-range edits that preserve comments and layout, and +proves each rewrite by compiling both editions to the same typed IR. The GQ +logic-test corpus, the user guide, the bundled skill and every stored query on +the graphs the maintainers operate are rewritten by that tool before the +release. The release that ships edition 3 still compiles edition 2, rewriting +a stored source at boot with a warning instead of quarantining its graph; the +release after removes the edition-2 grammar. + +What does not change: every semantic contract the language has today (static +closed typing, three-valued logic with no null literal in reads, pattern +conjunction, stratified blocks and the safety rule, reachability semantics for +bounded traversal, filter-before-rank, one total order, index required at +compile time, keyed insert as upsert, one commit per mutation, the +constructive-or-destructive rule, branch and commit statements, the `@` +namespace, the four-field diagnostic), the storage format, the `__manifest` +publication door, the HTTP contract version and the plan-acceptance contract. +Every slot's semantics stays with the RFC that owns it; this RFC owns the +grammar, the rewriter and the edition window, nothing else. + +## Motivation + +GQ is pre-stable, and the project has said so each time it changed the +grammar: the shared expression model's compatibility section states that "GQ is +pre-stable and a better language wins over compatibility wherever the two +conflict". It has paid for that position piecemeal. Reserving `and`, `or`, +`not`, `is` and `null` took the language from 1.0 to 2.0; reserving `in` +followed (PR #803); the operator-vocabulary amendment (PR #805) schedules +infix `contains` and `starts_with` to become calls through a deprecated rule +and two more majors (3.0, then 4.0); the analyzed lexical search RFC (PR #792) +schedules the removal of `search`, `match_text` and `fuzzy` at "the next GQ +language major"; RFC 0048 (PR #793) deprecates the positional `k` of `rrf()` +for one release; the polymorphic types RFC (PR #884) allocates a minor +version and an explain version of its own. Each is a break with its own +deprecation shim, lint warning, corpus edit, skill edit and fleet sweep, and +each is a release in which agents meet two spellings of one operation. + +Three facts make one early break cheaper than many late ones. + +1. **Stored text is small and mechanical to rewrite.** Stored queries are GQ + source re-parsed at boot (`crates/omnigraph-server/src/queries.rs`); a + parse failure quarantines the graph (`settings.rs`). The fleet the + maintainers operate is a few graphs whose stored queries one tool can + rewrite in one pass, and the compiler already emits byte-range + `suggestion` edits with `machine_applicable` applicability for exactly + this purpose ([Diagnostics](../user/queries/diagnostics.md)). +2. **The language's expressiveness gaps are the next several RFCs, and each + needs a slot.** Optional traversal (#658), block aggregates as values + (#1050), heterogeneous discovery (#648), one entity with its neighbourhood + (#649), fusion with weights and thresholds (RFC 0048's deferred list), a + derived relation other queries can bind to, and parameterised paging all + need a clause, a block, a declaration kind or a call form the grammar does + not have. Added one at a time, each is additive only if its position was + already reserved; added after a freeze, each is a slot filled. +3. **The agent is the author, and the author's cost is round trips, not + spelling.** Measured on this grammar, a general agent solved 45 of 45 tasks + with 37 of 45 first queries valid, and every first-try failure was one + missing `()` (RFC 0047, Evidence). Published comparisons of query languages + for LLM generation find a 10 to 45 point zero-shot gap between familiar + and unfamiliar spellings that five retrieved worked examples erase + (Sivasubramaniam et al., SM3-Text-to-Query, NeurIPS 2024), and that better + diagnosis, not more editing, doubles repair success (Olausson et al., ICLR + 2024). A spelling therefore costs an agent one repair turn once; a second + spelling for the same operation costs it on every query it reads. + +A local change cannot do this: the grammar is a compatibility surface +([Compatibility surfaces](2026-09-14-compatibility-surfaces.md)), its +change is wire-visible through every client that sends GQ text, and the +decision to make it the last incompatible change is a lasting contract. + +## User and operational behavior + +### The edition line + +A file may open with `gq 3;` before its settings lines. A file without the +line is read in the compiler's current edition, which this release makes 3. +`gq 2;` selects the edition-2 grammar for one release; any other edition is +refused at parse (`Q00x: unknown edition 4; this release compiles editions 2 +and 3`). The stored-query registry records the edition each source was +validated under, and `queries validate`, `cluster plan` and `lint` report an +edition-2 source with the rewrite that moves it. + +### What a reader sees changed + +Before and after, on one query: + +```gq +// edition 2 +query passages($matter: String, $q: String) { + match { + $r: SourceRevision { matter_number: $matter } + $r hasPassage $p + $p.text contains "tariff" + } + return { $r.path, $p.locator, bm25($p.text, $q) as score } + order { bm25($p.text, $q) desc } + limit 5 +} +``` + +```gq +gq 3; +query passages($matter: String, $q: String, $k: I64 = 5) { + match { + $r: SourceRevision { matter_number: $matter } + $r -hasPassage-> $p + contains($p.text, "tariff") + } + retrieve { $p.lex: bm25($p.text, $q) } + return { $r.path, $p.locator, $p.lex as score } + order { $p.lex desc } + limit $k +} +``` + +The write side: + +```gq +// edition 2 +query retract($slug: String) { + update Assertion set { retracted: true } where slug = $slug +} +``` + +```gq +gq 3; +mutation retract($slug: String) { + update Assertion set { retracted: true } where slug = $slug +} +``` + +### The freeze list + +Every row ships in the edition-3 release. The third column says what the +compiler does with the form in that release; the fourth names the RFC that +owns its semantics. A form marked *slot* parses and is refused by the type +checker with one code, `T6x` (allocated at implementation under the catalogue +rule in `crates/omnigraph-compiler/src/query/codes.rs`), whose `expected` +names the construct and whose `fix` names the owning RFC; a form marked +*works* has today's semantics in the new spelling. + +| Surface | Edition 3 | In this release | Owner of the semantics | +|---|---|---|---| +| Declaration kinds | `query` (reads only), `mutation` (writes), `view`, `contract`, `batch`; the schema kinds `node`, `edge`, `interface` in the same grammar | `query`, `mutation` and the schema kinds work; `view`, `contract`, `batch` are slots | derived relations RFC (`view`), the permissions RFCs (`contract`), the agent-first batch (`batch`) | +| Read clause order | `match` → `retrieve`? → `return` → `having`? → `order`? → `limit`? | `match`, `return`, `order`, `limit` work; `retrieve` works with today's rankings (below); `having` works (a Boolean expression over aggregates and group keys, one filter above the aggregate) | this RFC for `having`; RFC 0048 for everything `retrieve` gains later | +| Mutation clause order | statements → `returning`? | statements work; `returning` is a slot | writes RFC | +| Blocks in `match` | `not { }`, `exists { }`, `count|sum|min|max|avg(…) { } op expr`, `optional { }` | the four work; `optional` is a slot | derived relations RFC | +| Blocks as values | a block aggregate in operand position, `return { count { $n -knows-> $m } as knows }`, `exists { … } as employed` | parses; a slot | derived relations RFC | +| Bindings | `$x: Type { … }` as today; `$x: Name($arg, …) { … }` binds a declared `view` with arguments; `$x: Interface` binds an abstract type | the first works; a binding with arguments is a slot; an interface name is refused as today (`T1`) until its RFC | derived relations RFC (views), polymorphic types RFC (interfaces) | +| Traversal | `$a -knows-> $b`, `$b <-knows- $a`, `$a -knows- $b` (either direction), `$a -$e:knows-> $b`, `$a -(knows\|likes){1,3}-> $b`, `$a -*-> $b`, `$a -"in"-> $b` | works; the triple form `$a knows $b` is a parse error whose fix prints the arrow form | this RFC | +| Calls | `name(positional…, option: value…)` for every function; the registry holds `count`, `sum`, `avg`, `min`, `max`, `now`, `date`, `datetime`, `nearest`, `bm25`, `rrf`, `search`, `fuzzy`, `match_text`, `contains`, `starts_with`; `count(distinct x)` is the one special form | works; the infix `x contains y` and `x starts_with y` are parse errors whose fix prints the call; `rrf(a, b, 60)` is a type error whose fix prints `k: 60` | the operator-vocabulary amendment for the call rule; the function-registry work for `@Function` and new rows | +| Reserved words | `and`, `or`, `not`, `is`, `null`, `in`; every other keyword is contextual (word-bounded, legal as an identifier in operand and property position) | works; a grammar test pins the set | this RFC | +| Null | no literal in a read; `null` legal as the value of an `insert` or `set` assignment | works | this RFC | +| Parameters | `$x: Type? = default`; `limit $k` with a non-null `I64` parameter; the parameter kinds `Branch` and `Commit` | defaults and `limit $k` work; `Branch` and `Commit` parse and are slots | this RFC; the branch-statements RFC for binding them | +| Annotations | `@name(args)` on any declaration, retained as metadata; `@description`, `@instruction`, `@example`, `@if_commit($p)` are typed; `@budget`, `@approximate`, `@satisfice`, `@brief`, `@phase` are reserved | the typed four work (`@if_commit` binds the commit precondition the request carries today); the reserved five are slots | RFC 0048 and the agent-first batch | +| Meta-fields | `@id`, `@src`, `@dst`, `@type`, `@type_id`, `@distance`, `@score` | the first four work; `@type_id` is a slot; `@distance` and `@score` are the typed fields of a `retrieve` arm's column | polymorphic types RFC for `@type_id` | +| Catalogue names | `@Operator`, `@Pass`, `@Setting`, `@Engine`, `@LogicalKind` exist; `@Type`, `@Property`, `@Edge`, `@Interface`, `@Index`, `@Function`, `@View` reserved | the five exist; the seven are slots | the engine catalogue RFC's schema extension | +| Type references | `a::b::C`: zero or more lowercase segments then a type name; the grammar admits it everywhere a type name stands | one or more segments is a slot | schema namespaces RFC (`ns::Type`); a federation RFC for an outermost graph segment | +| Schema roles | `Vector(n, distance="l2"\|"cosine"\|"dot")`, `@analyzed(…)`, `@valid(from, to)`, `@embed(source, model=…, …)` | `distance` works (RFC 0048's rule: omitted means `l2`); `@analyzed` and `@valid` are accepted as SchemaIR features and refused at use | RFC 0048 (distance, recipes), the lexical RFC (`@analyzed`), the writes RFC (`@valid`) | +| Envelope | `graph_commits` (a map keyed by graph id, one entry today) beside `graph_commit_id`; `notifications`, `why_not`, `complete`, `incomplete_reason`, `continuation`, `estimate` | present where the owning stage has shipped, absent otherwise; all additive | the envelope stage RFCs | + +Two things are not on the list because they are additive by nature: a new +registry row (a function, an annotation name, a catalogue type) and a new plan +node behind existing syntax. + +### The `retrieve` clause in this release + +A ***retrieve arm*** binds a score column to a binding: `$p.lex: bm25($p.text, +$q)`. The column's name lives in the binding's namespace beside its +properties; a name that collides with a declared property is a type error. In +this release the arms are exactly today's rankings and keep today's rules: + +| Arm | Column type | Rule carried over | +|---|---|---| +| `$d.vec: nearest($d.embedding, $q)` | `F64?`, the squared L2 distance Lance ranks by; null for a row outside the window | `limit` required (today's `T17`); the window is the query's `limit`, widened by the overfetch ladder as today | +| `$d.lex: bm25($d.text, $q)` | `F64?`, the BM25 score; null for a non-match | the property needs a full-text index (`T27`); eligibility placement as today | + +`order` must lead with an arm or with `rrf(arm, arm [, k: n])`, exactly the +forms `order` accepts today for `nearest(…)`, `bm25(…)` and `rrf(…)`; `asc` on a +`bm25` arm or on `rrf` is refused (`T54`), as is any modifier on a `nearest` +arm. `return` may project an arm as a value (`$d.lex as score`), which is +today's `T33` projection without the repetition rule, since the arm is named +once. A filter in `match` may not read an arm (arms are bound after `match`); +`having` may, and `having { $d.vec < 0.6 }` is the one new capability this +release gives retrieval, because it falls out of scores being values: it is +the same `Filter` the engine already applies when the overfetch ladder drops +rows, now written by the query. Everything else RFC 0048 adds (a `k` on an +arm, more than two `rrf` arms, weights, `norm()`, `exact: true`, a reranker) +is a registry row or an option on a call, not grammar, and lands with that +RFC. + +### The rewriter + +`omnigraph queries migrate` takes `.gq` and `.pg` files or a `--target`, parses +each source under edition 2, and writes edition 3: + +- as byte-range edits (the `suggestion` shape), so comments and layout + survive; +- adding `gq 3;` as the first line; +- refusing to write when the edited text does not compile to a typed IR equal + to the original's (`IRExpr`, `IROp` and their containers derive `Eq` and + `Hash` since the shared expression model), and naming the construct it + could not prove; +- idempotent: a second run changes nothing. + +| Edition 2 | Edition 3 | +|---|---| +| `query name(…) { insert … }` | `mutation name(…) { insert … }` | +| `$a knows $b` | `$a -knows-> $b` | +| `$a $b` | `$a -knows- $b` | +| `$a $e:knows $b` | `$a -$e:knows-> $b` | +| `$a (knows \| likes){1,3} $b` | `$a -(knows \| likes){1,3}-> $b` | +| `$x.title contains "g"` | `contains($x.title, "g")` | +| `$x.tags contains "rust"` (a list property) | `"rust" in $x.tags` | +| `$x.title starts_with "O"` | `starts_with($x.title, "O")` | +| `order { nearest($d.e, $q) }` | `retrieve { $d.nearest: nearest($d.e, $q) } … order { $d.nearest }` | +| `order { bm25($d.t, $q) desc }` | `retrieve { $d.bm25: bm25($d.t, $q) } … order { $d.bm25 desc }` | +| `order { rrf(nearest(…), bm25(…), 60) }` | two arms, `order { rrf($d.nearest, $d.bm25, k: 60) }` | +| `return { …, nearest($d.e, $q) as score }` | `return { …, $d.nearest as score }` | + +Everything not in the table is unchanged text. The generated arm names are +`nearest`, `bm25` and, for a second arm of the same kind, `nearest2`; a name +that collides with a declared property gets a numeric suffix. A human may +rename them afterwards; the rewriter never has to. + +### Operators + +The edition-3 release boots an edition-2 stored source by rewriting it in +memory and logging the edits once per query (a warning with the query's name +and each edit), so no graph is quarantined for this change. `cluster plan` +prints the same edits as a pre-upgrade finding. The release after this one +compiles edition 3 only; an edition-2 source then fails validation with a +diagnostic whose fix is the `queries migrate` command. The bundled skill, the +user guide and the GQT corpus are edition 3 in the release that ships it; the +skill's command index and condensed reference are regenerated, not edited. + +### Refusals + +Every refusal keeps the four-field diagnostic. New codes, allocated at +implementation: + +- `Q00x` unknown edition; `Q00x` an edition line after another statement; +- the triple traversal, the infix string operators and `query` with a mutation + body are parse or type errors whose `fix` is the rewritten text (a + `machine_applicable` suggestion); +- `T6x` reserved construct: `view`, `contract`, `batch`, `optional { }`, a + binding with arguments, a block aggregate as a value, `returning`, + `@type_id`, the reserved annotations and catalogue names, a namespaced type + reference, a `Branch` or `Commit` parameter; `expected` + names the construct, `fix` names the owning RFC and says "not yet served in + this release"; +- a `retrieve` arm name colliding with a property; an arm read in `match`; an + `order` that does not lead with an arm or an `rrf` over arms when a + `retrieve` clause is present. + +## Design + +### One grammar for schema and queries + +`query.pest` and `schema.pest` become one grammar with one `file` rule: + +```pest +file = { SOI ~ edition? ~ setting_stmt* ~ (statement | declaration*) ~ EOI } +edition = { kw_gq ~ integer ~ ";" } +declaration = { schema_decl | query_decl | mutation_decl | view_decl | contract_decl | batch_decl } +``` + +The schema rules move unchanged except for `type_name`, which becomes a +`type_ref` admitting namespace segments (below), and `vector_type`, which +admits `distance=`. A `.pg` file is a `file` holding only `schema_decl`s; the +loader keeps the extension and `cluster.yaml` keeps its `schema:` key, both +of which now parse with the one grammar. Nothing a `.pg` file accepted today +is refused. + +### Declarations + +```pest +query_decl = { kw_query ~ ident ~ param_list ~ annotation* ~ "{" ~ read_body ~ "}" } +mutation_decl = { kw_mutation ~ ident ~ param_list ~ annotation* ~ (as_commit_clause | as_unattached_clause)? ~ "{" ~ mutation_body ~ "}" } +view_decl = { kw_view ~ type_ref ~ param_list ~ annotation* ~ "{" ~ read_body ~ "}" } +contract_decl = { kw_contract ~ type_ref ~ annotation* ~ balanced_block } +batch_decl = { kw_batch ~ ident ~ annotation* ~ "{" ~ declaration* ~ "}" } +param_list = { "(" ~ (param ~ ("," ~ param)*)? ~ ")" } +param = { variable ~ ":" ~ type_ref_or_kind ~ ("=" ~ literal)? } +``` + +`query` with a mutation body, or `mutation` with a read body, is a type error +with the rewrite as its fix; the parser accepts both so the message can name +the construct. A `view` has a type name because it is bound as one. A +`contract` body is a `balanced_block`, a brace-balanced span the parser keeps +as text and the type checker refuses; the permissions RFC defines what it +holds, and until then nothing inside it is interpreted. `batch` holds +declarations; it is refused as a whole. + +### Read body and the retrieve clause + +```pest +read_body = { match_clause ~ retrieve_clause? ~ return_clause ~ having_clause? ~ order_clause? ~ limit_clause? } +retrieve_clause = { kw_retrieve ~ "{" ~ retrieve_arm+ ~ "}" } +retrieve_arm = { variable ~ "." ~ ident ~ ":" ~ call ~ ","? } +having_clause = { kw_having ~ "{" ~ expr ~ "}" } +limit_clause = { kw_limit ~ (integer | variable) } +clause = { negation | optional_block | subquery_predicate | exists_block | binding | traversal | filter } +optional_block = { kw_optional ~ "{" ~ clause+ ~ "}" } +binding = { variable ~ ":" ~ type_ref ~ ("(" ~ (expr ~ ("," ~ expr)*)? ~ ")")? ~ ("{" ~ prop_match_list ~ "}")? } +block_value = { (agg_name ~ ("(" ~ expr ~ ")")? | kw_exists) ~ subquery_block } +``` + +A `binding` with an argument list names a `view`; the type checker refuses it +in this release. `block_value` joins `operand` (below) so a block aggregate +can stand where a value stands; the type checker refuses it in this release +and the derived relations RFC gives it the `SubqueryAggregate` the engine +already runs for the predicate form. Both are listed so that compositional +closure, the capability the next RFC adds, needs no grammar change: a view +declaration, a view binding, an optional block, a block as a value and +`having` all parse today. + +An arm's column is a `PropAccess` whose property is the arm name; the type +checker records it in the binding's scope with the arm's result type and +refuses a read of it inside `match`. Lowering maps an arm to today's logical +nodes (`Nearest`, `TextSearch`, `RankFuse` in `crates/omnigraph-planner/src/logical.rs`) +and the arm's column to the score column the ranked scan already appends +(`_distance`, `_score`, `_rrf`), under the arm's name. The planner's search +node selection (`optimizer.rs` `search_node`) reads the leading `order` key as +it does today, finding an arm reference or an `rrf` over arm references where +it found a call. `having` lowers to a `Filter` above `Aggregate`, or above the +retrieval when the query has no aggregate; both are nodes the planner has. +Plan acceptance gains the arm as a named score origin (RFC 0047's "score +meaning" requirement already binds a projected score to its producer; the arm +is that binding made explicit). The explain document version advances because +score columns are named by the query. + +### Traversal + +```pest +traversal = { variable ~ edge_part ~ variable } +edge_part = { "-" ~ edge_core ~ "->" | "<-" ~ edge_core ~ "-" | "-" ~ edge_core ~ "-" } +edge_core = { edge_binding? ~ edge_sel ~ traversal_bounds? } +edge_sel = { edge_alternation | edge_wildcard | edge_ident | string_lit } +``` + +The three parts carry direction, so `undirected_edge`'s angle brackets +disappear and the type checker's endpoint rule (`T5`) checks the written +direction against the declared endpoints instead of inferring it. `-e-` +keeps today's rule for the undirected form: same node type at both ends. The +grammar has no `-` operator in edition 3, so no ambiguity with arithmetic +exists in this release; the function-registry work that adds arithmetic +inherits one rule, stated here so it is not rediscovered: inside `match`, a +`-` or `<-` directly before an `edge_core` and followed by `->` or `-` is a +traversal, tried before `filter` in the ordered choice as `traversal` is +today; an arithmetic minus takes a value operand (`$x.p`, a literal, a call, a +parenthesised expression), never a bare edge identifier, so the two never +match the same text. + +### Calls and the reserved set + +```pest +call = { call_name ~ "(" ~ call_args? ~ ")" } +call_name = @{ !reserved ~ ident } +call_args = { positional ~ ("," ~ positional)* ~ ("," ~ option)* | option ~ ("," ~ option)* } +option = { ident ~ ":" ~ expr } +count_call = { "count" ~ "(" ~ kw_distinct ~ expr ~ ")" } +operand = { "(" ~ expr ~ ")" | block_value | count_call | call | prop_access | meta_field | variable | literal | reserved_property | expr_ident } +reserved = { kw_and | kw_or | kw_not | kw_is | kw_null | kw_in } +``` + +The rules `now_call`, `search_call`, `fuzzy_call`, `match_text_call`, +`bm25_call`, `rrf_call`, `rank_expr`, `agg_call`, `agg_func`, +`nearest_ordering`, `date_lit` and `datetime_lit` are deleted; `date("…")` and +`datetime("…")` are calls the compiler folds to literals as `ir/fold.rs` folds +constants today. A call resolves in the type checker against a ***function +registry***: name, positional parameter types, named options with their types +and ranges, result type rule, kind (`scalar`, `aggregate`, `predicate`, +`ranking`), and the two planner facts `can_fail` and `volatile`. In this +release the registry holds exactly the functions the grammar held, with their +current type rules (`T8`, `T19`, `T20`, `T21`, `T27`, `T33`, `T38` keep their +texts, re-anchored to registry rows), and `rrf` admits `k:` as its one option. +The registry is a Rust table in `omnigraph-compiler`; the catalogue type +`@Function` that exposes it is a reserved name here and lands with the schema +catalogue. Block aggregates (`count { } > n`) keep their keyword form: they +are logic over a pattern, not calls, as the operator-vocabulary amendment +classifies them. `count(distinct x)` keeps its special form because `distinct` +is contextual and the form is the one every comparator spells the same way. + +A grammar test pins the reserved set: every keyword outside the six is used +as a property name after a dot, as an edge name in a traversal, as a return +alias and as a bare property in a mutation `where`, and parses. + +### Annotations, meta-fields, catalogue names, type references + +```pest +annotation = { "@" ~ ident ~ ("(" ~ annotation_args ~ ")")? } +meta_field = @{ "@" ~ ident } +type_ref = @{ (ident ~ "::")* ~ "@"? ~ ASCII_ALPHA_UPPER ~ (ASCII_ALPHANUMERIC | "_")* } +``` + +Annotations are generic on every declaration and resolve against an +***annotation registry*** of the same shape as the function registry (typed +arguments; `scalar` or `declaration` scope); an unknown name is retained as +metadata as `.pg` does today. Meta-fields resolve against a closed list in the +type checker; `@type_id` is in the list and refused. Catalogue type names +resolve against the catalogue registry; the seven reserved names are in it and +refused. A `type_ref` with a namespace segment resolves to a refusal until the +schema namespaces RFC lands; the grammar admits it so that RFC adds no rule, +and the federation reservation is the sentence that an outermost segment may +later name a graph. + +### Parameters, limit, null + +`param` gains a default literal and the kinds `Branch` and `Commit` in +`type_ref_or_kind`; the binder refuses the two kinds in this release. `limit` +takes an `integer` or a `variable` declared `I64` non-null; a null or negative +value is a bind-time refusal. `null` is admitted by the grammar only as the +whole value of an assignment (`mutation_assignment` and `prop_match`), where it +is `Literal::Null` of the property's type, which the write path already +handles for a null constant result; a read filter containing `null` keeps +today's parse error. + +### Envelope + +`ReadOutput` gains `graph_commits: BTreeMap` keyed by graph id +(the request's graph when served, the store's graph identity when embedded), +with one entry today, beside the existing `graph_commit_id`; the other names +in the freeze list are reserved as optional fields and omitted while their +stage has not shipped. All are additive under the HTTP conventions +([HTTP API conventions](2026-10-19-http-api-conventions.md)); the HTTP contract +version does not change, because the edition travels inside the query text. +`openapi.json` regenerates. + +### The rewriter + +`queries migrate` is a compiler-owned pass over the edition-2 AST, which the +compiler keeps for one release behind the `edition` token. Each rewrite is a +list of `(start, end, replacement)` edits in UTF-8 byte offsets of the original +source, the `suggestion` shape; the tool applies them highest offset first, +prepends the edition line, parses the result under edition 3, lowers both to +IR and compares. Comments survive because the pass never re-prints. The server +runs the same pass at boot for an edition-2 stored source, keeps the rewritten +text in memory for the registry, and logs the edits; the on-disk source in +the cluster bundle is the operator's to rewrite with the tool. + +### What is inherited and what is added + +Nothing here reads or writes Lance; no storage surface changes. Added by +Omnigraph: the grammar, the edition token, the two registries as compiler +tables, the arm-as-column lowering onto existing plan nodes, the `having` +filter placement, the rewriter and its IR-equality proof, and the boot-time +rewrite. + +## Invariants + +- **9, query semantics are typed structures.** Strengthened: functions, + annotations and catalogue names leave the grammar for typed registries; a + score is a typed column with a named producer; nothing new travels in + strings. +- **8, integrity failures are loud.** Every slot refuses with a stable code + and a fix; the rewriter refuses to write a rewrite it cannot prove; an + edition it does not know is a refusal, never a guess. +- **13, evidence matches the boundary.** The grammar's evidence is the corpus + rewritten mechanically and green, plus IR equality; the spelling's evidence + is the competence instrument on the new spelling before the break. +- **3 and 4, one coherent view, one publication.** Untouched; no read or write + path changes. +- **11, bounded resources.** Untouched; `limit $k` binds to the same `Limit`. + +Deny-list items touched: none. The rewriter is a compiler pass, not a side +channel; `having` is a `Filter` node, not a string. + +## Compatibility and reversibility + +- **This RFC is the break.** The spellings it removes are the triple + traversal, `query` for a write, the infix string operators and the positional + `rrf` constant; every other edition-2 text parses unchanged under edition 3 + (the reserved set does not grow, every new keyword is contextual). +- **One release of overlap.** The edition-3 release compiles edition 2 and + rewrites it at boot with a warning; the next release removes the edition-2 + grammar and refuses it with the `queries migrate` fix. `GQ_LANGUAGE_VERSION` + becomes the edition and moves to `(3, 0)`; the compatibility-surfaces hash + file, when it lands, pins the edition-3 rules. +- **Wire.** Additive envelope fields; no HTTP contract bump; explain and + bound-plan versions advance once (named score columns, `having`). +- **Storage.** None. +- **Stored queries.** Every stored query in the fleet is rewritten by the tool + before the release ships (Evidence); a stored source the sweep missed is + rewritten at boot in the overlap release. +- **Skill, docs, corpus.** Rewritten by the tool in the same release; + `skills/omnigraph` byte identity is checked by the existing owner. +- **Reverting.** Before the overlap release ends, revert restores the + edition-2 grammar and the old spellings' rules; the rewriter has no inverse + and none is built, since a stored edition-3 source would have to be + rewritten by hand. After the edition-2 grammar is removed, reverting is a new + edition. + +## Alternatives + +| Alternative | Why not | +|---|---| +| Additive first, break last: ship every new form beside the old one with a deprecation, remove the old ones in a final major | Every construct built twice, every corpus and skill edit made twice, agents reading two spellings for a release per construct; three open RFCs already schedule four majors this way | +| Keep rankings in `order` (RFC 0048's "adds no clause and no stage") | One token selects the population and sorts the rows; a score cannot be a filter; `candidates`, `weight`, `exact` and `k` become grammar on the call; the aggregate and `distinct` cases need a special rule. A named arm makes every one of those an expression or an option | +| Keep the triple traversal and let the schema fix direction | Direction invisible at the reading site; `T5` needs a schema lookup to understand; `(p)-[:WORKS_AT]->(c)` is what the model's prior reads | +| Keep `query` for writes | Agents trained on GraphQL write `mutation { }` and the skill documents the trap; a keyword lets lint classify before the body | +| A separate `where { }` clause for filters | Rejected by the shared expression model for reasons that hold: conditions belong beside the pattern; retrieval is different because an arm is not a condition on one row | +| `graph@ns::Type` for the federation segment | A second delimiter where the namespace RFC's `::` already nests; an outermost segment is enough and needs no grammar now | +| Permanent two-edition compiler | Not needed for one break; the registry records the edition, and a second break, if one comes, builds the N-1 arm then | +| One RFC for grammar and semantics (the PR #606 shape) | Closed on 2026-09-28 because it could not be accepted or evidenced piecemeal; this RFC owns the grammar and the window and names the owner of every slot | +| Do nothing | The next five RFCs each break the language once | + +## Evidence and tests + +Existing owners, per [the testing map](../dev/testing.md): + +- **Compiler parser tests** (`query/parser_tests.rs`, `schema/parser_tests.rs`): + one test per freeze-list row; the reserved-set test; the traversal-versus- + minus rule as a parser test even though no minus exists yet; `balanced_block` + nesting; the edition line in every position. +- **Type-check tests**: every slot's refusal with its code and fix; arm + collision with a property; an arm read in `match`; `query` with a mutation + body. +- **GQT corpus** (`crates/omnigraph-gqt/cases/`): the whole corpus rewritten by + `queries migrate` with zero hand edits, run green; a hand edit is a rewriter + defect, fixed in the tool. New cases: `having` over an aggregate and over a + retrieval; `limit $k`; `null` in an assignment; one `expect error` case per + slot, written now so the owning RFC flips it. +- **Rewriter tests**: IR equality for every corpus case and every fleet stored + query; idempotence; comment and layout preservation on a fixture with every + construct; refusal on an unprovable rewrite. +- **Planner acceptance** (`crates/omnigraph-planner/src/validate/tests.rs`): + the arm as a score origin; `having` placement; explain version. +- **Server** (`stored_queries`, `boot_settings`): an edition-2 stored source + boots with the warning and the rewritten registry entry; `queries validate` + and `cluster plan` print the edits; the release-after refusal. +- **Skill and docs**: `integration/cli_schema_config.rs` byte identity of the + exported skill; `scripts/check-docs.py`; the user guide's every example + compiled by the existing doc-example owner. +- **The competence instrument** (RFC 0047, Evidence): run on a parser-only + edition-3 build before this RFC is accepted; thresholds in `blocked_on`. + Its task set stays outside the repository as before; the parser build and + its results are recorded in this RFC's decision log. + +Surveyed and not changed: Lance surfaces (none read); the bound-plan envelope +(version advance only). + +## Rollout + +1. **Parser-only build and measurement.** The edition-3 grammar behind the + `edition` token with every slot refusing; the competence instrument run on + it; the freeze list amended if the measurement says so. `implementation` + stays `not-started`. +2. **Compiler, registries, lowering, rewriter.** One PR series: grammar, the + function and annotation registries holding today's functions and + annotations, arm lowering and `having`, the edition token, the rewriter with + IR-equality proof. `implementation` to `in-progress`. +3. **Corpus, docs, skill, fleet.** The corpus rewritten by the tool; the user + guide and skill rewritten; `cluster plan` on the maintainers' graphs clean. +4. **Release N.** Edition 3 the default; edition 2 rewritten at boot with a + warning. `implementation` to `partial`. +5. **Release N+1.** Edition-2 grammar removed; `GQ_LANGUAGE_VERSION` `(3, 0)` + is the edition. `implementation` to `complete`. + +Each stop is independently safe: step 1 changes nothing a user sees; step 2 +is inert until a file says `gq 3;` or the default moves; steps 3 and 4 are +the release; step 5 is the removal every deprecation ends in. + +## Unresolved questions + +1. The arm spelling: `$d.lex: bm25($d.text, $q)` (column first, as this draft + writes it) or `bm25($d.text, $q) as $d.lex` (call first, mirroring `return`). + The parser-only measurement decides. +2. Whether `count(distinct x)` stays the one special call form or becomes a + registry function `count_distinct(x)`; the former is kept here for + familiarity. +3. The `graph_commits` key for an embedded target with no graph id. + +## Decision log + +- 2026-10-28: drafted from the evaluation of GQ against SQL, GQL, Cypher, + GraphQL, Datalog, Elasticsearch, LanceDB and Vespa, its literature review, + and the maintainer's decision to make every incompatible change first and + once. The freeze list is the set of syntactic decisions the next RFCs + (derived relations and optional patterns, the RFC 0048 rewrite, the lexical + RFC, the writes RFC, the permissions RFCs, the polymorphic types RFC, schema + namespaces) would otherwise each break the grammar to make. diff --git a/docs/rfcs/README.md b/docs/rfcs/README.md index 5b8afb7a7..58deb5108 100644 --- a/docs/rfcs/README.md +++ b/docs/rfcs/README.md @@ -254,3 +254,4 @@ then dated RFCs by date. | [2026-10-19](2026-10-19-http-api-conventions.md) | HTTP API conventions | maintainer | accepted | complete | | [2026-10-19](2026-10-19-storage-outcomes-and-restartable-ownership.md) | Storage outcomes per operation and restartable ownership | maintainer | accepted | in-progress | | [2026-10-20](2026-10-20-operator-connections-and-credential-providers.md) | Operator profiles and credential providers | maintainer | draft | in-progress | +| [2026-10-28](2026-10-28-gq-3-grammar.md) | GQ 3 grammar: the one break and the slots it freezes | maintainer | draft | not-started | From 6e8f38dadc9adfac160b13a845947dfaf1b22513 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Thu, 29 Oct 2026 09:38:16 -0700 Subject: [PATCH 02/13] docs(rfc): record the GQ 3 grammar RFC's discussion link --- docs/rfcs/2026-10-28-gq-3-grammar.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/rfcs/2026-10-28-gq-3-grammar.md b/docs/rfcs/2026-10-28-gq-3-grammar.md index 315272d89..ec42ae7c6 100644 --- a/docs/rfcs/2026-10-28-gq-3-grammar.md +++ b/docs/rfcs/2026-10-28-gq-3-grammar.md @@ -8,7 +8,7 @@ authors: - Ragnor Comerford (@ragnorc) created: 2026-10-28 updated: 2026-10-29 -discussion: null +discussion: "https://github.com/ModernRelay/omnigraph/pull/1202" supersedes: [] superseded_by: [] blocked_on: From 44cd4b34d5e5914d906b5f6d75d61b91cb70cf89 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Thu, 29 Oct 2026 09:39:54 -0700 Subject: [PATCH 03/13] docs(rfc): keep the RFC registry in date order --- docs/rfcs/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/rfcs/README.md b/docs/rfcs/README.md index 4376fbc2d..b494c9966 100644 --- a/docs/rfcs/README.md +++ b/docs/rfcs/README.md @@ -254,6 +254,6 @@ then dated RFCs by date. | [2026-10-19](2026-10-19-http-api-conventions.md) | HTTP API conventions | maintainer | accepted | complete | | [2026-10-19](2026-10-19-storage-outcomes-and-restartable-ownership.md) | Storage outcomes per operation and restartable ownership | maintainer | accepted | in-progress | | [2026-10-20](2026-10-20-operator-connections-and-credential-providers.md) | Operator profiles and credential providers | maintainer | draft | in-progress | -| [2026-10-29](2026-10-29-remote-optimize-through-the-serving-owner.md) | Remote optimize through the serving owner | maintainer | draft | not-started | | [2026-10-28](2026-10-28-gq-3-grammar.md) | GQ 3 grammar: the one break and the slots it freezes | maintainer | draft | not-started | +| [2026-10-29](2026-10-29-remote-optimize-through-the-serving-owner.md) | Remote optimize through the serving owner | maintainer | draft | not-started | ||||||| ea895066 From 52e631ff2f93108c14280a7a7c5dbf9d6eda32fa Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Thu, 29 Oct 2026 09:40:16 -0700 Subject: [PATCH 04/13] docs(rfc): remove a leftover merge marker from the registry --- docs/rfcs/README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/rfcs/README.md b/docs/rfcs/README.md index b494c9966..dca5c7c20 100644 --- a/docs/rfcs/README.md +++ b/docs/rfcs/README.md @@ -256,4 +256,3 @@ then dated RFCs by date. | [2026-10-20](2026-10-20-operator-connections-and-credential-providers.md) | Operator profiles and credential providers | maintainer | draft | in-progress | | [2026-10-28](2026-10-28-gq-3-grammar.md) | GQ 3 grammar: the one break and the slots it freezes | maintainer | draft | not-started | | [2026-10-29](2026-10-29-remote-optimize-through-the-serving-owner.md) | Remote optimize through the serving owner | maintainer | draft | not-started | -||||||| ea895066 From 1c3584b2cea94a72a549bae16019ef3b1bfc4fee Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Thu, 29 Oct 2026 09:45:03 -0700 Subject: [PATCH 05/13] docs(rfc): drop the reserved contract declaration until permissions are decided --- docs/rfcs/2026-10-28-gq-3-grammar.md | 38 ++++++++++++++++++---------- 1 file changed, 24 insertions(+), 14 deletions(-) diff --git a/docs/rfcs/2026-10-28-gq-3-grammar.md b/docs/rfcs/2026-10-28-gq-3-grammar.md index ec42ae7c6..ea1ffc382 100644 --- a/docs/rfcs/2026-10-28-gq-3-grammar.md +++ b/docs/rfcs/2026-10-28-gq-3-grammar.md @@ -39,7 +39,7 @@ leaving `order` to sort values; makes every function a generic call resolved against a registry, so no function remains in the grammar; closes the reserved-word set at the six logic words; lets `limit` take a parameter; admits a generic `@name(args)` annotation on any declaration; reserves the -declaration kinds `view`, `contract` and `batch`, the block `optional { }`, the +declaration kinds `view` and `batch`, the block `optional { }`, the clauses `having { }` and `returning { }`, the meta-fields `@type_id`, `@distance` and `@score`, the catalogue type names the schema catalogue will use, and the namespaced type reference `a::b::C`; reserves the binding form `$x: Name($arg)` and a block aggregate in @@ -191,7 +191,7 @@ names the construct and whose `fix` names the owning RFC; a form marked | Surface | Edition 3 | In this release | Owner of the semantics | |---|---|---|---| -| Declaration kinds | `query` (reads only), `mutation` (writes), `view`, `contract`, `batch`; the schema kinds `node`, `edge`, `interface` in the same grammar | `query`, `mutation` and the schema kinds work; `view`, `contract`, `batch` are slots | derived relations RFC (`view`), the permissions RFCs (`contract`), the agent-first batch (`batch`) | +| Declaration kinds | `query` (reads only), `mutation` (writes), `view`, `batch`; the schema kinds `node`, `edge`, `interface` in the same grammar | `query`, `mutation` and the schema kinds work; `view` and `batch` are slots | derived relations RFC (`view`), the agent-first batch (`batch`) | | Read clause order | `match` → `retrieve`? → `return` → `having`? → `order`? → `limit`? | `match`, `return`, `order`, `limit` work; `retrieve` works with today's rankings (below); `having` works (a Boolean expression over aggregates and group keys, one filter above the aggregate) | this RFC for `having`; RFC 0048 for everything `retrieve` gains later | | Mutation clause order | statements → `returning`? | statements work; `returning` is a slot | writes RFC | | Blocks in `match` | `not { }`, `exists { }`, `count|sum|min|max|avg(…) { } op expr`, `optional { }` | the four work; `optional` is a slot | derived relations RFC | @@ -293,7 +293,7 @@ implementation: - the triple traversal, the infix string operators and `query` with a mutation body are parse or type errors whose `fix` is the rewritten text (a `machine_applicable` suggestion); -- `T6x` reserved construct: `view`, `contract`, `batch`, `optional { }`, a +- `T6x` reserved construct: `view`, `batch`, `optional { }`, a binding with arguments, a block aggregate as a value, `returning`, `@type_id`, the reserved annotations and catalogue names, a namespaced type reference, a `Branch` or `Commit` parameter; `expected` @@ -312,7 +312,7 @@ implementation: ```pest file = { SOI ~ edition? ~ setting_stmt* ~ (statement | declaration*) ~ EOI } edition = { kw_gq ~ integer ~ ";" } -declaration = { schema_decl | query_decl | mutation_decl | view_decl | contract_decl | batch_decl } +declaration = { schema_decl | query_decl | mutation_decl | view_decl | batch_decl } ``` The schema rules move unchanged except for `type_name`, which becomes a @@ -328,7 +328,6 @@ is refused. query_decl = { kw_query ~ ident ~ param_list ~ annotation* ~ "{" ~ read_body ~ "}" } mutation_decl = { kw_mutation ~ ident ~ param_list ~ annotation* ~ (as_commit_clause | as_unattached_clause)? ~ "{" ~ mutation_body ~ "}" } view_decl = { kw_view ~ type_ref ~ param_list ~ annotation* ~ "{" ~ read_body ~ "}" } -contract_decl = { kw_contract ~ type_ref ~ annotation* ~ balanced_block } batch_decl = { kw_batch ~ ident ~ annotation* ~ "{" ~ declaration* ~ "}" } param_list = { "(" ~ (param ~ ("," ~ param)*)? ~ ")" } param = { variable ~ ":" ~ type_ref_or_kind ~ ("=" ~ literal)? } @@ -336,11 +335,17 @@ param = { variable ~ ":" ~ type_ref_or_kind ~ ("=" ~ literal)? } `query` with a mutation body, or `mutation` with a read body, is a type error with the rewrite as its fix; the parser accepts both so the message can name -the construct. A `view` has a type name because it is bound as one. A -`contract` body is a `balanced_block`, a brace-balanced span the parser keeps -as text and the type checker refuses; the permissions RFC defines what it -holds, and until then nothing inside it is interpreted. `batch` holds -declarations; it is refused as a whole. +the construct. A `view` has a type name because it is bound as one. `batch` +holds declarations; it is refused as a whole. + +No declaration kind is reserved for permissions. The permissions model is +not accepted (two drafts, PR #951 and PR #1197, propose different units of +authority), and nothing is lost by waiting: a file-level declaration keyword +is additive whenever it arrives, because a file can never begin with a bare +identifier, so a later `contract`, `policy` or any other kind joins +`declaration` as one more ordered-choice alternative without a break. The +freeze list reserves only forms whose later addition *would* break stored +text or whose shape the next RFCs already depend on. ### Read body and the retrieve clause @@ -561,8 +566,8 @@ Existing owners, per [the testing map](../dev/testing.md): - **Compiler parser tests** (`query/parser_tests.rs`, `schema/parser_tests.rs`): one test per freeze-list row; the reserved-set test; the traversal-versus- - minus rule as a parser test even though no minus exists yet; `balanced_block` - nesting; the edition line in every position. + minus rule as a parser test even though no minus exists yet; the edition + line in every position. - **Type-check tests**: every slot's refusal with its code and fix; arm collision with a property; an arm read in `match`; `query` with a mutation body. @@ -628,5 +633,10 @@ the release; step 5 is the removal every deprecation ends in. and the maintainer's decision to make every incompatible change first and once. The freeze list is the set of syntactic decisions the next RFCs (derived relations and optional patterns, the RFC 0048 rewrite, the lexical - RFC, the writes RFC, the permissions RFCs, the polymorphic types RFC, schema - namespaces) would otherwise each break the grammar to make. + RFC, the writes RFC, the polymorphic types RFC, schema namespaces) would + otherwise each break the grammar to make. +- 2026-10-29: removed the reserved `contract` declaration kind at the + maintainer's direction, since the permissions model is not accepted; a + file-level declaration keyword can be added later without a break, as the + Design section now states. The permissions RFCs are no longer named as slot + owners. From bf92611304e80db1d0a24ff55007d4b05a970775 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Thu, 29 Oct 2026 11:01:15 -0700 Subject: [PATCH 06/13] docs(rfc): say why GQ is being revised as a whole --- docs/rfcs/2026-10-28-gq-3-grammar.md | 40 ++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/docs/rfcs/2026-10-28-gq-3-grammar.md b/docs/rfcs/2026-10-28-gq-3-grammar.md index ea1ffc382..24330b5a6 100644 --- a/docs/rfcs/2026-10-28-gq-3-grammar.md +++ b/docs/rfcs/2026-10-28-gq-3-grammar.md @@ -70,6 +70,46 @@ grammar, the rewriter and the edition window, nothing else. ## Motivation +GQ grew one capability at a time. Each change answered the request in front +of it: a logic word reserved when a filter needed it, a ranking function placed +in `order` because `order` was where a score was first wanted, a search +function added as a keyword, an operator vocabulary amended in place, an +explain version allocated per feature, a deprecation shim per rename. Each was +right on its own terms, and none was asked to fit a shape the next five +changes would also fit. The language therefore carries decisions whose only +justification is the order they arrived in: retrieval spelled as a sort key, +functions that are grammar productions instead of registry entries, a reserved +set that grew by one word per release, and several open RFCs that each +schedule a break of their own. A whole-language review against SQL, GQL, +Cypher, GraphQL, Datalog and the query languages of Elasticsearch, LanceDB and +Vespa found the semantics sound and the accretion the problem: every +expressiveness gap is a position the grammar never reserved, and every +scheduled break is a decision that could be made once. + +This RFC opens a revision that treats the language as a whole rather than as +a queue of requests. Its aim is a foundation with four properties, each of +which a later section of this document or a named follow-up RFC is +responsible for. + +- **Evolvable.** Every later capability lands without a break because its + position already exists: a reserved clause, block, declaration kind, call + form or envelope slot ([The freeze list](#the-freeze-list)). +- **Maintainable.** One grammar for schema and queries, one registry for + functions, one annotation form, one diagnostic contract, so a change has + one place to land and one surface to document ([Design](#design)). +- **Expressive.** The forms the open gaps need, derived relations, optional + patterns, block aggregates as values, retrieval as typed columns, post-group + filters and pattern-scoped writes, have slots whose semantics the closure, + retrieval and write RFCs fill without touching the grammar. +- **Extensible.** A registry row or a declaration is something an agent can + author against a graph; a new function, view or annotation then extends the + language without a compiler release, which is the property that matters as + agents become the main authors of GQ. + +The rest of this section records why the project has not had these +properties, and why one early break buys them more cheaply than the breaks +already scheduled. + GQ is pre-stable, and the project has said so each time it changed the grammar: the shared expression model's compatibility section states that "GQ is pre-stable and a better language wins over compatibility wherever the two From 6dc6315d3bc73217723a5e916312bd94b4970943 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Thu, 29 Oct 2026 13:25:12 -0700 Subject: [PATCH 07/13] docs(rfc): one arrow per traversal, one spelling per meaning, TypeDB review --- docs/rfcs/2026-10-28-gq-3-grammar.md | 115 ++++++++++++++++++++++++--- 1 file changed, 102 insertions(+), 13 deletions(-) diff --git a/docs/rfcs/2026-10-28-gq-3-grammar.md b/docs/rfcs/2026-10-28-gq-3-grammar.md index 24330b5a6..83551bb45 100644 --- a/docs/rfcs/2026-10-28-gq-3-grammar.md +++ b/docs/rfcs/2026-10-28-gq-3-grammar.md @@ -33,7 +33,8 @@ row, a plan node behind existing syntax, or the filling of a slot, and none of those breaks a stored query. The break is defined by a ***freeze list***. It renames the write declaration -`mutation`; spells a traversal with a visible arrow (`$p -worksAt-> $c`); gives +`mutation`; spells a traversal with one visible arrow (`$p -worksAt-> $c`, source on +the left, `-e-` for either orientation); gives retrieval its own clause (`retrieve { }`) that binds typed score columns, leaving `order` to sort values; makes every function a generic call resolved against a registry, so no function remains in the grammar; closes the @@ -87,7 +88,7 @@ expressiveness gap is a position the grammar never reserved, and every scheduled break is a decision that could be made once. This RFC opens a revision that treats the language as a whole rather than as -a queue of requests. Its aim is a foundation with four properties, each of +a queue of requests. Its aim is a foundation with five properties, each of which a later section of this document or a named follow-up RFC is responsible for. @@ -105,6 +106,13 @@ responsible for. author against a graph; a new function, view or annotation then extends the language without a compiler release, which is the property that matters as agents become the main authors of GQ. +- **One way.** One spelling per meaning, in Go's sense ("a language needs + only one conditional control flow construct"): a respelling that adds no + meaning is absent from the grammar and named by the diagnostics; a form + that falls out of a general construct is canonicalised by the formatter + ([One spelling per meaning](#one-spelling-per-meaning)). An agent that + retrieves one example per operation, and reads one spelling in every stored + query, is the agent the evidence below measures as reliable. The rest of this section records why the project has not had these properties, and why one early break buys them more cheaply than the breaks @@ -237,7 +245,7 @@ names the construct and whose `fix` names the owning RFC; a form marked | Blocks in `match` | `not { }`, `exists { }`, `count|sum|min|max|avg(…) { } op expr`, `optional { }` | the four work; `optional` is a slot | derived relations RFC | | Blocks as values | a block aggregate in operand position, `return { count { $n -knows-> $m } as knows }`, `exists { … } as employed` | parses; a slot | derived relations RFC | | Bindings | `$x: Type { … }` as today; `$x: Name($arg, …) { … }` binds a declared `view` with arguments; `$x: Interface` binds an abstract type | the first works; a binding with arguments is a slot; an interface name is refused as today (`T1`) until its RFC | derived relations RFC (views), polymorphic types RFC (interfaces) | -| Traversal | `$a -knows-> $b`, `$b <-knows- $a`, `$a -knows- $b` (either direction), `$a -$e:knows-> $b`, `$a -(knows\|likes){1,3}-> $b`, `$a -*-> $b`, `$a -"in"-> $b` | works; the triple form `$a knows $b` is a parse error whose fix prints the arrow form | this RFC | +| Traversal | `$a -knows-> $b` (source on the left, the one directed form), `$a -knows- $b` (whichever orientation the endpoint types admit), `$a -$e:knows-> $b`, `$a -(knows\|likes){1,3}-> $b`, `$a -*-> $b`, `$a -"in"-> $b` | works; the triple form `$a knows $b` and the left arrow `$b <-knows- $a` are parse errors whose fix prints the `->` form with the ends in schema order; an arrow written against the schema is `T5` with the swapped clause as its fix; `T22` narrows to an orientation the endpoint types leave ambiguous | this RFC | | Calls | `name(positional…, option: value…)` for every function; the registry holds `count`, `sum`, `avg`, `min`, `max`, `now`, `date`, `datetime`, `nearest`, `bm25`, `rrf`, `search`, `fuzzy`, `match_text`, `contains`, `starts_with`; `count(distinct x)` is the one special form | works; the infix `x contains y` and `x starts_with y` are parse errors whose fix prints the call; `rrf(a, b, 60)` is a type error whose fix prints `k: 60` | the operator-vocabulary amendment for the call rule; the function-registry work for `@Function` and new rows | | Reserved words | `and`, `or`, `not`, `is`, `null`, `in`; every other keyword is contextual (word-bounded, legal as an identifier in operand and property position) | works; a grammar test pins the set | this RFC | | Null | no literal in a read; `null` legal as the value of an `insert` or `set` assignment | works | this RFC | @@ -330,9 +338,14 @@ Every refusal keeps the four-field diagnostic. New codes, allocated at implementation: - `Q00x` unknown edition; `Q00x` an edition line after another statement; -- the triple traversal, the infix string operators and `query` with a mutation - body are parse or type errors whose `fix` is the rewritten text (a - `machine_applicable` suggestion); +- the triple traversal, the left-arrow traversal, the infix string operators + and `query` with a mutation body are parse or type errors whose `fix` is + the rewritten text (a `machine_applicable` suggestion); +- `T5` for an arrow written against the schema, whose `fix` is the clause + with its ends swapped; `T22` only for an undirected traversal whose + orientation the endpoint types leave ambiguous; +- `L…` lint for `count { } > 0` and `count { } = 0`, whose `fix` is the + `exists` or `not` block; - `T6x` reserved construct: `view`, `batch`, `optional { }`, a binding with arguments, a block aggregate as a value, `returning`, `@type_id`, the reserved annotations and catalogue names, a namespaced type @@ -362,6 +375,36 @@ loader keeps the extension and `cluster.yaml` keeps its `schema:` key, both of which now parse with the one grammar. Nothing a `.pg` file accepted today is refused. +### One spelling per meaning + +The freeze list is checked against one rule, in Go's sense: the grammar has +one form for one meaning, and the formatter canonicalises what the grammar +cannot forbid. Concretely: + +- A respelling that adds no meaning is not in the grammar. The triple + traversal, the left arrow, the infix `contains` and `starts_with`, the + positional `k` of `rrf`, and `query` around a write are each a parse or + type error whose `fix` prints the one form. The diagnostics know every + respelling an agent trained on Cypher, SQL or GraphQL is likely to write. +- A form that falls out of a general construct and has a canonical instance + is rewritten to that instance by `omnigraph queries fmt`, the rewriter's + second job (`gofmt -s` is the precedent), and named by lint: `count { P } > + 0` becomes `exists { P }` and `count { P } = 0` becomes `not { P }`; once + the polymorphic types RFC adds `is`, `$x.@type = "Person"` becomes + `$x is Person`. The grammar keeps `exists` because ISO GQL has only + `EXISTS` and the planner's semi-join is the honest plan for it. +- Forms that look alike but differ in meaning stay: `return { $a.slug }` + (a column) and `return { $a { slug } }` (one object); `$e.@src` (stored + orientation) and the pattern's left end (written orientation); `limit 10` + and `limit $k`; `count(distinct $x)` and `return distinct`; `having` and a + `match` filter. +- One argument has one form: a registry function declares each argument + positional (required) or named (an option), and a caller cannot choose. +- A later RFC's Alternatives section answers this rule for every form it + adds; `@if_commit($p)` and the request's precondition field are the first + case, and the writes RFC must say the field is derived from the + annotation, never a second source. + ### Declarations ```pest @@ -429,19 +472,47 @@ score columns are named by the query. ```pest traversal = { variable ~ edge_part ~ variable } -edge_part = { "-" ~ edge_core ~ "->" | "<-" ~ edge_core ~ "-" | "-" ~ edge_core ~ "-" } +edge_part = { "-" ~ edge_core ~ "->" | "-" ~ edge_core ~ "-" } +left_arrow = { "<-" ~ edge_core ~ "-" } // recognised only to print the fix edge_core = { edge_binding? ~ edge_sel ~ traversal_bounds? } edge_sel = { edge_alternation | edge_wildcard | edge_ident | string_lit } ``` -The three parts carry direction, so `undirected_edge`'s angle brackets -disappear and the type checker's endpoint rule (`T5`) checks the written -direction against the declared endpoints instead of inferring it. `-e-` -keeps today's rule for the undirected form: same node type at both ends. The +There is one directed form, source on the left. The left arrow +`$c <-worksAt- $p` would be a second spelling of the same clause with no +meaning of its own, because a GQ traversal is one hop (`variable edge_part +variable`, no chains) and so never needs to continue a line in the other +direction; `left_arrow` is in the grammar only as a recognizer, so that the +parse error's fix prints the clause with its ends swapped, as the triple +form's fix prints the arrow. `undirected_edge`'s angle brackets disappear. + +The type checker's endpoint rule (`T5`) checks the written arrow against the +declared endpoints instead of inferring the orientation, and when the arrow +is written against the schema (`$c: Company -worksAt-> $p`) its `fix` is the +swapped clause, `machine_applicable`. This is the deterministic correction +the Cypher ecosystem bolts on after generation (LangChain's +`CypherQueryCorrector`, built because models "can sometimes mess up the +direction of the relationship"; CypherBench counts a reversed relation in +about one failure in six for gpt-4o) moved into the compiler, with two +differences: it is loud, so the agent learns in one turn, and it never +guesses. When the two endpoint types are the same (`Person -knows-> Person`, +`Company -subsidiaryOf-> Company`) the arrow is the only carrier of meaning +and nothing can check it against the schema; the edge's `@description` and +the skill's schema card say in words which end is which. + +`-e-` means whichever orientation the endpoint types admit. For a same-type +edge that is both orientations, deduplicated, as `` is today. For an +alternation or wildcard whose members run in different orientations between +the two bound types (`$p: Person -(owns | ownedBy)- $d: Doc`, which the triple +form resolves per member today) it is each member's declared orientation, +and the bound edge's `@src` and `@dst` keep reporting the stored orientation. +`T22` ("undirected traversal needs a same-endpoint-type edge") narrows to the +one case that stays ambiguous: an asymmetric edge whose endpoints are not +typed well enough to fix the orientation. The grammar has no `-` operator in edition 3, so no ambiguity with arithmetic exists in this release; the function-registry work that adds arithmetic inherits one rule, stated here so it is not rediscovered: inside `match`, a -`-` or `<-` directly before an `edge_core` and followed by `->` or `-` is a +`-` directly before an `edge_core` and followed by `->` or `-` is a traversal, tried before `filter` in the ordered choice as `traversal` is today; an arithmetic minus takes a value operand (`$x.p`, a literal, a call, a parenthesised expression), never a bare edge identifier, so the two never @@ -598,6 +669,10 @@ channel; `having` is a `Filter` node, not a string. | `graph@ns::Type` for the federation segment | A second delimiter where the namespace RFC's `::` already nests; an outermost segment is enough and needs no grammar now | | Permanent two-edition compiler | Not needed for one break; the registry records the edition, and a second break, if one comes, builds the N-1 arm then | | One RFC for grammar and semantics (the PR #606 shape) | Closed on 2026-09-28 because it could not be accepted or evidenced piecemeal; this RFC owns the grammar and the window and names the owner of every slot | +| Keep the left arrow `$b <-knows- $a` beside `$a -knows-> $b` | Two spellings of one clause; GQ has no path chains, so the left arrow never continues a line; the one-spelling rule removes it and the parse error's fix prints the swap | +| Correct a wrong arrow silently against the schema (LangChain's `CypherQueryCorrector`) | A silent rewrite teaches the agent nothing and must guess, or return nothing, when both orientations fit; the typed error with a machine-applicable swap is the same determinism made loud | +| Role-named relation ends instead of direction (TypeQL `links (employee: $p, employer: $c)`) | Right for n-ary relations, which GQ does not have; for a binary edge the role names document `src` and `dst`, which `@description` does; a role name on an edge declaration stays additive if a later RFC wants it | +| Stage pipelines as the closure mechanism (TypeQL 3: `match`, `reduce`, `match`, `insert` in any order) | Closed by construction, but the text order is the plan order, each stage's output is its input crossed with its matches, and reads and writes mix in one transaction; the reserved `view`, `optional`, block values and `having` give the measured closure cases under one planner-owned shape and one commit | | Do nothing | The next five RFCs each break the language once | ## Evidence and tests @@ -610,7 +685,11 @@ Existing owners, per [the testing map](../dev/testing.md): line in every position. - **Type-check tests**: every slot's refusal with its code and fix; arm collision with a property; an arm read in `match`; `query` with a mutation - body. + body; the left arrow's parse error and the swapped-clause fix; `T5` with + the swapped fix for an arrow against the schema; `-e-` over an alternation + of opposed members (the issue 659 shape) resolving per member; `T22` only + when the endpoints leave the orientation ambiguous; the `count { } > 0` and + `count { } = 0` lint with its fix. - **GQT corpus** (`crates/omnigraph-gqt/cases/`): the whole corpus rewritten by `queries migrate` with zero hand edits, run green; a hand edit is a rewriter defect, fixed in the tool. New cases: `having` over an aggregate and over a @@ -680,3 +759,13 @@ the release; step 5 is the removal every deprecation ends in. file-level declaration keyword can be added later without a break, as the Design section now states. The permissions RFCs are no longer named as slot owners. +- 2026-10-29: investigated TypeDB 3 (TypeQL pipelines, functions, `try`, type + variables, roles, the 2-to-3 migration) and how nine languages express + edge direction, with the measured direction-error evidence from Cypher + generation. TypeDB confirms the slot strategy and the rewriter and changes + no slot; its type-variable binding and row-valued parameters are additive + and noted for the polymorphism and a later RFC. Direction: the left arrow + is removed as a second spelling, a wrong arrow becomes `T5` with a swapped + fix, `-e-` resolves orientation per member and `T22` narrows. Adopted the + maintainer's requirement of one spelling per meaning as a stated rule, with + the formatter canonicalising `count { } > 0` and `count { } = 0`. From f67be6bea5845cf031efa5a5600b0eda6ebdd948 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Thu, 29 Oct 2026 16:53:58 -0700 Subject: [PATCH 08/13] docs(rfc): commit the GQ evaluation as the grammar RFC's evidence; schema queryable at any size --- .typos.toml | 2 + docs/rfcs/2026-10-28-gq-3-grammar.md | 43 +- .../2026-10-28-gq-3-grammar-evaluation.md | 4184 +++++++++++++++++ 3 files changed, 4221 insertions(+), 8 deletions(-) create mode 100644 docs/rfcs/assets/2026-10-28-gq-3-grammar-evaluation.md diff --git a/.typos.toml b/.typos.toml index 7a9c86952..4ab4ff965 100644 --- a/.typos.toml +++ b/.typos.toml @@ -18,6 +18,8 @@ extend-ignore-re = ['\bmis-\w+'] AIMD = "AIMD" # additive-increase/multiplicative-decrease retry policy; Lance's lance_aimd_max_retries option AKS = "AKS" # Azure Kubernetes Service (RFC 0029 assets) BA = "BA" # AB/BA measurement-order labels (RFCs 0023, 0031); also the hex fragment of the short commit SHAs 2aab48ba (RFC 0019) and da466ba (vocabulary guard test). Cost: a bare "ba" typed for "by" passes +EDN = "EDN" # Extensible Data Notation, Datomic's data format (GQ 3 grammar RFC evaluation) +FO = "FO" # first-order logic, the standard abbreviation in the expressiveness ladder (GQ 3 grammar RFC evaluation) fof = "fof" # friend-of-friend: pattern variable in test queries and the query reference unparseable = "unparseable" # accepted variant; appears in operator-facing error strings and docs diff --git a/docs/rfcs/2026-10-28-gq-3-grammar.md b/docs/rfcs/2026-10-28-gq-3-grammar.md index 83551bb45..8e079cf9d 100644 --- a/docs/rfcs/2026-10-28-gq-3-grammar.md +++ b/docs/rfcs/2026-10-28-gq-3-grammar.md @@ -81,14 +81,15 @@ changes would also fit. The language therefore carries decisions whose only justification is the order they arrived in: retrieval spelled as a sort key, functions that are grammar productions instead of registry entries, a reserved set that grew by one word per release, and several open RFCs that each -schedule a break of their own. A whole-language review against SQL, GQL, +schedule a break of their own. [A whole-language +review](assets/2026-10-28-gq-3-grammar-evaluation.md) against SQL, GQL, Cypher, GraphQL, Datalog and the query languages of Elasticsearch, LanceDB and Vespa found the semantics sound and the accretion the problem: every expressiveness gap is a position the grammar never reserved, and every scheduled break is a decision that could be made once. This RFC opens a revision that treats the language as a whole rather than as -a queue of requests. Its aim is a foundation with five properties, each of +a queue of requests. Its aim is a foundation with six properties, each of which a later section of this document or a named follow-up RFC is responsible for. @@ -113,6 +114,17 @@ responsible for. ([One spelling per meaning](#one-spelling-per-meaning)). An agent that retrieves one example per operation, and reads one spelling in every stored query, is the agent the evidence below measures as reliable. +- **Self-describing at any size.** The schema is data in the same language: + the catalogue types `@Type`, `@Property`, `@Edge`, `@Interface`, `@Index`, + `@Function` and `@View`, joined by catalogue edges, are bound, traversed, + searched and counted with the same `match`, `retrieve` and `return` as user + data. A schema of a few dozen types fits in a context window; one of a few + thousand (every source system's types, every agent's views) does not, and + the agent then finds the type it needs the way it finds a row: by name, by + the text of its `@instruction`, by its neighbourhood. The evaluation's + literature review found the full schema wins only while it fits + (R1 in §9.1); past that, a schema the language cannot query is a schema + the agent cannot use. The rest of this section records why the project has not had these properties, and why one early break buys them more cheaply than the breaks @@ -252,7 +264,7 @@ names the construct and whose `fix` names the owning RFC; a form marked | Parameters | `$x: Type? = default`; `limit $k` with a non-null `I64` parameter; the parameter kinds `Branch` and `Commit` | defaults and `limit $k` work; `Branch` and `Commit` parse and are slots | this RFC; the branch-statements RFC for binding them | | Annotations | `@name(args)` on any declaration, retained as metadata; `@description`, `@instruction`, `@example`, `@if_commit($p)` are typed; `@budget`, `@approximate`, `@satisfice`, `@brief`, `@phase` are reserved | the typed four work (`@if_commit` binds the commit precondition the request carries today); the reserved five are slots | RFC 0048 and the agent-first batch | | Meta-fields | `@id`, `@src`, `@dst`, `@type`, `@type_id`, `@distance`, `@score` | the first four work; `@type_id` is a slot; `@distance` and `@score` are the typed fields of a `retrieve` arm's column | polymorphic types RFC for `@type_id` | -| Catalogue names | `@Operator`, `@Pass`, `@Setting`, `@Engine`, `@LogicalKind` exist; `@Type`, `@Property`, `@Edge`, `@Interface`, `@Index`, `@Function`, `@View` reserved | the five exist; the seven are slots | the engine catalogue RFC's schema extension | +| Catalogue names | `@Operator`, `@Pass`, `@Setting`, `@Engine`, `@LogicalKind` exist; `@Type`, `@Property`, `@Edge`, `@Interface`, `@Index`, `@Function`, `@View` reserved; catalogue edges between them spelled with the same prefix, `$t: @Type -@property-> $p: @Property`, `$e: @Edge -@from-> $t`, `$t -@implements-> $i: @Interface`; a catalogue type is a node type for every purpose of the grammar (binding, traversal, a `retrieve` arm over its text fields, aggregation) | the five exist; the seven and the `@`-prefixed edge selector are slots | the engine catalogue RFC's schema extension, which declares the catalogue's own edges and which of its text fields are searchable | | Type references | `a::b::C`: zero or more lowercase segments then a type name; the grammar admits it everywhere a type name stands | one or more segments is a slot | schema namespaces RFC (`ns::Type`); a federation RFC for an outermost graph segment | | Schema roles | `Vector(n, distance="l2"\|"cosine"\|"dot")`, `@analyzed(…)`, `@valid(from, to)`, `@embed(source, model=…, …)` | `distance` works (RFC 0048's rule: omitted means `l2`); `@analyzed` and `@valid` are accepted as SchemaIR features and refused at use | RFC 0048 (distance, recipes), the lexical RFC (`@analyzed`), the writes RFC (`@valid`) | | Envelope | `graph_commits` (a map keyed by graph id, one entry today) beside `graph_commit_id`; `notifications`, `why_not`, `complete`, `incomplete_reason`, `continuation`, `estimate` | present where the owning stage has shipped, absent otherwise; all additive | the envelope stage RFCs | @@ -475,7 +487,8 @@ traversal = { variable ~ edge_part ~ variable } edge_part = { "-" ~ edge_core ~ "->" | "-" ~ edge_core ~ "-" } left_arrow = { "<-" ~ edge_core ~ "-" } // recognised only to print the fix edge_core = { edge_binding? ~ edge_sel ~ traversal_bounds? } -edge_sel = { edge_alternation | edge_wildcard | edge_ident | string_lit } +edge_sel = { edge_alternation | edge_wildcard | catalogue_edge | edge_ident | string_lit } +catalogue_edge = @{ "@" ~ ident } // an edge of the catalogue's own schema; a slot ``` There is one directed form, source on the left. The left arrow @@ -500,6 +513,15 @@ guesses. When the two endpoint types are the same (`Person -knows-> Person`, and nothing can check it against the schema; the edge's `@description` and the skill's schema card say in words which end is which. +A catalogue edge carries the `@` prefix for the reason catalogue types do: +a user schema may declare an edge named `property` or `from`, and one +spelling per meaning forbids resolving a bare name against two namespaces by +endpoint type. The engine catalogue RFC's schema extension declares the +catalogue's edges and which of its text fields a `retrieve` arm may search, +so that `match { $t: @Type } retrieve { $t.lex: bm25($t.instruction, $q) }` +is an ordinary query over an ordinary table; nothing in the grammar +distinguishes it from a query over user data, which is the point. + `-e-` means whichever orientation the endpoint types admit. For a same-type edge that is both orientations, deduplicated, as `` is today. For an alternation or wildcard whose members run in different orientations between @@ -747,8 +769,9 @@ the release; step 5 is the removal every deprecation ends in. ## Decision log -- 2026-10-28: drafted from the evaluation of GQ against SQL, GQL, Cypher, - GraphQL, Datalog, Elasticsearch, LanceDB and Vespa, its literature review, +- 2026-10-28: drafted from [the evaluation](assets/2026-10-28-gq-3-grammar-evaluation.md) + of GQ against SQL, GQL, Cypher, GraphQL, Datalog, Elasticsearch, LanceDB + and Vespa, its literature review, and the maintainer's decision to make every incompatible change first and once. The freeze list is the set of syntactic decisions the next RFCs (derived relations and optional patterns, the RFC 0048 rewrite, the lexical @@ -761,11 +784,15 @@ the release; step 5 is the removal every deprecation ends in. owners. - 2026-10-29: investigated TypeDB 3 (TypeQL pipelines, functions, `try`, type variables, roles, the 2-to-3 migration) and how nine languages express - edge direction, with the measured direction-error evidence from Cypher + edge direction (the evaluation's §12 to §14), with the measured direction-error evidence from Cypher generation. TypeDB confirms the slot strategy and the rewriter and changes no slot; its type-variable binding and row-valued parameters are additive and noted for the polymorphism and a later RFC. Direction: the left arrow is removed as a second spelling, a wrong arrow becomes `T5` with a swapped fix, `-e-` resolves orientation per member and `T22` narrows. Adopted the maintainer's requirement of one spelling per meaning as a stated rule, with - the formatter canonicalising `count { } > 0` and `count { } = 0`. + the formatter canonicalising `count { } > 0` and `count { } = 0`. Added the + maintainer's requirement that the schema stay queryable as it grows: + catalogue types are node types for every purpose of the grammar, and the + `@`-prefixed catalogue edge selector is reserved. The evaluation is + committed beside this RFC as its evidence. diff --git a/docs/rfcs/assets/2026-10-28-gq-3-grammar-evaluation.md b/docs/rfcs/assets/2026-10-28-gq-3-grammar-evaluation.md new file mode 100644 index 000000000..e15988081 --- /dev/null +++ b/docs/rfcs/assets/2026-10-28-gq-3-grammar-evaluation.md @@ -0,0 +1,4184 @@ +# GQ, evaluated from first principles + +**Subject:** the Omnigraph query language (`.gq`) and schema language (`.pg`) as +of 0.14.1 (`ea895066`), read from `query.pest`, `schema.pest`, the user guides, +the language RFCs (shared expression model, typed edge alternation, branch +statements, search plan truth, engine catalogue, bulk load) and the GQT corpus (425 +cases). +**Compared against:** SQL (with SQL:2023 SQL/PGQ, recursive CTEs, pgvector), +ISO GQL (39075:2024), Cypher (openCypher and Neo4j Cypher 25), GraphQL, +Datalog (classical, Soufflé, Datomic, CozoDB, Rel), Elasticsearch (Query DSL +and ES|QL), LanceDB (query builder and DataFusion SQL), and Vespa (YQL, +grouping, rank profiles). +**Status:** a design evaluation, not an RFC. Where it recommends, it ranks. +**Outcome:** the [GQ 3 grammar RFC](../2026-10-28-gq-3-grammar.md), which +cites this document as its evidence; §12 to §14 were added after that RFC +was drafted and are folded into it. + +--- + +## 0. Method + +Omnigraph's own systems paper (`docs/dev/systems.md`) defines a database as an +observable contract (state, operations, guarantees, interfaces) implemented by +physical mechanisms for a distribution of work. A query language is the +*interface* part of that contract: the notation in which a caller names a +subset of the state and the operations over it, and from which the guarantees +(typing, ordering, atomicity, bounds) are either derivable or not. + +So the evaluation does not ask "is GQ like Cypher". It asks, for each thing a +query language must do, what GQ chose, what the alternatives chose, and what +each choice costs after five more changes of the same kind. The axes: + +| # | Axis | The question | +|---|---|---| +| 1 | Model fidelity | Does the vocabulary name exactly the state model, no more, no less? | +| 2 | Compositional closure | What is the unit of composition, and does the language close over it? | +| 3 | Binding and scope | How are variables introduced, joined and scoped? | +| 4 | Typing | Static or dynamic; declared or inferred; how are coercions and nullability handled? | +| 5 | Absence | What does a missing value mean, and how is it tested? | +| 6 | Traversal | Reachability semantics, path identity, recursion bounds, direction. | +| 7 | Sub-pattern predicates | Negation, existence, correlated counting; the safety rule. | +| 8 | Aggregation | Grouping discipline, post-group filtering, numeric exactness. | +| 9 | Order and cut | Totality of order, paging, determinism. | +| 10 | Retrieval | Full-text, vector and fusion; how ranking interacts with predicates and traversal. | +| 11 | Writes | Mutation shape, atomicity, read-write mixing, idempotence. | +| 12 | Time and branches | Snapshots, history, branches, in or beside the language. | +| 13 | Reuse | Named queries, views, rules, fragments, functions. | +| 14 | Expressions | Arithmetic, scalar functions, conditionals, collections. | +| 15 | Result shape | Flat rows versus nested trees; whole entities; paths. | +| 16 | Self-description | Explain, catalogue, schema-as-data. | +| 17 | Refusal contract | Error codes, positions, fixes; suitability for machine authorship. | + +The workload GQ is built for is named in the README and the schema-design +guide: many agents sharing one typed world model, retrieving high-precision +context and writing reviewable changes. Every verdict below is relative to that +workload, not to analytics, OLTP or web APIs. + +--- + +## 1. GQ in one page + +```gq +set ann_nprobes = 40; + +query customer_evidence($slug: String, $q: String, $least: I64) { + match { + $c: Customer { slug: $slug } // typed binding with inline equality + $a: Assertion + $a about $c // directed edge, direction from the schema + $a $e:(statedIn | derivedFrom) $s // bound edge over an alternation + $s.published_at is not null + ($s.kind = "primary" or $s.kind = "audit") and not $a.retracted + count { $x supports $a } >= $least // correlated block, stratified + not { $a supersededBy $_ } + } + return { $a.statement, $e.@type as via, $s.title, count($x) as support } + order { nearest($a.embedding, $q) } // ranking lives in order + limit 10 +} +``` + +What the one shape fixes: + +- **One declaration form.** Every executable unit is `query name(params) { body }`. + The body is a read (`match` → `return` → `order`? → `limit`?) or a mutation + (`insert`/`update`/`delete` statements, constructive *or* destructive, never + both). A file is either declarations or exactly one statement (`branch …`, + `commit …`, `load csv …`, `show …`, `explain …`), optionally prefixed by `set` + lines. +- **Schema-first, statically typed.** Node and edge types, properties, keys, + nullability, enums, vectors and Blobs come from `.pg`; every binding, edge, + property and parameter is checked against it before planning. 51 stable `T` + codes, 5 `Q` codes, 3 `P` codes. +- **Pattern conjunction by juxtaposition.** A `match` block is a set of clauses: + bindings, triples (`$a edge $b`), Boolean filters, and correlated blocks + (`not { }`, `exists { }`, `count { } op v`, `sum(e) { } op v`). Clause order + is irrelevant to meaning except that a variable must be bound in a visible + scope. +- **Triples, not arrows.** `$p worksAt $c` is subject–predicate–object; the + schema fixes direction, so reversing a traversal is swapping the ends. + `` is either direction, `(a | b)` and `*` select edge types, `{m,n}` + bounds hops. +- **Reachability, not paths.** An unbound traversal has set semantics over + endpoint pairs; `{m,n}` means shortest-distance *m..n* from the source; no + path is ever bound; edge bindings are single-hop only. +- **Ranking is an order key.** `nearest`, `bm25`, `rrf` lead `order { }`, + select the retrieved population *and* sort it; the projected score must repeat + the order key; `limit` is required for `nearest`. +- **Writes are statements over one type with constant values.** Keyed insert is + upsert; `where` is the same expression language as a read filter; no + read-write mixing, no computed assignment, no edge update. +- **Control is in the language.** Branch create/delete/merge/list/diff/ + merge-base, unpublished commits (`as commit`, `commit publish|drop`), + unattached snapshots, CSV load and `explain` are GQ statements. +- **The engine is a graph.** `@Operator`, `@Pass`, `@Setting`, `@Engine`, + `@LogicalKind` are queryable node types. + +--- + +## 2. The axes + +### 2.1 Model fidelity + +**GQ.** The grammar names exactly the state model: `node`/`edge`/`interface` +types; `String`, `Bool`, four integer widths, two float widths, `Date`, +`DateTime`, `Vector(N)`, `Blob`, `enum(...)`, `[T]`, `T?`; `@key`, `@unique`, +`@index`, `@range`, `@check`, `@card`; system fields `@id`, `@src`, `@dst`, +`@type`. Nothing in the query language exists that the storage model does not +have (no maps, no untyped JSON, no arbitrary labels), and nothing in the storage +model is unreachable from the language except Blob *values*, which are +deliberately routed to a dedicated byte API. + +This is the tightest model–language fit of the nine systems. Cypher and GQL +describe a schema-*optional* graph (labels are sets, properties are open maps), +so their syntax carries machinery GQ does not need (label expressions `:A&B|C`, +`properties()`, `keys()`, dynamic property access `n[$k]`, `labels(n)`) and a +type system that is resolved at run time. SQL describes tables; SQL/PGQ layers a +graph *view* over them (`CREATE PROPERTY GRAPH … VERTEX TABLES … EDGE TABLES`), +which is powerful for existing relational data and heavy for a native graph. +GraphQL describes an API, not storage: its types are resolver signatures, so the +language is complete for shape and silent on filtering, ordering and +aggregation. Datalog describes relations; property graphs are an encoding +(Datomic's EAV, Soufflé's typed relations, Rel's modular relations), so every +"edge" is a predicate and every "property" another, which is maximally uniform +and minimally ergonomic. Elasticsearch and Vespa describe documents with +mappings/schemas and treat relationships as a special case (nested, parent–child, +imported fields). LanceDB describes Arrow tables and has no relationship +vocabulary at all. + +**Verdict.** Keep. The schema-first, closed-vocabulary model is the foundation +of everything else GQ does well (static typing, plan acceptance, exact +diagnostics, deterministic results). The one fidelity gap is that `.pg` is +declared but not *queryable* from GQ (see 2.16). + +### 2.2 Compositional closure + +This is the axis on which GQ is most unlike every comparator, and the one whose +consequences show up in the most open issues. + +**GQ.** A query is one fixed pipeline: `match` → `return` → `order` → `limit`. +Inside `match`, the only nesting is the correlated block, which is a predicate, +not a value. There is no construct that takes a query's result as input to +another query: no `WITH`/pipeline, no subquery in `return`, no `UNION`, no +view, no derived relation, no CTE. The result of a query is rows; nothing in the +language consumes rows. + +Every comparator closes over something: + +| System | Unit of closure | Mechanism | +|---|---|---| +| SQL | table | subquery, derived table, CTE (`WITH … AS`), view, `UNION`, lateral join, recursive CTE | +| Cypher | binding table (rows of variables) | `WITH` pipeline, `CALL { }` subquery, `UNION`, `UNWIND`, `COLLECT {}` | +| GQL | binding table | linear composition of statements, `CALL`, `UNION`/`EXCEPT`/`INTERSECT`, nested queries | +| GraphQL | selection set over a type | nesting; no closure over results (cannot query a result) | +| Datalog | relation | rules; a rule head is a relation any other rule may use; recursion | +| Elasticsearch | ranked hit list; bucket | retriever tree (`rrf` of retrievers of retrievers), sub-aggregations | +| LanceDB | Arrow table | DataFusion SQL closes; the fluent builder does not | +| Vespa | match set + rank | `rank()` composes operators; grouping pipelines compose over groups | + +What GQ buys by not closing: + +- The planner sees the whole query as *one* component shape. That is why plan + acceptance can check an "exact subset" against a canonical form, why every + read node lowers to exactly one operator, why `explain` can be a flat table, + and why plan replay is a contract. Closure would turn validation into a + per-stage problem. +- Every query is bounded by construction: one scan root per connected + component, deferred bindings, streaming `Expand`, one `Sort`, one `Limit`. + There is no place to write an unbounded cross-stage fan-out. +- An agent generating GQ cannot be wrong about *structure*; only about content. + +What it costs, with the issue that reports it: + +| Shape that needs closure | Issue / symptom | +|---|---| +| Optional traversal (keep the root when the edge is absent) | #658, P-high, needs-rfc | +| A block aggregate as a *value* (`count { } as n` in return) | #1050 | +| Top-N per group, "the latest assertion per customer" | not expressible; needs a second stage | +| Filter on a group aggregate (`HAVING`) | not expressible: block aggregates are per outer row, return aggregates are post-order | +| Union of heterogeneous types for discovery | #648 | +| One entity plus its neighbourhood in one answer | #649 | +| Reusing a derived relation across queries | no views/rules; see 2.13 | + +**Verdict.** The non-closure is a *deliberate* trade and it has been paying +for itself: plan truth, bounded execution and the diagnostics all rest on it. +But every open language-feature issue (#658 P-high and needs-rfc, #1050, +#648, #649) is a closure shape, which is the five-more-changes signal: each will be solved ad hoc (an `optional { }` +block here, a projected block there) unless one closure mechanism is chosen. +The recommendation in §4 is a *single* closure step, not SQL-style general +subqueries: a `with`-free linear pipeline is not needed; what is needed is (a) +correlated blocks that yield a value (which already have the operator, +`SubqueryAggregate`), (b) an optional block, and (c) derived relations as +stored, typed queries usable as bindings. Those three cover every row above +without making the planner's job per-stage. + +### 2.3 Binding and scope + +**GQ.** `$var` is introduced by a typed binding (`$p: Person`), by appearing +as a traversal endpoint (its type inferred from the edge's declared endpoints), +or by an edge binding (`$e:knows`). Join is by shared variable, Datalog-style: +mentioning `$p` twice conjoins. Re-declaring `$p: Person { x: 1 }` adds a +constraint rather than a second binding; `$_` is a fresh anonymous node each +time; blocks see outer variables declared *before* them in text (a later outer +declaration is not visible to an earlier block); `__`-prefixed names are +reserved. + +Cypher and GQL bind the same way inside one `MATCH` (shared variables conjoin; +anonymous nodes are `()`), but across clauses the rule is lexical and +*positional*: a variable survives only if `WITH` carries it. GQL made this +precise with the binding-table model. SQL binds by table alias and column; joins +are explicit or by `USING`. Datalog is the pure form: any shared variable in a +rule body is a join, there is one scope (the rule), and the safety condition +requires every head/negated variable to appear in a positive body atom. GraphQL +has no variables for data at all, only for arguments. ES/Vespa/LanceDB have +no binding variables; "joins" are structural (nested, parent) or absent. + +Two GQ specifics deserve a note: + +- *The type of an endpoint is inferred from the edge.* `$p worksAt $c` binds + `$c: Company` without a declaration. That is concise and correct (the schema + is closed), and it is also why wildcard and undirected traversals need both + endpoints declared: the edge no longer determines the types. The rule is + sound; the asymmetry ("sometimes I must declare, sometimes I needn't") is a + learning cost the diagnostics (`T5`, `T6`) carry well. +- *Text-order visibility into blocks.* "Later outer declarations are not + visible inside an earlier nested block" is a lexical rule in an otherwise + order-free conjunction. It is the one place where clause order changes meaning, + and the shared-expression RFC records the related trap (`T6` on a comparison + written before the traversal that binds its variable) and the rejected fix (a + separate `where`). Datalog and Cypher have no such rule; GQL does not either. + +**Verdict.** Keep the Datalog-style conjunction. Consider making block +visibility order-free (resolve all top-level bindings of the `match` block +before checking its blocks), which removes the one order-dependent rule without +adding syntax. + +### 2.4 Typing + +**GQ.** Static, declared, closed. Parameters are declared with scalar, list or +vector types and explicit nullability; a missing non-nullable parameter is +refused before execution. Every expression retains a compiler type; every +numeric comparison conversion is an explicit `Cast` node; the planner never +chooses a domain from values. Result schemas are inferred and checked (every +GQT rows step carries `--- expect shape`). Integer `sum` accumulates exactly in +128 bits; mixed signed/`U64` comparison is exact; literal-to-property +comparison uses the property's type when exactly representable, else a common +type. Nullability propagates (`Bool?` from a nullable operand; `is null` is +always `Bool`). Blob is a type but not a value. + +Comparators: + +- *GraphQL* is the nearest peer: a declared, closed type system with + nullability as a modifier (`String!`), variables typed in the operation + header, and static validation of the document against the schema. Where GQ + propagates nullability into results, GraphQL *bubbles* nulls (an error in a + non-null field nulls its parent), which is the single most-criticized + GraphQL semantic. +- *SQL* is statically typed but with implicit coercion rules that vary by + dialect, `NUMERIC` versus float versus integer overflow behaviour that + varies by engine, and no notion of a declared parameter list for an ad-hoc + statement. +- *Cypher* is dynamically typed. A property may hold any type on any node; + parameters are untyped placeholders; a property typo yields `null` rather + than an error. Cypher 25 adds `IS :: INTEGER` type predicates precisely + because the type is not known statically. *GQL* defines value types that + carry `NOT NULL` and *closed* graph types under which static checking is + possible, but the schema is optional (open graph types) and the one major + implementation, Neo4j, declares open graph types only. +- *Datalog* varies: Soufflé is statically typed with declared relation + schemas; Datomic/Datascript are dynamic (attributes have value types, but + queries are not type-checked); Rel is typed. Cozo is dynamic. +- *Elasticsearch* types come from the mapping; the DSL coerces leniently + (`"42"` to a long) and silently ignores unknown fields unless `strict`. + ES|QL is typed at plan time and reports type errors. +- *Vespa* is schema-typed including tensor types; query-time parameters + (`input.query(q) tensor(x[384])`) are declared in the rank profile. +- *LanceDB* is Arrow-typed, but the filter is a SQL string parsed by + DataFusion; the builder cannot type-check it before sending. + +**Verdict.** Keep, and this is GQ's clearest advantage over the graph +incumbents. The declared-parameter list with nullability is the feature that +makes stored queries into typed tools (RFC 0003, MCP) and makes #1191 +(codegen) straightforward. The one typing wart is that `limit` takes an +integer *literal* only, so page size cannot be a parameter (2.9). + +### 2.5 Absence + +**GQ.** Three-valued logic for comparisons and Boolean operators, SQL rules +exactly (`true or null = true`, `false and null = false`, `not null = null`); +a filter keeps rows whose expression is *true*; `is null` / `is not null` are +the only null tests and are always `Bool`; there is **no null literal**. A +nullable parameter bound to null makes every comparison on it null. `in []` +keeps no row. + +The no-null-literal choice is unusual and mostly right. It removes the whole +class of `x = null` bugs that SQL and Cypher both suffer, and it makes the +three-valued rules teachable in a four-row table. Its one cost is spelled out +in the docs: clearing a nullable property (or Blob) requires a nullable +parameter bound to null, because `set { x: null }` cannot be written. That is +a mutation-side ergonomics tax paid for a read-side safety gain. + +Comparators: SQL has `NULL`, 3VL, `IS [NOT] NULL`, `IS [NOT] DISTINCT FROM`, +`COALESCE`, `NULLIF`, and a long list of surprises (`NOT IN` with a null, +`NULL` in `GROUP BY` grouping together). Cypher has `null`, 3VL, `IS NULL`, +`coalesce()`, and the well-known pitfall that a missing property *is* null +(no "property absent" vs "property null" distinction). GQL inherits Cypher's +model and adds `IS [NOT] UNKNOWN`. GraphQL has `null` as a value and the +bubbling rule. Datalog has no nulls at all: absence of a fact is false under +the closed world, which is cleaner than 3VL but forces every "optional +attribute" into a separate relation or an explicit default (Datomic's pull +simply omits the key; `missing?` is a predicate). Elasticsearch: a missing +field matches nothing, `exists` tests presence, `must_not exists` is the null +test; `null_value` in the mapping can substitute. Vespa: a missing attribute +reads as the type's default (0, empty string), which is a documented wart. +LanceDB: Arrow nulls, DataFusion 3VL. + +**Verdict.** Keep. GQ's model is SQL's 3VL with the footguns removed. Two +small additions would complete it without reintroducing them: a `coalesce` +(or default-on-null) expression once scalar functions arrive (2.14), and a +spelled way to assign null in a mutation that is not "declare a nullable +parameter and omit it", for instance `set { x: null }` *only* in assignment +position, where it is unambiguous. + +### 2.6 Traversal + +**GQ.** Directed typed edges; direction comes from the schema, so a triple is +written in schema orientation and reversed by swapping the ends. `` for +either direction (same-type endpoints only). `(a | b)` alternation and `*` +wildcard over compatible types, resolved against the captured catalog. +`{m,n}` bounds: finite, `1 <= m <= n`, meaning *shortest-distance* m..n from the +source (BFS with one visited set per source); omitted bounds mean exactly one +hop; `*` never means repetition. Unbound traversals have set semantics on +endpoint pairs; bound edges (`$e:knows`) yield one row per edge, single hop +only, with `$e.@id/@src/@dst/@type`. Self-loops count once. No path binding, +no shortest-path function, no unbounded recursion. Alternation/wildcard +statements carry a `traversal_work_limit` admission budget. + +This is a *reachability* language, not a *path* language, and that is the +most consequential traversal decision: + +- Cypher's variable-length pattern `(a)-[:KNOWS*2..3]->(b)` returns one row per + *path* under relationship isomorphism (no relationship repeated in a path), + so a diamond yields two rows for the same `b`. `shortestPath`/`allShortestPaths` + and (in Cypher 25) quantified path patterns `((a)-[:KNOWS]->(b)){2,3}` with + path modes exist because the default blows up combinatorially. Users must + add `DISTINCT` to get GQ's default. +- GQL standardizes exactly this problem space: path modes `WALK`, `TRAIL`, + `SIMPLE`, `ACYCLIC`, path search prefixes `ANY`, `ALL SHORTEST`, `ANY + SHORTEST`, quantifiers `{m,n}`, `+`, `*`, and `KEEP` clauses. The + standard's size here is the measure of how hard the path-semantics problem + is when paths are first-class. +- SQL/PGQ (`GRAPH_TABLE … MATCH`) inherits GQL's path patterns, with + `ANY SHORTEST` etc. +- Datalog: transitive closure is a recursive rule with *set* semantics (a + derived fact is derived once), so Datalog's default is GQ's default; distance + requires an explicit counter argument and stratification care; unbounded + recursion is the norm and termination follows from finiteness. +- GraphQL: depth is fixed by the document; no recursion; no path. +- Elasticsearch/Vespa/LanceDB: no traversal in the language. + +GQ's `{2,2}` therefore means "nodes at distance exactly 2", while Cypher's +`*2..2` means "paths of length 2". Both are defensible; they differ on every +graph with a cycle or a diamond. GQ's choice is the right one for its workload +(agents want the neighbourhood, not the path multiset) and it is what makes +the BFS bounded. The costs: no "how many distinct routes", no path +projection, no edge properties along a multi-hop (edge binding is single hop), +no `ALL SHORTEST`, and the `{2,2}` ≠ Cypher trap that LLMs trained on Cypher +will fall into. The direction-from-schema rule is elegant and hides direction +at the reading site; `(p)-[:WORKS_AT]->(c)` is self-documenting where +`$p worksAt $c` needs the schema, and the undirected form `` being +restricted to same-type edges is correct but surprising. + +**Verdict.** Keep reachability semantics; document the Cypher difference in +the skill as prominently as the `mutation {}` trap. Two bounded additions would +cover most path demands without becoming a path language: (a) a `distance` +meta-field on the far end of a `{m,n}` traversal (`$b.@distance`), which the +BFS already knows; (b) multi-hop edge bindings as a *list* of edge objects once +collection results exist (2.15). Unbounded `*` recursion should stay refused: +`traversal_work_limit` is the honest form of the same promise. + +### 2.7 Sub-pattern predicates + +**GQ.** `not { }`, `exists { }`, `count { } op v`, `sum|min|max|avg(e) { } op v`. +Each block is matched once per outer row; each must read an outer-bound +variable (`T39`, the safety rule); the comparison's right side is a constant +(literal, `now()`, parameter); the aggregate's argument is scalar in the +block's scope. `not` is `count = 0`; `exists` is `count > 0`; empty `sum/min/ +max/avg` satisfy nothing. Blocks narrow rows *before* `order`/`limit`, so +paged results filtered by relationship count are exact. The planner picks +`correlated` or `key_set` strategy; a single-hop filter-free `count` reads the +CSR degree. + +This is stratified negation and aggregation sub-goals, the Datalog shape, with +Datalog's safety rule made explicit. SQL's equivalents are `NOT EXISTS`, +`EXISTS`, and correlated scalar subqueries (`(SELECT count(*) …) > 2`), which +are more general (the right side can be another column, the subquery can +return a value to project) and harder to plan (decorrelation is a research +area). Cypher 5 added `EXISTS { }`, `COUNT { }` and `COLLECT { }` subqueries +with the same block syntax GQ uses; Cypher's `COUNT { }` is a *value* usable +anywhere, which is what #1050 asks of GQ. GQL has `EXISTS { }` only; `COUNT { }` and +`COLLECT { }` are Cypher-only. GraphQL has nothing. Datalog: `not` (stratified), aggregates +in rule heads (`count`, `sum`, with grouping by the other head variables), +Soufflé's `count : { … }` as a value, Datomic's aggregates in `:find`. ES: +`must_not`, `exists`, `nested` with `inner_hits`; counting a relationship is a +`children` aggregation. Vespa: `!` negation, parent–child counts via grouping. + +GQ's two restrictions, versus Cypher's `COUNT {}`: + +1. *A block is a predicate, not a value.* So `return { count { $n knows $m } }` + is refused; it costs one query per relationship group (#1050 measured 92 + requests for one node page). The operator (`SubqueryAggregate`) and the + degree shortcut already exist; this is a grammar and result-schema change. +2. *The right side is a constant.* `count { $a knows $x } > $a.min_friends` is + refused. Allowing an outer property on the right side costs nothing in the + planner (the comparison is evaluated after the fold) and removes a + round-trip. + +**Verdict.** Keep the stratified-block design and the safety rule; they are +the correct foundation. Lift the two restrictions, both of which are expressed +in Cypher 5 and Datalog and both of which have their mechanism in the engine. + +### 2.8 Aggregation + +**GQ.** Aggregates `count`, `sum`, `avg`, `min`, `max` in `return`; grouping +is implicit on every non-aggregate return item (Cypher's rule); `count(distinct +$p)` / `count(distinct $p.x)`; `return distinct { }`; a Boolean expression +over aggregates under an alias (`count($p) > 0 as any`); an aggregate-only +return over zero rows yields one row. No `HAVING`: a group cannot be filtered +by its aggregate (the Boolean projection *labels* the group, it does not drop +it). No nested or two-stage aggregation. No `collect`/list aggregation. Exact +128-bit integer `sum` rounded once to `F64`; `avg` in `F64`; `min`/`max` keep +the column type including Bool and dates. + +Implicit grouping is shared with Cypher, Datalog heads, Datomic, and +Vespa's grouping (which groups by the `group()` expression). GQL chose an +explicit `GROUP BY`, and Cypher 25 added one in 2026.07 beside the implicit +rule. SQL requires an explicit `GROUP BY` and offers `HAVING`, `ROLLUP`, `GROUPING SETS`, window +functions; its explicitness is verbose but makes "group by X, order by +aggregate, keep groups with aggregate > n" one statement. The implicit rule's +known pitfall (adding a column to `RETURN` silently changes the grouping) GQ +shares; its diagnostics mitigate it (`T58`, `T9`) but cannot remove it. + +Numeric exactness is distinctive. SQL engines differ: PostgreSQL `sum(bigint)` +returns exact `numeric`; MySQL returns `decimal`; SQLite returns integer with +overflow error; DuckDB `HUGEINT`. Cypher's integers are 64-bit and `sum` does not +widen (`sum(null)` is `0`, where SQL's is `NULL`). GQ's choice (exact accumulate, one rounding, F64 result) is +coherent and documented to the bit, but it has a surprising edge: a sum of two +integers that is exactly representable in `I64` still comes back as `F64`, so +`sum` of an `I64` column is never an `I64`. SQL's answer (an exact wider +integer or decimal) is what a client with 64-bit integers would prefer. + +**Verdict.** Keep implicit grouping (it is the graph-language convention) and +the exactness. Add post-group filtering, which is one `Filter` above +`Aggregate` in the plan and needs only grammar: either a `having { }` clause +or, more in GQ's style, allow a Boolean aggregate expression as a *filter* +rather than only as a projection. Reconsider returning an exact integer type +for integer `sum` (`I64` with overflow refusal, `exact_integer` internally), +since F64 is a lossy public type for a quantity the engine computed exactly. + +### 2.9 Order and cut + +**GQ.** `order { key [asc|desc], … }` with property keys, system fields, +return aliases, or expressions textually present in `return`; a leading search +function; nulls first ascending, last descending; a *total* order with entity-id +tiebreaks appended automatically (binding-name order), omitted only where rows +are indistinguishable. `limit N`, integer literal. No `offset`/`skip`, no +cursor, no parameterized limit. Without `order`, `limit` is any N rows. + +The total-order guarantee is stronger than every comparator's default. SQL's +`ORDER BY` is a partial order unless the user adds a unique key; `LIMIT … +OFFSET` paging over a partial order skips and repeats rows, a widely documented +bug class. Cypher's `ORDER BY … SKIP … LIMIT` has the same property; Neo4j +documents that ties are returned in an unspecified order. GQL: `ORDER BY … +OFFSET … LIMIT`, same. GraphQL has no ordering in the core; Relay's connection +spec standardizes *cursor* paging (`first`/`after`, opaque cursors, `pageInfo`), +which is the right abstraction and which GQ's deterministic order would make +trivial to implement. Elasticsearch: `sort` plus `search_after` with a +point-in-time, explicitly because `from`/`size` breaks past 10,000; it adds +`_shard_doc` as the tiebreaker, which is GQ's id tiebreak by another name. +Vespa: `order by` then `offset`/`hits`, with a documented limit. LanceDB: +`limit`/`offset` on the builder. + +GQ has the determinism and lacks the paging primitive. Keyset paging is +expressible by hand (`$p.name > $last or ($p.name = $last and $p.@id > $last_id)`) +now that `or` exists, but the client must know the tiebreak rule and the limit +cannot vary per call because it is a literal. + +**Verdict.** Three small changes, in order of value: (1) `limit $n` with an +`I64` parameter, (2) an opaque continuation token in the read envelope, +derived from the total order the engine already computes (the engine's own +`commit list --page-token` is the precedent), (3) `offset` only if a client +needs random-access pages, since it is the weakest form. + +### 2.10 Retrieval + +**GQ.** Full-text predicates `search`, `fuzzy`, `match_text` (positive, in +`match`, index required at compile time `T27`), rankings `nearest` (L2 or text +via the configured embedder), `bm25`, and `rrf(a, b [, k])`, each written as +the *leading order key*. Filters in `match` are applied before ranking; `limit` +is required for `nearest`; projected scores must repeat the order key +(`T33`); `rrf` arms must rank one binding or unconnected bindings (`T59`); a +ranking may root a traversal at the ranked binding regardless of declaration +order; the result order is score, then the remaining keys, then ids. Under +aggregate or `distinct` returns, the leading search function *selects the +population and does not order*. `ann_nprobes`, an overfetch ladder and a +prefilter gate are declared plan policy, visible in `explain`, and the read +envelope reports retrievals and metric kinds with `recall: approximate|exact`. + +Compared shapes: + +- *pgvector in SQL*: `ORDER BY embedding <-> $q LIMIT k` is exactly GQ's + shape, but with an HNSW or IVFFlat index the `WHERE` is applied *after* the + index scan, so a filtered top-k can return fewer than `k` rows + (`hnsw.iterative_scan` is the mitigation); hybrid is a hand-written join of + two CTEs with `1/(60+rank)`; scores are ordinary expressions so weighting and + thresholds are free. +- *Neo4j Cypher 25*: the procedures (`db.index.vector.queryNodes`, + `db.index.fulltext.queryNodes`) are deprecated as of 2026.04 and 2026.10 in + favour of a `SEARCH` subclause of `MATCH`: `SEARCH m IN (VECTOR INDEX idx FOR + $q WHERE m.year > 2000 LIMIT 5) SCORE AS s`. Its inner `WHERE` prefilters + inside the index on declared filter properties, AND-only; labels and the outer + `WHERE` post-filter; the pattern is limited to one bound variable and at most + one hop; there is no built-in fusion. GQL standardizes nothing here. +- *Elasticsearch*: scoring is the query tree (`bool` with `must`/`should` and + `filter` for non-scoring), `knn` with `num_candidates` and filters, and the + retriever tree (`rrf`, `linear`, `text_similarity_reranker`) composing + retrievers of retrievers. Most expressive hybrid surface of the nine; + scoring is opaque unless `explain: true`. +- *Vespa*: the cleanest separation. `where` decides the *match set* + (`nearestNeighbor(embedding, q)` with `targetHits`, `weakAnd`, `userInput`), + and the *rank profile* in the schema decides order with first/second/global + phases over typed rank features (`bm25(title)`, `closeness(field, + embedding)`, tensor expressions, ONNX). Hybrid is `rank(nearestNeighbor…, + userInput…)` plus an expression. `order by` is separate again. +- *LanceDB*: `.search(vec).where("…", prefilter=True).limit(k)`, FTS via + `query_type="fts"`, hybrid via reranker objects (RRF, linear, cross-encoder). + Prefilter is an explicit flag because post-filter is the historical default. +- *Datalog/Cozo*: `~docs:idx{ … | query: $q, k: 10, bind_distance: d }` as a + special relation inside a rule body; the result is just more atoms, so + joining with graph patterns is natural, and ordering is a separate `:order`. + +GQ's position: its filter-before-rank contract, exact total order, compile-time +index requirement, and plan-declared approximation policy are *correctness* +properties none of the search engines promise and the graph databases do not +have at all. Its weakness is *expressiveness*, and the weakness has one root: +retrieval is modelled as an order key, so the population-selecting role and the +sorting role are one syntactic object. Consequences: + +- Fixed fusion: only `rrf`, only two arms, no weights, no linear combination, + no reranker stage, no score arithmetic. +- No score threshold: `nearest(...) < 0.3` as a *filter* is not writable + because `nearest` is not a value in `match`; "everything within radius" + is inexpressible. +- The aggregate/`distinct` rule ("selects the population, does not order") + is the overload made visible: the same token does two jobs and in one + context does only one of them. +- Multi-field text (`bm25` over title *and* body) needs two arms; three + fields are impossible. +- No phrase/proximity/boosting; `fuzzy` is the one analyzer control. +- Heterogeneous search across types (#648) has no home, because a ranking + belongs to one binding's property. + +Vespa's split is the model to learn from, adapted: a *retrieval* position that +yields typed score columns (`$d.@score`, `$d.@distance`) over a declared +candidate window, and a plain `order` over any expression including those +columns. RFC 0048 (retrieval algebra, PR #793) appears to be this; this +evaluation endorses its direction. + +**Verdict.** Keep the contracts (prefilter, total order, `T27`, declared +policy, descriptors). Move retrieval out of `order` into its own position so +scores become values; then weights, thresholds, more arms and rerankers are +expressions, not grammar. + +### 2.11 Writes + +**GQ.** Inside a `query` declaration: `insert T { … }`, `update T set { … } +where expr`, `delete T where expr`. Several inserts/updates *or* several +deletes per query (D2 rule), never both; one query is one commit; a no-match +mutation publishes no commit. Assignment values are constants per invocation +(literals, parameters, `now()`, Boolean/comparison expressions over them); +keyed insert is upsert by derived id; unkeyed insert is strict; edges have no +`update`; `where` is the read expression language over the target type's own +properties and `@id/@src/@dst`; no binding variables (`T14`), no aggregates or +search calls (`T44`). `--if-commit` gives read-modify-write; `as commit` and +`as unattached branch` give proposals and private snapshots. Loads are `load +csv` statements or JSONL with `append|merge|overwrite`. + +Comparators mix reads and writes freely: Cypher `MATCH (p)-[r:KNOWS]->(q) +WHERE … DELETE r`, `SET p.visits = p.visits + 1`, `MERGE` with `ON CREATE`/`ON +MATCH`, `FOREACH`; GQL `INSERT`, `SET`, `REMOVE`, `DELETE`, `DETACH DELETE` +chained after `MATCH`; SQL `UPDATE … FROM`, `INSERT … SELECT`, `MERGE`, +`DELETE … USING`, `RETURNING`. GraphQL mutations are opaque resolver calls +executed serially. Datomic transacts datoms plus transaction functions for +computed writes; Soufflé is batch-only; Cozo `:put`/`:rm` take a query's +output as the rows to write (query-to-write composition). Elasticsearch +`update_by_query` with Painless scripts; `_bulk`; optimistic concurrency via +`if_seq_no`/`if_primary_term`. Vespa document API with partial updates and +`condition` test-and-set. LanceDB `merge_insert` (`when_matched_update_all`, +`when_not_matched_insert_all`), `update(where, values)`, `delete(where)`. + +GQ is the most restrictive write language of the nine, and the restriction is +load-bearing: + +- A mutation's effect set is computable from its text and parameters before + any read, so Cedar policy, `write_max_bytes`, the 8,192-entity bound and the + removed-ID budget are all checkable up front. +- Constants per invocation make retry idempotent and make `as commit` a + replayable proposal. +- No read in a write means no read-set to conflict on except the predicate's + own rows; `--if-commit` is the explicit, whole-graph version of that. +- D2 keeps the staging protocol a two-case problem. + +What it cannot do, and what agents do instead: + +| Need | GQ today | Cost | +|---|---|---| +| Increment/append (`visits + 1`, `tags += "x"`) | read, compute client-side, update with `--if-commit` | 2 round trips, retry on conflict | +| Delete by pattern ("edges from `$a` to anyone retracted") | read ids, then `delete E where @id in $ids` | 2 round trips | +| Update an edge's property | reinsert keyed edge, or delete + insert unkeyed | fine for keyed; two mutations (two commits) for unkeyed, which D2 forbids in one | +| Write what a query computed (materialize) | export + load | out of the language | +| Clear a property to null | nullable param, omitted | awkward but sound | + +**Verdict.** Keep statement-shaped writes, constants per invocation, upsert by +key and D2 for now. The RFC's own note ("the mutation `where` becoming the same +expression type as a read filter is the step a later RFC needs" toward +`MATCH … DELETE`) names the right next step: a *pattern-scoped* mutation, +`delete $e where { $a: Person { slug: $s } $a $e:knows $b $b.retracted }`, +that keeps the write's effect set visible (one target type) while letting the +pattern select it. Computed assignment (`set { visits: visits + 1 }`) waits on +arithmetic (2.14) and needs a read-set rule; it should be designed with +`--if-commit` as the default, not an exception. Edge `update` for keyed edges +is a gap with no principled reason. + +### 2.12 Time and branches + +**GQ.** Branch create/delete/merge/list/diff/merge-base as statements; merge +`dry run`; `as commit` (unpublished, publish/drop later); `as unattached +branch` (private, never publishes, chainable `on`); historical reads by +`--snapshot `; `--if-commit` preconditions; change feeds (`changes +poll`), commit list/show paging. Branch *selection* for a read or write is +outside the text (`--branch`, request field), branch *control* is inside. + +No comparator has this in the language. SQL has none (Dolt adds `dolt_*` +procedures and `AS OF`; Postgres has transaction snapshots, not versions). +Cypher/GQL: none (GQL has session/transaction statements; Neo4j has no +time travel). GraphQL: none. Datomic is the closest relative: a database is a +value, `as-of`, `since`, `history` produce filtered database values, and the +query runs against a value, so "which version" is a *parameter of the +query*, not text in it, which is GQ's `--snapshot`. Datomic has no branches. +Cozo has validity-time triples. LanceDB has dataset versions, `checkout`, +`restore`, tags and branches, all at the *table* level; its docs say it +"does not reconcile one branch's history with another or detect conflicts", +and merge is Enterprise-only and limited to added columns; there is no +cross-table coordination and no language. Elasticsearch and +Vespa have none. + +**Verdict.** Keep; this is a genuine differentiator and the statement form was +the right call (one transport, testable in GQT). One asymmetry to resolve +eventually: `--snapshot`/`--branch` as request fields versus `branch …` as +text means a stored `.gq` file cannot say which branch it reads, while it can +say which branch it merges. A `read on `/`at commit ""` clause on +a declaration would close that, and Datomic's precedent suggests keeping it a +*parameter* (`$branch: Branch`) rather than a literal, so stored queries stay +branch-agnostic. + +### 2.13 Reuse + +**GQ.** The unit of reuse is the named, parameterized query, stored in the +cluster's registry and invoked by name, exposed as MCP tools with +`@description`/`@instruction`, and bound to CLI aliases. There is no view, no +rule, no fragment, no user function, and no way for a query to use another +query. + +SQL: views, materialized views, CTEs, functions, procedures. Cypher: none +built in (Neo4j has user procedures in Java and APOC). GQL: procedures and +`CALL`. GraphQL: fragments (reuse of *selection shape*), schema stitching. +Datalog: rules are the whole language, and a rule is a reusable derived +relation; this is Datalog's defining strength, and Rel/Logica/Soufflé all build +large programs from them. Vespa: rank profiles inherit, query profiles store +parameters, fieldsets group fields. Elasticsearch: search templates +(Mustache), stored scripts, index aliases. LanceDB: none. + +For GQ's workload the gap matters more than for a general database. The +schema-design guide's principle 7 ("store what was observed or decided; +compute the rest") and principle 9 (`GraphPolicy` with `check_query`) both +depend on *shared derived relations*: "a former customer", "a contested +claim", "an orphan assertion". Today each is a stored query that returns +rows; no other query can *bind* to it. An agent that wants "contested claims +about customer X" rewrites the contested-ness pattern inline, and two agents +write it two ways, which is exactly the convergence failure the guide warns +about. + +**Verdict.** This is the highest-leverage missing feature after optional +match. A *typed view*: a stored read query whose result columns are declared +like a node type, usable as a binding (`$c: ContestedClaim` where +`ContestedClaim` is `view ContestedClaim($since: Date) = query …`). It is the +Datalog rule in GQ clothing, it is non-recursive (so stratification is +trivial), and it reuses the one closure step recommended in 2.2: a view is +inlined at compile time, so the planner still sees one component shape. + +### 2.14 Expressions + +**GQ.** Leaves: property access, system fields, literals (`"s"`, `1`, `1.5`, +`true`, `date("…")`, `datetime("…")`, `[…]`), parameters, `now()`. Operators: +six comparisons, `contains` (substring or list membership), `starts_with`, +`in`, `is [not] null`, `and`, `or`, `not`. Calls: the five aggregates and the +six search/ranking functions. That is the entire expression language. No +arithmetic, no string functions (`lower`, `trim`, `length`, concat), no date +arithmetic or extraction (`year(d)`, `d + 7d`), no `case`/`if`, no `coalesce`, +no list construction or indexing, no map/struct literals, no type conversion. + +Every comparator has all of these. SQL has the largest library; Cypher/GQL +have arithmetic, string, list, map, temporal and `CASE`; Datomic allows any +Clojure function in a predicate; Soufflé has arithmetic and string functors; +ES has Painless scripts and `runtime_fields`; ES|QL has `EVAL` with a function +library; Vespa has rank expressions and `userInput` grammars; LanceDB's filter +string is DataFusion SQL with its function library. + +The shared-expression RFC chose this scope deliberately and recorded the rule +the next step needs: a `can_fail` property per node so pushdown keeps a +fallible conjunct behind its guards. Three consequences of the gap for agents: +every derived value is computed client-side (`age` from `birth_date`, +`amount * rate`), so it cannot be filtered or ordered on server-side; string +normalization for keys (`lower(email)`) is impossible, which the schema-design +guide's convergence goal would want at write time; and `@check(p, "regex")` is +the only regular-expression surface, and it is write-time only. + +**Verdict.** Add the smallest total library first: arithmetic (`+ - * /` with +typed overflow refusal and explicit division semantics), `lower`/`upper`/ +`trim`/`length`, date extraction and `now() - duration`, `coalesce`, and a +`case` expression. Each is total or has one documented failure (division by +zero), so the `can_fail` rule stays simple. Resist a large library; resist +user-defined functions until views (2.13) exist, since a view covers most +"function" requests without a new execution surface. + +### 2.15 Result shape + +**GQ.** Flat rows of Arrow-typed columns, serialized with `arrow-json` +conventions (null keys omitted, dates as strings, `F32` at 32-bit width, +integers bare). A bare `$p` projects one struct column with `@id` and every +non-Blob, non-Vector property. Edge bindings cannot be projected bare; +blocks cannot be projected; no lists of nodes, no nested objects, no paths. +The envelope carries `columns`, `rows`, `graph_commit_id`, `retrievals`, +`metrics`, `usage`, and `profile` on inspected runs. + +GraphQL is the opposite pole: the result is a tree whose shape *is* the query, +which is why clients love it (one request, exactly the needed fields) and why +servers suffer (N+1, no server-side joins). Cypher returns rows whose cells may +be nodes, relationships, paths, lists and maps, so `RETURN p, collect(f) AS +friends` gives one row per `p` with a nested list. GQL likewise. SQL returns +flat rows; nesting is `ARRAY_AGG`/`JSON_AGG`. Datomic's *pull* API nests by +pattern (`[:person/name {:person/friends [:person/name]}]`), and `:find` with +pull returns nested maps in a flat tuple. Elasticsearch returns hits with +`_source` (arbitrary JSON) plus `inner_hits` for nested matches and an +aggregation tree. Vespa returns hits with summary fields, including structs and +arrays, and grouping results as a tree. LanceDB returns Arrow, so nested +columns are natural. + +Flat rows are the right *canonical* answer for a columnar engine and the right +thing for GQT to compare; the question is whether one nested level is worth +having. The evidence that it is: #649 (entity plus neighbourhood), #1050 +(entity plus counts), the N+1 pattern in the schema-design guide's "node page" +discussion, and the fact that `return { $p }` already projects a struct, so +`list` (Arrow-native) is one step away. + +**Verdict.** Keep flat rows as the model and add exactly one aggregation that +produces a collection: `collect($f)`/`collect($f.name)` as `[T]` or +`[struct]` columns under the existing implicit-grouping rule. With a projected +`count { }` and an optional block, that covers the entity-page shape in one +query, with no tree result model and no change to the row envelope. + +### 2.16 Self-description + +(Requirement added 2026-10-29: the schema must stay queryable as it grows, +through the same language as the data. The catalogue types are that answer; +the GQ 3 RFC reserves catalogue edges between them and makes a catalogue type +a node type for every purpose of the grammar, including a `retrieve` arm over +its `@instruction` text.) + +**GQ.** `explain` as rows (logical, physical, DataFusion trees, passes, +assumptions, validation scope); `profile` rows for inspected runs; the engine +catalogue as `@` node types joinable with user types; `omnigraph schema show` +for the schema; `lint` with an `operation` descriptor (`result`, `reads`, +`writes`) per query; `show all` for settings; read envelopes with retrieval +and metric descriptors. + +SQL: `EXPLAIN [ANALYZE]`, `information_schema`, `pg_catalog`, all queryable +tables. Cypher: `EXPLAIN`/`PROFILE`, `SHOW INDEXES|CONSTRAINTS|PROCEDURES`, +`db.schema.visualization()`. GQL: `CALL`-based catalog access, implementation +defined. GraphQL: `__schema`/`__type` introspection, the best-in-class example +of a schema fully queryable in the language itself, on which every GraphQL +IDE, codegen and client depends. Datomic: the schema *is* data (attributes are +entities with `:db/valueType`, `:db/cardinality`), so `[:find ?a :where [?a +:db/valueType]]` lists it, the purest form. Cozo: `::relations`, `::columns`, +`::explain`. ES: `_mapping`, `_explain`, `_validate/query`, profile API. +Vespa: `trace.level`, no schema query. LanceDB: `schema`, `explain_plan()`, +`analyze_plan()`. + +GQ's `explain` and catalogue are strong; the schema is the omission. There is +no `match { $t: @NodeType }` or `$p: @Property` with `type`, `nullable`, +`key`, `indexed`, `description`, `instruction`, nor `@EdgeType` with `src`, +`dst`, `card`. The catalogue RFC built the mechanism (registry-derived +read-only `@` types); the schema is the registry that agents need most, and +`@instruction` text is already written *for* agents. + +**Verdict.** Add `@NodeType`, `@EdgeType`, `@Property` (and `@Constraint`) +catalogue types from the accepted schema, snapshot-pinned like everything else. +It gives agents schema discovery in one query, lets a `GraphPolicy` check +query join schema facts to data, and answers #1058's "which properties does +this commit hold" as a query. + +### 2.17 Refusal contract + +**GQ.** Every refusal: a stable code (`Q…` parser, `T…` typechecker, `P…` +planner; meaning frozen, text may improve, numbers never reused), a position +(line, column, byte) or stage and expression, an `expected` line, one `fix`, +and optionally `suggestion` edits with `machine_applicable`/`needs_review` +applicability. A query-shape refusal is 400, never 500; an index prerequisite +is 409 with a typed detail. The grammar is word-bounded (`nothing`, `android` +are identifiers), reserved words are six and named in their error, keywords are +lowercase, every block is braces. The measured LLM run: 45/45 correct, 8/45 +first attempts refused, all for one shape, now `Q002` with a machine-applicable +edit. + +SQL errors: SQLSTATE codes exist but position and fix quality vary by engine; +PostgreSQL gives a caret position and `HINT`. Cypher: `Neo.ClientError. +Statement.SyntaxError` with line/column and a caret, GQLSTATUS codes in Cypher +25, generally good. GQL defines GQLSTATUS. GraphQL: validation errors with +`locations` and `path`, well specified, and the schema makes most errors +static. Datomic: exceptions with EDN data, variable quality. Soufflé: compiler +errors with positions. ES: verbose JSON `reason` chains, no stable codes. +Vespa: often opaque (`Could not create query from YQL`). LanceDB: DataFusion +parse errors as Python exceptions. + +For machine authorship, two things matter beyond errors: *regularity* (one +way to write each thing) and *familiarity* (the author's prior). GQ wins on +regularity and loses on familiarity: `$e.@type`, lowercase-first edge names +derived from PascalCase declarations, `{m,n}` as distance, `count { } > 2`, +`nearest` in `order`, no `null` literal, no `mutation` wrapper, and the +direction-from-schema rule are each one more thing a model trained on Cypher +and SQL must be told. The skill carries them; the diagnostics repair them; but +the novelty tax is real and compounds with every new spelling. + +**Verdict.** Keep, and treat the four-field diagnostic as a language feature +with the same compatibility discipline as the grammar. Spend novelty +deliberately: where a GQ spelling differs from Cypher/SQL *without* a semantic +reason, prefer the familiar one (for instance `!=` and `is null` already do; +`offset` and `collect` should, if added). Where it differs *for* a reason +(`{m,n}` distance, no `null`), say the reason in the error text, as the +`Q002` fix does. + +--- + +## 3. Head to head + +Each section: what GQ shares with the language, where GQ is stronger, where +it is weaker, and what is worth borrowing. Facts about the comparators were +checked against their primary documentation on 2026-10-28; the few that could +not be are marked *recalled*. + +### 3.1 SQL (SQL:2023, SQL/PGQ, pgvector and friends) + +**Shared.** Static typing over a declared schema; three-valued logic with the +same truth tables (GQ took SQL's `AND`/`OR`/`NOT` rules and dropped the `NULL` +literal); `ORDER BY … LIMIT` as the cut, and pgvector's `ORDER BY embedding <-> +$q LIMIT k` as the exact shape of GQ's `order { nearest(…) } limit k`; exact +widening of integer sums (PostgreSQL `sum(bigint) → numeric`, GQ's 128-bit +accumulator); `EXPLAIN` as rows and `information_schema`/`pg_catalog` as +queryable catalogs (GQ's `explain` rows and `@` types); stable five-character +SQLSTATE classes (GQ's `Q`/`T`/`P` codes); typed prepared-statement parameters +(`PREPARE p (int, text) AS …`; GQ's declared `$params`); one statement, one +snapshot, one commit for a data-modifying CTE (GQ's one-commit mutation). + +**GQ stronger.** For a graph workload, SQL/PGQ is a *view* (`CREATE PROPERTY +GRAPH … VERTEX TABLES … EDGE TABLES`) over tables, and `GRAPH_TABLE … COLUMNS` +must flatten every match to scalars; its adoption is fragile (Oracle spells +labels `IS`, DuckDB `:`; PostgreSQL merged `GRAPH_TABLE` for PG19 in March 2026 +and reverted it in September). Every join is spelled; clause order is not +evaluation order; `NOT IN` with a null returns nothing, `sum()` of nothing is +`NULL`, and nulls group together but do not compare equal, the warts Date and +Darwen called "inherently flawed" and which GQ's subset avoids. `LIMIT … +OFFSET` over a partial order skips and repeats rows unless the author adds a +unique key; GQ's order is total. pgvector and DuckDB VSS apply `WHERE` *after* +the approximate index scan, so a filtered top-k can return fewer than `k` +rows (`hnsw.iterative_scan` and ACORN are the mitigations); GQ's +filter-before-rank contract and overfetch ladder make the filled `limit` a +promise. SQL has system-versioned tables (`FOR SYSTEM_TIME AS OF`) and no +branches; parameter nullability is not declarable. + +**SQL stronger.** Closure and composition: subqueries, derived tables, CTEs +(recursive, with `SEARCH`/`CYCLE`), views, functions, `UNION`/`EXCEPT`, +`LATERAL`, `UNNEST`, `JSON_TABLE`. Aggregation: explicit `GROUP BY`, +`HAVING`, `FILTER (WHERE …)`, ordered-set aggregates, and window functions, +which answer every "top-N per group" and running-total request in one +statement. The whole expression library. Writes: `MERGE` with `WHEN MATCHED +… BY SOURCE`, `INSERT … SELECT`, `UPDATE … FROM`, `RETURNING`. Fifty years of +tooling and training data. + +**Borrow.** (1) `HAVING` by any spelling. (2) `RETURNING`: a mutation that +answers the ids it touched saves the read that follows it. (3) A parameterised +`LIMIT` and `OFFSET`. (4) The exact-integer result type for integer `sum`. +(5) Views. (6) SQL:2023's `UNIQUE NULLS [NOT] DISTINCT` is a reminder to state +in the `.pg` guide how `@unique` treats a nullable member: the validator drops +a null key from the unique image (nulls never collide), and the guide does +not say so. + +### 3.2 ISO GQL (39075:2024) + +**Shared.** The pattern vocabulary: GQ's `$a knows{1,3} $b` and GPML's +`(a)-[:Knows]->{1,3}(b)` are the same quantifier spelling; `EXISTS { }`; +`OPTIONAL MATCH` in GQL and the requested `optional { }` in GQ (#658); value +types that carry `NOT NULL` (GQ's `T?` inverted); closed graph types (GQ is +always closed); `INSERT` rather than `CREATE`; in-language control statements +(GQL: `SESSION SET`, `START TRANSACTION`, `COMMIT`; GQ: `branch`, `commit`, +`set`); standardised GQLSTATUS codes (GQ's frozen codes). Both are young +languages written after Cypher with a chance to fix it. + +**GQ stronger.** GQL standardises no full-text, vector or ranking, no +constraints beyond graph types, no indexes, no `EXPLAIN`, no views, and no +`MERGE`/upsert (Neo4j: "MERGE, FOREACH, and LOAD CSV" are "not in GQL"). GQL v1 +has no `STARTS WITH`/`CONTAINS`/`IN` (Neo4j lists them as Cypher-only); GQ has +`starts_with`, `contains` and `in`. GQL's path machinery, modes `WALK`/`TRAIL`/ +`SIMPLE`/`ACYCLIC` times selectors `ANY`/`ALL SHORTEST`/`SHORTEST k GROUPS` times +match modes `REPEATABLE ELEMENTS`/`DIFFERENT EDGES`, is what path-as-value +costs; implementations already diverge on the defaults (Ultipa `TRAIL`, Spanner +`REPEATABLE ELEMENTS`, Neo4j `DIFFERENT RELATIONSHIPS`) and on `YIELD`, so "GQL" +is a family at launch. GQ's reachability semantics need none of it. The +standard runs to 610 pages; GQ's grammar is 158 pest rules with an executable +corpus of 425 cases. + +**GQL stronger.** Linear composition: a statement produces a working table +and `NEXT` hands it to the next (`MATCH … RETURN a, count(p) AS n GROUP BY a +NEXT FILTER n >= 3 RETURN …`), which is `HAVING`, top-N per group and +multi-stage aggregation in one mechanism. `LET` (non-blocking) and `FILTER` +(standalone) are cleaner than Cypher's `WITH`. Explicit `GROUP BY` removes the +implicit-grouping trap. `OPTIONAL MATCH`. Paths, records and lists as +first-class result values. `IS [NOT] UNKNOWN`. An open ANTLR grammar. + +**Borrow.** (1) `OPTIONAL MATCH` semantics as the specification for #658: +an outer join whose inline filters belong to the pattern, nullable bindings +in the result schema. (2) `FILTER` after a grouped `return` as the HAVING +form, which fits GQ's one pipeline without `NEXT`. (3) A documented warning +that GQ's `{m,n}` means *distance* where GPML's identical spelling means +*path length*; the shared spelling with different meaning is the single most +likely cross-language error. + +### 3.3 Cypher (openCypher, Neo4j Cypher 25) + +**Shared.** Implicit grouping on non-aggregated return items; `EXISTS { }` and +`COUNT { }` block syntax (GQ's blocks are Cypher 5's with the braces and +without the keyword in the `not` case); `IS NULL`; three-valued logic with +"anything that is not `true` is false" in a filter; `STARTS WITH`/`CONTAINS` +case-sensitive (GQ `starts_with`/`contains`); `EXPLAIN`/`PROFILE`; stable +GQLSTATUS codes; and, since 2026.01, Neo4j's `MATCH (m:Movie) SEARCH m IN +(VECTOR INDEX idx FOR $q WHERE m.year > 2000 LIMIT 5) SCORE AS s`, a +declarative, in-index-prefiltered ranking inside `MATCH`, which is Cypher +moving toward GQ's shape and deprecating its procedures (`db.index.vector. +queryNodes` 2026.04, full-text 2026.10). + +**GQ stronger.** Typing: Cypher parameters are untyped, properties are +untyped per node, and a property typo returns `null` instead of an error, +which Neo4j's own documentation names as the recurring LLM failure; GQ refuses +with `T2`. Schema: Cypher constraints are unique/exists/type/key only, no +range or check; GQ has `@range`, `@check`, `@card`, enums. Order: Cypher's +`ORDER BY` is partial; GQ's is total. Traversal: `*1..3` is paths with +`03N91` warnings for unbounded ones; GQ's `{m,n}` is distance with a work +budget. Retrieval: `SEARCH` prefilters only on declared filter properties, +AND-only, one binding, at most one hop, everything else post-filters, and +hybrid is hand-built RRF over `CALL { } UNION ALL`; GQ prefilters on the whole +pattern and has `rrf`. `OPTIONAL MATCH` chains fan out rows ("hey LLM, you're +using OPTIONAL MATCH wrong" is a Neo4j blog title); `MERGE` on a whole pattern +silently duplicates sub-parts. Two live language versions with monthly +deprecations; GQ's codes are frozen and its corpus executable. No branches, +no time travel. + +**Cypher stronger.** Composition: `WITH`, `LET`, `FILTER`, `NEXT`, `CALL +(scope) { }`, `UNION`, `UNWIND`/`FOR`, list/map/pattern comprehensions, +`reduce`. `OPTIONAL MATCH`. `COUNT { }` and `COLLECT { }` as *values* usable +anywhere. Results as nodes, relationships, paths, lists, maps with map +projection `n { .id, .name }`. The expression library (`CASE`, temporal, list, +string, `coalesce`). Writes: `MATCH … SET/DELETE`, `MERGE ON CREATE/ON MATCH`, +`FOREACH`, `WHEN … THEN { } ELSE { }` conditional queries, `CALL (n) { … } IN +TRANSACTIONS OF 2 ROWS ON ERROR CONTINUE` batching. Quantified path patterns +with group variables and in-traversal `WHERE`; `SHORTEST k`. Vector index DDL +with quantization. A decade of training data. + +**Borrow.** (1) `COUNT { }`/`COLLECT { }` as values. (2) `optional { }` with +Cypher's rule that the block's own filters are part of the pattern. (3) Map +projection as a partial node object: `$p { name, age }` between bare `$p` (all +properties) and one property at a time. (4) Planner *notifications* beside +errors: Neo4j's `03N90` (cartesian product) and `03N91` (unbounded pattern) +are the warning channel GQ lacks; GQ plans a `CrossJoin` silently and scans a +whole ranked type for a destination-ranked traversal (#758) with no word to the +author. (5) `CALL IN TRANSACTIONS`' idea, not its syntax: a batched mutation +form for the 8,192-entity bound. + +### 3.4 GraphQL + +**Shared.** Schema-first, closed type system; declared, typed variables with +explicit nullability (`$episode: Episode`, `String!` versus GQ's `T?`); static +validation of the document against the schema before execution; named +operations as the unit of exposure; metadata on types for clients +(`@deprecated`, descriptions; GQ's `@description`, `@instruction`); one +spelling for a reserved namespace (`__typename`, `__schema`; GQ's `@id`, +`@Operator`). Of the nine, GraphQL is the one GQ most resembles in *posture*: +both are contracts written for machine consumers first. + +**GQ stronger.** GraphQL has no predicate language, no ordering, no +aggregation and no joins; every filter is a schema author's convention +(Hasura's `where: { name: { _eq: … } }` is one of dozens), so a client must +learn each API. GQ has one expression language, one grouping rule, one order +contract. GraphQL's nulls *bubble*: an error in a non-null field nulls its +parent up to the nearest nullable ancestor, which is why most schemas make +fields nullable "only to limit how far error propagation spreads" (GAP-49, +semantic nullability); GQ's nullability is a type fact, not an error-containment +device. GraphQL's N+1 is structural (every nested field is a resolver call) and +fixed by an ecosystem tool (DataLoader); GQ's traversal is a planned join. +GraphQL has no time travel and takes "a strong opinion on avoiding versioning"; +GQ has branches, snapshots and commits in the language. + +**GraphQL stronger.** Result shape. The selection set *is* the response: one +request returns the entity, its friends' names and their companies as a tree, +in requested order (§7.2.2). GQ returns flat rows and needs N queries or a +client-side join for the same page (#649, #1050). Introspection: `__schema` +and `__type` are queryable in the language, and the whole tooling ecosystem +(codegen, IDE validation, Relay compiler) stands on it; GQ's schema is +`schema show` output, not a queryable relation (2.16). Fragments reuse +selection *shape* across operations; GQ has no reuse below a whole query. +Pagination: Relay's connection spec (`first`/`after`, opaque cursors, +`pageInfo`) is a standard clients implement once; GQ has `limit N` and no +cursor (2.9). + +**Borrow.** (1) Introspection as query: `@NodeType`/`@EdgeType`/`@Property`. +(2) One nested level in results (`collect`), not a tree. (3) Relay-style +opaque continuation tokens derived from the total order. (4) The discipline +that every result field's nullability is a declared type fact is already GQ's; +keep GraphQL's *explicit non-null marker* idea in mind if `T?` ever becomes the +default rather than the exception. + +### 3.5 Datalog (classical, Soufflé, Datomic, CozoDB, Rel) + +**Shared.** More than any other comparator, GQ's `match` block *is* a Datalog +rule body: typed unary atoms (`$p: Person`), binary atoms (`$a knows $b`), +built-in predicates (filters), stratified negation (`not { }`), aggregate +sub-goals with a safety rule (`T39` is range restriction), join by variable +repetition, set semantics on unbound traversals, and a head (`return`) whose +non-aggregated variables are the implicit group keys, which is exactly Cozo's +and Logica's rule-head aggregation. Datomic's `(not-join [?a] …)` requiring +the correlation variable is GQ's `T39`; Cozo's "negated atoms cannot introduce +bindings" is GQ's block rule; Soufflé's rejection of `A(x) :- !B(x). B(x) :- +!A(x).` is the stratification GQ gets for free by having no recursion. + +**GQ stronger.** Typing at the query surface: Datomic's `:in` is untyped and +attributes are checked against data, not a declared schema; Cozo's columns are +typed but queries are dynamically checked; only Soufflé and Rel type-check +statically as GQ does. Ordering: Datomic has no `ORDER BY` at all ("handle +this in your application code"); Soufflé none; Cozo only on the entry rule. +Null semantics: Datomic has absence-only (clean but forces every optional +attribute into `missing?`/`get-else` dances); Cozo admits a totally ordered +`Null`; GQ's SQL-style 3VL with no literal is the most teachable of the three. +Bounded cost: Datalog recursion is unbounded by design and the engines warn +about it (Cozo "large recursive rules can be expensive", infinite relations +the compiler may accept; Rel non-monotone recursion "may oscillate forever"); +GQ's finite `{m,n}` and `traversal_work_limit` make cost a contract. Search: +Cozo's `~idx{ … | query: $q, k: 10 }` is a body atom, elegant, but ranking is +then a plain `:order` with no total-order or prefilter contract; GQ's +retrieval contracts are stronger. + +**Datalog stronger.** Reuse and recursion. A rule head is a relation every +other rule can use; a program is many rules; recursion is the normal case. +GQ has neither derived relations nor recursion, so "contested claim" cannot be +defined once and bound to (2.13), and transitive closure needs an explicit +finite bound. Schema as data: Datomic attributes *are* entities, so the +schema is queried with the same language as the data; GQ cannot (2.16). +Time: Datomic's `as-of`/`since`/`history` make the database version a *value* +passed to the query, and `history` exposes retractions as rows; GQ's +`--snapshot` is the same idea for a point, and the change feed is the history +read, but neither is composable inside a query. Cozo's `Validity` as a key +column is bitemporal modelling in the data model; GQ has transaction time +(commits) and leaves valid time to the user's schema, which the schema-design +guide recommends. Aggregates as values: Soufflé's `c = count : { … }` and +Cozo's head aggregates make a count a thing to project or compare to another +variable; GQ's blocks are predicates only (2.7). + +**Borrow.** (1) Non-recursive derived relations as typed views, the one +Datalog feature GQ's planner can absorb by inlining. (2) Aggregate blocks as +values with a variable, not only a constant, on the comparison's right side. +(3) Datomic's "database as a value": a `$branch`/`$at` parameter on a stored +query. (4) Cozo's `::describe` (human- and agent-facing documentation stored +beside a relation) is already GQ's `@instruction`; make it queryable. +(5) Do *not* borrow unbounded recursion; borrow Datalog's *set* semantics +confidence and say so in the docs when explaining `{m,n}`. + +### 3.6 Elasticsearch (Query DSL, ES|QL) + +**Shared.** An index is required for full text (a `text` mapping; GQ's +`@index`, refused otherwise by `T27`); a `filter` inside the `knn` clause is a +pre-filter applied during the search so that `num_candidates` survive (GQ's +contract); `_shard_doc` as the implicit tiebreak under `search_after` (GQ's id +tiebreak); `rrf` with `rank_constant` 60 and a `rank_window_size` (GQ's `k` +60 and the nearest window); `_score` as a named column; `_validate/query` +(GQ's `lint`); `explain`/`profile`; ES|QL's typed `columns` and named +`?params` (GQ's `columns` and `$params`); `matched_queries` naming which +clause matched (GQ's `retrievals` descriptors). + +**GQ stronger.** One language: Elasticsearch has four (Query DSL, ES|QL, EQL, +KQL) with different type systems; ES|QL remaps `float` to `double`, treats +`text` as `keyword`, returns `null` for a function over a multivalued field, +and caps output at 10,000 rows whatever `LIMIT` says. Semantics by position: a +clause scores under `must` and not under `filter`; `minimum_should_match` +defaults to 1 or 0 depending on its siblings; the *same* filter is a pre-filter +inside `knn` and a lossy post-filter one level up. GQ's positions are typed by +context rules and refuse rather than reinterpret. Hard ceilings (`index. +max_result_window` 10k, 1000 fields, 65,536 terms) versus GQ's declared and +settable budgets. Schema inferred on first sight and then frozen (`dynamic: +strict` opt-in) versus declared. Relationships are physical choices (nested +hidden documents, same-shard `join`, a lookup index with `index.mode: lookup`) +each with "a significant tax"; GQ traverses typed edges. Near-real-time +visibility (1s refresh, `refresh=wait_for`) and per-document CAS versus one +atomic commit with read-your-commit. `lenient` and `ignore_unmapped` swallow +type errors silently; `cardinality` is HyperLogLog and `track_total_hits` stops +at 10,000; GQ counts exactly. Search templates are Mustache text substitution; +GQ's stored queries are typed. No versioning. + +**Elasticsearch stronger.** Retrieval expressiveness, by a wide margin: +`bool` with `should`/`minimum_should_match`/boosts, `function_score` decays, +phrase, proximity, fuzzy, wildcard, regexp, `multi_match`, highlighting, +`collapse`, `semantic_text`, and the retriever tree (`rrf` with per-arm +`weight`, `linear` with `minmax`/`l2_norm` normalizers, `rescorer`, +`text_similarity_reranker`, `diversify`, `pinned`). Aggregations: bucket, +metric and pipeline families nesting without limit, `composite` paging, +`bucket_selector` as `HAVING`. Deep paging with `search_after` over a +point-in-time. Runtime fields. ES|QL's `STATS … BY`, `EVAL`, `LOOKUP JOIN`, +`INLINE STATS`, and its hybrid pipeline `FORK … | FUSE RRF|LINEAR | RERANK`. +EQL's `sequence by … with maxspan` for temporal event patterns, an axis no +graph language here has. `_explain` says *why one document* matched. + +**Borrow.** (1) The retriever tree as the model for retrieval composition: +fusion is a node over retrievals, arms carry weights, normalizers are explicit. +(2) `similarity`/`min_score` floors: a score threshold predicate. (3) +`search_after` over a PIT is exactly a continuation token over a pinned +snapshot, which GQ already has for free. (4) Per-row contribution: GQ's +descriptors say what a score column *means*; Elasticsearch's `explain: true` +on `rrf` says what each arm contributed to *this row*, which an agent choosing +between results can use. (5) Snippets (`TOP_SNIPPETS`, highlighting): for +passages, the matched span is the answer, not the row. + +### 3.7 LanceDB and Lance + +**Shared.** The substrate: GQ runs on Lance, so LanceDB is the comparison +with GQ's own storage spoken directly. Both name `_distance` and `_score`, +prefilter by default, label quantised distances approximate until refined +(GQ's `recall: approximate`), serve unindexed rows by a flat fallback while +indexes lag (GQ's "indexes are derived"), version every write (Lance versions; +GQ graph commits), expose `explain_plan`/`analyze_plan` (GQ `explain`/ +`profile`), return Arrow, upsert by key (`merge_insert`; GQ keyed `insert`), +and attach the embedding model to the schema so a text query is embedded +automatically (LanceDB's registry in table metadata; GQ's `@embed` and text +`nearest`). + +**GQ stronger.** A language. LanceDB's query is a builder chain whose filter +is "supplied as an SQL query string": no bound parameters (injection is the +caller's problem), no validation before execution, no stored or named queries, +no `ORDER BY`, no aggregation, no join in the builder (those are DataFusion +SQL or the client). No relationships. Versions are *per table*; LanceDB's +branches "do not reconcile one branch's history with another or detect +conflicts", and merge is Enterprise-only and "promotes only added columns", +whereas GQ's branch is a whole-graph lineage with three-way merge, merge-base, +diff and dry run in the language. No uniqueness or key constraints +(`merge_insert` "relies on the caller keeping keys unique"); GQ has `@key`, +`@unique`, `@card`, `@range`, `@check`, enums. No policy. The filter dialect +has its own warts ("field names containing periods are NOT supported", the +plain FTS string "doesn't support `OR`, `AND`"). + +**LanceDB stronger.** The DataFusion expression library in filters (`LIKE`, +`regexp_match`, `CAST`, `array_has_any`, nested paths, date literals). +Structured full-text queries (`MatchQuery` with `fuzziness` and `boost`, +`PhraseQuery(slop)`, `MultiMatchQuery(columns, boosts)`, `BooleanQuery` with +`MUST_NOT`). Hybrid with pluggable rerankers (RRF, linear, cross-encoder, +Cohere, ColBERT, Jina, …) and an explicit `normalize: rank|score`. +`distance_range(lower, upper)` as a threshold cut. `refine_factor`, `ef`, +`search_effort`, `bypass_vector_index` knobs. `add_columns` filled by a SQL +expression (computed backfill at schema time). Full `MERGE` including +`when_not_matched_by_source_delete`. `offset`. + +**Borrow.** (1) `distance_range` as a `nearest` threshold predicate. (2) +Phrase and multi-field text queries. (3) A reranker *stage* as a plan node +with a provider named in `cluster.yaml`, the way embedding providers already +are. (4) SQL-fill `add_columns` as the backfill expression `schema apply` +needs for adding a required property to a populated type (#721). (5) Nothing +on branches: GQ's whole-graph lineage is the thing LanceDB's own docs +disclaim; say so in positioning. + +### 3.8 Vespa (YQL, grouping, rank profiles) + +**Shared.** Schema-first with typed fields including tensors (GQ's +`Vector(N)`); a required `targetHits`/`limit` for nearest-neighbour; exact +fallback when a filter is too selective (Vespa's `approximateThreshold`, GQ's +whole-type exact pass); prefilter as the default; a declared, inspectable +ANN/filter strategy (Vespa's `postFilterThreshold`, GQ's `rrf_plan` and +overfetch policy in `explain`); parameters kept out of the query string +(`@param`, `input.query(q)`; GQ's `$params`); a `relevance`/score per hit. + +**GQ stronger.** Vespa has no null predicate ("there is no way to query for a +field that is not set") while its *write-side* selection language has `== +null`, two languages with two semantics; GQ has one expression language with +`is null`. Vespa's counts are approximate by default under `nearestNeighbor`, +`wand` and `weakAnd`, group counts are HyperLogLog, and the session cache +"might cause inaccuracies"; GQ's counts are exact and its `nearest` is labelled +`approximate` honestly in the envelope. Vespa's YQL `timeout` is milliseconds +while the `timeout` parameter is seconds; `in` works only on `match: word| +exact` fields; string semantics depend on schema `match` modes invisible in the +query. GQ has one typed expression model with no schema-dependent operator +meaning. Vespa has no joins beyond child-to-parent attribute imports on +globally replicated parents; GQ traverses typed edges in both directions with +bounds. Vespa has no versioning, no snapshots, no transactions beyond a +test-and-set `condition`. + +**Vespa stronger.** Ranking as a first-class, separate, composable surface. +`where` decides the *match set*; `rank(a, b, …)` matches on `a` only and +scores on all; the rank profile decides *order* in four declared phases +(match-phase degradation → first → second → global) with declared budgets +(`rerank-count`, `rank-score-drop-limit`); rank features are typed +expressions (`bm25(title)`, `closeness(field, embedding)`, tensor algebra, +ONNX); hybrid is composed from primitives (`or`, `rank()`, +`reciprocal_rank_fusion`, `normalize_linear`) rather than one `rrf` operator; +`labeled()`/`itemRawScore` give per-subquery scores. Everything GQ cannot say +in 2.10 (weights, thresholds, three arms, rerank, multi-field) Vespa says as +an expression. Grouping as a pipeline (`| all(group(f) each(output(count())))`) +with nested groups and continuation tokens is more expressive than implicit +grouping, at the price of a third DSL. Query profiles are server-side, typed, +inheritable parameter sets that can hold the YQL itself with `%{name}` +splicing: stored queries with defaults and variants. + +**Borrow.** (1) The match/rank split: a retrieval position that yields score +columns, and `order` over expressions. (2) Declared cost budgets for ranking +phases as *language* (GQ has `traversal_work_limit`, `ann_nprobes` and the +overfetch ladder; a `rerank N` stage would follow the same pattern). (3) +Continuation tokens for grouped results. (4) Do *not* borrow annotation- +before-term syntax, three bracket conventions, or ranking defined in the +schema rather than the query: GQ's stored queries already give ranking a home +beside the data, in one language. + +--- + +## 4. Verdicts + +### 4.1 Where GQ sits + +Read against the nine, GQ is a *Datalog rule body* (typed atoms, join by +variable, stratified blocks, set semantics, implicit head grouping) with +*SQL's* typing and null logic minus the null literal, *pgvector's* ranking +shape, *Cypher 5's* block syntax, *Datomic's* "database as a value" for +snapshots, and *Vespa's* idea that cost budgets are part of the language. +Three things it has that none of the nine has together: whole-graph branch, +merge, diff and proposal as statements; a plan-truth contract (acceptance, +replay, declared switches, descriptors) that makes "what will this query do" +a checkable artifact; and a refusal contract designed for a machine author. + +Its one structural choice that no comparator makes is **non-closure**: the +language does not consume its own results. That choice is why the three +differentiators above hold, and it is the root of every expressiveness gap the +open issues report. + +### 4.2 Keep + +These are the load-bearing walls. Each has been argued in an RFC or is +derivable from the invariants, and each has a cost that is already paid. + +1. Schema-first, closed vocabulary; nothing in the query the storage model + lacks. +2. Static typing with declared, nullable parameters; explicit casts; result + schema inference. +3. Three-valued logic without a null literal; `is null` as the only null test. +4. Reachability semantics for traversal: set semantics on pairs, finite + `{m,n}` as distance, `traversal_work_limit` admission. +5. Stratified correlated blocks with the safety rule. +6. Filter-before-rank, one total order, index required at compile time, + approximation declared in the plan and the envelope. +7. Statement-shaped writes with constants per invocation, keyed upsert, D2, + one commit per query. +8. Control as statements: branches, commits, proposals, snapshots, load, + explain, settings. +9. One declaration shape; braces everywhere; word-bounded keywords; six + reserved words. +10. The four-field diagnostic with frozen codes and machine-applicable edits. + +### 4.3 Fix, in order of leverage + +Each item names the gap, the comparator that shows the shape, and what it +costs the planner. Items 1 to 3 remove most of the open language issues; the +rest are smaller and independent. + +| # | Change | Shows the shape | Issues | Planner cost | +|---|---|---|---|---| +| 1 | **Retrieval out of `order`.** A retrieval position yields typed score columns (`$d.@distance`, `$d.@score`) over a declared candidate window; `order` sorts any expression; fusion, weights, thresholds, more arms and a rerank stage become expressions and plan nodes, not grammar. Keeps every contract in 4.2 item 6. | Vespa `where`/rank split; ES retriever tree; LanceDB `distance_range`; RFC 0048 | #648 (partly), #917, the aggregate/`distinct` overload in 2.10 | The ranked `Scan` and `RankFuse` nodes exist; the change is where the compiler puts them and that scores become columns | +| 2 | **One closure step, three forms, all inlined:** (a) block aggregates as values in `return` (`count { } as n`, `exists { } as b`); (b) `optional { }` with Cypher/GQL outer-join semantics and nullable bindings; (c) typed views: a stored read declared as a binding type (`view ContestedClaim(...) = query …`), inlined at compile time. No `WITH`, no general subquery. | Cypher `COUNT {}`/`COLLECT {}`; GQL `OPTIONAL MATCH`; Datalog rules; SQL views | #1050, #658, #649, 2.13 | (a) `SubqueryAggregate` exists; (b) one new join kind (`LeftExpand`) and nullability propagation; (c) compile-time only | +| 3 | **Post-group filter and `collect`.** A Boolean aggregate expression as a filter after `return` (spelling: `having { }` or GQL's `filter`), and `collect($x)`/`collect($x.p)` producing `[T]`/`[struct]` under implicit grouping. | SQL `HAVING`; GQL `FILTER`; Cypher `collect`; Datomic pull | R3 in §5, #649 | One `Filter` above `Aggregate`; one Arrow list aggregate | +| 4 | **Smallest total expression library:** `+ - * /` with typed overflow refusal, `lower/upper/trim/length`, date extraction and `now() - duration`, `coalesce`, `case`. Each total or with one documented failure, so the `can_fail` pushdown rule stays one line. Then computed assignment in `update … set`, under `--if-commit` by default. | Every comparator | 2.14, 2.11 increment row | Expression evaluator arms; one `can_fail` flag | +| 5 | **Paging:** `limit $n` (I64 parameter), an opaque continuation token in the read envelope derived from the total order over the pinned snapshot, `offset` last. | ES `search_after` + PIT; Relay cursors; GQ's own `commit list --page-token` | 2.9 | Token = serialized last sort key + snapshot pin | +| 6 | **Schema catalogue types** `@NodeType`, `@EdgeType`, `@Property`, `@Constraint`, snapshot-pinned, carrying `@description`/`@instruction`. | GraphQL `__schema`; Datomic schema-as-data; `@Operator` precedent | #1058, 2.16 | Registry-derived rows, the catalogue mechanism | +| 7 | **Writes:** pattern-scoped selection (`delete $e where { pattern }`) with one target type; `update` for keyed edges; `set { x: null }` in assignment position; `returning` the affected ids. | Cypher `MATCH … DELETE`; SQL `RETURNING` | #660 lineage, 2.11 table | The mutation `where` already shares the read expression type; a pattern needs the read planner's `Expand` under the write door | +| 8 | **Traversal extras:** `$b.@distance` on the far end of `{m,n}`; multi-hop edge bindings as a list once `collect` exists. | GQL group variables; Cypher QPP lists | 2.6 | BFS already knows the distance | +| 9 | **Planner notifications** beside errors: a cross join of unconnected bindings, a destination-ranked scan of a whole type, a `bm25` without `limit`, a wildcard without a budget. Stable `N…` codes, same four fields. | Neo4j `03N90`/`03N91` | #758, #1057 | The planner knows each already | +| 10 | **Exact integer `sum`** returns an exact integer type (`I64` with overflow refusal) rather than `F64`. | PostgreSQL `numeric`, DuckDB `HUGEINT` | 2.8 | A result type change; `expect shape` in 33 cases | + +Four further items are about *evolvability* rather than any one gap and are +argued in §7.5: generic call syntax over a typed function registry, interface +bindings, a generic declaration annotation, and an edition pragma with an N-1 +compiler. They rank with items 1 and 2 above once agents are the authors. + +### 4.4 Resist + +Each of these is requested somewhere, has a comparator that does it, and would +cost one of the walls in 4.2. + +- **Unbounded recursion** (`*` as repetition, Datalog fixpoints). It breaks + the work-budget contract; finite `{m,n}` and a larger `traversal_work_limit` + are the honest form. +- **Path as a value** with modes and selectors (GQL/Cypher). Its cost is the + 610 pages; its benefit for agents is small; `@distance` and edge lists cover + the demand. +- **General subqueries and `WITH` pipelines** (SQL, Cypher, GQL `NEXT`). The + three closure forms in 4.3 item 2 cover the reported shapes without making + plan acceptance per-stage. +- **Read-write mixing in general** (Cypher `MATCH … SET`, SQL `UPDATE … FROM`). + Pattern-scoped selection with one target type is the bounded form. +- **A `null` literal in reads.** The mutation-side assignment case is the one + exception worth making. +- **User-defined functions** before views exist; most requests are views. +- **Schema-resident ranking** (Vespa rank profiles). Stored queries already + give ranking a home beside the data, in one language. +- **Open labels and dynamic properties** (Cypher, GQL open graph types). The + closed model is what makes the typing and the diagnostics possible. +- **Tree results** (GraphQL). One `collect` level, flat rows underneath. +- **Semantics by position** (Elasticsearch `must`/`filter`, `knn` filter + placement). Every GQ position is typed by a context rule; keep it that way + when retrieval moves. + +### 4.5 Two questions the evaluation could not settle from the sources + +1. How `@unique` treats a nullable member. The validator's unique image + (`validate/kernels.rs`) drops a row whose key is null, so nulls never + collide, SQL's default `NULLS DISTINCT`; the `.pg` guide does not say so, + and SQL:2023 made the choice explicit (`UNIQUE NULLS [NOT] DISTINCT`) + because users guess it wrong. +2. Whether a `view` should be inlined (planner sees one shape, no new + validation scope) or materialised (a derived table with its own commits). + The schema-design guide's principle 7 argues for inlining; the issue-#1050 + measurements (92 requests per node page) argue that some derived relations + will want materialisation later. Inline first; the compile-time form costs + nothing to reverse. + + +--- + +## 5. Rosetta stone: six shapes in nine notations + +The schema, in `.pg`: `Person { email @key, name, age? }`, `Company { slug +@key, name }`, `WorksAt: Person -> Company { role? }`, `Knows: Person -> +Person`, `Author`, `Post { published: Bool }`, `Authored: Author -> Post`, +`Matter { number @key }`, `Passage { locator, text @index, embedding: +Vector(N) @index }`, `PassageOf: Passage -> Matter`. Document stores +(Elasticsearch, Vespa, LanceDB) denormalize the one-hop relationships onto the +child document, which is how they are used in practice; the snippets say so +where it changes the meaning. + +### R1. Keyed lookup, one hop, filter, order, page + +*People over 30 at company `$slug`, by name, first 20.* + +```gq +query staff($slug: String) { + match { $c: Company { slug: $slug } $p worksAt $c $p.age > 30 } + return { $p.name } + order { $p.name asc } + limit 20 +} +``` + +```sql +-- SQL +SELECT p.name FROM person p +JOIN works_at w ON w.src = p.id JOIN company c ON c.id = w.dst +WHERE c.slug = $1 AND p.age > 30 +ORDER BY p.name, p.id LIMIT 20; -- p.id added by hand for a total order + +-- SQL/PGQ (SQL:2023) +SELECT name FROM GRAPH_TABLE (g + MATCH (p:Person)-[:WorksAt]->(c:Company WHERE c.slug = $1) + WHERE p.age > 30 + COLUMNS (p.name AS name)) +ORDER BY name FETCH FIRST 20 ROWS ONLY; +``` + +```cypher +// Cypher and GQL are the same here +MATCH (p:Person)-[:WORKS_AT]->(c:Company {slug: $slug}) +WHERE p.age > 30 +RETURN p.name ORDER BY p.name LIMIT 20 +``` + +```graphql +# GraphQL, Hasura-style conventions (the spec defines none of where/order_by/limit) +query Staff($slug: String!) { + company(where: {slug: {_eq: $slug}}) { + employees(where: {age: {_gt: 30}}, order_by: {name: asc}, limit: 20) { name } + } +} +``` + +```clojure +;; Datomic: no order, no limit in the query; the caller sorts +[:find ?name :in $ ?slug + :where [?c :company/slug ?slug] [?p :person/works-at ?c] + [?p :person/age ?age] [(> ?age 30)] [?p :person/name ?name]] +``` + +``` +# CozoScript +?[name] := *company{slug: $slug, id: c}, *works_at{src: p, dst: c}, + *person{id: p, name, age}, age > 30 +:order name :limit 20 +``` + +```json +// Elasticsearch, company denormalized onto the person document +{"query": {"bool": {"filter": [{"term": {"company.slug": "acme"}}, {"range": {"age": {"gt": 30}}}]}}, + "sort": [{"name.keyword": "asc"}, {"_id": "asc"}], "size": 20, "_source": ["name"]} +``` + +``` +-- ES|QL +FROM people | WHERE company.slug == "acme" AND age > 30 | SORT name | LIMIT 20 | KEEP name +``` + +```sql +-- Vespa YQL, company imported through a parent reference +select name from person where company_slug contains "acme" and age > 30 order by name limit 20 +``` + +```python +# LanceDB: the filter is a SQL string; the builder has no ORDER BY, so order comes from SQL or the client +tbl.search().where("company_slug = 'acme' AND age > 30").select(["name"]).limit(20).to_arrow() +``` + +What the row shows: GQ's direction-from-schema makes `$p worksAt $c` the +whole join; every other notation spells the join (`JOIN … ON`, `-[:X]->`, +nested field, repeated variable). GQ and ES (`_shard_doc`/`_id`) are the only +two whose order is total without the author adding a key; SQL and Cypher need +`p.id` for a stable page. + +### R2. Negation + +*People with no employer.* + +```gq +match { $p: Person not { $p worksAt $_ } } return { $p.name } +``` + +```sql +SELECT p.name FROM person p WHERE NOT EXISTS (SELECT 1 FROM works_at w WHERE w.src = p.id); +``` + +```cypher +MATCH (p:Person) WHERE NOT EXISTS { (p)-[:WORKS_AT]->() } RETURN p.name // Cypher 5 / GQL +``` + +```graphql +person(where: {_not: {works_at: {}}}) { name } # Hasura convention +``` + +```clojure +[:find ?name :where [?p :person/name ?name] (not [?p :person/works-at _])] +``` + +``` +?[name] := *person{id: p, name}, not *works_at{src: p} # Cozo +Unemployed(p) :- Person(p), !WorksAt(p, _). # Soufflé +``` + +```json +{"query": {"bool": {"must_not": {"exists": {"field": "company"}}}}} // Elasticsearch +``` + +```sql +select name from person where employed = false -- Vespa: no existence test; a sentinel field +``` + +```python +tbl.search().where("company_slug IS NULL") # LanceDB +``` + +Every notation has it except Vespa. GQ's block spelling is Cypher 5's +`EXISTS { }` with the braces and without the keyword; Datalog's `not` is the +same construct with the same safety rule. + +### R3. Relationship count as a filter, plus the count as a value + +*Authors with at least three published posts, with the count, most prolific +first.* + +```gq +query prolific() { + match { + $a: Author + count { $a authored $p $p.published = true } >= 3 // the filter + $a authored $q $q.published = true // the same pattern again, for the value + } + return { $a.name, count($q) as posts } + order { posts desc, $a.name } + limit 20 +} +``` + +```sql +SELECT a.name, count(*) AS posts FROM author a +JOIN authored x ON x.src = a.id JOIN post p ON p.id = x.dst +WHERE p.published GROUP BY a.id, a.name +HAVING count(*) >= 3 ORDER BY posts DESC, a.name LIMIT 20; +``` + +```cypher +MATCH (a:Author)-[:AUTHORED]->(p:Post {published: true}) +WITH a, count(p) AS posts WHERE posts >= 3 +RETURN a.name, posts ORDER BY posts DESC, a.name LIMIT 20 +``` + +``` +-- GQL: linear composition with NEXT and FILTER +MATCH (a:Author)-[:Authored]->(p:Post {published: true}) +RETURN a, count(p) AS posts GROUP BY a +NEXT FILTER posts >= 3 +RETURN a.name, posts ORDER BY posts DESC, a.name LIMIT 20 +``` + +```clojure +;; Datomic: the count, grouped by name; no HAVING, no order +[:find ?name (count ?p) :with ?p + :where [?a :author/name ?name] [?a :author/posts ?p] [?p :post/published true]] +``` + +``` +# Cozo: a second rule is the HAVING +cnt[a, count(p)] := *authored{src: a, dst: p}, *post{id: p, published: true} +?[name, c] := cnt[a, c], c >= 3, *author{id: a, name} +:order -c, name :limit 20 +``` + +``` +-- ES|QL +FROM posts | WHERE published | STATS posts = COUNT(*) BY author +| WHERE posts >= 3 | SORT posts DESC | LIMIT 20 +``` + +```json +// Elasticsearch Query DSL: a bucket_selector pipeline aggregation is the HAVING +{"size": 0, "query": {"term": {"published": true}}, + "aggs": {"by_author": {"terms": {"field": "author", "size": 20, "order": {"_count": "desc"}}, + "aggs": {"min3": {"bucket_selector": {"buckets_path": {"c": "_count"}, "script": "params.c >= 3"}}}}}} +``` + +```sql +-- Vespa grouping +select * from post where published = true | all(group(author) order(-count()) max(20) each(output(count()))) +``` + +What the row shows: GQ is the only notation that writes the pattern twice, +because a block is a predicate and a return aggregate cannot be filtered. Each +other language has one of: `HAVING`, a `WITH`/`NEXT` stage, a second rule, a +pipeline aggregation. This is 2.2 and 2.8 in one example. + +### R4. Bounded reachability + +*People at distance exactly two from `$email`.* + +```gq +match { $p: Person { email: $e } $p knows{2,2} $f } return { $f.name } +``` + +```cypher +// Cypher: paths of length 2; to get GQ's answer, exclude distance-1 nodes and the source +MATCH (p:Person {email: $e})-[:KNOWS*2..2]->(f) +WHERE NOT (p)-[:KNOWS]->(f) AND f <> p +RETURN DISTINCT f.name +``` + +``` +-- GQL: quantified path pattern; same exclusions needed for distance semantics +MATCH (p:Person {email: $e}) ((x)-[:Knows]->(y)){2} (f) +WHERE NOT EXISTS { MATCH (p)-[:Knows]->(f) } AND f <> p +RETURN DISTINCT f.name +``` + +```sql +-- SQL: two derived tables and an anti-join +WITH d1 AS (SELECT dst FROM knows WHERE src = $1), + d2 AS (SELECT k.dst FROM knows k JOIN d1 ON k.src = d1.dst) +SELECT DISTINCT p.name FROM d2 JOIN person p ON p.id = d2.dst +WHERE d2.dst NOT IN (SELECT dst FROM d1) AND d2.dst <> $1; +``` + +```graphql +person(email: $e) { knows { knows { name } } } # a tree of paths: duplicates, and the source itself +``` + +``` +# Cozo: Datalog's set semantics give GQ's answer once distance-1 is subtracted +d1[f] := *knows{src: $p, dst: f} +d2[f] := d1[m], *knows{src: m, dst: f}, not d1[f], f != $p +?[name] := d2[f], *person{id: f, name} +``` + +Elasticsearch, Vespa and LanceDB cannot express it. What the row shows: GQ's +`{2,2}` is a *distance*; Cypher's `*2..2` and GQL's `{2}` are *path lengths*, +and the two differ on every graph with a triangle. Datalog agrees with GQ. + +### R5. Hybrid retrieval scoped by a traversal + +*Passages of matter `$n`, ranked by RRF of vector similarity and BM25, top 5.* + +```gq +query passages($n: String, $v: Vector(1536), $q: String) { + match { $m: Matter { number: $n } $p passageOf $m } + return { $p.locator, rrf(nearest($p.embedding, $v), bm25($p.text, $q)) as score } + order { rrf(nearest($p.embedding, $v), bm25($p.text, $q)) } + limit 5 +} +``` + +```sql +-- PostgreSQL with pgvector and tsvector: RRF by hand +WITH scope AS (SELECT p.* FROM passage p JOIN passage_of x ON x.src = p.id + JOIN matter m ON m.id = x.dst WHERE m.number = $1), +v AS (SELECT id, row_number() OVER (ORDER BY embedding <-> $2) AS r FROM scope LIMIT 50), +t AS (SELECT id, row_number() OVER (ORDER BY ts_rank_cd(tsv, plainto_tsquery($3)) DESC) AS r + FROM scope WHERE tsv @@ plainto_tsquery($3)) +SELECT s.locator, coalesce(1.0/(60+v.r), 0) + coalesce(1.0/(60+t.r), 0) AS score +FROM scope s LEFT JOIN v USING (id) LEFT JOIN t USING (id) +WHERE v.id IS NOT NULL OR t.id IS NOT NULL +ORDER BY score DESC, s.id LIMIT 5; +``` + +```cypher +// Neo4j Cypher 25: two SEARCH subclauses in a UNION ALL, RRF assembled by hand (abridged). +// SEARCH prefilters only on declared filter properties of the index; the traversal to Matter is a post-filter. +CALL () { + MATCH (p:Passage) SEARCH p IN (VECTOR INDEX passage_embedding FOR $v LIMIT 50) SCORE AS s + MATCH (p)-[:PASSAGE_OF]->(:Matter {number: $n}) + WITH p, s ORDER BY s DESC WITH collect(p) AS ps UNWIND range(0, size(ps)-1) AS r RETURN ps[r] AS p, r + UNION ALL + MATCH (p:Passage) SEARCH p IN (FULLTEXT INDEX passage_text FOR $q LIMIT 50) SCORE AS s + MATCH (p)-[:PASSAGE_OF]->(:Matter {number: $n}) + WITH p, s ORDER BY s DESC WITH collect(p) AS ps UNWIND range(0, size(ps)-1) AS r RETURN ps[r] AS p, r +} +RETURN p.locator, sum(1.0 / (60 + r)) AS score ORDER BY score DESC LIMIT 5 +``` + +```json +// Elasticsearch retriever tree, matter denormalized onto the passage +{"retriever": {"rrf": {"rank_constant": 60, "rank_window_size": 50, "retrievers": [ + {"knn": {"field": "embedding", "query_vector": [/*…*/], "k": 50, "num_candidates": 100, + "filter": {"term": {"matter": "M-1"}}}}, + {"standard": {"query": {"bool": {"must": {"match": {"text": "…"}}, + "filter": {"term": {"matter": "M-1"}}}}}}]}}, + "size": 5, "_source": ["locator"]} +``` + +```sql +-- Vespa: retrieval in the query, fusion in the schema's rank profile +select locator from passage +where matter_number contains "M-1" + and (({targetHits: 50}nearestNeighbor(embedding, q)) or userInput(@q)) +-- rank-profile hybrid { global-phase { expression: reciprocal_rank_fusion(closeness(field, embedding), bm25(text)) } } +-- &input.query(q)=…&ranking=hybrid&hits=5 +``` + +```python +# LanceDB, matter denormalized +(tbl.search(query_type="hybrid").vector(v).text(q) + .where("matter = 'M-1'", prefilter=True).rerank(reranker=RRFReranker()).limit(5).to_pandas()) +``` + +``` +# Cozo: the search atoms join with the scope; RRF needs ranks, which Datalog has no window for +scope[p] := *passage_of{src: p, dst: m}, *matter{id: m, number: $n} +?[locator, d, s] := scope[p], ~passage:emb{p | query: $v, k: 50, bind_distance: d}, + ~passage:fts{p | query: $q, k: 50, bind_score: s}, *passage{id: p, locator} +:order d, -s :limit 5 # an inner join of the two arms, not a fusion +``` + +GQL has no standard retrieval; GraphQL leaves it to a resolver argument. What +the row shows: GQ, Elasticsearch, Vespa and LanceDB have a *first-class* +hybrid; GQ's is the shortest and the only one where the scope is a graph +traversal rather than a denormalized field; Vespa's is the most expressive; +Elasticsearch's composes most freely; SQL's is honest about what RRF is. +GQ's score appears twice because the projected rank must repeat the order key. + +### R6. Upsert a keyed node and an edge + +```gq +query hire($email: String, $name: String, $slug: String, $role: String) { + insert Person { email: $email, name: $name } // keyed: upsert + insert WorksAt { from: $email, to: $slug, role: $role } // one commit for both +} +``` + +```sql +BEGIN; +INSERT INTO person (email, name) VALUES ($1, $2) + ON CONFLICT (email) DO UPDATE SET name = EXCLUDED.name; +INSERT INTO works_at (src, dst, role) + SELECT p.id, c.id, $4 FROM person p, company c WHERE p.email = $1 AND c.slug = $3; +COMMIT; +``` + +```cypher +MERGE (p:Person {email: $email}) SET p.name = $name +WITH p MATCH (c:Company {slug: $slug}) +MERGE (p)-[w:WORKS_AT]->(c) SET w.role = $role +``` + +```graphql +mutation Hire($email: String!, …) { upsertPerson(input: {…}) { id } hire(input: {…}) { id } } # root fields run serially +``` + +```clojure +;; Datomic: unique identity on :person/email makes this an upsert; a lookup ref names the company +[{:person/email "ada@x" :person/name "Ada" :person/works-at [:company/slug "acme"]}] +``` + +``` +?[email, name] <- [[$email, $name]] :put person {email => name} # Cozo, then a second :put for the edge +``` + +```json +// Elasticsearch: id = email, company denormalized; "doc_as_upsert" +POST /person/_update/ada@x {"doc": {"name": "Ada", "company": {"slug": "acme", "role": "Engineer"}}, "doc_as_upsert": true} +``` + +```json +// Vespa: create-if-absent put with field operations +PUT /document/v1/ns/person/docid/ada@x?create=true +{"fields": {"name": {"assign": "Ada"}, "company_ref": {"assign": "id:ns:company::acme"}}} +``` + +```python +tbl.merge_insert("email").when_matched_update_all().when_not_matched_insert_all().execute(rows) # LanceDB +``` + +What the row shows: GQ, Datomic and LanceDB make "keyed insert is upsert" a +property of the schema, so the statement carries no merge vocabulary; SQL and +Cypher spell the merge every time. GQ's edge endpoints are *keys*, not +internal ids, which removes SQL's `SELECT p.id … WHERE p.email = $1` dance. +GQ alone has no computed value anywhere in the statement. + +--- + +## 6. Scorecard + +● strong, ◐ partial, ○ weak or absent. The mark is relative to the axis as +defined in §0, for a typed-property-graph, many-agent workload; a ○ is not a +defect in a system built for another workload (Elasticsearch has no traversal +because it is not a graph database). Datalog is marked for its strongest +member on each row, named where it matters. + +| Axis | GQ | SQL | GQL | Cypher | GraphQL | Datalog | ES | LanceDB | Vespa | +|---|---|---|---|---|---|---|---|---|---| +| 1 Model fidelity (typed property graph) | ● | ◐ view | ◐ optional schema | ◐ open | ○ API | ◐ encoding | ○ | ○ | ○ | +| 2 Compositional closure | ○ | ● | ● | ● | ○ | ● | ◐ retrievers, aggs | ◐ via SQL | ◐ | +| 3 Binding and scope | ● | ◐ explicit joins | ● | ● | ○ | ● | ○ | ○ | ○ | +| 4 Static typing | ● | ◐ | ◐ closed types optional | ○ | ● | ◐ Soufflé/Rel ●, Datomic ○ | ◐ ES\|QL | ◐ string filter | ◐ | +| 5 Absence | ● | ◐ | ◐ | ◐ | ◐ bubbling | ● closed world | ◐ | ◐ | ○ no null test | +| 6 Traversal | ◐ reachability only | ◐ recursive CTE, PGQ | ● paths, modes | ● | ○ | ● recursion | ○ | ○ | ○ parent–child | +| 7 Sub-pattern predicates | ◐ predicate, constant RHS | ● | ◐ EXISTS only | ● EXISTS/COUNT/COLLECT | ○ | ● | ◐ | ○ | ○ | +| 8 Aggregation | ◐ no HAVING, no collect | ● | ● | ● | ○ | ◐ varies | ● | ○ via SQL | ● approximate counts | +| 9 Order and cut | ◐ total order, no paging | ◐ | ◐ | ◐ | ◐ cursor spec only | ○ | ● search_after | ◐ | ◐ | +| 10 Retrieval (contracts / expressiveness) | ● / ○ | ◐ / ◐ | ○ / ○ | ◐ / ◐ | ○ | ◐ Cozo | ◐ / ● | ● / ● | ● / ● | +| 11 Writes | ◐ bounded, no computed | ● | ◐ no MERGE | ● | ◐ opaque | ◐ Datomic tx fns ● | ◐ no tx | ◐ merge_insert | ◐ | +| 12 Time and branches | ● | ◐ system time | ○ | ○ | ○ | ● Datomic, Cozo | ○ | ◐ table versions, no merge | ○ | +| 13 Reuse | ◐ named queries | ● | ◐ procedures | ◐ | ◐ fragments | ● rules | ◐ templates | ○ | ◐ profiles | +| 14 Expressions | ○ | ● | ● | ● | ○ | ● | ● | ● | ◐ | +| 15 Result shape | ◐ flat, node struct | ◐ | ● | ● | ● tree | ◐ pull ● | ● | ◐ | ● | +| 16 Self-description | ◐ explain ●, schema ○ | ● | ○ | ◐ | ● | ● Datomic | ◐ | ◐ | ○ | +| 17 Refusal contract, machine authorship | ● | ◐ | ◐ GQLSTATUS | ◐ | ● | ◐ | ◐ | ○ | ○ | + +Reading down GQ's column: every ● is a contract (model, typing, nulls, +binding, retrieval contracts, time, refusals) and every ○ or ◐ is +expressiveness (closure, traversal breadth, blocks as values, HAVING, paging, +retrieval breadth, computed writes, reuse, expressions, result nesting, schema +introspection). That split is the finding. The contracts are the hard part and +they are done; the expressiveness items in 4.3 are additions that none of them +has to give up. + +--- + +## 7. Expressiveness and evolvability: is there a ceiling? + +The question this section answers: does GQ's initial structure (one +declaration shape, one pipeline, closed types, no closure) put a ceiling on how +far it can be extended, and how does the arrival of much stronger coding +agents change what the language should optimise for? + +### 7.1 Two kinds of expressiveness + +Expressiveness has to be split before it can be judged, because the two halves +move in opposite directions. + +- **Grammar expressiveness**: what can be said in one query. GQ is low here + (§2.2, §2.14, §6), and every item in §4.3 raises it. +- **System expressiveness**: what can be *defined once and reused* by every + author without a release of the engine. SQL has views and functions, Datalog + has rules, GraphQL has fragments and directives, Vespa has rank and query + profiles, Elasticsearch has templates and scripts. GQ has stored queries, + which other queries cannot bind to (§2.13). GQ is lowest of the nine here, + and this is the half that decides whether there is a ceiling. + +A language with high grammar expressiveness and low system expressiveness +(Cypher before procedures, Elasticsearch's Query DSL) grows by adding grammar +until the grammar is the ceiling; Elasticsearch's answer was a second language +(ES|QL), Vespa accreted three DSLs, SQL accumulated several hundred reserved +words and a dialect per vendor. A language with a small grammar and high system +expressiveness (Datalog: every rule is a new relation in the same notation) +grows by adding *definitions*, and its grammar can stay small for decades. The +no-ceiling structure is the second one. GQ is currently neither: its grammar is +small *and* its definitions are closed. + +### 7.2 Where GQ's extension points are, and which ones are grammar + +Walking the surface for what a new capability would touch: + +| Surface | Extension today | Grammar change needed? | Breaks stored text? | +|---|---|---|---| +| New statement kind (`branch`, `commit`, `load csv`, `explain`) | ordered choice at file level, word-bounded keyword | yes, additive | no | +| New clause in `match` (`optional { }`) | ordered choice in `clause` | yes, additive | no | +| New clause after `limit` (`having`, `offset`) | optional suffix | yes, additive | no | +| New Boolean/comparison operator | expression ladder | yes, additive unless a word is reserved | only if reserved (as `and/or/not/is/null/in` were, GQ 2.0) | +| New scalar, aggregate, search or ranking **function** | each is its own grammar rule today: `search_call = { "search" ~ … }`, `agg_func = { "count" \| "sum" \| … }`, `rrf_call` fixed at two `rank_expr` arms | **yes, every time** | no, but every function is a release | +| New session setting | `DEFINITIONS` registry; `set = ` is generic | no | no | +| New schema annotation | unknown annotations retained as metadata | no | no | +| New schema type (`Map`, `Struct`, `Json`) | `base_type` list, `type_ref` | yes, additive | no | +| New engine catalogue type | `@`-prefixed registry rows | no | no | +| New plan node / operator | `physical_nodes!` and `logical_kinds!` registries | no | no | +| A derived relation (a view) | none | would be additive (`view … = query …`) | no | +| Polymorphic binding over an interface | `Catalog.interfaces` exists "for Phase 2 polymorphic queries"; `$x: Named` is `T1` today | no grammar change (a type name is a type name) | no | +| A query hint or option (`@timeout`, `@cache`, `@approximate`) | only `@description`/`@instruction` on a declaration | the annotation rule is closed; a generic `@ident(args)` would be additive | no | +| Language version | `GQ_LANGUAGE_VERSION` is `(2, 3)` in the compiler; no edition pragma in a file; the compiler has one grammar at a time | n/a | a breaking bump re-parses every stored query at boot and **quarantines the graph** on failure | + +Three ceilings stand out, and none is structural: + +1. **Functions are grammar.** A `lower()` or a third `rrf` arm is a pest rule, a + parser arm, a type-check arm and a release. A generic call form + (`ident "(" args ")"`) resolved against a typed function registry, with a + `@Function` catalogue type listing name, signature, nullability rule and + `can_fail`, turns every future function into a registry row. The registry + is exactly how `@Operator`, `@Pass` and `@Setting` already work. +2. **Definitions are closed.** No view, no rule, no user-named relation. The + fix is 4.3 item 2(c); its importance is larger than §2.13 said once agents + are the authors (7.4). +3. **Versioning is all-or-nothing.** Stored queries persist as text and are + recompiled at boot; a breaking grammar change is a graph outage unless the + operator rewrote every stored query first (the shared-expression RFC's + reserved-word break is the recorded instance). The compiler already carries + a version; what is missing is a per-file or per-registration edition + (`gq 2;` as the first settings line, or the registry recording the version + a stored query was validated under) and a compiler that accepts edition + N-1 for one release. Rust editions, Neo4j's `CYPHER 25` prefix with a + per-database default, and GraphQL's `@deprecated`-first rule are the three + working precedents; the compatibility-surfaces RFC's per-rule hashes are the + detection half and need this as the migration half. + +What is *not* a ceiling, checked against the planner model in `execution.md`: + +- The one-scan-root-per-component rule survives inlined views (they are + patterns), `optional { }` (a new join kind), projected blocks (the operator + exists) and interface bindings (a scan over several tables with a common + projection, the shape wildcard traversal already lowers to). +- Flat Arrow rows survive `collect` (`list` is Arrow-native) and would + survive a nested result model later; the envelope's `columns` already carry + Arrow types. +- Plan acceptance survives every additive clause because acceptance checks a + plan against the checked query's requirements, not against a fixed grammar; + only the exact-subset fragment is closed, by design and versioned. +- The `@` namespace (meta-fields, catalogue types) and the `__` reservation + give engine-owned names a home that can never collide with user schema, + which is the discipline GraphQL's `__` and SQL's `pg_` prefixes show + paying off over decades. + +### 7.3 Author-protecting versus system-protecting restrictions + +GQ's refusals fall into two classes, and the distinction decides which ones +should loosen as authors get better. + +**System-protecting** (they hold a bound or a contract; author skill is +irrelevant): `nearest` requires `limit`; `traversal_work_limit` admission; +finite `{m,n}`; constants per invocation in writes; D2; one commit per query; +`T27` index required for full text; filter-before-rank; `T59` one scan root +for `rrf` arms; the mutation `where` taking no binding; wildcard endpoints +declared. These should stay whatever the author is. + +**Author-protecting** (they exist because a human, or a 2025 model, gets the +construct wrong): `T43` a Boolean in `return` needs an alias; `T42` an order +key must appear in `return`, chosen "because an explicit error with an obvious +fix costs one cheap retry while a silent row change costs a wrong answer"; +non-associative comparison (`1 < $x < 3` refused); no `null` literal; `T38` a +search predicate must stand alone or be `= true`; a block's right side a +constant; implicit grouping with `T58`/`T9` guards. Each is a good default for +a human and a defensible one for a model that retries; none protects the +engine. + +The rule this gives: as agents improve, **lift author-protecting restrictions +when they block a round trip, keep system-protecting ones, and never confuse +the two**. §4.3 already asks for the block right side and the order-key rule +to loosen; `T43` and the chained comparison are harmless to keep. The `null` +literal stays out of reads because the three-valued rules it would import are +a correctness hazard for any author, which makes it system-protecting after +all. + +### 7.4 What stronger coding agents change + +The design brief in the README is that agents and UIs become "thin stateless +consumers of durable state". Stronger agents change four things about the +language that serves them, and only one of them points toward *less* +expressiveness. + +1. **The novelty tax falls; the round-trip tax does not.** GQ's bet in §2.17 + (regularity and diagnostics over familiarity) ages well: a model that reads + the grammar, the skill and a four-field refusal, and repairs in one turn, + does not need Cypher's training-data prior. Cypher's bet ages badly (two + live versions, monthly deprecations, the documented `OPTIONAL MATCH` + failure). But every missing construct is still a round trip, and a round + trip costs tokens, latency and a consistency window regardless of how + capable the author is. The 92-request node page (#1050) is not an author + problem. So the balance shifts *toward* grammar expressiveness, not away + from it. +2. **Agents will route around a ceiling, and the detour lands in the wrong + place.** An agent that cannot write "contested claims about X" as one + query writes three queries and the join in its own context. That logic is + then unversioned, unreviewed, unshared and recomputed by every agent in its + own words, which is precisely the convergence failure the schema-design + guide warns about. The only way to keep agent-authored logic in the graph, + where branches, review and merge apply to it, is for the language to have a + place to put it: views, stored queries that other queries bind to, and + functions in a registry. This is why system expressiveness (7.1) matters + more under agents than under humans, and why it ranks above most grammar + additions. +3. **Agents extend the system through data, not releases.** A human adds a + function by shipping a compiler; an agent adds a view by writing a `.gq` + file on a branch, linting it, and merging. Every extension point that is a + registry row or a stored declaration (views, functions in a registry, + catalogue types, settings, annotations) is agent-reachable; every one that + is a grammar rule is not. The no-ceiling design is therefore the one where + the *grammar* stays a small fixed core and *everything else* is a + declaration the same agents can author under the same policy and review. + Datalog is the proof that this scales; `GraphPolicy` nodes with a + `check_query` are GQ already doing it by hand. +4. **Verification becomes the product.** The more autonomously an agent acts, + the more it needs to *prove* a query before running it: `lint` with the + operation descriptor, `explain` with validation scope, typed result + columns, `recall: approximate` in the envelope, `--if-commit`, `as commit` + proposals, `branch merge … dry run`. These are GQ's strongest assets for an + agent workload and no comparator has the set. They should be extended with + the one thing agents cannot get today: a *warning* channel (4.3 item 9), + since a refusal is a stop and a notification is a chance to choose. + +A fifth change arrived while this was being written. Parameswaran, Shankar, +Zaharia, Cheung, Crooks, Gonzalez, Hellerstein and Stoica ("Intelligence is +Free, Now What? Data Systems for, of, and by Agents", BAIR, July 2026) describe +the agent workload as *agentic speculation*: "a high-volume, heterogeneous +stream of work spanning schema introspection, columnar exploration, partial and +then full query formulation", thousands of queries per user request, of which +"only 10–20% of the sub-plans are distinct", with task success rising with the +number of attempts. Their asks of a data system map onto this evaluation +almost line by line: cheap in-language schema introspection (4.3 item 6); +reuse across overlapping sub-plans (a planner concern GQ's `logical_hash` +already identifies); batches of queries with per-query approximation +requirements (the declared `recall` and `ann_nprobes` policy, generalised); +"higher-level primitives rather than requiring agents to list each SQL query +explicitly" (views and registry functions, 7.5); and a *proactive* system that +"instead of executing an expensive query … could first provide the agent a +latency estimate", since "an agent can accept any form of textual feedback" +(the notification channel, 4.3 item 9, and `explain` as the estimate). On the +"data systems of agents" side they name the two things GQ already has and most +systems do not: that "knowledge graphs suffer from … their lack of structured +search", answered by typed, indexed properties rather than keyword or +embedding retrieval alone; and that under thousands of concurrent agents "the +effects of the vast majority of these transactions need to be rolled back, +with only the one 'correct' transaction's result persisting", which is the +branch, `as commit` and `as unattached branch` model with a merge. Their +"data systems by agents" section (engines synthesised per workload) implies +that the durable asset of a database is its *observable contract*, the +language and its executable corpus, not its implementation; that is an +argument for spending on GQ's semantics and the GQT corpus over any one +operator. + +The single counter-current: agents are also better at *decomposition*, so one +could argue the language can stay small and let the agent be the compiler. The +argument fails on points 2 and 4: decomposition in the agent's context is +invisible to review, to policy, to plan truth and to other agents. Expressiveness +belongs in the shared language exactly to the extent that the shared language +can still verify it, which is the boundary 7.3 draws. + +### 7.5 The no-ceiling shape, concretely + +Putting 7.2 to 7.4 together, the structure that has no ceiling is already +mostly present and needs five moves, in this order: + +1. **Generic call syntax plus a typed function registry** (`@Function` + catalogue type; `can_fail`, nullability and type signature as data). After + this, scalar functions, new aggregates, new rankings and new fusion arms + are rows, not grammar; the reserved-word list stays at six. +2. **Views as declarations** (4.3 item 2(c)), inlined, typed like node types, + bindable, versioned with the schema, reviewable on a branch. After this, + derived relations are agent-authored data. +3. **Interface bindings** (`$x: Named`), the "Phase 2 polymorphic queries" the + catalog already reserves space for. After this, heterogeneous discovery + (#648) and uniform results over several types need no `UNION`. +4. **A generic declaration annotation** (`@ident(args)` on `query`, kept as + metadata unless the engine knows it), the GraphQL-directive move, so hints, + caching, timeouts and client metadata have a home before each is designed. +5. **An edition pragma and an N-1 compiler**, so the one breaking change a + decade needs is a migration, not a quarantine. + +With those, the fixed core is: one declaration shape, typed pattern +conjunction with stratified blocks, one expression ladder, one pipeline with +additive clauses, statements for control, `@` for the engine. Everything +agents will want to add lives outside that core as declarations. That is the +same split Datalog has had since 1977 and the opposite of the one that made +Elasticsearch write a second language. + +### 7.6 Is there a formalism for this? Yes, two, and GQ's place in them + +The grammar/system split in 7.1 is a design lens, not a theory. Two bodies of +theory stand behind it and give the words precise meanings. + +**Expressive power of query languages** (database theory). A query language's +expressive power is the class of queries it can define, measured against logics +and complexity classes, independent of syntax. The standard ladder: + +```text +CQ ⊂ UCQ ⊂ FO (relational algebra, "Codd-complete") ⊂ FO + TC / Datalog ⊂ LFP (= PTIME on ordered structures) +``` + +Codd (1972) defined relational completeness as first-order power; Aho and +Ullman (1979) proved first-order logic cannot express transitive closure, the +reason every language that wants reachability adds recursion (`WITH RECURSIVE`, +Datalog rules, Cypher `*`); Chandra and Harel (1980–85) placed Datalog and its +negation/aggregation extensions; Vardi (1982) and Immerman separated *data* +complexity (fixed query, growing data) from *combined* complexity and showed +Datalog with least fixpoint captures PTIME on ordered structures. For graphs, +Angles, Arenas, Barceló, Hogan, Reutter and Vrgoč (ACM CSUR 2017) organise the +path side: RPQ (regular path queries), CRPQ (conjunctions of them), UCRPQ, and +data paths (Libkin, Martens, Vrgoč, JACM 2016); Francis et al. gave Cypher a +formal semantics (SIGMOD 2018), Deutsch et al. did the same for GQL and +SQL/PGQ patterns (SIGMOD 2022), and the pattern calculus GPC (PODS 2023) is the +formal core both standards share. Two further measures matter beside power: +*succinctness* (how long the shortest expression of a query is; two languages +with equal power can differ exponentially) and *data complexity* (whether the +class is tractable at all). + +GQ's read core on this ladder, from `query.pest` and the type rules: + +- A `match` block is a **conjunctive query** over a fixed typed signature: + unary atoms (`$p: Person { … }`), binary atoms (`$a knows $b`), and built-in + comparisons. Join is by variable repetition. +- `not { }`, `exists { }` and `count|sum|min|max|avg(…) { } op c` are + **stratified negation and aggregation sub-goals** with the Datalog safety + (range-restriction) rule, `T39`. No recursion, so stratification is trivial. +- `return` with implicit grouping is a **rule head with aggregation**. +- Disjunction is **restricted**: Boolean `or` over one tuple's values and edge + alternation `(a | b)`; there is no union of two *patterns*, so GQ is not + UCQ-complete (it cannot say "match pattern A or pattern B" over different + bindings in one query). +- `{m,n}` over a label set is a **bounded regular path query**: `(a|b)^{m..n}`, + a strict fragment of RPQ that is still first-order (bounded repetition + unfolds), with *shortest-distance set semantics* rather than path + semantics; unbounded `*` is refused, so transitive closure is inexpressible. +- `order`/`limit` and `nearest`/`bm25`/`rrf` are **ordered, top-k** semantics, + outside the pure relational ladder (which is unordered); top-k with + approximate `nearest` is a separate dimension the theory treats under + ranking/preference queries, not expressive power. +- No arithmetic or functions, so the built-in predicate set is comparisons + only. + +So: GQ's read language is **non-recursive stratified Datalog with aggregation +over a fixed typed schema, minus arithmetic, plus bounded label-set paths with +distance semantics and top-k ranking**. Because `not { }` blocks nest (the +corpus has a double-negation case), De Morgan gives disjunction over one +pattern's bindings, so for Boolean conditions on a conjunctive pattern GQ is +first-order complete with aggregates; what it cannot do is a *union at the +projection*, a UCQ whose arms bind different variables ("a Person or a Company +matching …" as one result). Data complexity is TC⁰ (AC⁰ without aggregates), +so every query is constant-depth parallel and LOGSPACE; combined complexity is +NP-complete for the conjunctive fragment. It sits below Cypher/GQL (no +recursion, no path values, no projection-level union) and far below SQL with +recursion (Turing-complete in practice). Standard examples: it *can* express "people +with no employer" (negation), "authors with ≥ 3 posts" (aggregate sub-goal), +"distance-2 neighbours" (bounded RPQ); it *cannot* express "everyone +reachable from X" (transitive closure), "the latest assertion per customer" +(needs a second stage or window), "a Person or a Company matching …" as one +projected result (union at the head), "the number of nodes is even" (no +arithmetic, so no `count mod 2`), or "balance × rate" (arithmetic). Every +addition in §4.3 and §7.5 except unbounded recursion keeps data complexity in +TC⁰ and the language first-order with counting, which is the property that +keeps plan acceptance tractable; unbounded recursion would move it to +NLOGSPACE (reachability) or PTIME (general fixpoint), a cost the operator +should declare. + +**Expressive power of programming languages** (PL theory). Felleisen's "On +the Expressive Power of Programming Languages" (Science of Computer +Programming, 1991) gives the criterion the 7.2 table was reaching for: a +construct is *macro-expressible* in a language if it can be eliminated by a +local, compositional rewrite into the rest of the language; adding a +macro-expressible construct is a **conservative extension** (it changes +succinctness, not power), while a construct that cannot be locally rewritten +increases expressive power and forces every program analysis to learn a new +case. Steele's "Growing a Language" (OOPSLA 1998) is the design consequence: +a language should be small enough to learn and let its *users* define new +words that look like primitives, so the language grows by library, not by +grammar; Racket's "languages as libraries" (PLDI 2011) is the engineering of +it. + +Under Felleisen's criterion, GQ's planned extensions sort cleanly: + +| Extension | Felleisen class | What it means for the planner | +|---|---|---| +| `exists { }` (today) | macro: `count { } > 0` | already lowered that way | +| Inlined views | macro: textual substitution of a pattern | nothing new to accept; one shape | +| Scalar functions, `coalesce`, `case` | conservative over the expression ladder once a generic call form exists | evaluator arms, `can_fail` | +| Projected block aggregates | conservative: the `SubqueryAggregate` operator exists, the value is its column | result schema inference | +| `collect` | new aggregate kind, same `Aggregate` node | Arrow list type | +| `optional { }` | **not** macro-expressible in GQ today (no outer join) | one new join kind; genuine power increase, bounded | +| Pattern union | **not** macro-expressible (UCQ > CQ) | a second component shape, or interface bindings as the bounded form | +| Unbounded recursion | **not** macro-expressible (FO < TC) | breaks the work budget; refused | + +The design rule this gives is the one §7.5 implied: **admit extensions in +order of Felleisen class**: sugar and conservative extensions freely, each +genuine power increase through an RFC that names the new plan node and the +bound that keeps it tractable. Non-closure stays a property of the *kernel*; +expressiveness grows in the *library*. + +### 7.7 How to make GQ extensible: the design + +A concrete layering, from what exists today, with the acceptance test for +each tier. Everything below is additive to the current grammar; the one +breaking element (editions) is what makes future breaking changes survivable. + +**Tier 0: the kernel, frozen in shape.** One declaration form; typed pattern +conjunction with stratified blocks; one expression ladder; one pipeline +(`match` → `return` → `order` → `limit`) with additive clauses; statements for +control; the `@` namespace for the engine; the six reserved words. Changes +here are RFC-only and bump the edition. The compiler's `QueryIR` is the kernel +language; everything in tiers 1–3 lowers to it, so plan acceptance, replay and +`explain` never learn a tier-1 or tier-2 construct. + +**Tier 1: sugar, lowered in the compiler.** Any surface form that is +macro-expressible into the kernel: `exists`, a future `having { }` (a `Filter` +over `Aggregate`), map projection `$p { name, age }` (a struct of properties), +`optional { }` *once* the kernel has the join kind, `$b.@distance` (a column the +BFS already computes). Acceptance test: the lowering is a local rewrite and +the `.gqt` case for the sugar is the same case as for its expansion. No edition +bump; `GQ_LANGUAGE_VERSION` minor. + +**Tier 2: registries, data not grammar.** Four registries, each already +partly built, each exposed as a `@` catalogue type so agents can read it: + +1. *Functions.* Replace the per-function grammar rules (`search_call`, + `bm25_call`, `agg_func`, `now_call`) with one `call = { ident ~ "(" ~ args + ~ ")" }` resolved against a typed registry: name, parameter types with + nullability, result type rule, kind (scalar | aggregate | predicate | + ranking), `can_fail`, `volatile` (`now()`), pushdown capability (which + Lance/DataFusion function it maps to, if any). The existing six search + functions and five aggregates become rows; `@Function` lists them; `lint` + resolves against the snapshot's registry; the planner's pushdown rule reads + `can_fail`. A new function is a Rust implementation plus a row, never a + grammar change, never a reserved word. +2. *Views (derived relations).* `view ContestedClaim($since: Date) { match … + return { … } }` declared in `.gq` files beside queries, registered with the + stored queries, typed by its `return` list as a node type is by its + properties, bindable as `$c: ContestedClaim($since)`, inlined at compile + time, non-recursive (a view may bind other views; a cycle is `T…`), + validated at `cluster plan` like stored queries today (a schema change that + invalidates a view is a pre-apply finding). `@View` lists them with their + columns. This is the Datalog rule and the SQL view in one, and it is the + place agent-authored logic lives under branches and review. +3. *Types.* Interface bindings (`$x: Named`) over the `interfaces` the catalog + already holds, lowering to a multi-table scan with the common projection + (the wildcard-traversal lowering, which already normalises common + properties). Later: `Struct`/`Map` scalars, each an additive `base_type` + and an Arrow type. +4. *Annotations.* A generic `@ident(args)` on `query`/`view` declarations, + retained as metadata like unknown `.pg` annotations are, with engine-known + names typed (`@timeout(ms)`, `@approximate`, `@cache(ttl)`). `@Annotation` + lists the known ones. This is GraphQL's directive mechanism, and it gives + every future knob a home before the knob is designed. + +Acceptance test for tier 2: the kernel grammar is unchanged, the new thing is +a row in a registry the catalogue exposes, and `lint` can refuse a wrong use +from the registry alone. No edition bump. + +**Tier 3: kernel changes.** New plan nodes with new bounds: `LeftExpand` for +optional, a partitioned top-k (`limit 3 per $c`) for top-N per group, a rerank +stage, pattern union if interface bindings prove insufficient. Each is an RFC +that names the node, its bound, its acceptance check and its `.gqt` cases, as +the typed-edge-alternation RFC did (budget, admission, replay). Minor edition +bump when additive, major when a produced shape changes. + +**Editions.** `gq 2;` as an optional first settings line (absent means the +compiler's current edition); the stored-query registry records the edition +each stored source was validated under; the compiler compiles the current +edition and the previous one; `queries validate` reports sources on the old +edition with the migration; a graph is never quarantined for an edition it +still compiles. This turns the reserved-word kind of break from an outage into +a deprecation cycle. Precedents: Rust editions, Neo4j's `CYPHER 25` prefix +with a per-database default, GraphQL's additive-only rule. + +**Governance, so the tiers hold.** Additive-only within an edition; every +removal deprecated one release first (the compatibility-surfaces RFC's +markers); the `.gqt` corpus is the executable contract and every tier-1 sugar +cites its expansion case; the skill's reference pages are generated from the +registries (the command index already is from the clap tree), so +documentation cannot drift from what the binary accepts; refusal codes are +counted in served metrics so the language's roadmap is read off which `T` +codes fire most. + +**What stays out, and why.** User-defined functions executed in the engine +(WASM or otherwise) are the tempting shortcut and the one tier-2 item to +refuse for now: a view covers most "function" requests, an opaque function +breaks pushdown and plan truth (Froid's lesson: inline or lose the +optimizer), and executing agent-written code inside the write path needs a +policy story first. Revisit when views have been in use long enough to show +what they cannot express. + +--- + +## 8. What the literature says about query surfaces for agents + +Scope: peer-reviewed work and preprints from major labs, read on 2026-10-28, +on three questions. What does the agent workload look like? Which properties +of a query language measurably change how well an LLM uses it? What do agent +memory and retrieval systems ask of the engine, and with which metrics? Each +entry carries the finding, the metric it was measured with, and the +implication for GQ. The question the user put, "do we optimise for precision, +recall and throughput?", is answered in 8.6 from the evidence. + +### 8.1 The agent workload: Berkeley's agent-first thesis + +**Parameswaran, Agarwal, Akillioglu, Shankar, Zeighami, Iyer, Zaharia, Cheung, +Crooks, Gonzalez, Hellerstein, Stoica. "Intelligence is Free, Now What? Data +Systems for, of, and by Agents." BAIR blog, 7 July 2026.** A perspective from +the EPIC and DSF groups. Three claims: agents become the dominant workload and +behave unlike humans or applications ("agentic speculation"); agent swarms need +a substrate for shared state, coordination and failure, where "files are all +you need" and knowledge graphs both fall short, the latter "due to their lack of +structured search"; and agents will synthesise disposable, workload-specific +engines, so verification of a specification becomes the bottleneck. On shared +state: with thousands of agents trying transactions in response to one +request, "the effects of the vast majority of these transactions need to be +rolled back, with only the one 'correct' transaction's result persisting", and +the failure mode to avoid is a "livelock" of compensating actions. +*Implication for GQ:* the three things the post asks for that most systems lack +are typed, faceted retrieval over structured attributes; cheap fork, speculate, +roll back, merge; and a verifiable contract independent of implementation. GQ +has the first two (typed indexed properties; branches, `as commit`, `as +unattached branch`) and the third in the GQT corpus and plan acceptance. The +post's own structured-memory proposal (attributes with `*` wildcards and +natural-language corrective instructions) is a `.pg` schema with enums and a +`GraphPolicy`-style node, which the schema-design guide already recommends. + +**Liu, Ponnapalli, Shankar, Zeighami, Zhu, Agarwal, Chen, Suwito, Yuan, Stoica, +Zaharia, Cheung, Crooks, Gonzalez, Parameswaran. "Supporting Our AI Overlords: +Redesigning Data Systems to be Agent-First." UC Berkeley. CIDR 2026; +arXiv:2509.00997.** The paper behind the blog's first claim, with +measurements. On BIRD with 50 parallel attempts per task, "the number of +distinct sub-plans of each size is often a small fraction of less than 10–20% +of the total". On 44 traces of o3 against two-backend tasks, actions phase +from metadata and sample exploration, to column statistics, to partial then +full solutions, with every phase present throughout; "proactively provided +grounding pertinent to the task" cuts the queries an agent issues by more +than 20% depending on phase. The design they propose: *probes* instead of +queries, each a batch of queries plus a natural-language *brief* stating goal, +phase, approximation needs and priorities; semantic-similarity operators over +*any* data or metadata ("impossible to address within SQL"); a proactive side +channel of "auxiliary information" (related tables, why-not explanations for +empty results) and "cost estimates … even before executing a query"; a probe +optimizer that *satisfices* rather than maximises throughput ("our goal is not +to optimize overall throughput as in traditional databases, but to evaluate +probes enough such that agents can make their decision on how to proceed in +the next turn"); an agentic memory store as a queryable semantic cache; and +*branched updates* with "multi-world isolation", citing Neon's observation +that agents "created 20× more branches, and performed 50× more rollbacks, +relative to humans", with forks that "must eventually reconcile, not just with +the mainline, but with each other". *Metrics used:* distinct versus total +sub-plans, steps to solution, task success rate under repeated attempts. +*Implication for GQ:* the strongest external statement of what GQ's workload +is. Four of its asks are already in the language or one step from it: schema +introspection (`@` catalogue types, 4.3 item 6), cost before execution +(`explain` with `validation` and statistics; a notification channel, 4.3 item +9), declared approximation (`recall`, `ann_nprobes`, the overfetch ladder) and +branched updates with three-way merge. Two are not: shared-work reuse across +the 80–90% redundant sub-plans (a planner concern the `logical_hash` makes +tractable), and a search operator over metadata and data at once (#648 and +interface bindings, 7.7). The paper also reframes the throughput question: the +objective is time-to-next-decision for the agent, not queries per second. + +**Zeighami et al. "Arming Data Agents with Tribal Knowledge." UC Berkeley. +arXiv:2602.13521 (February 2026).** Tk-Boost indexes +"tribal knowledge", corrective instructions learned from an agent's own +mistakes on a database, with *applicability conditions* on query features, and +feeds it back during generation; +16.9 points on Spider 2.0 and +13.7 on BIRD +across NL2SQL agents. *Metric:* execution accuracy. *Implication for GQ:* +corrective knowledge keyed by schema element and operation is a graph of +`(type, property, operation) → instruction` edges; GQ's `@instruction` on +types and edges is the static half of this, and a `GraphPolicy`-shaped node +type is the learned half. The paper is also evidence that the gap between a +model's prior and a specific schema is closed by *schema-attached* text, which +favours keeping `@description`/`@instruction` in the schema over a separate +prompt. + +### 8.2 How language design changes what an LLM gets right + +**Benchmarks, and what moved the numbers.** Spider (Yu et al., Yale, EMNLP +2018) is saturated at about 91% execution accuracy; BIRD (Li et al., HKU and +Alibaba DAMO, NeurIPS 2023) dropped the same model family to the fifties by +adding 33 GB databases, dirty values and external evidence, and introduced the +Valid Efficiency Score so a correct slow query scores below a correct fast +one (human R-VES is now matched, correctness is not); Spider 2.0 (Lei et al., +HKU XLang with Google and Salesforce, ICLR 2025) dropped the best agent to 21% +on real warehouses with 1,000-column schemas and multi-dialect workflows. +Spider 2.0's failure analysis: query planning 17.7%, **dialect-specific +function usage 10.3%**, schema linking 27.6% (columns 16.6%, tables 10.1%), +**JOIN errors 8.3%, often from missing foreign-key declarations**. The two +surveys (Liu et al., HKUST/Tsinghua/Renmin, arXiv:2408.05109; Hong et al., +PolyU, IEEE TKDE 2025) fix the metric vocabulary (EX, EM, VES, query-variance +robustness) and a two-level error taxonomy (which clause; which cause). +*Implication for GQ:* one dialect, declared edges and explicit composition +remove two of the top five error classes outright; and a new language needs an +execution oracle and a clause-level error taxonomy from day one, which the GQT +corpus and the `Q`/`T`/`P` codes are. + +**Execution feedback is the lever; feedback-free self-correction is not.** +DIN-SQL (Pourreza and Rafiei, Alberta, NeurIPS 2023): schema linking is the +largest error class, JOINs second; *feedback-free* self-correction adds 0.9 +points for GPT-4 and a generic "buggy SQL / fixed SQL" prompt *hurts* it by +4.2. Self-Debug (Chen, Lin, Schärli, Zhou, Google DeepMind, ICLR 2024): +unit-test execution feedback 72.2 → 75.2 on MBPP, up to +12% on TransCoder, +matching baselines that sampled ten times more candidates. MAC-SQL (Wang et +al., Beihang and Tencent, COLING 2025): the Refiner that executes, catches +exceptions *and empty results*, and loops is the single most valuable +component (−4.6 points without it). Olausson et al. (MIT CSAIL and Microsoft +Research, ICLR 2024), the cleanest measurement of *feedback quality* as a +variable: holding the repairing model fixed, better feedback doubles repair +success (GPT-3.5 4.7% → 9.6% with GPT-4's diagnosis; CodeLlama 0.4% → +17.6%); human feedback lifts GPT-4 from 33.3% to 52.6%; GPT-4's own diagnosis +was wrong 32 times in 80. *Implication for GQ:* the bottleneck in repair is +diagnosis, not editing; every byte of precise, located diagnosis the engine +emits substitutes for a stronger model. GQ's four-field refusal is the right +shape, and MAC-SQL's result says an *empty result* should carry a reason too +(why-not provenance; the agent-first paper asks for the same). + +**Language comparisons.** Text2Cypher (Ozsoy et al., Neo4j, GenAIK at COLING +2025): 44,387 pairs, fine-tuning gains shrink as the base model improves; +evaluated by Google-BLEU and exact match, not execution. SM3-Text-to-Query +(Sivasubramaniam et al., ZHAW, NeurIPS 2024), the only peer-reviewed head-to- +head of SQL, Cypher, SPARQL and MongoDB MQL on the same 408 question templates +and data: Llama3-70B zero-shot with schema SQL 47.1, Cypher 34.5, MQL 21.6, +SPARQL 1.0; with five BM25-retrieved worked examples (GPT-3.5) SQL 88.6, +Cypher 78.3, MQL 78.7, SPARQL 75.8. The schema in the prompt doubles zero-shot +accuracy for SQL, Cypher and MQL and barely moves SPARQL. SPINACH (Liu et al., +Stanford, Findings of EMNLP 2024): on Wikidata's open schema the winning +interface is *exploratory and incremental* (look up, probe a neighbourhood, +run a fragment, refine), beating one-shot generation by 38 F1. Pipe syntax +(Shute et al., Google, VLDB 2024): the LLM benefit of linear composition is +stated as explicit *speculation*; the measured benefits are readability, +local edits and that every prefix of a pipe query is itself runnable. +*Implication for GQ:* the zero-shot prior of a well-known language is worth +10–45 points and **retrieved worked examples erase most of it**, so a young +language is viable if its tooling injects examples (the skill, the GQT corpus +as an example bank, a `@Example` annotation on types); SPARQL's floor shows +that *opaque identifiers*, not grammar, are what schema text cannot fix, +which favours GQ's human-readable keys and names; and partial queries and +schema introspection should be cheap and in-language. + +**Schema and context.** CHESS (Talaei et al., Stanford and Alberta, 2024): +pruning takes column precision 0.11 → 0.90 at recall 0.94 and uses five times +fewer tokens, but execution-driven revision is worth more (−6.8 without it +versus −2.7 to −6.1 for the pruning stages). "The Death of Schema Linking?" +(Maamari et al., Distyl, NeurIPS 2024 TRL workshop): aggressive linking +forfeits up to 22.6% of questions; when the full typed schema fits the window, +frontier models link it themselves and score higher with the whole schema. +DTS-SQL (Pourreza and Rafiei, Findings of EMNLP 2024): for 7B models the +picture inverts, a smaller visible schema is worth 3–7 points. DAIL-SQL (Gao et +al., Alibaba, VLDB 2024): including foreign keys adds 0.6–2.9 points, a terse +output rule up to 3. Lost in the Middle (8.5) applies to a 300-column schema +dump as much as to documents. *Implication for GQ:* the design target is a +schema whose complete typed description *fits*, with explicit edges (which GQ +has) and a cheap typed way to fetch a sub-schema by type or neighbourhood +(the `@NodeType`/`@EdgeType` catalogue, 4.3 item 6) for smaller models and +larger graphs. + +**Agents acting on data.** TAG (Biswal, Patel et al., Berkeley and Stanford, +CIDR 2025): on questions needing world knowledge or semantic reasoning, +Text2SQL scores 17%, RAG 0%, and pipelines with *semantic operators* 55%. +Semantic operators (Patel et al., Stanford and Berkeley, VLDB 2025): +`sem_filter`, `sem_join`, `sem_topk`, `sem_agg` with natural-language +parameters, optimised with statistical accuracy guarantees (proxy models plus +verification), up to 1,000× speedups. DocETL (Shankar et al., Berkeley and +Columbia, VLDB 2025): an agent-driven optimiser that rewrites a declared +pipeline and synthesises its own validation prompts; 25–80% more accurate than +hand-built baselines. Palimpzest (Liu et al., MIT CSAIL, CIDR 2025): a +declarative program compiled against a cost, time and *quality* policy, picking +cheaper physical operators for exploration and exact ones for the answer. +*Implication for GQ:* three ideas transfer directly. A query should be able to +carry a declared accuracy/cost policy (GQ has `recall` and `ann_nprobes` as +declared plan policy; the generalisation is a per-query `@approximate` or +`@budget` annotation, 7.7). Natural-language predicates belong in the *typed +plan* where they can be bounded, not in strings; GQ's text `nearest` is the +first such operator, and a `sem_filter`-style predicate over a text property +with a declared verifier is the natural second. And the engine, not the +caller, should be allowed to rewrite a query into a cheaper plan under a +stated target, which is what plan acceptance already permits for exact plans. + +### 8.3 Theory and the database community's vision papers + +**Expressive power.** Codd (1972, relational completeness: calculus equals +algebra, first-order logic as the yardstick); Aho and Ullman (POPL 1979): +relational algebra cannot express transitive closure, so unbounded +reachability can never be a sugar; Chandra and Harel (JCSS 1980, 1982; JLP +1985): the ladder CQ ⊂ UCQ ⊂ FO (non-recursive stratified Datalog) ⊂ FO+LFP +⊂ while ⊂ computable, with positive Datalog incomparable to FO; Vardi (STOC +1982) and Immerman (1986): data versus combined complexity, FO in AC⁰ data +complexity, FO+LFP equals PTIME on ordered structures; Grohe and Schweikardt +(LMCS 2005): *succinctness* as a distinct measure, equi-expressive formalisms +differing exponentially in shortest expression. For graphs: Angles et al. (ACM +CSUR 2017) separate *patterns* (CQs, NP-complete combined) from *navigation* +(RPQ, 2RPQ, CRPQ, UCRPQ; NLOGSPACE data complexity; path-output semantics can +be intractable even in data complexity); Libkin, Martens and Vrgoč (JACM 2016, +the 2026 Alonzo Church Award) formalise data paths and show cross-position +value comparison is the expensive extension; Francis et al. (SIGMOD 2018) give +Cypher a denotational semantics as a pipeline of binding tables; Deutsch et al. +(SIGMOD 2022) present the pattern kernel GQL and SQL/PGQ share, with +restrictors (`TRAIL`, `ACYCLIC`, `SIMPLE`) and selectors (`ANY SHORTEST`, `ALL +SHORTEST`) that make infinite path sets finite; the GQL digest (ICDT 2023) +lists the open expressiveness questions path values create; GPC (PODS 2023) +types pattern variables as singleton, group or conditional so repetition feeds +aggregation coherently. *Implication for GQ:* the two instruments are +expressiveness (rung on the ladder) and succinctness (how much shorter); +GQ's `{m,n}`, implicit grouping and top-k are succinctness gains over their +first-order expansions, not new power, and that is the right thing to say in +their defence. If repetition ever becomes unbounded, GPC's typing and +GQL's restrictor/selector vocabulary are the published blueprint. + +**Expressiveness and extensibility in programming languages.** Felleisen +(Science of Computer Programming, 1991): macro-expressibility and conservative extension, +the formal test 7.6 applies; Steele (OOPSLA 1998 keynote, HOSC 1999): design +to grow, users add vocabulary, designers promote proven idioms; Tobin-Hochstadt +et al., "Languages as Libraries" (PLDI 2011): whole languages, Typed Racket +included, as compile-time expansions into a small core with access to binding +and type information; Pombrio and Krishnamurthi, "Resugaring" (PLDI 2014): +desugarings must be compositional and position-preserving so evaluation, and +by extension errors and plans, read back in surface terms. *Implication for +GQ:* the tier-1 sugar layer of 7.7 should be a schema-aware expander whose +desugarings keep source positions, so `explain` rows and `T` refusals name the +construct the author wrote. + +**Extensibility in database systems.** Froid (Ramachandra et al., PVLDB 2017): +inlining imperative UDFs into relational expressions turned row-at-a-time +callbacks into optimisable plans with speedups of orders of magnitude; the +lesson is that an extension executed as a callback is a performance cliff by +construction. Hirn and Grust (SIGMOD 2021), Duta et al. (CIDR 2020): compiling +PL/SQL into recursive CTEs works only because the kernel has recursion, so a +kernel without recursion cannot absorb iterative extensions by compilation. +LogicBlox (Aref et al., SIGMOD 2015) and Soufflé (CAV 2016): typed Datalog as +the single language for schema, constraints, queries and views, with derived +predicates inlined into worst-case-optimal joins, the best-evidenced +extensibility mechanism in the field. Volcano and Cascades (Graefe, 1993, +1995): extensibility lives in the rule set over typed operators; a new +operator with rules and properties joins the search space on equal terms. +*Implication for GQ:* inlined typed views and registry functions (7.7 tier 2) +are the Froid and LogicBlox shape; semantic filters, approximate top-k and a +rerank stage should enter as typed logical operators with rules and +properties, never as string rewrites or external callbacks; and the one thing +a non-recursive kernel cannot absorb is iteration, which is why unbounded +recursion is a kernel decision, not a library one. + +**Vision papers.** Stonebraker and Pavlo, "What Goes Around Comes Around… +And Around…" (SIGMOD Record 2024): no new data model displaced the relational +one; graph systems' ideas are being absorbed through SQL/PGQ (DuckDB's PGQ +beats a leading graph DBMS by up to 10×) and they "do not expect specialized +graph DBMSs to be a large market"; a new language's lasting value is the +features it exports. Pan, Wang and Li, "Survey of Vector DBMSs" (VLDBJ 2024): +open problems are the semantics and cost of hybrid filter-plus-similarity +queries, approximation without declared accuracy, and the absence of a +composable algebra for ranking operators. Madden, Cafarella, Franklin and +Kraska, "Databases Unbound" (PVLDB 2024): natural language is a front end and +a structured declarative language is the optimizer's intermediate +representation; propose `convert` and `enrich` operators and a multi-objective +optimizer over cost, latency and quality. Castro Fernandez, Elmore, Franklin, +Krishnan and Tan, "How LLMs Will Disrupt Data Management" (PVLDB 2023): +design pressure moves from human ergonomics to *verifiability*; expose +schemas, constraints and provenance in machine-consumable form. Eckmann and +Binnig, "A Vision for Autonomous Data Agent Collaboration" (CIDR 2026): +per-source agents and a collaboration protocol replacing a central planner. +None of these argues for natural language as the sole interface; all put an +optimisable declarative layer beneath it. *Implication for GQ:* three +consequences. GQ's durable asset is a kernel of typed operators that would +survive translation into a SQL/PGQ host, so each feature should be stated as +an operator a host could absorb. The vector survey's three open problems are +exactly the contracts GQ already states (prefilter semantics, declared +approximation) plus the one it lacks (a composable ranking algebra, 4.3 item +1). And verifiability, the property the 2023 paper predicts matters most, is +where GQ's static types, compile-time refusals and plan acceptance are ahead +of every comparator. + +### 8.4 Agent memory systems: what they ask of the engine + +Every entry was confirmed against its arXiv or publisher record; numbers are +the papers' own. + +**Packer et al., "MemGPT: Towards LLMs as Operating Systems", UC Berkeley, +arXiv:2310.08560 (2023).** Two-tier memory (working context plus recall and +archival stores on PostgreSQL and pgvector); the agent's whole interface is a +handful of typed calls (append, replace, search, insert) with paging. Deep +Memory Retrieval accuracy 92.5% versus 32.1% for fixed-context GPT-4. +*Implication:* the memory API is a few keyed reads and writes plus a vector +index, issued as sequential round trips, so per-call latency and paging shape +task time more than any single-query recall figure. + +**Sumers, Yao, Narasimhan, Griffiths, "Cognitive Architectures for Language +Agents" (CoALA), Princeton, TMLR 2024, arXiv:2309.02427.** Episodic, semantic +and procedural long-term stores; retrieval already means a *fused* score over +recency, importance and similarity (Generative Agents' scoring), not one ranked +list. *Implication:* an engine serving agents needs several memory kinds with +different access patterns and a fusion that can take time and salience as +inputs, which is an argument for scores as values (4.3 item 1). + +**Rasmussen et al., "Zep: A Temporal Knowledge Graph Architecture for Agent +Memory", arXiv:2501.13956 (2025).** Graphiti: episodes, entities with fact +edges, communities; facts are **bi-temporal** (`t_valid`/`t_invalid` on the +event line, created/expired on the ingestion line); a contradicting fact +*invalidates* the old edge; search is cosine plus BM25 plus bounded +breadth-first expansion, reranked by RRF, MMR or a cross-encoder. LongMemEval +with gpt-4o: full context 60.2% at 28.9 s median and about 115k tokens; Zep +71.2% at 2.58 s and about 1.6k tokens. *Implication:* the closest published +shape to GQ (typed graph, vector, BM25, RRF, time). Its differentiating +operation is invalidation, not retrieval, and its win was 10× latency and 70× +fewer tokens, not recall. + +**Chhikara et al., "Mem0: Building Production-Ready AI Agents with Scalable +Long-Term Memory", arXiv:2504.19413 (2025).** A vector store of extracted +facts, and Mem0^g, a labelled graph in Neo4j; the write is retrieve-top-10 then +one of ADD, UPDATE, DELETE, NOOP; obsolete relations are marked invalid. LoCoMo +LLM-judge: full context 72.9, Mem0^g 68.4, Mem0 66.9, Zep 66.0, best RAG 61.0; +p95 latency 1.4 s (Mem0), 2.6 s (graph), 17.1 s (full context); retrieved +context about 1.8k tokens versus 26k. *Implication:* the dominant write is a +read-then-decide keyed upsert; the graph variant buys two points for 1.5–2× +latency; the quality headroom is small next to the latency and token gap. + +**Xu et al., "A-Mem: Agentic Memory for LLM Agents", Rutgers, NeurIPS 2025, +arXiv:2502.12110.** Zettelkasten notes whose neighbours' derived attributes +are rewritten in place when a new note arrives; retrieval is plain top-k +cosine; 85–93% fewer tokens per question than MemGPT. *Implication:* structure +is exploited at write time; the engine needs cheap in-place property update +with versioning more than richer read queries. + +**Lin et al., "Sleep-time Compute", Letta / UC Berkeley, arXiv:2504.13171 +(2025).** Between turns the agent rewrites its memory (`rethink_memory`); about +5× less test-time compute for equal accuracy. *Implication:* heavy background +rewrite traffic interleaved with latency-sensitive reads; isolating in-progress +consolidation from readers is a first-class need, which is what a branch or an +unattached snapshot is. + +**Zhang et al., "A Survey on the Memory Mechanism of LLM-based Agents", +Renmin / Huawei, arXiv:2404.13501 (2024; TOIS acceptance stated by the +authors).** Three write-side operations (write, manage, forget) for every read; +evaluation "an open problem". *Implication:* benchmark coverage of the write +side is essentially absent (8.5). + +### 8.5 Graph retrieval, fusion, filtered ANN, and the benchmarks + +**Edge et al., "From Local to Global: A Graph RAG Approach to Query-Focused +Summarization", Microsoft Research, arXiv:2404.16130.** Leiden communities with +LLM summaries; 72–83% win rates over vector RAG on comprehensiveness and +diversity for *global* questions, none on directness; indexing MuSiQue cost +115.5M input tokens (HippoRAG 2's measurement). *Implication:* the win is a +materialised-view workload (expensive LLM indexing, cheap root-level reads), +not a traversal problem. + +**Gutiérrez et al., "HippoRAG", Ohio State / Stanford, NeurIPS 2024, +arXiv:2405.14831; "HippoRAG 2 / From RAG to Memory", ICML 2025, +arXiv:2502.14802.** Personalized PageRank over an extracted graph seeded at +query entities. Recall@5 on 2WikiMultiHopQA 68.2 → 89.1 versus ColBERTv2, but +*worse* on HotpotQA (79.3 → 77.7). HippoRAG 2, the most careful graph-versus- +vector measurement: average F1 across seven datasets 59.8 versus NV-Embed-v2 +57.0, GraphRAG 49.6, LightRAG 6.6; the only graph method matching dense +retrieval on simple QA while leading multi-hop; indexing 9.2M input tokens +versus GraphRAG's 115.5M. *Implication:* graph structure must *augment* a +dense or lexical ranking, not replace the corpus; the winning design is +passage nodes carrying the embedding with edges carrying associations, which +is a property graph with per-node vectors, and a bounded walk should be a +ranking primitive the planner can choose, not the default path. + +**Xiang et al., "When to use Graphs in RAG" (GraphRAG-Bench), ICLR 2026, +arXiv:2506.05690.** Vanilla RAG with rerank leads on fact retrieval (60.9 +versus LightRAG 58.6, MS-GraphRAG 49.3); HippoRAG 2 leads complex reasoning +(53.4 versus 42.9); prompt tokens per query: vanilla ~879, HippoRAG 2 ~1,008, +LightRAG ~101k, MS-GraphRAG global ~331k. *Implication:* traversal pays only +when the question spans concepts; the token cost of the result set is a +first-order metric. + +**Pan et al., "Unifying LLMs and Knowledge Graphs: A Roadmap", IEEE TKDE 2024, +arXiv:2306.08302.** KGs valued for explicit, editable, interpretable structure +and provenance. *Implication:* typed schema, keyed identity and lineage over +raw traversal speed. + +**Cormack, Clarke, Büttcher, "Reciprocal Rank Fusion…", SIGIR 2009.** `k = 60` +was fixed "during a pilot investigation" on TREC runs, never tuned for +lexical-plus-dense fusion. **Bruch, Gai, Ingber, "An Analysis of Fusion +Functions for Hybrid Retrieval", ACM TOIS 42(1) 2023, arXiv:2210.11934.** A +convex combination of min-max-normalised scores beats RRF on every dataset +tried (MS MARCO nDCG@1000 0.454 versus 0.425; NQ 0.542 versus 0.514) and is +"largely agnostic" to the normalisation; RRF is parameter-sensitive (tuning +`k` gains on one dataset and loses on another); one tuned weight needs only a +handful of labelled queries. **Thakur et al., "BEIR", NeurIPS 2021.** Zero-shot, +BM25 beats most dense retrievers on about half of 18 datasets; BM25 plus a +cross-encoder reranker is the strongest configuration (+11%). *Implication:* +keep `rrf` as the zero-config default and expose weighted, normalised fusion +and a rerank stage; the 2–3 nDCG points on the table are larger than most +graph-versus-vector deltas, and they are exactly what 4.3 item 1 unlocks. + +**Patel, Kraft, Guestrin, Zaharia, "ACORN", Stanford, SIGMOD 2024, +arXiv:2403.04871.** Pre-filtering is competitive at low selectivity but scales +poorly; post-filtering "failed to reach high recall" on high-cardinality +predicates; traversing the predicate-induced subgraph of a denser HNSW gives +2–50× QPS at recall 0.9. **Gollapudi et al., "Filtered-DiskANN", Microsoft, +WWW 2023.** Label-aware index construction for low-cardinality labels; post- +filtering collapses at low specificity. **Simhadri et al., "Results of the +Big-ANN NeurIPS'23 Competition", arXiv:2409.17424.** The filtered track fixes +recall@10 ≥ 0.9 and ranks on QPS; the winner branches its strategy on tag +cardinality. *Implication:* the recall of a filtered vector query depends on +selectivity and on its correlation with the embedding space; the engine must +choose pre-, post- or in-graph filtering from cardinality estimates and report +the recall it achieved. GQ's prefilter gate, overfetch ladder and `recall: +approximate` descriptor are this contract; node type is the natural +low-cardinality label for a label-aware index. + +**Benchmarks.** **LoCoMo** (Maharana et al., ACL 2024): the best configuration +retrieved five dense LLM-extracted observations, beating 25 raw turns; no +scoring of writes or contradictions; scores are now reported under +incompatible metrics (F1 versus LLM-judge). **LongMemEval** (Wu et al., ICLR +2025): GPT-4o drops from 0.870 with oracle evidence to 0.606 with the full +115k history; the only benchmark scoring *knowledge updates* and *abstention*; +fact-augmented keys +9.4% recall@k, time-aware query expansion +6.8–11.3% +temporal recall. **RAGAS** (EACL 2024) and **ARES** (NAACL 2024): the deployed +metric is *context relevance*, the fraction of retrieved content that was +needed, a precision metric. **BEIR/MTEB**: nDCG@10 for ranked quality, +recall@100 for first-stage coverage; nothing measures freshness, update +visibility or consistency under concurrent writes. + +**Context precision.** **Liu et al., "Lost in the Middle", TACL 2024.** With 20 +retrieved documents GPT-3.5 scores 75.8% with the answer first, 53.8% in the +middle (below the 56.1% closed-book score), 63.2% last; reader accuracy +saturates near 20 documents while retriever recall keeps rising. **Hsieh et +al., "RULER", NVIDIA, COLM 2024.** Distractors, not length, drive degradation. +**Hong, Troynikov, Huber, "Context Rot", Chroma technical report, 2025.** +Across 18 models a focused ~300-token prompt beats the full 113k-token +LongMemEval history. **Anthropic, "Introducing Contextual Retrieval", 2024.** +Enriching the indexed key cut top-20 retrieval failures from 5.7% to 2.9% with +contextual BM25 and 1.9% with a reranker. *Implication:* beyond roughly 20 +items more recall is worthless to the reader; the engineering target is the +smallest, most relevant context, in a deliberate order. + +### 8.6 So what should the engine optimise? Precision, recall, throughput, revisited + +The three named objectives are necessary and misordered, and the literature +adds two the three leave out. In evidence order: + +1. **Precision at a small k, and the token cost of the result set.** Readers + saturate near 20 items and lose up to 22 points when the needed fact sits + mid-context (Lost in the Middle); distractors, not length, drive the drop + (RULER, Context Rot); LoCoMo's best system retrieved five dense + observations rather than 25 turns; the deployed RAG metric (RAGAS, ARES) is + *context relevance*, a precision measure; Zep's headline gain was 70× fewer + tokens and 11 points, Mem0's and A-Mem's were 85–93% token reductions. + Recall beyond what the reader will use is a cost. *For GQ:* fused top-k + precision with typed and temporal predicates is the primary quality metric, + and "tokens returned per correct answer" is the headline number; flat typed + rows that the caller hydrates are the right default, which is what GQ has. +2. **Diagnosis quality, because it decides the repair loop.** Feedback-free + self-correction is worth under one point for a frontier model and can be + negative (DIN-SQL); execution feedback is worth +4 to +12 (Self-Debug, + MAC-SQL, CHESS); holding the model fixed, *better diagnosis alone* doubles + repair success (Olausson et al.). An empty result deserves a reason + (MAC-SQL loops on it; the agent-first paper asks for why-not provenance). + *For GQ:* the four-field refusal is a quality metric to measure (turns to a + valid query; share of refusals carrying a machine-applicable edit), and the + next two items on that axis are why-not answers for empty results and the + planner notification channel. +3. **Round trips per task, measured as time to the agent's next decision.** + Agent memory is sequences of small calls (MemGPT paging, Mem0's + retrieve-then-upsert, CoALA's loop); HippoRAG matched iterative retrieval at + 6–13× lower latency by collapsing hops into one query; grounding hints cut + queries per task 18%; repeated attempts raise success 14–70% and 80–90% of + their sub-plans are duplicates. The agent-first paper's objective is "to + evaluate probes enough such that agents can make their decision on how to + proceed in the next turn", not queries per second. *For GQ:* every construct + in §4.3 that removes a round trip (blocks as values, optional, views, + HAVING, collect, computed writes) is a throughput feature in this sense; + shared-sub-plan reuse via `logical_hash` and a cost estimate before + execution are the engine-side halves. +4. **Write correctness, temporal validity and isolation under speculation, + which no benchmark measures yet.** Every production memory system + invalidates rather than deletes and resolves contradictions at write time + (Zep, Mem0, A-Mem); LongMemEval is the only benchmark scoring knowledge + updates and abstention, and it is where systems fail; background + consolidation interleaves with reads (sleep-time compute); agents make 20× + the branches and 50× the rollbacks of humans and "the vast majority" of + their transactions must be rolled back. *For GQ:* keyed upsert, `--if- + commit`, branches, `as unattached branch`, three-way merge and snapshot + reads are the differentiators the literature asks for and cannot score; + add the metrics it lacks: update visibility latency, staleness of returned + facts, merge conflict rate, wasted-speculation cost. +5. **Recall at k as a declared bound, not a maximum, above all under + filters.** Filtered ANN recall collapses under naive post-filtering at low + selectivity (ACORN, Filtered-DiskANN); the Big-ANN contract fixes recall@10 + ≥ 0.9 and then ranks on throughput; the winner picks its strategy from tag + cardinality. *For GQ:* the prefilter gate, overfetch ladder and `recall: + approximate` descriptor are already this contract; the missing piece is + reporting the recall *achieved* on a filtered `nearest`, and using node + type as the label for a label-aware index. +6. **Fusion quality over fusion default.** Hybrid beats either index alone + everywhere; zero-shot BM25 beats most dense retrievers on half of BEIR; + weighted, normalised fusion beats RRF by 2–3 nDCG points and RRF's `k = 60` + was fixed in a 2009 pilot; reranking adds the largest marginal gain + (BEIR +11%, Anthropic's contextual retrieval). *For GQ:* keep `rrf` as the + zero-config path and expose weighted fusion and a rerank stage, which is + what moving retrieval out of `order` (4.3 item 1) makes an expression. +7. **Graph traversal as a planner choice, not a default.** PPR wins on + associative multi-hop (+21 recall@5 on 2WikiMultiHopQA) and loses on single- + passage QA; community summaries win only on global questions and are a + materialised-view workload; the cheapest graph method costs 15% more tokens + than vector RAG, the others 100–400×. *For GQ:* "scope by traversal, then + rank inside the scope" (the schema-design guide's rule) is the shape the + measurements support; a bounded PPR-style ranking would be a ranking + function in the registry, not a default path. +8. **Latency and raw throughput last among the quality metrics, but a floor.** + Published wins sit at p95 under about 3 s for memory reads and 10× over + full-context; past that floor further throughput buys little, while each + item above buys measured accuracy. Under agentic speculation the throughput + that matters is *shared*: 80–90% of sub-plans are redundant, so reuse beats + raw queries per second. + +The direct answer: **precision, yes, at small k and measured as context +relevance and tokens per correct answer; recall, as a bounded guarantee the +engine reports rather than a number to maximise; throughput, redefined as +round trips and time-to-next-decision with shared-work reuse, with raw latency +as a floor.** The two missing objectives are diagnosis quality and write +correctness under concurrent speculation, and both are places where GQ is +ahead of the comparators and where the literature has no benchmark, which is +an opportunity to publish one. The proposed metric set for GQ, in the order +above: tokens returned per correct answer; nDCG@10 and recall@k on the fused +ranking (comparable with BEIR and MTEB); turns to a valid query and share of +refusals with a machine-applicable fix; round trips per task on an agent +benchmark (LoCoMo and LongMemEval as read-path regression suites, with the +caveat that their scores are not comparable across papers); declared versus +achieved recall under filters; update visibility latency and staleness; merge +conflict and wasted-speculation rates; p95 latency; shared-sub-plan hit rate. + +--- + +## 9. GQ redesigned without compatibility + +The question: if nothing stored had to keep parsing, what would GQ be, derived +from the workload (§8) and the axes (§2) rather than from the sequence of RFCs +that produced today's grammar? The answer keeps more than it changes. Every +contract in §4.2 survives; what changes is the shape the contracts are +delivered in. Working name below: **GQ 3**. Each decision cites the evidence +that drove it. + +### 9.1 The requirements, from the evidence + +| # | Requirement | Evidence | +|---|---|---| +| R1 | Closed, typed schema the agent can read, that fits in context while small and is queried through the language (bound, traversed, searched, counted) once it does not; edges declared | SM3 (schema doubles zero-shot accuracy), DAIL (explicit FKs +0.6–2.9), Spider 2.0 (8.3% JOIN failures from missing FKs), Death of Schema Linking (full schema wins when it fits) | +| R2 | Diagnosis as a product: located, typed refusals with fixes; reasons for empty results; warnings | Olausson (feedback quality doubles repair), MAC-SQL (empty result is a signal), Self-Debug, GQ's own 45/45 measurement | +| R3 | Fewer round trips: one query for one question; batches under one snapshot; shared work | agent-first paper (12.7 queries per task, 80–90% duplicate sub-plans), HippoRAG (one-shot PPR at 6–13× lower latency), #1050 (92 requests per node page) | +| R4 | Small, precise results with their provenance; token cost is a quality variable | Lost in the Middle, RULER, Context Rot, Zep (70× fewer tokens), LoCoMo best config, RAGAS context relevance | +| R5 | Retrieval as an algebra: scoped by graph, fused with weights, thresholded, reranked, with declared and reported recall | Bruch (weighted fusion +2–3 nDCG over RRF), BEIR (rerank +11%), ACORN and Big-ANN (recall as a bound under filters), Vespa's match/rank split | +| R6 | Writes that invalidate rather than delete, upsert by key, compute under a precondition, and select by pattern | Zep, Mem0, A-Mem (invalidation; retrieve-then-upsert), LongMemEval (updates are where systems fail), GQ's own 2-round-trip table in 2.11 | +| R7 | Speculation as a first-class mode: fork, propose, roll back, merge, with isolation of background rewrites | agent-first paper (20× branches, 50× rollbacks, "roll back all but one"), sleep-time compute | +| R8 | Declared cost and accuracy per query; explicit, never silent, partial answers | Palimpzest, LOTUS, agent-first "satisfice", invariant 11 | +| R9 | Growth by declaration, not grammar: views, functions, interfaces, annotations; editions for the rare break | Felleisen, Steele, LogicBlox, Froid; Elasticsearch's second language as the counterexample | +| R10 | One language for schema, derived relations, reads, writes, control and introspection | RFC 0055's wider direction, Berkeley's "holistic source of truth", Datomic's schema-as-data | + +### 9.2 What stays exactly as it is + +Static closed typing with nullable parameters; three-valued logic with no +null literal in reads; pattern conjunction by juxtaposition with join by +variable; stratified blocks with the safety rule; reachability semantics for +bounded traversal; filter-before-rank, total order with id tiebreaks, index +required at compile time, approximation declared in plan and envelope; +keyed insert as upsert; one commit per mutation; D2 (constructive or +destructive per statement list: with invalidation as the write pattern, +deletes become rare and the two-case staging protocol stays worth its +simplicity); branches, proposals and snapshots as statements; the `@` +namespace; the four-field diagnostic with frozen codes; `$var`, braces, +lowercase keywords, word-bounded identifiers; flat Arrow rows as the +canonical result. + +### 9.3 What changes, and why + +| Today | GQ 3 | Driver | +|---|---|---| +| `query` for reads and writes alike | `query` for reads, `mutation` for writes, `view` for derived relations, `batch` for several under one snapshot | agents trained on GraphQL write `mutation { }` (the skill's documented trap); lint classifies by keyword, not body | +| `$p worksAt $c`, direction from the schema | `$p -worksAt-> $c`, `$c <-worksAt- $p`, `$a -knows- $b` undirected, `-$e:worksAt->` bound, `-(knows|likes){1,3}->` | direction visible to the reviewer and the model; `T5` becomes a reading error, not a schema lookup | +| Ranking functions as the leading `order` key | a `retrieve { }` clause that binds typed score columns; `order` sorts any expression | 2.10; Vespa; Bruch; thresholds, weights, more arms and rerank become expressions | +| `rrf(a, b)` only | `rrf(...)`, `norm(...)`, weighted sums, `rerank(model, text) k` as a stage, `where $a.vec.distance < 0.4` | Bruch (+2–3 nDCG), BEIR (+11% rerank), LanceDB `distance_range` | +| Blocks as predicates only, constant right side | blocks as values (`count { } as n`), any expression on the right | #1050, Cypher `COUNT {}`, Soufflé | +| No optional | `optional { }` with pattern-bound filters and nullable bindings | #658, GQL `OPTIONAL MATCH` | +| No post-group filter, no collection | `having { }`, `collect($x)` / `collect($x { a, b })` | R3 Rosetta row; Datomic pull; one nesting level | +| Functions are grammar rules | generic `name(args)` over a typed registry; `@Function` catalogue | 7.7 tier 2; Froid | +| No derived relations | `view Name($p: T) { match … return … }`, bindable as `$x: Name($p)`, inlined | 2.13; LogicBlox; convergence | +| No polymorphism | `$x: Named` over interfaces; `$x: *` over every node type with the common projection | #648; the catalog's reserved `interfaces` | +| Constant assignments only | computed assignments (`visits + 1`) allowed under an explicit precondition | 2.11; Mem0's retrieve-then-upsert | +| `where` over own properties only | pattern-scoped selection: `delete $e where { pattern }`, `update $a … where { pattern }`, bounded pattern-scoped `insert` | 2.11; Cypher `MATCH … DELETE` without the free mixing | +| Counts only on success | `returning { … }` ids and projected values | SQL `RETURNING`; one round trip saved per write | +| `limit N` literal; no paging | `limit $k`; continuation token in the envelope; no `offset` | 2.9; Relay; ES `search_after` | +| `--branch`/`--snapshot` outside the text | `at $commit` / `on $branch` typed parameters on a declaration | 2.12; Datomic's database-as-value | +| No cost or accuracy policy in the text | `@budget(rows, ms)`, `@approximate(recall)`, `@satisfice` annotations; refusal with estimate before running; `complete: false` with reason, never silent | R8; Palimpzest; agent-first | +| Empty result is empty | `why_not` funnel in the envelope (rows surviving each clause) | MAC-SQL; agent-first why-not provenance | +| Errors only | `N…` notifications beside `Q/T/P` refusals | Neo4j 03N90/03N91; agent-first "steer" | +| Schema not queryable | `@Type`, `@Property`, `@Edge`, `@Index`, `@Statistic`, `@Function`, `@View` beside `@Operator` | 2.16; GraphQL `__schema`; Datomic | +| `@description`/`@instruction` only | plus `@example` on types and views, served back as worked examples | SM3: five retrieved examples erase a 10–45 point prior | +| No temporal roles | `@valid(from, to)` on a type declares bitemporal validity; `invalidate` as a write verb; `at date(…)` on a binding | Zep; LongMemEval; the schema-design guide's axes | +| One grammar at a time | `gq 3;` edition line; compiler accepts N and N-1; registry records edition | 7.7; Rust editions; Neo4j `CYPHER 25` | +| Six hardcoded search functions, `.gq` and `.pg` separate | one file type holding `node`, `edge`, `interface`, `view`, `query`, `mutation`, `policy` declarations | R10 | + +### 9.4 The shape, in one file + +```gq +gq 3; + +// ---- schema: the same declarations, one file type +interface Named @instruction("Anything a human refers to by name") { name: String } + +node Customer implements Named { + slug: String @key + tier: enum(standard, strategic) @index + @example("$c: Customer { tier: strategic }") +} + +node Assertion @valid(asserted_at, retracted_at) + @instruction("One actor's claim from one source. Never edit the statement; supersede it.") { + slug: String @key // span + extractor + statement digest + statement: String @index // full-text + embedding: Vector(1536) @embed("statement") @index + asserted_at: DateTime + retracted_at: DateTime? + confidence: F64? @range(confidence, 0..1) +} + +edge About: Assertion -> Customer @key(@src, @dst) +edge Supports: Assertion -> Assertion +edge Supersedes: Assertion -> Assertion @card(0..1) + +// ---- a derived relation, declared once, bound anywhere +view Contested($since: DateTime) @description("Assertions attacked since a date") { + match { + $a: Assertion + $r: Assertion -supports-> $a // reuse: a 'Reason' would be its own type in a real schema + $r.confidence < 0.3 $r.asserted_at >= $since + } + return { $a as assertion, count($r) as attacks } +} + +// ---- a read: scope by graph, retrieve, fuse, filter on scores, project compactly +query evidence($slug: String, $q: String, $k: I64 = 10, $at: Commit?) + @approximate(recall: 0.9) @budget(rows: 200000, ms: 2000) { + match { + $c: Customer { slug: $slug } + $a: Assertion -about-> $c + not { $a -supersedes-> $_ } // only current claims + optional { $x: Contested(datetime("2026-01-01")) { assertion: $a } } + } + retrieve { + $a.vec: nearest($a.embedding, $q) k 200 + $a.lex: bm25($a.statement, $q) + } + return { + $a { slug, statement, asserted_at }, + $x.attacks, + count { $s -supports-> $a } as support, + 0.7 * norm($a.vec) + 0.3 * norm($a.lex) as score + } + having { $a.vec.distance < 0.6 or $a.lex is not null } + order { score desc } + limit $k +} + +// ---- a write: pattern-scoped, keyed, computed under a precondition, answering ids +mutation retract($slug: String, $why: String, $head: Commit) @if_commit($head) { + update $a: Assertion where { $a: Assertion { slug: $slug } } + set { retracted_at: now(), note: $why } + invalidate $e where { $a: Assertion { slug: $slug } -$e:supports-> $_ } + returning { $a.@id, count($e) as detached } +} + +// ---- speculation: the same statements as today, plus a batch under one snapshot +batch explore @brief("which assertion types mention pricing") @phase(explore) { + query types() { match { $t: @Type } return { $t.name, $t.instruction } } + query hits($q: String) @satisfice(ms: 300) { + match { $a: Assertion } + retrieve { $a.lex: bm25($a.statement, $q) } + return { $a.slug, $a.lex.score as s } order { s desc } limit 20 + } +} +``` + +Reading the file against the requirements: R1 is the schema with `@example` +and `@instruction` served through `@Type`; R2 is the diagnostic, `why_not` and +`N…` channel the envelope carries; R3 is `optional`, the block as a value, +`having`, the view and the batch, which turn the five-query node page into one +request; R4 is `$a { slug, statement, asserted_at }` (a partial object), typed +score columns and `limit $k` with a token; R5 is `retrieve` plus the score +expression and the `having` threshold under a declared recall; R6 is +`invalidate`, the computed `set` under `@if_commit`, the pattern-scoped +`update` and `returning`; R7 is the unchanged branch, proposal and snapshot +statements plus `batch`; R8 is `@budget`, `@approximate`, `@satisfice`; R9 is +`view`, the function registry behind `norm`, `nearest`, `bm25`, and the `gq 3;` +line; R10 is the one file. + +### 9.5 The retrieval clause, precisely + +`retrieve { $b.name: fn(args) [k N] … }` declares one or more *arms*, each +over one binding's indexed property, each binding a typed score column +(`$a.vec.distance: F64?`, `$a.lex.score: F64?`, null when the row is outside +that arm's window). Semantics, unchanged in substance from today's contracts: +the `match` block's filters and traversals are applied before any arm scores +(prefilter); a `nearest` arm is a top-`k` window over the eligible rows with +the declared recall, a `bm25` arm is the full matching population; the +retrieved population is the union of the arms' rows; `return`, `having` and +`order` may use the score columns as ordinary values, so `norm`, `rrf`, a +weighted sum and a threshold are expressions, not grammar; `rerank(model, +$a.statement, k 50)` is a stage declared after the arms, run over the fused +top-k with a provider named in `cluster.yaml` the way embedders are; a +retrieval forces a `limit`. The envelope reports per arm the recall contract +and, under filters, the recall achieved, and per row the arm contributions +behind a fused score. A ranking that scopes by traversal (`$a -about-> $c` +above) ranks only the reached rows, as today. + +### 9.6 Writes, precisely + +Three selection forms, one target type each, so the effect set stays visible: +`where expr` over the target's own properties (today's form); `where { pattern +}` binding the target and reaching it through edges; and for `insert`, a +bounded pattern that supplies endpoint values. Assignments admit expressions +over the target's current properties (`visits: visits + 1`) only when the +declaration carries `@if_commit` or the statement names a key equality, so a +computed write always has a stated read-set. `invalidate` is `update` that +sets the type's declared `@valid` end to `now()`; it exists because every +production memory system (Zep, Mem0, A-Mem) invalidates rather than deletes, +and because a graph that is also a history should not lose the fact that a +fact was once held. `delete` stays for genuine removal under D2. `returning` +projects over the affected rows after the write, inside the same commit. +Everything else about writes is today's: one commit, constants per invocation +for literals and parameters, `as commit`, `as unattached branch`. + +### 9.7 The envelope, precisely + +Beside `columns`, `rows` and `graph_commit_id`: `retrievals` and `metrics` +(today's descriptors, plus `recall_achieved` per filtered arm); +`notifications` (`N…` codes with the same four fields as refusals: a cross +join, a destination-ranked scan of a whole type, a wildcard without a budget, +a `bm25` arm without a cap); `why_not`, present when zero rows were returned +and cheap to produce from the profile, giving the rows that survived each +clause in plan order so the agent sees which clause emptied the result; +`complete` and `incomplete_reason` under `@satisfice`, so a partial answer is +never silent; `continuation` for paging over the pinned snapshot; and, when +the request was `explain`, `estimate` (rows, bytes, time) so a `@budget` +refusal is informative before any row is read. + +### 9.8 What the redesign still refuses + +Unbounded recursion without a declared work budget and a typed `exhausted` +outcome (and, with one, only as a kernel operator by RFC); path values with +modes and selectors; general subqueries and `WITH` pipelines (the three +closure forms plus `batch` cover the measured demand); free read-write mixing +(pattern-scoped selection with one target type is the bounded form); a `null` +literal in reads; executing user code in the engine (a registry function is a +Rust implementation plus a row; a provider-backed `judge($a.text, "about +pricing")` semantic predicate is the one exception worth designing, with LOTUS's +declared accuracy target, because TAG measured +38 points for it); schema- +resident ranking; open labels. Each is still a wall in §4.2 or a cost the +theory in §7.6 names. + +### 9.10 Room for federation + +Nothing above spans graphs, and a redesign is the moment to make sure nothing +above forecloses it. Federation has two shapes, and they need different +things from the language: + +- **Query federation**: one query reads from several graphs (one server or + several). The precedents are SPARQL 1.1 `SERVICE`, SQL foreign tables + (`postgres_fdw`, Trino connectors) and GraphQL Federation, where subgraphs + compose into one schema and an entity's `@key` is the join address across + services. +- **Replication federation**: a graph, or a view of it, is kept current + elsewhere. The precedents are git remotes (Dolt's `push`/`pull`), and in + this repository the actor-views RFC's *shares* (a recipient's view + materialised into its own graph from the source's change feed) and the + branch-operations RFC's deferred "Git-style remotes". + +What GQ 3 reserves so both stay possible, each a small rule now: + +1. **A graph qualifier in type references.** The schema-namespaces RFC gives + `crm::Contact`; a type reference grammar must admit one more segment for a + graph (`sales@crm::Contact`, or a `from graph sales` declaration clause) + without touching identity, exactly as the namespace RFC says the namespace + is "part of the name and never part of identity". +2. **Keys stay the cross-graph address.** The integer-identity RFC makes + `__id` engine-owned and packs type and row into 64 bits; those are + graph-local by construction. The same RFC keeps `@key` as the public + address, and federation depends on that staying true: a foreign reference + is a key, never a packed id. This is GraphQL Federation's `@key` rule. +3. **One snapshot per graph, pinned in the envelope.** A cross-graph read + cannot have one snapshot; it has one per graph. The envelope's + `graph_commit_id` becomes a map, and `batch` pins per graph. Invariant 3 + ("one coherent accepted view") holds per graph and the result says which. +4. **Views are the unit that binds to a foreign graph.** A `view` declared over + `sales@crm::Contact` is a foreign table; its result columns are its contract, + so the local planner treats it as an opaque, typed, pushdown-capable source + with its own budget. Pushdown rules are the planner's problem later; the + language already has the declaration kind. +5. **Writes stay single-graph.** A mutation targets one graph; cross-graph + consistency comes from change feeds and replication, never from a + distributed commit. The one-commit-per-mutation wall holds. +6. **Search across corpora is the polymorphism problem again.** BM25 + statistics are per table and per graph; the polymorphism RFC's one-corpus + contract for cross-member `bm25` is the same contract a cross-graph search + needs, and `rrf` over per-graph arms is the honest form until then. +7. **Policy per graph.** Cedar resources are graphs already; actor views are + per graph; a federated read is authorised per graph and the envelope's + `view.restricted` is per graph too. + +None of this is built in GQ 3; each is a sentence in the grammar or the +envelope that costs nothing now and is expensive to retrofit. + +### 9.9 How far this is from today's GQ + +Most of §9.3 is additive and already in §4.3 and §7.7: blocks as values, +`optional`, `having`, `collect`, views, the function registry, interface +bindings, annotations, catalogue types, paging, `returning`, pattern-scoped +writes, notifications, `why_not`, editions. Only four items are incompatible +with stored text and are the reason to ask the question at all: the arrow +spelling for traversals, `mutation` as a keyword, `retrieve` replacing ranking +functions in `order`, and `@valid`/`invalidate` as schema and verb. Under an +edition line all four ship as GQ 3 beside a GQ 2 the compiler keeps compiling +for one release, with `queries validate` printing the rewrite; the rewrite from +`$p worksAt $c` to `$p -worksAt-> $c` and from `order { nearest(…) }` to a +`retrieve` arm is mechanical, which is what the `suggestion` field with +`machine_applicable` edits was built for. + +--- + +## 10. The repository's RFCs: what to adopt, what to change, what is missing + +Read for this section, in full or in their design sections: RFC 0047 (search +plan truth, merged, in progress), RFC 0048 (search contracts, PR #793), the +analyzed lexical search RFC (PR #792), the shared expression model and its +operator-vocabulary amendment (PR #805), typed edge alternation, RFC 0055 +(branch statements) and RFC 0056 (one language, PR #670), the engine +catalogue, session settings, compatibility surfaces, bulk load as a plan, RFC +0041 and RFC 0003 (stored queries, MCP), RFC 0045 (GQT), RFC 0043, 0046 and +the index-facts and incremental-reconciliation RFCs (PR #1156), RFC 0012 and +0015 (embeddings), the polymorphic types RFC with its integer-identity +dependency (PR #884), schema namespaces (PR #1015), GQP query contracts (PR +#1197), actor views (PR #951) and branch operations v2 (PR #677). The +judgement is against §4, §7 and §9; "adopt" means the decision stands under +the redesign, "change" names what the redesign would alter, "missing" names a +decision no RFC makes. + +### 10.1 Search: 0047, 0048, the lexical RFC, and the index RFCs + +**RFC 0047, search plan validation and result guarantees. Adopt, nearly +whole.** Its seven items are the correctness contracts §4.2 keeps: the +four-field diagnostic with frozen codes and machine-applicable suggestions +(now shipped and measured), `T27` and `FullTextIndexRequired` (no silent +case-sensitive scan), the ranked binding rooting its component (declaration +order stops mattering), one total order with the fused score as a column, read +descriptors with `recall` as a contract rather than an observation, the served +read floor, and plan acceptance as the single gate for run, explain and +replay. Its alternatives table is right on every row, including the rejection +of a compile-time refusal of ranked destinations (`T26`) and of recording +retrieval in `QueryIR` as a second truth. Two things to change: (1) it fixes +`recall` as a function of retrieval *kind*; §8.5's evidence (ACORN, Big-ANN) +says the envelope should also carry the recall *achieved* under a filter, a +measured fact beside the contract; (2) its "served read floor" `usage` object +is the right place for the cost *estimate* the agent-first paper asks for +before execution, so `explain` should return the same object with +`estimated: true`. One thing it got right that the redesign keeps verbatim: +"the reader is often an agent that treats an error as the documentation it +acts on", which is §8.2's Olausson finding stated a year earlier. + +**RFC 0048, search contracts and retrieval algebra. Adopt the laws, the +windows, the weights, exact retrieval and the schema declarations; change the +surface.** The eleven laws (eligibility before selection, cuts are semantic, +windows are separate from the final limit, target identity, exact answers +independent of index state, every cut total, partitions never redefine the +population, meaning resolved from the accepted schema, one snapshot, separate +facts, one resource protocol) are the retrieval algebra §2.10 asked for and +every one survives the redesign. `candidates: n` as a window distinct from +`limit`, with the rule that "a downstream limit never sizes an upstream +window", fixes a real defect (an `rrf` vector arm fetching exactly `limit` +targets). Weighted `rrf` with 2 to 16 arms and projectable per-arm scores that +are null outside the arm's window is exactly §9.5's arm model. `nearest(…, +exact: true)` is the right spelling for the exact contract. `Vector(n, +distance="cosine")`, the recipe identity (model revision plus encoding +choices, hashed over a canonical typed form, never a label) and `coverage = +exact` are schema and envelope decisions the redesign adopts unchanged. + +What to change is the one sentence the RFC is proudest of: "It adds no clause +and no stage: the query surface is the Shared expression model, where search +predicates live in `match` and the leading ranking call lives in `order`." Its +own 2026-09-13 draft had a staged syntax (`rank … yield`, named sources, a +`filter` stage) and withdrew it because "stages would be a second surface for +the same plan". That is a compatibility argument, and §2.10 shows what it +costs: the ranking token does two jobs (select the population, sort the +rows), the aggregate and `distinct` cases make the overload visible, a score +cannot be a *filter* (`nearest(…) < 0.4` is unwritable), and every option the +RFC adds (`candidates`, `weight`, `exact`, `k`) is grammar on the ranking call +rather than a value in an expression. Under §9.5 the same laws hold over a +`retrieve { }` clause whose arms bind score columns; `candidates` becomes the +arm's `k`, `weight` becomes a coefficient in a score expression, the threshold +becomes a `having` conjunct, and `norm()` (min-max over the arm's window, +Bruch et al.) joins `rrf()` as a function because the evidence says weighted +normalised fusion beats RRF by 2–3 nDCG points. The RFC's deferred list (per- +group selection, a second ranking after a traversal, fusion across bindings, +discovery across node types, a reranker) is the list §9 unblocks, and the +RFC's own "evidence shape" example ("the 5 best documents, and all of their +passages") is the clearest case for a retrieval position separate from the +sort. + +One more change: the RFC's "Agent use" section says "an in-context competence +measurement decides between spellings" and cites the 45/45 run. Keep the +instrument, but §8.2 says what it will show: SM3 found a 10–45 point zero-shot +gap between spellings that five retrieved examples erased. Spelling choices +should be made on reviewability and regularity and tested for repairability, +not on first-try accuracy. + +**Analyzed lexical search (PR #792). Adopt the model; align the deprecation +with editions.** Schema-owned analyzer profiles (`standard_v1`, +`standard_folded_v1`, `english_v1`, immutable, every setting explicit, +NFC named as part of the profile), one typed lexical query `terms(q, mode:, +max_edits:)` consumed by both the predicate `match_terms` and the ranking +`bm25`, the membership laws (`matches(edits=0) ⊆ matches(edits=1) ⊆ +matches(edits=2)`; `matches(indexed) = matches(unindexed)`), the exact scan as +oracle and the field-corpus statistics that graph filters never rescope are +the correct separation of meaning from acceleration (invariant 7) and the +reason §8.5's "index state changes which rows a text query matches" defects +(#747, #748, #749) stop being possible. The custom `bm25_v1` scorer is a bold +choice; the RFC earns it with guards showing native fuzzy BM25 rewarding a +rare expansion 38× and rounding a common term's IDF to zero at 2^24 +documents. Two notes. First, `@analyzed` as a capability separate from +`@index` is right (searchability is semantics, the index is cost), and it is +the pattern the polymorphism RFC should reuse for "the full-text index is +declared on the interface property". Second, the RFC retires `search`, +`match_text` and `fuzzy` through a one-release deprecation and "the next GQ +language major"; that is exactly the migration an edition pragma (§7.7) +exists for, and the RFC should name it rather than a boot-time quarantine. + +**RFC 0043, 0046, index facts as planner input, incremental reconciliation +(PR #1156). Adopt.** These are the engine's side of "indexes are derived": +analyzer certificates bound to artifacts, a read-only index status surface +that observes and never builds, access paths decided from pinned index facts +and stated in the plan, and appended-row segments with a Lance 13 fence for +moved rows. The language consequence they share is already in §9: coverage is +a planning fact the envelope reports, never a semantics. One request: RFC +0046's `index status` should be a catalogue type (`@Index` with `state`, +`coverage`, `reason`) rather than a verb, so an agent asks it in GQ with the +rest. + +**RFC 0012 and 0015, embeddings. Adopt; make the lag visible.** Provider- +independent configuration with the embedding identity recorded in the schema +and a typed refusal of a mismatched query space is the right contract. 0015's +decision that an embedding is derived state materialised by a reconciler, not +a write-path network call, is invariant 7 applied correctly and matches how +every production memory system in §8.4 behaves (write the text, embed +later). Its cost for an agent is that a just-written memory is not a +`nearest` hit until reconciled; §9.7's envelope carries `embedding_coverage` +for exactly this, and the `@Index` catalogue row should report the pending +count so an agent can decide to wait or to fall back to `bm25`. + +### 10.2 Language: the expression model, alternation, one language, catalogue, settings, versions + +**Shared expression model and the operator-vocabulary amendment (PR #805). +Adopt, and take the amendment to its conclusion.** One expression type for +every clause, context rules as checks rather than second types, three-valued +logic with no null literal, the conjunct list placed by one pass, mutations +lowered through the same expression (deleting the string-built SQL): all +right, all kept. The amendment's three rules are the most important language- +design statement in the repository: the reserved set is closed and holds only +the logic of the language; every operation of one domain is a call, "so a new +capability adds a function to the type checker and no grammar"; options are +named. That is §7.7's tier-2 principle. What the amendment does not yet say, +and the redesign does: the calls are a *registry* exposed as `@Function`, not +match arms in the type checker, so a new function is a row and `lint` resolves +it from the snapshot. The amendment's own rollout (`contains` infix to +`contains()` call through a deprecated rule, `GQ_LANGUAGE_VERSION` 2.0 → 3.0 +→ 4.0 over two releases) is the third time the project has paid for not +having editions; §7.7's pragma is the generalisation. One alternative the RFC +rejected deserves a second look under §9: "a separate `where { expr }` clause +after `match`" was set aside to keep conditions beside the pattern. That is +the right call for filters, and it is also why §9 puts scores in their own +`retrieve` clause rather than beside the pattern: a retrieval is not a +condition on one row. + +**Typed edge alternation. Adopt as the template.** Exact `{1,1}` default, +finite bounds, common-property typing, `$e.@type`, a declared budget with +admission before the scan and a typed exhaustion error, replay validation of +every member pin: this RFC is the model every later kernel operator in §7.7 +tier 3 should follow, and §9.8's "unbounded recursion only with a declared +work budget and a typed `exhausted` outcome" is its rule generalised. + +**RFC 0055 and RFC 0056, branch statements and one language. Adopt the +direction; change two rules.** Every operation as a GQ statement, classified +by the compiler and entering through the door its classification names, is +the one-language requirement (R10) and the reason the GQT corpus can test +merges. Two of its rules should change. First, "a control statement takes no +parameters": that is a transport simplification that costs agents the ability +to store `branch merge $b into main` as a typed tool, and §9.3 gives +declarations typed `$branch`/`$commit` parameters instead. Second, 0056's +raw block `$$ … $$` for inline payloads is fine for schema text, but a load's +rows belong in a typed `load` statement with a source, as the shipped `load +csv` already does; the one-language RFC predates that statement and should +cite it. Its maintenance-as-job decision (RFC 0011 Decision 11) and the new +`maintain` Cedar action are right and overdue. + +**Engine catalogue. Adopt; extend to the schema.** Registry-derived `@` types +are the mechanism §7.7 builds every registry on. The polymorphism RFC proposes +"a structured, read-gated catalog route" for types, properties, keys, +interfaces and members; it should be `@Type`, `@Property`, `@Edge` and +`@Interface` catalogue types instead, for the reason the catalogue RFC itself +gives ("a query surface also gives the join for free"). + +**Session settings. Adopt; split request intent from process knobs.** The +scope model (request versus process), `show` as a statement and the refusal +of unknown names are right. §9.3's `@budget`, `@approximate` and `@satisfice` +are declaration-level intent that belongs on the query, not in a `set` line, +and the `@Setting` catalogue makes the split visible. + +**Compatibility surfaces. Adopt the hashes; add editions.** Per-rule hashes +and a `(major, minor)` per surface detect a change; they do not let two +grammars coexist. The lexical, operator-vocabulary and polymorphism RFCs each +schedule a "GQ language major"; without an edition pragma each is a boot-time +quarantine of every graph with a stored query that used the old spelling. + +**Bulk load as a query plan, RFC 0041 and RFC 0003, RFC 0045. Adopt.** A load +with an `explain`able plan is the right shape; stored queries as typed tools +with `@description`/`@instruction` projected through MCP are the agent door +§8.2 wants; and RFC 0045's amendment making "agent experience … the format's +design priority" is the right ordering. The one gap in 0041/0003 is the one +§2.13 names: a stored query is callable, not bindable; views are the missing +declaration kind. + +### 10.3 Compositional closure: no RFC owns it + +This is the clearest finding of the review. Every closure shape §2.2 names +has an issue and none has an RFC: + +| Closure shape | Issue | Nearest RFC text | +|---|---|---| +| Optional traversal | #658 (P-high, needs-rfc) | none; the shared expression RFC's alternatives mention nothing optional | +| Block aggregate as a value | #1050 | the shared expression RFC keeps blocks as predicates; 0048 fixes `rrf` arms as values but not blocks | +| Post-group filter (`having`) | none filed | none | +| `collect` / one nesting level | #649, #1050 | none | +| Derived relations (views) | none filed | 0041 (stored queries are callable, not bindable); #1197 names "definer-rights stored queries" as a *later* idea | +| Top-N per group | none filed | 0048 defers "per-group selection (at most `n` targets per group)" | +| Union at the projection | #648 | the polymorphism RFC (PR #884) is the one RFC that supplies a closure form: abstract bindings and inline unions `$x: A \| B` | + +The polymorphism RFC therefore deserves credit it does not claim: interface +and union bindings *are* the bounded union §7.6 recommends over general +pattern union, and "result schemas never depend on the member set" is the +principle that keeps it conservative. The rest needs one RFC, "Derived +relations and optional patterns": views as declarations inlined at compile +time, `optional { }` with pattern-bound filters and nullable bindings, blocks +as values, `having`, `collect`, and the Felleisen test (§7.6) as its +acceptance criterion for each. It is the highest-value RFC not yet written, +because §8.1's measured workload (one question per query, 80–90% redundant +sub-plans, 92 requests per node page) is a closure problem. + +### 10.4 Polymorphism: PR #884, with integer identity and namespaces + +**Adopt the query model; accept the storage dependency with eyes open; change +one refusal.** The RFC makes interfaces and inline unions abstract node types +a query can bind, traverse to and mutate through; resolves each use once per +invocation to an id-sorted type set; lowers to one union-scan node with per-arm +resolution (pins, filters, index facts, late materialisation); gives `$x.@type` +and `$x.@type_id` as query-only columns; adds `is`/`is not` type tests and +narrowing by rebinding; and keeps result schemas independent of the member set +(a column's type comes from the declared interface, never from the current +implementors). Its direction-inference rules, migration tiers (adding +`implements` is metadata-only; narrowing is validated; cross-kind name reuse +in one apply is refused), stored-query dependency pinning by `StableTypeId`, +and the `ChangeScopeMembershipChanged` feed refusal are careful. The +prototype findings it records (a bare-id rule dropping a row that shares a +key across types; single-table shortcuts opening `node:`) are the +right kind of evidence. This is §7.7 tier 2 item 3 and §9.3's interface +bindings, done properly. + +The storage dependency is the integer-identity RFC: `__id` becomes a `UInt64` +from a distributed sequence and every edge endpoint a packed reference (16-bit +`StableTypeId`, 48-bit row id), while `@key` stays the public address. That is +a format change justified by polymorphic endpoints (an edge row needs to say +which table its endpoint is in) and it is the right design for that purpose; +the evaluation's one condition is §9.10 item 2: keys must stay the only +cross-graph address, which the RFC states. The 16-bit type limit (65,536 +types) and 48-bit row limit are generous for the workload. + +The refusal to change: "`bm25` on a binding whose set has more than one member +is refused" until a one-corpus scanner option exists upstream. The reasoning +(scores from two tables' statistics are not comparable; normalisation is +"unstable across `k`, filters and pages") is sound for a *single* `bm25` score, +but it leaves #648 (discovery across types) with no answer until Lance +changes. §8.5's evidence offers the interim form the RFC's own alternatives +table dismisses: per-member arms fused by rank (`rrf`), which is how every +multi-index search engine answers a cross-corpus query, and which §9.5's +`retrieve` clause expresses as one arm per member. Rank fusion "ignores +absolute relevance", as the RFC says; for a discovery query that is the point. + +Two smaller notes. The RFC's "structured catalog route" should be catalogue +types (10.2). And `$x: *` (every node type) has no spelling; a schema can +declare a root interface, but an implicit `@Node` interface that every type +implements would give discovery one binding without a schema convention. + +**Schema namespaces (PR #1015). Adopt.** Namespaces as part of the name and +never of identity, moves as renames through `@rename_from`, `use ns;` with no +search order and two matches a type error, and the canonical string on every +wire surface: this is the qualifier §9.10 extends with a graph segment, and +its "composition is declared once, in `cluster.yaml`" rule is the right +precedent for declaring a federated source. + +### 10.5 Permissions: PR #1197 against PR #951 + +The two drafts present themselves as alternatives. Against the agent workload +they are two layers, and the review's recommendation is to keep both, each +narrowed to what it is for. + +**Actor views (PR #951) is the data layer.** Its contract is non-interference: +two graphs that give an actor the same view give the same answer to every +query, including counts, scores, ranks, errors and the result schema, enforced +at one door per mechanism (plan acceptance for reads, attempt validation and +the publication CAS for writes). Its model is graph data: `@principal`, +`@group`, `@member`, `@securable`, `@in(p)`, `@grant`, `@inherits`, +`@requires`, `@deny`, `@starts`/`@expires`; users write facts, never rules, and +the readable set is a stratified fixpoint with denial in one layer. That is +the schema-design guide's principle 9 ("keep the rules in the graph") applied +to access, it is the Zanzibar shape every comparable product converged on +(its Appendix B), and it is the only one of the two that answers the company- +brain case where thousands of audiences already exist in source systems. +`view.restricted` as a policy fact rather than a data count is the right +signal, and "a name the view hides does not exist for the actor" is the right +disclosure rule (the polymorphism RFC adopted it after withdrawing a refusal +that leaked a hidden member's existence). Its costs are real: twelve phases, +two measurement gates it is blocked on, exact restricted search (D10) that +refuses approximate `nearest` to restricted actors and bounds visible text by +`max_exact_search_rows`, and sixteen annotations. The redesign's view: accept +the contract and the model, ship the first scope (type and column visibility, +per-row placement by property, reads), and treat exact restricted search as +the price of non-interference rather than a flaw, since §8.5's precision-first +evidence makes a bounded exact search over the visible set a good default +for memory-sized corpora. + +**GQP (PR #1197) is the operation layer.** Its insight is agent-native: an +agent produces a program, so "validating that program is a productive +boundary"; a capability is a contract plus (for deployed tools) one exact +program revision; Cedar stays the only grant authority; trusted context +(`$auth.tenant_id`) is separate from parameters; preflight uses the execution +checker; diagnostics repair rather than refuse. That is §8's verifiability +thesis and §9.3's stored-queries-as-tools in policy form. Three changes. +First, GQP should not be a separate language: a `contract` is a declaration +kind in the one GQ file (§9.4), sharing the grammar, type checker and +registry, which the RFC's own "GQ language family" phrasing almost says. Second, +its initial fragment ("one concrete node binding"; no traversals, blocks, +negation or search) is too small for a graph: the planner's requirements +(RFC 0047) already retain every predicate, traversal and block, so guard +preservation can be checked over the kernel, not a sub-fragment; the fragment +should be the kernel minus retrieval. Third, its row scope is a guard the +author must write (`$ticket.tenant_id = $auth.tenant_id`) and the checker must +find; that is brittle where actor views' placement is structural. The clean +composition: #951 decides *which rows and columns exist* for an actor; #1197 +decides *which programs* an actor may run over them and binds trusted +context; a definer-rights view (which #951 calls an "answer capability" and +#1197 a "deployed capability") is the same object in both, and in §9 it is a +`view` with a `contract`. + +Both must be federation-aware (10.6): Cedar resources are graphs, views are +per graph, shares are the materialised form. + +### 10.6 Federation: what the RFCs leave room for + +No RFC proposes query federation, and three touch its preconditions. Schema +namespaces supply the qualifier that a graph segment extends. The integer- +identity RFC keeps `@key` as the public address, which is the cross-graph +join rule; its packed ids must never leak as foreign references, and its +export spelling (`from_type`/`to_type` with keys) already honours that. Actor +views' *shares* (a recipient's view materialised into a separate graph kept +current from the change feed, with the source as lineage) and branch +operations v2's deferred "Git-style remotes" are the replication form; both +would become one mechanism once a change feed can be consumed as a load +(RFC 0056's `load` statement with a feed source). The polymorphism RFC's +one-corpus contract for cross-member `bm25` is the cross-graph search +contract in waiting. What no RFC reserves and §9.10 asks for: a graph +qualifier in type references, per-graph commit pins in the envelope, views as +the foreign-source declaration, and the rule that writes stay single-graph. + +### 10.7 Verdicts + +| RFC | Verdict | The change, if any | +|---|---|---| +| 0047 search plan truth | adopt | report achieved recall beside the contract; `explain` returns the `usage` estimate | +| 0048 search contracts | adopt laws, windows, weights, exact, schema declarations; **change the surface** | a `retrieve` clause with score columns; `norm()` beside `rrf()`; thresholds as filters | +| Analyzed lexical search | adopt | ship the deprecation through an edition, not a quarantine | +| 0043, 0046, index facts, incremental reconciliation | adopt | `@Index` catalogue type instead of a verb | +| 0012, 0015 embeddings | adopt | pending counts in `@Index`; coverage in the envelope (exists) | +| Shared expression model + operator vocabulary | adopt; **go further** | functions as a registry (`@Function`), not type-checker arms | +| Typed edge alternation | adopt as the template for every budgeted operator | none | +| 0055 / 0056 one language | adopt direction | typed parameters on control statements; cite `load csv`; keep maintenance as jobs | +| Engine catalogue | adopt; extend | schema catalogue types | +| Session settings | adopt | declaration annotations for per-query intent | +| Compatibility surfaces | adopt hashes | add editions | +| Bulk load as plan, 0041, 0003, 0045 | adopt | views as the missing declaration kind | +| Polymorphic types + integer identity | adopt; accept the format change | `rrf` over member arms as the interim cross-member search; `@Node` root; catalogue types not a route | +| Schema namespaces | adopt | reserve a graph segment | +| Actor views (#951) | adopt the contract and model; narrow first scope | ship type/column/row visibility for reads first | +| GQP contracts (#1197) | adopt the capability idea | a `contract` declaration in GQ; fragment = kernel minus retrieval; compose with actor views | +| Branch operations v2 (#677) | out of this review's scope (storage) | its deferred remotes are federation's replication form | +| **Missing** | closure RFC | views, `optional`, blocks as values, `having`, `collect`, Felleisen test as acceptance | +| **Missing** | editions RFC | `gq N;`, N-1 compiler, registry records editions | +| **Missing** | federation reservation | qualifier, per-graph pins, views as foreign sources, single-graph writes | + +--- + +## 11. Staging and sequencing + +The order below is derived from three things: what each piece *depends on* +(a change cannot ship before its enabler), what each piece *is worth* to the +agent workload (§8.6's metrics: round trips per task, tokens per correct +answer, repair turns), and what each piece *breaks* (anything that breaks +stored text waits for editions). Each stage is independently shippable and +independently valuable; nothing in a later stage is needed to make an earlier +one pay. Sizes are relative (S, M, L), not dates. + +### 11.1 The dependency graph + +```text + ┌────────────── E editions (gq N;, N-1 compiler, registry records) ──────────────┐ + │ │ + M0 measure ──► S1 ───┼──► F function registry ──► R retrieve clause ──► W2 computed writes │ + (baseline, │ (generic call, (windows, weights, (arithmetic + precondition)│ + task corpus) │ @Function, can_fail) norm, thresholds, │ + │ rerank; rewrites 0048) │ + ├──► S schema catalogue types ──────────────────────────────────────┐ │ + ├──► N envelope side channel (notifications, why_not, estimate) ────┤ │ + ├──► P paging (limit $k, continuation token) ────────────────────────┤ ▼ + │ ├──► X edition-3 spelling flip + ├──► C closure (blocks as values, having, collect, optional, views) ─┤ (arrows, mutation, retrieve + │ │ │ default, deprecations run + │ └──► A permissions (contracts as declarations │ out: search/fuzzy/match_text, + │ over views; actor views first scope) ──────────┤ infix contains, rrf positional k) + ├──► L lexical (@analyzed, terms, match_terms, bm25_v1) ─────────────┤ + ├──► W1 writes (pattern-scoped, returning, @valid/invalidate) ────────┤ + │ │ + storage upgrade ─┴──► I integer identity ──► Poly polymorphic endpoints ─────────────┘ + (interface *bindings* need no storage and ship under C) + + Fed federation: reservation in E/S; build after Poly + views +``` + +Reading it: **E** is the root of every change that touches stored text, so it +is first and it is small. **F**, **S**, **N**, **P**, **C**, **L**, **W1** are +mutually independent and additive; they can run in parallel on separate +tracks. **R** needs F (scores as values) and E (it changes a surface). **X** is +the one deliberately breaking stage, and it is last among the language stages +because by then every new spelling has existed additively for a release and +every old one has had its deprecation. **I** and **Poly** are a storage +upgrade with their own RFC pair and their own clock; only interface +*bindings* (no storage change) are pulled forward into C because they are the +bounded union §7.6 wants. **A** waits for views (contracts bind to them). +**Fed** is reserved early and built late. + +### 11.2 The stages + +| Stage | Contents | Depends on | Breaks stored text? | Gate before ship | What an agent gets | +|---|---|---|---|---|---| +| **M0 Measure** (S) | Baseline the §8.6 metrics on the personal, CRM and engineering graphs: tokens per correct answer, round trips per task, first-try validity and repair turns on the 45-task instrument plus a node-page and a discovery task; count refusals by `T` code in served metrics; freeze a 30-task agent corpus with exact-query ground truth | nothing | no | the corpus and numbers are checked in beside the GQT corpus | nothing yet; this is what every later stage is judged by | +| **E Editions** (S) | `gq N;` optional first line (absent = current); compiler compiles N and N-1; stored-query registry records the edition at validation; `queries validate` and `cluster plan` report edition drift with the rewrite; `GQ_LANGUAGE_VERSION` becomes the edition; compatibility-surfaces hashes per edition; the grammar reserves a graph segment in type references (`Fed` reservation, unused) | nothing | no | a GQT case per edition path; a server boot test with a GQ 2 stored query under a GQ 3 default | nothing visible; the stage that makes X survivable | +| **S Schema catalogue** (S) | `@Type`, `@Property`, `@Edge`, `@Interface`, `@Index` (with coverage and pending counts), `@Function` (empty until F), `@View` (empty until C); `@example` annotation on types and views, stored as metadata and served through `@Type` | catalogue mechanism (exists) | no | GQT cases joining catalogue rows to user rows; `og_catalog` served | schema discovery in one query (#1058 answered); the agent-first paper's cheapest ask | +| **N Side channel** (S–M) | `N…` notifications with the four fields (cross join, destination-ranked scan, wildcard without budget, uncapped bm25); `why_not` funnel on zero rows from the profile; `estimate` on `explain` (`usage` with `estimated: true`); `complete`/`incomplete_reason` reserved for `@satisfice` | 0047's diagnostic type | no | the funnel checked against the profile's `actual_rows`; notification codes frozen in `codes.rs` | repair turns fall (Olausson); empty results explain themselves (MAC-SQL) | +| **P Paging** (S) | `limit $k` (I64 parameter, non-null); continuation token in the envelope derived from the total order over the pinned snapshot; `--after ` on CLI, `after` on HTTP | total order (exists) | no | token round trip across a write (snapshot pinned, no skip, no repeat); two-key tie cases | stable pages without hand-written keyset predicates | +| **F Function registry** (M) | generic `name(args, opt: v)` call rule; registry with signature, nullability rule, `can_fail`, `volatile`, pushdown mapping; the existing aggregates and search calls become rows; `contains()`/`starts_with()` land as the operator amendment specifies (infix kept, deprecated); first scalar library (`lower`, `upper`, `trim`, `length`, `coalesce`, date extraction, `case`); arithmetic with typed overflow refusal and the `can_fail` pushdown rule | E (for the infix deprecation) | no (infix stays one release) | `@Function` rows equal the compiler's registry (a test, not a doc); pushdown refuses to lift a fallible conjunct past its guard; GQT per function | derived values filter and order server-side; the end of client-side arithmetic | +| **C Closure** (L) | blocks as values (`count { } as n`, `exists { } as b`), any expression on a block's right side; `having { }`; `collect($x)` / `collect($x { a, b })`; `optional { }` with pattern-bound filters and nullable bindings (`LeftExpand`); `view` declarations inlined at compile time, bindable as `$x: View(args)`, registered and validated with stored queries, listed in `@View`; interface *bindings* `$x: Named` as a union scan with the common projection (from the polymorphism RFC, no storage change); the Felleisen test applied to each as an acceptance criterion in the RFC | S (for `@View`), registry infrastructure | no | GQT per form; plan acceptance updated for `LeftExpand` and the projected block; a view invalidation case under `cluster plan`; the node-page task in M0 goes from N requests to 1 | the largest round-trip reduction in the plan; derived relations agents can author on a branch | +| **L Lexical** (M–L) | `@analyzed` profiles in SchemaIR; `terms()`, `match_terms`; exact scan as oracle; `bm25_v1`; `search`/`fuzzy`/`match_text` compile to `match_terms` with a deprecation under E | E; F (named options) | no (deprecation window) | the RFC's oracle fixtures and the three Lance guards; #747/#748/#749 cases green at every index state | text answers that do not depend on index state | +| **R Retrieve** (L) | `retrieve { }` clause binding typed score columns; arms with `k`; `norm()`, `rrf()` with n arms and weights, `rerank(model, text, k)` as a provider-backed stage; score columns usable in `having`/`order`/`return`; `@approximate(recall)` declaration; achieved recall in the envelope; 0048's laws, `exact: true`, `Vector(n, distance=…)` and recipes; rankings in `order` keep compiling for one release and lint prints the rewrite | E, F, 0047 (shipped) | no until X | the fusion and vector oracles 0048 names; GQT for windows, weights, thresholds, rerank, n arms; the discovery task in M0 answered by `rrf` over interface arms | hybrid search with weights, thresholds and a reranker; the end of two-query hybrids | +| **W1 Writes** (M) | pattern-scoped `update $a … where { }` and `delete $e where { }` with one target type; `returning { }`; `@valid(from, to)` schema role and `invalidate`; `set { x: null }` in assignment position; bounded pattern-scoped `insert` | read planner under the write door (the shared expression RFC's "later RFC") | no | staging protocol unchanged (D2); GQT per form; affected receipts | invalidation as a verb; one round trip per write instead of read-then-write | +| **W2 Computed writes** (S) | `set { visits: visits + 1 }` under `@if_commit` or a key equality; the read-set rule | F (arithmetic), W1 | no | contention case under DST; retry idempotence | the increment without a round trip | +| **X Edition 3** (M) | `mutation` keyword; arrow traversal spelling (`-e->`, `<-e-`, `-e-`); `retrieve` as the only ranking position; removal of `search`/`fuzzy`/`match_text`, infix `contains`/`starts_with`, positional `rrf` `k`; the GQ 2 compiler kept for one more release; `queries validate` prints every rewrite as a machine-applicable suggestion | every deprecation above having lived one release | **yes, under the edition** | every stored query in the fleet rewritten by the suggestion tool and re-validated; the 45-task instrument re-run on the new spelling | one spelling per operation; direction visible; the novelty tax paid once | +| **I + Poly** (L, own clock) | integer identity storage upgrade; polymorphic endpoints; unions `A \| B`; type tests; `@type_id`; interface-wide mutations; `rrf` over member arms as the interim cross-member search | storage upgrade RFC pair; C (interface bindings already shipped) | no (schema feature; old binaries refuse) | the RFC's prototype findings as regressions; upgrade journey tests | edge families collapse; discovery across types | +| **A Permissions** (L) | actor views first scope (type and column visibility, per-row placement, reads); `contract` declarations in GQ bound to views and stored queries; Cedar grants on named capabilities; the fragment = kernel minus retrieval | C (views), S, both RFCs amended per §10.5 | no | #951's differential oracle for the first scope; #1197's adversarial matrix | query-only agents; shared graphs with audiences | +| **Fed Federation** (L, later) | graph qualifier in type references; per-graph commit pins in the envelope; views over a foreign graph; change-feed-as-load for shares | E/S reservation; C (views); Poly (one-corpus search) | no | nothing until designed | cross-graph reads; shares kept current | + +### 11.3 Three parallel tracks + +The stages group into tracks that touch different code and can run at once: + +- **Compiler and language track:** E, F, C (grammar, type checker, lowering, + inlining), X. Owner: `omnigraph-compiler`, the planner's `resolve_query`. +- **Planner and engine track:** N, P, R, W1/W2, the `LeftExpand` and union + scan nodes, acceptance updates, the fusion and vector oracles. Owner: + `omnigraph-planner`, `crates/omnigraph/src/engine`. +- **Server, schema and policy track:** S, L's SchemaIR features, I/Poly, A, + the registry's edition records, `cluster plan` validation of views and + contracts. Owner: `omnigraph-server`, `omnigraph-cluster`, `schema_ir`. + +Two things keep the tracks from colliding. The GQT corpus is the shared +contract: every stage lands its cases first, and a case another track's +change turns red is a conversation, not a merge. And the catalogue is the +shared interface between tracks: F publishes `@Function`, C publishes `@View`, +S publishes the schema types, and each other track reads them rather than the +crate internals. + +### 11.4 Rules of the sequence + +1. **Additive before breaking.** Every new spelling exists beside the old + one for a full release before X removes the old one, so an agent reading + the deprecation warning has one release to learn the new form and the + suggestion tool to apply it. +2. **Enabler before consumer.** E before anything that breaks; F before R; + C before A; S before anything that needs a registry visible to agents. +3. **Measure at every stage.** M0's numbers are re-run after each stage on + the same corpus. A stage that does not move round trips, tokens per + correct answer or repair turns is questioned before the next starts; + a stage that moves first-try validity down (a novelty tax) must show the + repair turns compensate. +4. **One RFC per stage, the umbrella names them.** The umbrella document + (§10.7) carries this table and nothing else; each stage's RFC carries its + grammar, semantics, acceptance checks and cases. 0047 is already the + shape: seven items, eight rollout steps, each a PR with its regression. +5. **Storage changes on their own clock.** I and Poly follow the storage- + upgrade matrix (genuine predecessor binaries, whole-root restore); the + language stages never wait on them, and interface bindings are pulled out + of Poly precisely so that C does not either. +6. **The fleet is part of the gate.** Before X, every stored query on the + personal, CRM and engineering graphs is rewritten by the suggestion tool + and validated under edition 3; a rewrite the tool cannot make is a defect + in the tool, not a reason to extend the window. + +### 11.5 What the first month looks like + +M0 and E in parallel, since M0 is measurement and E is compiler work; S and +N start the week E's grammar lands, because both are small and both change +what an agent sees immediately. F's RFC (the operator-vocabulary amendment +rewritten as a registry) and C's RFC (derived relations and optional +patterns) are drafted during the same weeks, since drafting is the long pole +and both can be written by agents against sections 7.7, 9 and 10 of this +evaluation. R's rewrite of 0048 waits until F's registry shape is accepted, +because `norm`, `rrf` and `rerank` are its first rows. Nothing in the first +month breaks stored text, and by its end the agent has schema introspection, +reasons for empty results, cost estimates and parameterised paging without a +single rewrite. + +### 11.6 Breaking first: the revised sequence + +Sections 11.1 to 11.5 put the breaking stage last, the compatibility- +preserving order. With a pre-stable language (the shared expression RFC's own +words: "a better language wins over compatibility wherever the two conflict") +and a fleet of a few graphs whose stored queries one tool can rewrite, the +better order is the reverse: **one break, as early as possible, then every +addition on the final grammar.** What it buys is concrete. No construct is +implemented twice (old spelling plus new); no deprecation shims, lint +warnings or dual-parse windows are written and then deleted; the GQT corpus, +the skill, the docs and the fleet are rewritten once; and agents learn one +spelling. What it costs is that the grammar must be decided before most of +its semantics exist, so the break must land *slots* whose behaviour arrives +later. This section supersedes rule 1 of 11.4 and the first-month plan in +11.5; the dependency graph, the stage contents and the tracks stand. + +#### The principle: break the grammar once, then only fill slots + +A later change is non-breaking exactly when the grammar already has a place +for it. So the break is defined by a **freeze list**: every syntactic +decision that could otherwise force a second break. Everything on it ships in +the one breaking release, even where its semantics are a stub; everything +not on it is additive forever after, by construction. + +| Frozen in the break | Decision | Semantics at break time | +|---|---|---| +| Declaration kinds | `query`, `mutation`, `view`, `contract`, `batch`, plus the schema kinds `node`, `edge`, `interface` in the same grammar; one file type, `.pg` accepted as the same grammar | `view`, `contract`, `batch` parse and are refused with a stable code ("reserved; not yet served") until their stage | +| Clause set and order | `match` → `retrieve`? → `return` → `having`? → `order`? → `limit`? → `returning`? (mutations: statements → `returning`?) | `retrieve` carries exactly today's power (`nearest`, `bm25`, two-arm `rrf`); `having` works immediately (one `Filter` over `Aggregate`); `returning` parses, refused until W1 | +| Block kinds in `match` | `not`, `exists`, `count`/`sum`/… blocks, `optional` | `optional` parses, refused until C | +| Traversal spelling | `-e->`, `<-e-`, `-e-`, `-$x:e->`, `-(a\|b){m,n}->`, `-*->` | today's semantics | +| Call syntax | `name(args, opt: v)`, generic, resolved against a registry; `contains()`, `starts_with()`, `terms()` reserved as names; no function in the grammar | today's functions become registry rows; `terms()` refused until L | +| Reserved words | the logic set only: `and`, `or`, `not`, `is`, `null`, `in`; every clause and declaration keyword contextual (word-bounded, position-based, as today) | closed; the amendment's rule becomes a grammar test | +| Null | no literal in reads; `null` legal in assignment position only | today's rules plus the assignment case | +| Parameters | `limit $k`; `$branch: Branch`, `$at: Commit` typed parameter kinds; defaults `($k: I64 = 10)` | `limit $k` works; `Branch`/`Commit` kinds parse and bind | +| Annotations | `@ident(args)` on any declaration, retained as metadata; engine-known names typed (`@budget`, `@approximate`, `@satisfice`, `@if_commit`, `@example`, `@description`, `@instruction`) | known ones that are stubs refuse with a code | +| Meta-fields and catalogue names | `@id`, `@src`, `@dst`, `@type`, `@type_id`, `@distance`, `@score` reserved; `@Type`, `@Property`, `@Edge`, `@Interface`, `@Index`, `@Function`, `@View` names reserved | `@distance`/`@score` are the `retrieve` columns; the catalogue types ship in S | +| Type references | `[graph@]ns::Type` admitted by the grammar | the graph segment refused ("federation reserved") | +| Edition line | `gq 3;` optional first line | the compiler is edition 3; edition 2 parses for one release through the rewriter | +| Schema roles | `@valid(from, to)`, `Vector(n, distance=…)`, `@analyzed(...)`, `@embed` recipe arguments | `@valid` and `@analyzed` are SchemaIR features whose enforcement arrives with W1 and L; distance ships | +| Envelope | `graph_commit_id` as a map keyed by graph (one entry today), `notifications`, `why_not`, `complete`, `continuation`, `retrievals.recall_achieved`, `estimate` reserved | present and empty until their stage; HTTP contract 0.14 → 0.15, explain and bound-plan versions bumped once | + +Two things are deliberately *not* on the list, because they are additive by +nature: new registry rows (functions, catalogue types, annotations) and new +plan nodes behind existing syntax. + +#### The revised order + +```text +M0 measure ─► B the break (grammar RFC, rewriter, corpus and fleet rewrite; one release) + │ + ├──► parallel, additive, on the final grammar: + │ S catalogue · N side channel · P paging · F function library + │ C closure (fills optional/view/having/collect slots) + │ L lexical (fills terms/@analyzed) · R retrieve semantics (n arms, weights, norm, rerank) + │ W1 writes (fills returning/invalidate) · W2 computed writes + │ + ├──► I + Poly on the storage-upgrade clock + ├──► A permissions after C + └──► Fed after Poly and views +``` + +**M0** is unchanged and becomes more important: the instrument must run on +the *new* spelling before the break, against a parser that accepts it and a +mock executor, so that the one break is made on evidence (first-try validity +and repair turns on the frozen grammar) rather than on taste. + +**B, the break**, is one release and one RFC ("GQ 3 grammar"), with four +deliverables: the grammar and type rules of the freeze list; a **rewriter** +(`omnigraph queries migrate`, the machine-applicable `suggestion` machinery +turned into a tool) that rewrites every edition-2 source mechanically +(`$p worksAt $c` → `$p -worksAt-> $c`, `order { nearest(…) }` → a `retrieve` +arm, `query` writes → `mutation`, infix `contains` → `contains()`); the GQT +corpus rewritten by that tool and re-run as the proof that the rewrite is +semantics-preserving; and the fleet's stored queries rewritten under `cluster +plan` with the same tool. Edition 2 keeps parsing for that one release so a +stored query that escaped the sweep is rewritten at boot with a warning rather +than quarantined; the release after removes the edition-2 grammar. The +registry records the edition from this release on, which is all the edition +mechanism the project needs until the next break, if there ever is one. + +**Everything after B is additive** and runs as 11.2 and 11.3 describe, with +one difference: no stage carries a deprecation, and the stages that were +gated on E (F, L, R) are gated on B instead. R shrinks to semantics only +(n arms, weights, `norm`, thresholds already work because scores are columns, +`rerank`), since its clause shipped in B. C fills slots the grammar already +parses, so a stored query written on day one with an `optional { }` block +starts working the day C ships, with no edit. + +#### What this changes in the rules + +Rule 1 of 11.4 becomes: **break once, before anything is built on the old +spelling; after the break, only fill slots.** Rule 2 (enabler before +consumer) now reads B before everything. Rule 6 (the fleet is part of the +gate) moves from the end of the plan to its start: the rewriter and the +rewritten fleet are B's acceptance criterion, not X's. + +#### Risks, and what covers them + +- *A frozen slot turns out wrong.* The slots are small (a keyword, a clause + position, a reserved name) and each is backed by §9's semantics, §8's + evidence and the instrument's run on the new spelling. The remaining risk + is the price of never breaking again. +- *Grammar designed ahead of semantics drifts.* Each slot's refusal code + names the stage that fills it, and the GQT case that will go green is + written at B as an `expect error` case; the stage flips it. +- *The rewriter cannot rewrite something.* Then the break has found a real + ambiguity in the old grammar; fix the rewriter, never widen the window. +- *The break lands before the metrics prove the spelling.* That is what M0 on + the new spelling is for; if the instrument shows first-try validity down + and repair turns not compensating, the freeze list is edited before B, not + after. + +#### The first month, revised + +M0 on both spellings (the current grammar, and the frozen grammar through a +parser-only build), the GQ 3 grammar RFC, and the rewriter, all at once; the +GQT corpus rewritten by the rewriter as the RFC's evidence section. S, N and +P start the day the grammar RFC is accepted, since all three are additive and +independent of it, and ship in the same release as B so the first edition-3 +release already gives an agent schema introspection, reasons for empty +results, cost estimates and parameterised paging. F, C, L, R and W follow on +the final grammar with nothing to deprecate. + +--- + +## 12. TypeDB and TypeQL 3 + +Added 2026-10-29, after the grammar RFC was drafted, to answer one question: +does TypeDB change the design? Facts come from the TypeQL 3 grammar +(`rust/parser/typeql.pest` on `typedb/typeql`), the TypeQL reference at +typedb.com/docs, the GitHub release notes (TypeDB 3.0.0 GA 2024-12-20; 3.14.1 +on 2026-10-22), TypeDB's `llms-full.txt`, and twelve dated TypeDB blog posts; +the Dorn and Pribadi paper ("TypeQL: A Type-Theoretic & Polymorphic Query +Language", Proc. ACM Manag. Data 2(2), SIGMOD 2024, DOI 10.1145/3651611) is +behind ACM's bot check, so only its abstract and the documentation's statement +of its model are cited, and its formal results are not claimed here. + +### 12.1 What TypeDB is + +TypeDB is a schema-first database over the PERA model, "Polymorphic +Entity-Relation-Attribute", which its documentation calls "an extension of the +ER model with dependent typing". Three kinds, each declared explicitly +(`entity person`, `relation friendship`, `attribute name`), single-inheritance +`sub` within a kind, `@abstract`. A relation declares roles (`relates employer, +relates employee`), is n-ary (zero or more players per role), and can itself +play a role in another relation. Attributes are shared value instances: "two +attributes are considered equal if they have the same associate value and have +the same (origin) type", ownerless attributes are removed unless +`@independent`, and attributes cannot own attributes. Annotations are a closed +grammar list: `@key`, `@unique`, `@card(N..M)`, `@range`, `@regex`, `@values`, +`@abstract`, `@independent`, `@doc`, `@meta`, and three the docs mark "planned +feature and not yet available" (`@cascade`, `@subkey`, `@distinct`). + +TypeQL 3 is a **pipeline**: `with fun` preambles, an optional `given` stage of +typed input rows (3.12.0), one or more of `match`, `insert`, `put`, `update`, +`delete`, `select`, `sort`, `distinct`, `offset`, `limit`, `require`, +`reduce`, an optional terminal `fetch`, an optional `end;`. The semantics are +a row stream: "each stage receives the stream of rows output by the previous +stage, and only the variables in those rows are available to it"; the first +stage gets the unit row; a `match` cross-products its input rows with its +matches; `reduce … groupby` keeps only reduced and grouping variables and "need +not be the last stage", so `reduce` followed by `match` is the HAVING; `insert` +after other stages runs per row and emits "input rows extended with the newly +inserted values", so `match → reduce → match → insert` and `put → match → +insert` are documented pipelines. Answers are rows, or JSON documents from +`fetch`, which is terminal and builds nested documents from `$x.A`, +`[ $x.A ]`, `{ $x.* }`, expressions, function calls, and bracketed +sub-pipelines (`[ match … fetch { … } ]`, `( match … return count; )`). + +**Functions replaced rules in 3.0.** `fun name($a: T) -> { T, U }` returns a +stream, `-> T, U` at most one row; a body is a read pipeline ending in +`return { … }`, `return first …`, `return count;`, `return check;`. Calls are +`let $x in f(…)` for streams, `let $x = f(…)` for singles, and scalar calls +inside expressions. Recursion is allowed through stream functions, with +"single return functions may not call themselves recursively" (3.5.0), a +documented stratification principle for negation, suspension to break cycles, +and a planner that "just sets the recursive call cost to 1". Names are +globally unique with no overloading. `with fun …` declares query-scoped +functions. + +**Patterns**: conjunction, `{ } or { }` (a true logical or, branches +unordered), `not { }` (nestable, giving "a form of for all"), and `try { }`, +the optional block (read side 3.4.4, 2025-08-05; write side 3.7.0). Type +variables stand in every type position: `$x isa $t`, `$t sub person`, `$o owns +email`, `user plays $r`; `isa!`/`sub!` are the exact forms; `is` is identity. +Expressions have `==`, `!=`, `<`…, `contains` (case-insensitive) and `like` +(Rust regex), arithmetic with precedence, `let`. Lists and structs are in the +grammar and announced for 3.0 in September 2024, and the docs on 2026-10-29 +still say "planned feature and not yet available"; the `list` reducer is +"unimplemented" in TypeQL 3.8.2. + +**Validation**: type inference runs before execution, "if no valid +combinations of types can be inferred for the variables in body pattern of +the stage the query will be rejected"; expression types resolve at compile +time; errors carry a code (`[INF11] Type-inference was unable to find +compatible types…`, `[TQL03]`); an `analyze` HTTP endpoint (3.5.0) returns the +type-annotated structure without executing; `typeql-check` validates syntax +offline. Transactions are `read`, `write`, `schema` with snapshot isolation. +**There is no branching, versioning or time travel**, no vector or full-text +search in any release note or grammar rule (an alpha `cosine_similarity_search` +preview for invited testers, 2026-08-18; the features page marks vector +indexing "in development"), and string search is `contains` and `like`. + +**The 2 to 3 break** had no query-migration tool: export the 2.x data, +"manually update your schema definition file", import, then "update your +queries using from 2.x syntax to 3.x syntax". Rules were removed, `get` became +`select`, `group`/`aggregate` became `reduce`, `?` value variables became `$` +with `let`, `long` became `integer`, root types went, `$r (role: $p) isa T` +changed meaning. Inside 3.x, `within` was renamed `groupby` in TypeQL 3.0.1, +a patch release, and `end;` arrived in 3.2.0 because "without `end`, TypeQL +parses the longest valid query it can". + +**On agents**, TypeDB's own measurements (blog, 2026-09-17, 3,024 runs over +Reactome with Claude Sonnet 5 and DeepSeek V4 Pro): bare TypeQL 24.8% against +SQL 62.4%; with a skill, examples and four retries TypeQL reaches 92%, and +"in-context learning can almost entirely compensate for baseline training +data deficit". A companion post (2026-10-14) reports first attempts: SQL 82% +correct and 17% silently wrong, TypeDB 75% correct, 23% a visible error and 2% +silently wrong. The docs ship `llms-full.txt`, a skill, `typeql-check`, an +MCP server, and a `given` stage sold as the anti-injection parameterisation. + +### 12.2 TypeDB against the axes, and against GQ 3 + +| Axis | TypeQL 3 | GQ 3 (RFC) | Reading | +|---|---|---|---| +| Closure (2.2) | Linear: every stage consumes the previous stage's rows, any order, kinds repeat; functions are named read pipelines | Non-closed kernel plus four reserved forms: `view` (a named pattern with typed columns, inlined), `optional`, block aggregates as values, `having` | Same demand, two mechanisms; see 12.3 | +| Polymorphism (2.1, 10.4) | Type variables in every type position; `isa` includes subtypes, `isa!` does not; parametric queries "valid over any schema" | Interface bindings, `@type`/`@type_id`, `is`; catalogue types `@Type` for the schema as data | TypeQL is more uniform; GQ's split (bind by interface, read the schema by catalogue) is the same power with two spellings | +| Direction (2.6) | None: relation ends are roles | Written arrow, checked against the schema | §13 | +| Optional (2.5) | `try { }`; unbound optionals fail later stages; `require` filters them | `optional { }` slot | Same form, same rule; TypeDB's "may only occur in a single try-block" is the scoping rule the closure RFC needs | +| Aggregation (2.8) | `reduce … groupby`, then any stage | `return` with aggregates, then `having` | GQ's is one fixed shape; TypeQL's allows re-matching on the reduced value, which `view` gives GQ | +| Result shape (2.15) | Rows, or nested JSON from terminal `fetch` with sub-pipelines | Rows with node objects; a partial object projection and a nested block in `return` are additive (parse errors today) | No slot needed; the nested-document demand (#649) has a precedent | +| Writes (2.11) | Pipelines mix reads and writes; `insert` emits augmented rows; `put` is match-or-insert of the whole pattern; `update` only for cardinality-1 capabilities | Reads and writes separate; keyed insert as upsert; pattern-scoped writes with `returning` | GQ keeps the wall; `returning` covers the read-after-write TypeDB gets from a trailing `match` | +| Parameters | `given $u: username, $phone: string?;` typed input rows | scalar typed parameters, `limit $k` | A row-valued parameter is additive in GQ (a type in parameter position); worth an RFC when batch loads of lookups appear | +| Reuse (2.13) | `define fun`, `with fun` query-scoped; no overloading; `@doc`/`@meta` | `view`, `@Function` registry, generic annotations | Same shape; adopt the no-overloading rule for the registry | +| Refusal (2.17) | Codes, pre-execution type inference, `analyze`, `typeql-check` | Four-field diagnostics with fixes, explain, lint | Equal; GQ's `fix` field is the extra | +| Time and branches (2.12) | None | Branches, commits, snapshots, change feed | GQ's defining advantage stays | +| Retrieval (2.10) | None shipped | `retrieve` arms over `nearest`, `bm25`, `rrf` | Not comparable | +| Recursion | Stream functions recurse, cost 1, suspension, crash fixes through 3.14 | Refused without a budget | TypeDB's release notes are the evidence for the refusal | + +### 12.3 Pipelines versus the non-closed kernel + +TypeQL's pipeline is the cleanest statement of the alternative GQ did not take, +so it deserves the argument in full. A pipeline makes the language closed by +construction: `reduce` then `match` is HAVING, `match` then `insert` then +`match` is read-after-write, and every stage is a function from rows to rows, +so the agent never needs a second construct for "do something with the result +of that". What it costs: + +1. **The query's text order is the plan's order.** "Sorting requires the + server to fetch all results, sort them, then proceed"; a `match` after a + `reduce` runs after it; a `limit` placed early cuts early. The optimizer + orders statements *inside* a `match`, not stages. GQ's one shape + (`match → retrieve → return → having → order → limit`) is the shape the + planner reorders freely under a declared result; that is what plan truth + and the replay contract (§2.17, RFC 0047) rest on. A stage pipeline moves + the planner's job into the author's hands, which is the SQL `WITH` trade + GQ refused in §4.4. +2. **Rows multiply.** A stage's output is its input "augmented": two input + rows times two matches is four rows. TypeQL documents this and gives + `select`, `distinct` and `$_` to manage it. GQ's pattern conjunction is one + join with one result schema; `optional`, `view` and the block value add the + closure cases without a cross product the author has to remember to prune. +3. **Reads and writes mix.** A `match → insert → match` pipeline is coherent + under snapshot isolation inside one transaction, which TypeDB has and GQ + does not want: GQ's mutation is one commit, with the D2 constructive-or- + destructive rule and a `returning` slot for what the agent needs back. + +Where TypeQL is right and GQ 3 already agrees: functions as the unit of +reuse, declared with a typed signature and called by binding (`let $x in +f($y)` is `$x: Name($y)`); `try` as the optional form with "may only occur in +a single try-block" scoping; reductions followed by a filter (`having`); +no overloading; query-scoped function declarations (`with fun`), which GQ's +`view` inside a `batch` covers. Where TypeQL is right and GQ 3 does not yet +say so: a **type variable in binding position** (`$x: $t` with `$t` ranging +over node types, with `$t` itself returnable) is the parametric query the +polymorphism review (§10.4) asked for as `$x: *`; it is additive (a variable in +`type_ref` position is a parse error today), so it needs no slot, and the +polymorphism RFC should take the TypeQL form rather than invent a wildcard. +Likewise a row-valued parameter (`given`) is additive. + +**Verdict on closure**: TypeDB does not change the design. It supplies the +best-documented version of the mechanism §9.8 rejected, and the reasons to +reject it (plan truth, row multiplication, read-write mixing) are the ones its +own documentation states as behaviour. + +### 12.4 Where TypeDB confirms the RFC + +- **Slots that refuse loudly.** TypeQL's grammar carries lists, structs, + `@cascade`, `@subkey` and `@distinct`; its documentation says "not yet + available in TypeDB"; TypeQL 3.8.2 adds "a Collector variant of Reducer to + accommodate the unimplemented list reducer". That is the freeze list's + strategy, running in production for two years: parse the form, refuse it + with a message, fill it when the semantics land. +- **One break, done manually, is the alternative the rewriter avoids.** The + 2 to 3 migration is export, hand-rewritten schema, import, hand-rewritten + queries. TypeDB could afford it with rules removed and the data model + changed; GQ's break changes text only, which is why `queries migrate` with + IR equality is cheap and worth building. +- **Breaks inside a major are what the edition line prevents.** `within` + became `groupby` in a patch release, `end;` changed how consecutive + pipelines parse in a minor, and TypeDB's own LLM summary disagrees with its + grammar on `=` versus `==`. An edition number in the file, a closed reserved + set and one spelling per meaning (§14) are the answers. +- **Retries beat first-try accuracy, and visible errors beat silent ones.** + TypeDB's 24.8% to 92% with a skill plus four retries, and 23% visible versus + 2% silent wrong, are the same shape as §8.3 and RFC 0047's 45-of-45 with + 8 refusals: optimise the diagnostic and the loop, and spend the prior on + the few spellings that matter. +- **Annotations as documentation for agents.** `@doc` and `@meta` on types, + capabilities and functions (3.12.0) are `@description`, `@instruction` and + the generic `@name(args)`. +- **Validation before execution with a structured answer** (`analyze`) is + GQ's lint and explain; the `fix` field remains GQ's edge. + +### 12.5 What GQ should take from TypeDB, and what to resist + +Take: the no-overloading rule for the function registry; the type-variable +binding as the polymorphism RFC's discovery form; the single-origin rule for +optional variables; a row-valued parameter as a later additive RFC; the +practice of shipping a grammar validator (`typeql-check`) as a standalone +tool, which `omnigraph lint` already is and the skill should name. + +Resist: roles instead of direction for binary edges (§13); n-ary relations +(a reified node with typed edges says the same thing with the planner's +existing join, and every comparator in §3 is binary); attributes as shared +value instances (a storage model, not a language gain, and it is why TypeDB +needs `@independent` and the dangling-attribute rule); recursion through +functions without a budget (TypeDB's own fixes through 3.14 for suspended +recursive calls are the cost); pipelines (12.3). + +## 13. Expressing direction + +Added 2026-10-29. Facts from the Cypher 25 manual and openCypher 9, the +opengql ANTLR grammar generated from the ISO 39075 BNF, Deutsch et al. +"Graph Pattern Matching in GQL and SQL/PGQ" (arXiv 2112.06217), Oracle 23 +`GRAPH_TABLE` and DuckPGQ's tests, the TinkerPop reference, SPARQL 1.1, +Datomic's query and pull references, Soufflé's relations page, the TypeQL 3 +grammar and reference, the GraphQL specification, LangChain's +`CypherQueryCorrector` source and its origin pull request, the Cypher +direction competition, three Neo4j developer-blog posts, and CypherBench +(arXiv 2412.18702, Figure 6 read from the published image). + +### 13.1 Three ways to say which way an edge goes + +Every language that has edges answers the same three questions: where does the +orientation of a binary relation live, how is "either way" written, and what +happens when the author writes it the wrong way round. + +| Language | Where orientation lives | Written form | Either way | Wrong way round | +|---|---|---|---|---| +| Cypher 25 | On each stored relationship ("all relationships have a direction"); no schema | `-[r]->`, `<-[r]-` | `-[r]-` matches "relationships of any direction"; a bound undirected pattern "in both directions", returning both bindings | No match: an empty result, no error | +| ISO GQL | Schema: `edgeKind : DIRECTED \| UNDIRECTED` with source and destination node types | seven forms: `<-[ ]-`, `~[ ]~`, `-[ ]->`, `<~[ ]~`, `~[ ]~>`, `<-[ ]->`, `-[ ]-`, plus abbreviations | `-[ ]-` is "left, undirected or right"; `<-[ ]->` either orientation but not undirected; `~[ ]~` undirected edges only | Unspecified in the sources fetched | +| SQL/PGQ (Oracle, DuckPGQ) | Edge table `SOURCE KEY … DESTINATION KEY …` | `-[ ]->`, `<-[ ]-` | `<-[ ]->` or `-[ ]-`: "any directed edge (right or left)", a `UNION ALL` of both orientations in DuckPGQ's test | Matches the other stored orientation; no error documented | +| Gremlin | Per edge | `out()`, `in()`, `outE()`, `inE()`, `outV()`, `inV()` | `both()`, `bothE()`, `otherV()` | No traversers; no error | +| SPARQL | The triple's order, subject to object | `?s p ?o`; inverse path `^p` "object to subject" | `(p \| ^p)` | No match | +| Datomic | The ref attribute's orientation, entity to value | `[?e :a ?v]`; reverse in pull as `:ns/_attr` | Two clauses | No match | +| Soufflé, Datalog | Argument position | `Edge(x, y)` | A symmetry rule `E(a,b) :- E(b,a)` | A different relation | +| GraphQL | The schema's field | a field | a second field, by convention only (the spec says nothing) | No such field: a validation error | +| TypeQL 3 | Nowhere: a relation has roles, `relates employer, relates employee`; `links (employee: $p, employer: $c)` | the role name | `rel ($a, $b)` with roles inferred; a symmetric relation is one role played twice (`relates friend @card(2)`) | A role the player's type cannot play: a type-inference refusal | +| GQ 2 (today) | The schema: `edge WorksAt: Person -> Company`; position in the triple | `$p worksAt $c`, source on the left | `$p $f`, same-type edges only (`T22`) | `T5` when the endpoint types do not fit; for a same-type edge, the other orientation | + +The table has three designs, not nine. **Arrow-written** (Cypher, GQL, PGQ, +Gremlin, SPARQL's `^`): the author writes the orientation every time, the +schema may or may not know it, and a wrong arrow is a silent empty result. +**Schema-decided** (Datomic, GraphQL, Soufflé, GQ today): the orientation is a +fact about the type; the author positions the ends, and the compiler knows +which way the edge runs. **Role-named** (TypeQL): the relation has no +orientation at all; its ends have names, and a binary relation is the special +case of an n-ary one. + +### 13.2 What the agent evidence says + +Direction is a measured error class, and it is the one error class the +ecosystem has decided to fix by machine. + +- CypherBench (gpt-4o and llama3.1-8b, 50 sampled failures each, multiple + errors per instance): "Reversed Direction" is among the categories that + "frequently occur in both models". Read off Figure 6, it is 9 of 55 error + marks for gpt-4o and 13 of 71 for llama3.1-8b, the second-largest category + for both after the pattern-level ones, ahead of syntax errors for gpt-4o. +- Neo4j's own prompt for GPT-4 carries "Relationship point from source to + target nodes" and "Make sure to respect relationship types and directions", + and the post concludes "sometimes it messes up the relationship direction + (not frequently, though)". +- LangChain's `GraphCypherQAChain(validate_cypher=True)` runs + `CypherQueryCorrector`, "used to correct relationship direction in generated + Cypher statements", built from the graph's structured schema and copied + from "the winner's submission to the Cypher competition" that Neo4j + sponsored in 2023 because LLMs "can sometimes mess up the direction of the + relationship". The author's claim is the important one: "the direction of + the relationship can be deterministically [fixed] after the LLM generates a + Cypher statement based on the provided schema". Its limits are also stated: + "we only catch if the relationship direction doesn't fit the schema, we + don't catch anything else", and a path that cannot be made to fit returns + an empty query. +- Neo4j's iterative-refinement study lists "Rule-Based Relation Direction + Correction" as a verification class and finds it "has low capacity (i.e., + only fixing relation direction)"; CyVer ships a "Wrong direction in path" + test. Ozsoy et al. and SM3 do not report direction at all. + +Two conclusions follow. First, when the two endpoint types differ, the arrow +carries **no information the schema does not already have**: a corrector can +rewrite it deterministically, and does. Second, when the endpoint types are +the same (`Person knows Person`, `Company subsidiaryOf Company`), the arrow is +the **only** carrier of meaning, and this is exactly where the models fail: +CypherBench's example is `subsidiaryOf` written from the parent. No +post-processor can fix that case, because both orientations fit the schema. + +### 13.3 What follows for GQ 3 + +The RFC made direction visible (`$p -worksAt-> $c`) for the reader and for the +model's prior, and made `T5` check the written arrow against the schema. The +evidence sharpens that into four rules. + +1. **Write the arrow once, source on the left: `->` only.** The left arrow + `$c <-worksAt- $p` is a second spelling of the same clause. GQ has no path + chains (a traversal is `variable edge variable`), so `<-` is never needed + to continue a line; it is pure redundancy, the Go test's first casualty + (§14). The parser knows it: `<-e-` is a parse error whose fix prints the + clause with the ends swapped, as the triple form's fix prints the arrow. +2. **A wrong arrow is a typed error with a machine-applicable swap**, never + an empty result and never a silent rewrite. `$c: Company -worksAt-> $p` is + `T5` with `fix` = the swapped clause. This is LangChain's corrector moved + into the compiler, where it belongs, with the two differences the evidence + asks for: it is loud (the agent learns in one turn, which Olausson's + result in §8.3 says is worth more than the edit) and it never guesses when + both orientations fit. +3. **Same-type edges are where direction is meaning, and position carries + it.** `$a -subsidiaryOf-> $b` means `$a` is the subsidiary. No rule can + check that against the schema; what GQ can do is make the schema say which + end is which in words, so the error is visible: `@description` on the edge + today, and the skill's schema card prints `Company -subsidiaryOf-> + Company (subsidiary -> parent)`. +4. **`-e-` means "whichever orientation the endpoint types admit".** For a + same-type edge that is both orientations, deduplicated, as `` is + today. For an alternation or wildcard over edges of differing orientation + (`$p: Person -(owns | ownedBy)- $d: Doc`, the issue 659 case, which today's + triple resolves per member) it is each member's declared orientation. The + only refusal left is the genuinely ambiguous one: an asymmetric edge whose + endpoints are not typed well enough to fix the orientation. Today's `T22` + ("undirected traversal needs a same-endpoint-type edge") narrows to that + case. The bound edge's `@src` and `@dst` keep reporting the stored + orientation, which is why they are not redundant with the pattern's ends. + +What GQ 3 does not take from TypeQL is roles. For a binary edge, `employee` +and `employer` are documentation of `src` and `dst`, and GQ puts +documentation in `@description`; for an n-ary relation, which GQ does not +have (§12.5), roles are the right design and a reified node with typed +edges is GQ's spelling of it. A role name on an edge declaration +(`edge WorksAt: employee Person -> employer Company`) would be additive if a +later RFC wanted it, since a word before a type in that position is a parse +error today. + +## 14. One way to say it + +Added 2026-10-29 at the maintainer's request: an overall requirement that +there be, ideally, one way of expressing something, in the spirit of Go. + +### 14.1 The principle, as Go states it + +Go's FAQ, on why there is no `?:`: "the language's designers had seen the +operation used too often to create impenetrably complex expressions. The +`if-else` form, although longer, is unquestionably clearer. A language needs +only one conditional control flow construct." On layout: "the advantages of a +single, programmatically mandated format for all Go programs greatly outweigh +any perceived disadvantages of the particular style", and the proverb +"Gofmt's style is no one's favorite, yet gofmt is everyone's favorite". On +design: "keep the concepts orthogonal … Orthogonality makes it easier to +understand what happens when things combine", and Pike's "Less is +exponentially more". Go is not absolutist (`var x = 1` and `x := 1` both +exist) but it has two mechanisms, and they are the two GQ should adopt: + +- **The grammar has one form for one meaning.** A respelling that adds no + meaning is not an alternative, it is absent, and the compiler names the one + form when it sees the other. +- **The formatter canonicalises what falls out of general rules.** `gofmt -s` + rewrites `s[a:len(s)]` to `s[a:]`: a form the grammar cannot forbid, + because it is an instance of a general construct, is rewritten to its + canonical instance by a tool everyone runs. + +For an agent author the principle has a measured value: §8.3's finding that +retrieved examples erase the unfamiliar-spelling gap only works if there is +one spelling to retrieve, and the RFC's own motivation ("a second spelling +for the same operation costs it on every query it reads") is this principle +applied to deprecation shims. + +### 14.2 The test applied to the freeze list + +Every pair of forms in GQ 3 that could mean the same thing, and the ruling. + +| Pair | Same meaning? | Ruling | +|---|---|---| +| `$a -knows-> $b` and `$b <-knows- $a` | Yes, exactly | Grammar: `->` only; `<-` is a parse error with the swapped clause as its fix (§13.3) | +| `$a knows $b` (triple) and the arrow | Yes | Grammar: arrow only; already in the RFC | +| `x contains y` and `contains(x, y)` | Yes | Grammar: the call only; already in the RFC | +| `rrf(a, b, 60)` and `rrf(a, b, k: 60)` | Yes | Grammar: named option only; already in the RFC | +| `exists { P }` and `count { P } > 0`; `not { P }` and `count { P } = 0` | Yes | The grammar cannot forbid the count form without forbidding `count { } > 2`. Formatter: `omnigraph queries fmt` (the rewriter's second job) canonicalises the two thresholds to `exists`/`not`, and lint names it. The grammar keeps `exists` because GQL has only `EXISTS` and the planner's semi-join is the honest plan | +| `$x.@type = "Person"` and `$x is Person` (polymorphism RFC) | Yes, once `is` exists | The polymorphism RFC owns `is`; the formatter canonicalises the string comparison to it, so one way survives the addition | +| `$x: Interface` and a wildcard `$x: *` | Yes | Grammar: no wildcard; §12.3's type-variable binding is the discovery form and it is not a respelling (it binds the type) | +| `return { $a.slug, $a.name }` and `return { $a { slug, name } }` | No: flat columns versus one nested object | Both stay; different result shapes | +| `$e.@src` and the left variable of the pattern | No under rule 13.3.4: `@src` reports stored orientation, the pattern's end is the written one | Both stay | +| `limit 10` and `limit $k` | No: a literal and a parameter | Both stay | +| `count(distinct $x)` and `return distinct` | No: a column's domain versus the row set | Both stay | +| `having { }` and a filter in `match` | No: above and below the aggregate | Both stay | +| `optional { }` and a `view` bound with an outer join | No: `view` has no outer-join form | One way | +| `@if_commit($head)` and the request's precondition field | Same fact, two surfaces | The annotation is the declaration; the request field is its transport. The writes RFC should say the field is derived from the annotation, never a second source | +| positional and named call arguments | No: one argument has one form, positional if required, named if an option | Rule for the registry: a function declares which; a caller cannot choose | + +Two cautionary examples from §12 show what skipping the test costs. TypeQL +lets a single-return function be consumed as `let $x = f()` or `let $x in +f()` ("a general `in` used for streamed results is also allowed"), and its +own LLM summary spells equality `=` where the grammar says `==`; its relation +patterns have a full form (`links (…)`) and a shorthand, both documented. Each +is a second form an agent has to recognise. + +### 14.3 The rule, for the RFC + +A form enters the grammar only if no existing form has its meaning; a form +that falls out of a general construct and has a canonical instance is +rewritten to that instance by the formatter and named by lint; the +diagnostics know every respelling an agent is likely to write (the triple, +the left arrow, the infix operators, GraphQL's `mutation` for a read) and +print the one form as a machine-applicable fix. The freeze list is checked +against the rule once, in this RFC, and every later RFC's "Alternatives" +section answers it for the form it adds. + +## 15. Sources + +Sections 12 to 14 (added 2026-10-29) read, on that day: the TypeQL 3 grammar +(`rust/parser/typeql.pest`, `typedb/typeql` master), the TypeQL reference at +typedb.com/docs (data model, pipelines and every stage page, functions, +patterns, optionals, annotations, fetch, schema, transactions, the 2-vs-3 +diff and process pages, `llms-full.txt`), the GitHub release notes of +`typedb/typedb` and `typedb/typeql` from 3.0.0-alpha-1 to 3.14.1, twelve +TypeDB blog posts dated 2026-01-07 to 2026-10-26, Crossref for Dorn and +Pribadi (DOI 10.1145/3651611; the paper's body is behind ACM's bot check and +is not cited for content); the Cypher 25 manual (patterns, MATCH, CREATE, +scalar functions, structural types, grammar appendix, GQL conformance), +openCypher 9, the opengql `GQL.g4` grammar, Deutsch et al. arXiv 2112.06217, +Libkin's LDBC slides, Spanner and Fabric GQL pattern pages, Oracle 23.2 +`GRAPH_TABLE`, DuckPGQ's README and `undirected_edges.test`, the TinkerPop +reference, SPARQL 1.1, Datomic query and pull references, Soufflé relations, +the GraphQL October 2021 specification, Elasticsearch `join` and `nested`, +Vespa parent-child; LangChain's `cypher_utils.py` and `cypher.py`, pull +request langchain-ai/langchain#11078, the `cypher-direction-competition` +README, Neo4j developer-blog posts (ChatGPT-4 on any schema, iterative +refinement for Text2Cypher, model struggles, CyVer), CypherBench arXiv +2412.18702 with its Figure 6 image, Ozsoy et al. (ACL Anthology +2025.genaik-1.11), SM3 arXiv 2411.05521; the Go FAQ (go.dev/doc/faq), the Go +proverbs, and Pike's "Less is exponentially more" (2012). Items the sheets +could not confirm are marked in the text (the SQL/PGQ `ANY DIRECTED` +keywords, GQL's wrong-arrow behaviour, CyDRA's categories). + +Omnigraph, at `ea895066`: `crates/omnigraph-compiler/src/query/query.pest`, +`src/schema/schema.pest`, `src/query/codes.rs`; `docs/user/queries/*`, +`docs/user/schema/index.md`, `docs/user/search/*`, `docs/user/mutations/*`, +`docs/user/branching/index.md`; RFCs 0041, 0045, 0047, 0055, +2026-09-14 (compatibility surfaces), 2026-09-16 (session settings), +2026-09-24 (shared expression model), 2026-09-30 (typed edge alternation), +2026-10-15 (bulk load), 2026-10-16 (engine catalogue); `docs/dev/execution.md`, +`docs/dev/systems.md`; `skills/omnigraph/references/queries.md`, +`schema-design.md`; issues #485, #563, #565, #648, #649, #658, #721, #749, +#758, #917, #1050, #1057, #1058, #1165, #1191; the GQT corpus +(`crates/omnigraph-gqt/cases/`, 425 files). + +Comparators, primary documentation read on 2026-10-28: PostgreSQL 18 +(queries-with, functions-aggregate, functions-window, sql-merge, sql-prepare, +errcodes), pgvector README, DuckDB VSS and DuckPGQ, Oracle 23 `GRAPH_TABLE`, +Spanner Graph ISO conformance; ISO/IEC 39075:2024 via the opengql ANTLR +grammar, Neo4j's GQL conformance appendix and Spanner's statement reference; +Neo4j Cypher 25 manual (clauses, patterns, subqueries, `SEARCH`, aggregating +functions, status codes, the OPTIONAL MATCH blog); GraphQL draft spec and +graphql.org learn pages, Relay connections spec, GAP-49; Datomic query/pull/ +schema/time-filter references, Soufflé rules/aggregates/types, CozoDB queries/ +aggregations/vector/timetravel/execution, Rel cheatsheet, Logica, Dedalus +(EECS-2009-173); Elasticsearch 9.x Query DSL (bool, knn, nested, join, +retrievers, RRF, paginate, sort, collapse) and ES|QL (syntax, limitations, +stats, lookup-join, fuse, fork, rerank); LanceDB docs (search, filtering, +hybrid, reranking, indexing, versioning, branching, schema, update) and +lance.org (transactions, tags and branches, DataFusion); Vespa docs (query +language reference, grouping, ranking, phased ranking, rank features, schema +reference, nearest-neighbor guides, document-v1, query profiles, consistency). +Facts the sources could not confirm are marked *recalled* in §3. + +Literature (§8), read on 2026-10-28 and confirmed against arXiv, ACL Anthology, +Crossref or the venue's proceedings; full citations are in the entries. The +annotated bibliographies the section condenses were produced by three +research passes whose dropped candidates (unconfirmable or outside the source +policy: CypherBench, FeedbackEval, blog-only language comparisons) are not +cited. Two venue caveats stand: Text2Cypher is a COLING workshop paper and +"The Death of Schema Linking?" a NeurIPS workshop oral; both are included for +being the only direct measurements of their question. The BAIR post +"Intelligence is Free, Now What?" is a perspective, not a paper, and is cited +as such. From 7c44b23d683921d7ed84104615f3525d0d33b441 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Fri, 30 Oct 2026 01:16:19 -0700 Subject: [PATCH 09/13] docs(rfc): keep the triple traversal; an arrow is a decoration the one-spelling rule removes --- docs/rfcs/2026-10-28-gq-3-grammar.md | 135 ++++++++++-------- .../2026-10-28-gq-3-grammar-evaluation.md | 90 ++++++------ 2 files changed, 122 insertions(+), 103 deletions(-) diff --git a/docs/rfcs/2026-10-28-gq-3-grammar.md b/docs/rfcs/2026-10-28-gq-3-grammar.md index 8e079cf9d..4100f9c03 100644 --- a/docs/rfcs/2026-10-28-gq-3-grammar.md +++ b/docs/rfcs/2026-10-28-gq-3-grammar.md @@ -33,8 +33,8 @@ row, a plan node behind existing syntax, or the filling of a slot, and none of those breaks a stored query. The break is defined by a ***freeze list***. It renames the write declaration -`mutation`; spells a traversal with one visible arrow (`$p -worksAt-> $c`, source on -the left, `-e-` for either orientation); gives +`mutation`; keeps the triple traversal (`$p worksAt $c`, source on the left, +`` for either orientation), rejecting an arrow spelling as decoration; gives retrieval its own clause (`retrieve { }`) that binds typed score columns, leaving `order` to sort values; makes every function a generic call resolved against a registry, so no function remains in the grammar; closes the @@ -213,7 +213,7 @@ gq 3; query passages($matter: String, $q: String, $k: I64 = 5) { match { $r: SourceRevision { matter_number: $matter } - $r -hasPassage-> $p + $r hasPassage $p contains($p.text, "tariff") } retrieve { $p.lex: bm25($p.text, $q) } @@ -255,16 +255,16 @@ names the construct and whose `fix` names the owning RFC; a form marked | Read clause order | `match` → `retrieve`? → `return` → `having`? → `order`? → `limit`? | `match`, `return`, `order`, `limit` work; `retrieve` works with today's rankings (below); `having` works (a Boolean expression over aggregates and group keys, one filter above the aggregate) | this RFC for `having`; RFC 0048 for everything `retrieve` gains later | | Mutation clause order | statements → `returning`? | statements work; `returning` is a slot | writes RFC | | Blocks in `match` | `not { }`, `exists { }`, `count|sum|min|max|avg(…) { } op expr`, `optional { }` | the four work; `optional` is a slot | derived relations RFC | -| Blocks as values | a block aggregate in operand position, `return { count { $n -knows-> $m } as knows }`, `exists { … } as employed` | parses; a slot | derived relations RFC | +| Blocks as values | a block aggregate in operand position, `return { count { $n knows $m } as knows }`, `exists { … } as employed` | parses; a slot | derived relations RFC | | Bindings | `$x: Type { … }` as today; `$x: Name($arg, …) { … }` binds a declared `view` with arguments; `$x: Interface` binds an abstract type | the first works; a binding with arguments is a slot; an interface name is refused as today (`T1`) until its RFC | derived relations RFC (views), polymorphic types RFC (interfaces) | -| Traversal | `$a -knows-> $b` (source on the left, the one directed form), `$a -knows- $b` (whichever orientation the endpoint types admit), `$a -$e:knows-> $b`, `$a -(knows\|likes){1,3}-> $b`, `$a -*-> $b`, `$a -"in"-> $b` | works; the triple form `$a knows $b` and the left arrow `$b <-knows- $a` are parse errors whose fix prints the `->` form with the ends in schema order; an arrow written against the schema is `T5` with the swapped clause as its fix; `T22` narrows to an orientation the endpoint types leave ambiguous | this RFC | +| Traversal | unchanged from edition 2: `$a knows $b` (source on the left, the one directed form), `$a $b` (whichever orientation the endpoint types admit), `$a $e:knows $b`, `$a (knows\|likes){1,3} $b`, `$a * $b`, `$a "in" $b` | works; an arrow form (`$a -knows-> $b`, `$b <-knows- $a`, `$a -knows- $b`) is a parse error whose fix prints the triple with the ends in schema order; a triple written against the schema is `T5` with the swapped clause as its fix; `T22` narrows to an orientation the endpoint types leave ambiguous | this RFC | | Calls | `name(positional…, option: value…)` for every function; the registry holds `count`, `sum`, `avg`, `min`, `max`, `now`, `date`, `datetime`, `nearest`, `bm25`, `rrf`, `search`, `fuzzy`, `match_text`, `contains`, `starts_with`; `count(distinct x)` is the one special form | works; the infix `x contains y` and `x starts_with y` are parse errors whose fix prints the call; `rrf(a, b, 60)` is a type error whose fix prints `k: 60` | the operator-vocabulary amendment for the call rule; the function-registry work for `@Function` and new rows | | Reserved words | `and`, `or`, `not`, `is`, `null`, `in`; every other keyword is contextual (word-bounded, legal as an identifier in operand and property position) | works; a grammar test pins the set | this RFC | | Null | no literal in a read; `null` legal as the value of an `insert` or `set` assignment | works | this RFC | | Parameters | `$x: Type? = default`; `limit $k` with a non-null `I64` parameter; the parameter kinds `Branch` and `Commit` | defaults and `limit $k` work; `Branch` and `Commit` parse and are slots | this RFC; the branch-statements RFC for binding them | | Annotations | `@name(args)` on any declaration, retained as metadata; `@description`, `@instruction`, `@example`, `@if_commit($p)` are typed; `@budget`, `@approximate`, `@satisfice`, `@brief`, `@phase` are reserved | the typed four work (`@if_commit` binds the commit precondition the request carries today); the reserved five are slots | RFC 0048 and the agent-first batch | | Meta-fields | `@id`, `@src`, `@dst`, `@type`, `@type_id`, `@distance`, `@score` | the first four work; `@type_id` is a slot; `@distance` and `@score` are the typed fields of a `retrieve` arm's column | polymorphic types RFC for `@type_id` | -| Catalogue names | `@Operator`, `@Pass`, `@Setting`, `@Engine`, `@LogicalKind` exist; `@Type`, `@Property`, `@Edge`, `@Interface`, `@Index`, `@Function`, `@View` reserved; catalogue edges between them spelled with the same prefix, `$t: @Type -@property-> $p: @Property`, `$e: @Edge -@from-> $t`, `$t -@implements-> $i: @Interface`; a catalogue type is a node type for every purpose of the grammar (binding, traversal, a `retrieve` arm over its text fields, aggregation) | the five exist; the seven and the `@`-prefixed edge selector are slots | the engine catalogue RFC's schema extension, which declares the catalogue's own edges and which of its text fields are searchable | +| Catalogue names | `@Operator`, `@Pass`, `@Setting`, `@Engine`, `@LogicalKind` exist; `@Type`, `@Property`, `@Edge`, `@Interface`, `@Index`, `@Function`, `@View` reserved; catalogue edges between them spelled with the same prefix, `$t: @Type @property $p: @Property`, `$e: @Edge @from $t`, `$t @implements $i: @Interface`; a catalogue type is a node type for every purpose of the grammar (binding, traversal, a `retrieve` arm over its text fields, aggregation) | the five exist; the seven and the `@`-prefixed edge selector are slots | the engine catalogue RFC's schema extension, which declares the catalogue's own edges and which of its text fields are searchable | | Type references | `a::b::C`: zero or more lowercase segments then a type name; the grammar admits it everywhere a type name stands | one or more segments is a slot | schema namespaces RFC (`ns::Type`); a federation RFC for an outermost graph segment | | Schema roles | `Vector(n, distance="l2"\|"cosine"\|"dot")`, `@analyzed(…)`, `@valid(from, to)`, `@embed(source, model=…, …)` | `distance` works (RFC 0048's rule: omitted means `l2`); `@analyzed` and `@valid` are accepted as SchemaIR features and refused at use | RFC 0048 (distance, recipes), the lexical RFC (`@analyzed`), the writes RFC (`@valid`) | | Envelope | `graph_commits` (a map keyed by graph id, one entry today) beside `graph_commit_id`; `notifications`, `why_not`, `complete`, `incomplete_reason`, `continuation`, `estimate` | present where the owning stage has shipped, absent otherwise; all additive | the envelope stage RFCs | @@ -316,10 +316,6 @@ each source under edition 2, and writes edition 3: | Edition 2 | Edition 3 | |---|---| | `query name(…) { insert … }` | `mutation name(…) { insert … }` | -| `$a knows $b` | `$a -knows-> $b` | -| `$a $b` | `$a -knows- $b` | -| `$a $e:knows $b` | `$a -$e:knows-> $b` | -| `$a (knows \| likes){1,3} $b` | `$a -(knows \| likes){1,3}-> $b` | | `$x.title contains "g"` | `contains($x.title, "g")` | | `$x.tags contains "rust"` (a list property) | `"rust" in $x.tags` | | `$x.title starts_with "O"` | `starts_with($x.title, "O")` | @@ -350,10 +346,10 @@ Every refusal keeps the four-field diagnostic. New codes, allocated at implementation: - `Q00x` unknown edition; `Q00x` an edition line after another statement; -- the triple traversal, the left-arrow traversal, the infix string operators +- an arrow traversal (`-e->`, `<-e-`, `-e-`), the infix string operators and `query` with a mutation body are parse or type errors whose `fix` is the rewritten text (a `machine_applicable` suggestion); -- `T5` for an arrow written against the schema, whose `fix` is the clause +- `T5` for a triple written against the schema, whose `fix` is the clause with its ends swapped; `T22` only for an undirected traversal whose orientation the endpoint types leave ambiguous; - `L…` lint for `count { } > 0` and `count { } = 0`, whose `fix` is the @@ -393,11 +389,14 @@ The freeze list is checked against one rule, in Go's sense: the grammar has one form for one meaning, and the formatter canonicalises what the grammar cannot forbid. Concretely: -- A respelling that adds no meaning is not in the grammar. The triple - traversal, the left arrow, the infix `contains` and `starts_with`, the - positional `k` of `rrf`, and `query` around a write are each a parse or +- A respelling that adds no meaning is not in the grammar. The arrow + traversal (`-e->`, `<-e-`, `-e-`), the infix `contains` and `starts_with`, + the positional `k` of `rrf`, and `query` around a write are each a parse or type error whose `fix` prints the one form. The diagnostics know every respelling an agent trained on Cypher, SQL or GraphQL is likely to write. + The rule also cuts the other way: a decoration on the one spelling that + carries no information is not added, which is why the traversal keeps its + triple and gains no arrowhead ([Traversal](#traversal)). - A form that falls out of a general construct and has a canonical instance is rewritten to that instance by `omnigraph queries fmt`, the rewriter's second job (`gofmt -s` is the precedent), and named by lint: `count { P } > @@ -483,35 +482,44 @@ score columns are named by the query. ### Traversal ```pest -traversal = { variable ~ edge_part ~ variable } -edge_part = { "-" ~ edge_core ~ "->" | "-" ~ edge_core ~ "-" } -left_arrow = { "<-" ~ edge_core ~ "-" } // recognised only to print the fix -edge_core = { edge_binding? ~ edge_sel ~ traversal_bounds? } -edge_sel = { edge_alternation | edge_wildcard | catalogue_edge | edge_ident | string_lit } -catalogue_edge = @{ "@" ~ ident } // an edge of the catalogue's own schema; a slot +traversal = { variable ~ edge_binding? ~ (undirected_edge | edge_sel) ~ traversal_bounds? ~ variable } +undirected_edge = { "<" ~ edge_sel ~ ">" } +edge_sel = { edge_alternation | edge_wildcard | catalogue_edge | edge_ident | string_lit } +catalogue_edge = @{ "@" ~ ident } // an edge of the catalogue's own schema; a slot +arrow_traversal = { variable ~ ("<-" | "-") ~ edge_binding? ~ edge_sel ~ traversal_bounds? ~ ("->" | "-") ~ variable } // recognised only to print the fix ``` -There is one directed form, source on the left. The left arrow -`$c <-worksAt- $p` would be a second spelling of the same clause with no -meaning of its own, because a GQ traversal is one hop (`variable edge_part -variable`, no chains) and so never needs to continue a line in the other -direction; `left_arrow` is in the grammar only as a recognizer, so that the -parse error's fix prints the clause with its ends swapped, as the triple -form's fix prints the arrow. `undirected_edge`'s angle brackets disappear. - -The type checker's endpoint rule (`T5`) checks the written arrow against the -declared endpoints instead of inferring the orientation, and when the arrow -is written against the schema (`$c: Company -worksAt-> $p`) its `fix` is the -swapped clause, `machine_applicable`. This is the deterministic correction -the Cypher ecosystem bolts on after generation (LangChain's -`CypherQueryCorrector`, built because models "can sometimes mess up the -direction of the relationship"; CypherBench counts a reversed relation in -about one failure in six for gpt-4o) moved into the compiler, with two -differences: it is loud, so the agent learns in one turn, and it never -guesses. When the two endpoint types are the same (`Person -knows-> Person`, -`Company -subsidiaryOf-> Company`) the arrow is the only carrier of meaning -and nothing can check it against the schema; the edge's `@description` and -the skill's schema card say in words which end is which. +The traversal keeps edition 2's spelling: a triple, source on the left, +`` for either orientation. This RFC's first draft replaced it with an +arrow (`$p -worksAt-> $c`, `$p -worksAt- $c`) for the reader and for the +model's Cypher prior, and the direction evidence removed both reasons. The +arrowhead carries no information: when the two endpoint types differ the +schema fixes the orientation, and when they are the same +(`Person knows Person`, `Company subsidiaryOf Company`) the position does, +in a triple exactly as in an arrow form. It is a decoration on the one +spelling, and the one-spelling rule, applied from the other side, asks what +a decoration buys. A reader reads `$p worksAt $c` in source-to-target order, +as a sentence; the one in-context measurement GQ has is on the triple +(RFC 0047, Evidence: 45 of 45 tasks, 37 first tries valid, no direction +error), while CypherBench counts a reversed relation in about one gpt-4o +failure in six on the arrow form; and keeping the triple removes the single +largest rewrite from the break, since every stored query has a traversal, +and the arithmetic-minus rule with it. `arrow_traversal` is in the grammar +only as a recognizer, so that an agent writing `$p -worksAt-> $c`, +`$c <-worksAt- $p` or `$a -knows- $b` from a Cypher habit gets a parse error +whose fix prints the triple with its ends in schema order. + +The type checker's endpoint rule (`T5`) checks the written triple against +the declared endpoints, and when the triple is written against the schema +(`$c: Company worksAt $p`) its `fix` is the swapped clause, +`machine_applicable`. This is the deterministic correction the Cypher +ecosystem bolts on after generation (LangChain's `CypherQueryCorrector`, +built because models "can sometimes mess up the direction of the +relationship") moved into the compiler, with two differences: it is loud, so +the agent learns in one turn, and it never guesses. For a same-type edge the +position is the only carrier of meaning and nothing can check it against the +schema; the edge's `@description` and the skill's schema card say in words +which end is which. A catalogue edge carries the `@` prefix for the reason catalogue types do: a user schema may declare an edge named `property` or `from`, and one @@ -522,23 +530,18 @@ so that `match { $t: @Type } retrieve { $t.lex: bm25($t.instruction, $q) }` is an ordinary query over an ordinary table; nothing in the grammar distinguishes it from a query over user data, which is the point. -`-e-` means whichever orientation the endpoint types admit. For a same-type -edge that is both orientations, deduplicated, as `` is today. For an -alternation or wildcard whose members run in different orientations between -the two bound types (`$p: Person -(owns | ownedBy)- $d: Doc`, which the triple -form resolves per member today) it is each member's declared orientation, +`` means whichever orientation the endpoint types admit. For a same-type +edge that is both orientations, deduplicated, as today. For an alternation +or wildcard whose members run in different orientations between the two +bound types (`$p: Person $d: Doc`; the directed triple +resolves these per member today) it is each member's declared orientation, and the bound edge's `@src` and `@dst` keep reporting the stored orientation. `T22` ("undirected traversal needs a same-endpoint-type edge") narrows to the one case that stays ambiguous: an asymmetric edge whose endpoints are not -typed well enough to fix the orientation. The -grammar has no `-` operator in edition 3, so no ambiguity with arithmetic -exists in this release; the function-registry work that adds arithmetic -inherits one rule, stated here so it is not rediscovered: inside `match`, a -`-` directly before an `edge_core` and followed by `->` or `-` is a -traversal, tried before `filter` in the ordered choice as `traversal` is -today; an arithmetic minus takes a value operand (`$x.p`, a literal, a call, a -parenthesised expression), never a bare edge identifier, so the two never -match the same text. +typed well enough to fix the orientation. No traversal token collides with +an operator the function-registry work may add: the ordered choice tries +`traversal` before `filter` inside `match`, as today, so `$a $b` is +never read as a comparison. ### Calls and the reserved set @@ -656,8 +659,8 @@ channel; `having` is a `Filter` node, not a string. ## Compatibility and reversibility -- **This RFC is the break.** The spellings it removes are the triple - traversal, `query` for a write, the infix string operators and the positional +- **This RFC is the break.** The spellings it removes are `query` for a + write, the infix string operators and the positional `rrf` constant; every other edition-2 text parses unchanged under edition 3 (the reserved set does not grow, every new keyword is contextual). - **One release of overlap.** The edition-3 release compiles edition 2 and @@ -685,14 +688,13 @@ channel; `having` is a `Filter` node, not a string. |---|---| | Additive first, break last: ship every new form beside the old one with a deprecation, remove the old ones in a final major | Every construct built twice, every corpus and skill edit made twice, agents reading two spellings for a release per construct; three open RFCs already schedule four majors this way | | Keep rankings in `order` (RFC 0048's "adds no clause and no stage") | One token selects the population and sorts the rows; a score cannot be a filter; `candidates`, `weight`, `exact` and `k` become grammar on the call; the aggregate and `distinct` cases need a special rule. A named arm makes every one of those an expression or an option | -| Keep the triple traversal and let the schema fix direction | Direction invisible at the reading site; `T5` needs a schema lookup to understand; `(p)-[:WORKS_AT]->(c)` is what the model's prior reads | +| An arrow spelling for traversals (`$p -worksAt-> $c`, `-e-`), as in Cypher, GQL and SQL/PGQ | This RFC's first draft, for readability and the model's prior; rejected on 2026-10-30: the arrowhead carries nothing the position and the schema do not, the one measurement on the triple shows no direction errors while CypherBench shows arrows do not prevent them, and it was the break's largest rewrite | | Keep `query` for writes | Agents trained on GraphQL write `mutation { }` and the skill documents the trap; a keyword lets lint classify before the body | | A separate `where { }` clause for filters | Rejected by the shared expression model for reasons that hold: conditions belong beside the pattern; retrieval is different because an arm is not a condition on one row | | `graph@ns::Type` for the federation segment | A second delimiter where the namespace RFC's `::` already nests; an outermost segment is enough and needs no grammar now | | Permanent two-edition compiler | Not needed for one break; the registry records the edition, and a second break, if one comes, builds the N-1 arm then | | One RFC for grammar and semantics (the PR #606 shape) | Closed on 2026-09-28 because it could not be accepted or evidenced piecemeal; this RFC owns the grammar and the window and names the owner of every slot | -| Keep the left arrow `$b <-knows- $a` beside `$a -knows-> $b` | Two spellings of one clause; GQ has no path chains, so the left arrow never continues a line; the one-spelling rule removes it and the parse error's fix prints the swap | -| Correct a wrong arrow silently against the schema (LangChain's `CypherQueryCorrector`) | A silent rewrite teaches the agent nothing and must guess, or return nothing, when both orientations fit; the typed error with a machine-applicable swap is the same determinism made loud | +| Correct a wrong triple silently against the schema (LangChain's `CypherQueryCorrector`) | A silent rewrite teaches the agent nothing and must guess, or return nothing, when both orientations fit; the typed error with a machine-applicable swap is the same determinism made loud | | Role-named relation ends instead of direction (TypeQL `links (employee: $p, employer: $c)`) | Right for n-ary relations, which GQ does not have; for a binary edge the role names document `src` and `dst`, which `@description` does; a role name on an edge declaration stays additive if a later RFC wants it | | Stage pipelines as the closure mechanism (TypeQL 3: `match`, `reduce`, `match`, `insert` in any order) | Closed by construction, but the text order is the plan order, each stage's output is its input crossed with its matches, and reads and writes mix in one transaction; the reserved `view`, `optional`, block values and `having` give the measured closure cases under one planner-owned shape and one commit | | Do nothing | The next five RFCs each break the language once | @@ -707,8 +709,8 @@ Existing owners, per [the testing map](../dev/testing.md): line in every position. - **Type-check tests**: every slot's refusal with its code and fix; arm collision with a property; an arm read in `match`; `query` with a mutation - body; the left arrow's parse error and the swapped-clause fix; `T5` with - the swapped fix for an arrow against the schema; `-e-` over an alternation + body; an arrow traversal's parse error and its triple fix; `T5` with the + swapped fix for a triple against the schema; `` over an alternation of opposed members (the issue 659 shape) resolving per member; `T22` only when the endpoints leave the orientation ambiguous; the `count { } > 0` and `count { } = 0` lint with its fix. @@ -796,3 +798,10 @@ the release; step 5 is the removal every deprecation ends in. catalogue types are node types for every purpose of the grammar, and the `@`-prefixed catalogue edge selector is reserved. The evaluation is committed beside this RFC as its evidence. +- 2026-10-30: withdrew the arrow spelling at the maintainer's direction after + the question "do we still need the arrow?": the triple traversal stays + unchanged from edition 2, an arrow form is a recognised parse error with + the triple as its fix, and the Traversal section records why the arrowhead + failed the one-spelling rule. The rewriter table and the compatibility + section lose the traversal rows; the break no longer touches every stored + query. diff --git a/docs/rfcs/assets/2026-10-28-gq-3-grammar-evaluation.md b/docs/rfcs/assets/2026-10-28-gq-3-grammar-evaluation.md index e15988081..71a267dcc 100644 --- a/docs/rfcs/assets/2026-10-28-gq-3-grammar-evaluation.md +++ b/docs/rfcs/assets/2026-10-28-gq-3-grammar-evaluation.md @@ -2832,7 +2832,7 @@ canonical result. | Today | GQ 3 | Driver | |---|---|---| | `query` for reads and writes alike | `query` for reads, `mutation` for writes, `view` for derived relations, `batch` for several under one snapshot | agents trained on GraphQL write `mutation { }` (the skill's documented trap); lint classifies by keyword, not body | -| `$p worksAt $c`, direction from the schema | `$p -worksAt-> $c`, `$c <-worksAt- $p`, `$a -knows- $b` undirected, `-$e:worksAt->` bound, `-(knows|likes){1,3}->` | direction visible to the reviewer and the model; `T5` becomes a reading error, not a schema lookup | +| `$p worksAt $c`, direction from the schema | `$p -worksAt-> $c`, `$c <-worksAt- $p`, `$a -knows- $b` undirected, `-$e:worksAt->` bound, `-(knows|likes){1,3}->` | direction visible to the reviewer and the model; `T5` becomes a reading error, not a schema lookup. **Withdrawn 2026-10-30 (§13.3): the triple stays; the arrowhead carries nothing position and schema do not** | | Ranking functions as the leading `order` key | a `retrieve { }` clause that binds typed score columns; `order` sorts any expression | 2.10; Vespa; Bruch; thresholds, weights, more arms and rerank become expressions | | `rrf(a, b)` only | `rrf(...)`, `norm(...)`, weighted sums, `rerank(model, text) k` as a stage, `where $a.vec.distance < 0.4` | Bruch (+2–3 nDCG), BEIR (+11% rerank), LanceDB `distance_range` | | Blocks as predicates only, constant right side | blocks as values (`count { } as n`), any expression on the right | #1050, Cypher `COUNT {}`, Soufflé | @@ -3072,7 +3072,7 @@ Most of §9.3 is additive and already in §4.3 and §7.7: blocks as values, bindings, annotations, catalogue types, paging, `returning`, pattern-scoped writes, notifications, `why_not`, editions. Only four items are incompatible with stored text and are the reason to ask the question at all: the arrow -spelling for traversals, `mutation` as a keyword, `retrieve` replacing ranking +spelling for traversals (withdrawn on 2026-10-30, §13.3), `mutation` as a keyword, `retrieve` replacing ranking functions in `order`, and `@valid`/`invalidate` as schema and verb. Under an edition line all four ship as GQ 3 beside a GQ 2 the compiler keeps compiling for one release, with `queries validate` printing the rewrite; the rewrite from @@ -3523,7 +3523,7 @@ bounded union §7.6 wants. **A** waits for views (contracts bind to them). | **R Retrieve** (L) | `retrieve { }` clause binding typed score columns; arms with `k`; `norm()`, `rrf()` with n arms and weights, `rerank(model, text, k)` as a provider-backed stage; score columns usable in `having`/`order`/`return`; `@approximate(recall)` declaration; achieved recall in the envelope; 0048's laws, `exact: true`, `Vector(n, distance=…)` and recipes; rankings in `order` keep compiling for one release and lint prints the rewrite | E, F, 0047 (shipped) | no until X | the fusion and vector oracles 0048 names; GQT for windows, weights, thresholds, rerank, n arms; the discovery task in M0 answered by `rrf` over interface arms | hybrid search with weights, thresholds and a reranker; the end of two-query hybrids | | **W1 Writes** (M) | pattern-scoped `update $a … where { }` and `delete $e where { }` with one target type; `returning { }`; `@valid(from, to)` schema role and `invalidate`; `set { x: null }` in assignment position; bounded pattern-scoped `insert` | read planner under the write door (the shared expression RFC's "later RFC") | no | staging protocol unchanged (D2); GQT per form; affected receipts | invalidation as a verb; one round trip per write instead of read-then-write | | **W2 Computed writes** (S) | `set { visits: visits + 1 }` under `@if_commit` or a key equality; the read-set rule | F (arithmetic), W1 | no | contention case under DST; retry idempotence | the increment without a round trip | -| **X Edition 3** (M) | `mutation` keyword; arrow traversal spelling (`-e->`, `<-e-`, `-e-`); `retrieve` as the only ranking position; removal of `search`/`fuzzy`/`match_text`, infix `contains`/`starts_with`, positional `rrf` `k`; the GQ 2 compiler kept for one more release; `queries validate` prints every rewrite as a machine-applicable suggestion | every deprecation above having lived one release | **yes, under the edition** | every stored query in the fleet rewritten by the suggestion tool and re-validated; the 45-task instrument re-run on the new spelling | one spelling per operation; direction visible; the novelty tax paid once | +| **X Edition 3** (M) | `mutation` keyword; arrow traversal spelling (`-e->`, `<-e-`, `-e-`; withdrawn 2026-10-30, §13.3); `retrieve` as the only ranking position; removal of `search`/`fuzzy`/`match_text`, infix `contains`/`starts_with`, positional `rrf` `k`; the GQ 2 compiler kept for one more release; `queries validate` prints every rewrite as a machine-applicable suggestion | every deprecation above having lived one release | **yes, under the edition** | every stored query in the fleet rewritten by the suggestion tool and re-validated; the 45-task instrument re-run on the new spelling | one spelling per operation; direction visible; the novelty tax paid once | | **I + Poly** (L, own clock) | integer identity storage upgrade; polymorphic endpoints; unions `A \| B`; type tests; `@type_id`; interface-wide mutations; `rrf` over member arms as the interim cross-member search | storage upgrade RFC pair; C (interface bindings already shipped) | no (schema feature; old binaries refuse) | the RFC's prototype findings as regressions; upgrade journey tests | edge families collapse; discovery across types | | **A Permissions** (L) | actor views first scope (type and column visibility, per-row placement, reads); `contract` declarations in GQ bound to views and stored queries; Cedar grants on named capabilities; the fragment = kernel minus retrieval | C (views), S, both RFCs amended per §10.5 | no | #951's differential oracle for the first scope; #1197's adversarial matrix | query-only agents; shared graphs with audiences | | **Fed Federation** (L, later) | graph qualifier in type references; per-graph commit pins in the envelope; views over a foreign graph; change-feed-as-load for shares | E/S reservation; C (views); Poly (one-corpus search) | no | nothing until designed | cross-graph reads; shares kept current | @@ -3617,7 +3617,7 @@ not on it is additive forever after, by construction. | Declaration kinds | `query`, `mutation`, `view`, `contract`, `batch`, plus the schema kinds `node`, `edge`, `interface` in the same grammar; one file type, `.pg` accepted as the same grammar | `view`, `contract`, `batch` parse and are refused with a stable code ("reserved; not yet served") until their stage | | Clause set and order | `match` → `retrieve`? → `return` → `having`? → `order`? → `limit`? → `returning`? (mutations: statements → `returning`?) | `retrieve` carries exactly today's power (`nearest`, `bm25`, two-arm `rrf`); `having` works immediately (one `Filter` over `Aggregate`); `returning` parses, refused until W1 | | Block kinds in `match` | `not`, `exists`, `count`/`sum`/… blocks, `optional` | `optional` parses, refused until C | -| Traversal spelling | `-e->`, `<-e-`, `-e-`, `-$x:e->`, `-(a\|b){m,n}->`, `-*->` | today's semantics | +| Traversal spelling | the triple stays (`$a e $b`, ``); the first draft's arrow forms were withdrawn on 2026-10-30 (§13.3) | today's semantics | | Call syntax | `name(args, opt: v)`, generic, resolved against a registry; `contains()`, `starts_with()`, `terms()` reserved as names; no function in the grammar | today's functions become registry rows; `terms()` refused until L | | Reserved words | the logic set only: `and`, `or`, `not`, `is`, `null`, `in`; every clause and declaration keyword contextual (word-bounded, position-based, as today) | closed; the amendment's rule becomes a grammar test | | Null | no literal in reads; `null` legal in assignment position only | today's rules plus the assignment case | @@ -3815,7 +3815,7 @@ MCP server, and a `given` stage sold as the anti-injection parameterisation. |---|---|---|---| | Closure (2.2) | Linear: every stage consumes the previous stage's rows, any order, kinds repeat; functions are named read pipelines | Non-closed kernel plus four reserved forms: `view` (a named pattern with typed columns, inlined), `optional`, block aggregates as values, `having` | Same demand, two mechanisms; see 12.3 | | Polymorphism (2.1, 10.4) | Type variables in every type position; `isa` includes subtypes, `isa!` does not; parametric queries "valid over any schema" | Interface bindings, `@type`/`@type_id`, `is`; catalogue types `@Type` for the schema as data | TypeQL is more uniform; GQ's split (bind by interface, read the schema by catalogue) is the same power with two spellings | -| Direction (2.6) | None: relation ends are roles | Written arrow, checked against the schema | §13 | +| Direction (2.6) | None: relation ends are roles | Triple, source on the left, checked against the schema | §13 | | Optional (2.5) | `try { }`; unbound optionals fail later stages; `require` filters them | `optional { }` slot | Same form, same rule; TypeDB's "may only occur in a single try-block" is the scoping rule the closure RFC needs | | Aggregation (2.8) | `reduce … groupby`, then any stage | `return` with aggregates, then `having` | GQ's is one fixed shape; TypeQL's allows re-matching on the reduced value, which `view` gives GQ | | Result shape (2.15) | Rows, or nested JSON from terminal `fetch` with sub-pipelines | Rows with node objects; a partial object projection and a nested block in `return` are additive (parse errors today) | No slot needed; the nested-document demand (#649) has a precedent | @@ -3998,39 +3998,49 @@ post-processor can fix that case, because both orientations fit the schema. ### 13.3 What follows for GQ 3 -The RFC made direction visible (`$p -worksAt-> $c`) for the reader and for the -model's prior, and made `T5` check the written arrow against the schema. The -evidence sharpens that into four rules. - -1. **Write the arrow once, source on the left: `->` only.** The left arrow - `$c <-worksAt- $p` is a second spelling of the same clause. GQ has no path - chains (a traversal is `variable edge variable`), so `<-` is never needed - to continue a line; it is pure redundancy, the Go test's first casualty - (§14). The parser knows it: `<-e-` is a parse error whose fix prints the - clause with the ends swapped, as the triple form's fix prints the arrow. -2. **A wrong arrow is a typed error with a machine-applicable swap**, never - an empty result and never a silent rewrite. `$c: Company -worksAt-> $p` is - `T5` with `fix` = the swapped clause. This is LangChain's corrector moved - into the compiler, where it belongs, with the two differences the evidence - asks for: it is loud (the agent learns in one turn, which Olausson's - result in §8.3 says is worth more than the edit) and it never guesses when - both orientations fit. +The RFC's first draft made direction visible with an arrow +(`$p -worksAt-> $c`) for the reader and for the model's prior, and made `T5` +check the written arrow against the schema. The evidence above and the +one-spelling test of §14 reversed that on 2026-10-30. Four rules remain. + +1. **The triple stays, source on the left, and gains no arrowhead.** The + arrowhead carries no information: when the endpoint types differ the + schema fixes the orientation (13.2's first conclusion), and when they are + the same the position does, in `$a subsidiaryOf $b` exactly as in + `$a -subsidiaryOf-> $b`. It is a decoration on the one spelling, and its + two justifications did not survive: `$p worksAt $c` reads in + source-to-target order as a sentence does, and the only in-context + measurement GQ has is on the triple (RFC 0047: 45 of 45, 37 first tries + valid, no direction error) while CypherBench shows the arrow form does not + prevent reversals. Dropping it also removes the break's largest rewrite, + since every stored query has a traversal, and the arithmetic-minus rule. + The arrow forms stay in the grammar as a recognizer only: `$p -worksAt-> + $c`, `$c <-worksAt- $p` and `$a -knows- $b` are parse errors whose fix + prints the triple with the ends in schema order. +2. **A triple written against the schema is a typed error with a + machine-applicable swap**, never an empty result and never a silent + rewrite. `$c: Company worksAt $p` is `T5` with `fix` = `$p worksAt $c`. + This is LangChain's corrector moved into the compiler, where it belongs, + with the two differences the evidence asks for: it is loud (the agent + learns in one turn, which Olausson's result in §8.3 says is worth more + than the edit) and it never guesses when both orientations fit. 3. **Same-type edges are where direction is meaning, and position carries - it.** `$a -subsidiaryOf-> $b` means `$a` is the subsidiary. No rule can - check that against the schema; what GQ can do is make the schema say which - end is which in words, so the error is visible: `@description` on the edge - today, and the skill's schema card prints `Company -subsidiaryOf-> - Company (subsidiary -> parent)`. -4. **`-e-` means "whichever orientation the endpoint types admit".** For a - same-type edge that is both orientations, deduplicated, as `` is - today. For an alternation or wildcard over edges of differing orientation - (`$p: Person -(owns | ownedBy)- $d: Doc`, the issue 659 case, which today's - triple resolves per member) it is each member's declared orientation. The - only refusal left is the genuinely ambiguous one: an asymmetric edge whose - endpoints are not typed well enough to fix the orientation. Today's `T22` - ("undirected traversal needs a same-endpoint-type edge") narrows to that - case. The bound edge's `@src` and `@dst` keep reporting the stored - orientation, which is why they are not redundant with the pattern's ends. + it.** `$a subsidiaryOf $b` means `$a` is the subsidiary. No rule can check + that against the schema; what GQ can do is make the schema say which end + is which in words, so the error is visible: `@description` on the edge + today, and the skill's schema card prints `Company subsidiaryOf Company + (subsidiary -> parent)`. +4. **`` means "whichever orientation the endpoint types admit".** For a + same-type edge that is both orientations, deduplicated, as today. For an + alternation or wildcard over edges of differing orientation + (`$p: Person $d: Doc`, the issue 659 case, which the + directed triple resolves per member today) it is each member's declared + orientation. The only refusal left is the genuinely ambiguous one: an + asymmetric edge whose endpoints are not typed well enough to fix the + orientation. Today's `T22` ("undirected traversal needs a + same-endpoint-type edge") narrows to that case. The bound edge's `@src` + and `@dst` keep reporting the stored orientation, which is why they are + not redundant with the pattern's ends. What GQ 3 does not take from TypeQL is roles. For a binary edge, `employee` and `employer` are documentation of `src` and `dst`, and GQ puts @@ -4080,8 +4090,8 @@ Every pair of forms in GQ 3 that could mean the same thing, and the ruling. | Pair | Same meaning? | Ruling | |---|---|---| -| `$a -knows-> $b` and `$b <-knows- $a` | Yes, exactly | Grammar: `->` only; `<-` is a parse error with the swapped clause as its fix (§13.3) | -| `$a knows $b` (triple) and the arrow | Yes | Grammar: arrow only; already in the RFC | +| `$a knows $b` and a reversed arrow `$b <-knows- $a` | Yes, exactly | Grammar: the triple only, source on the left; every arrow form is a parse error with the swapped triple as its fix (§13.3) | +| `$a knows $b` (triple) and `$a -knows-> $b` | Yes, and the arrowhead adds nothing | Grammar: triple only, decided 2026-10-30 (§13.3); the first draft had chosen the arrow | | `x contains y` and `contains(x, y)` | Yes | Grammar: the call only; already in the RFC | | `rrf(a, b, 60)` and `rrf(a, b, k: 60)` | Yes | Grammar: named option only; already in the RFC | | `exists { P }` and `count { P } > 0`; `not { P }` and `count { P } = 0` | Yes | The grammar cannot forbid the count form without forbidding `count { } > 2`. Formatter: `omnigraph queries fmt` (the rewriter's second job) canonicalises the two thresholds to `exists`/`not`, and lint names it. The grammar keeps `exists` because GQL has only `EXISTS` and the planner's semi-join is the honest plan | @@ -4109,7 +4119,7 @@ A form enters the grammar only if no existing form has its meaning; a form that falls out of a general construct and has a canonical instance is rewritten to that instance by the formatter and named by lint; the diagnostics know every respelling an agent is likely to write (the triple, -the left arrow, the infix operators, GraphQL's `mutation` for a read) and +the arrow forms, the infix operators, GraphQL's `mutation` for a read) and print the one form as a machine-applicable fix. The freeze list is checked against the rule once, in this RFC, and every later RFC's "Alternatives" section answers it for the form it adds. From 95328a2192f7ea3207180c2ddb60185cfd71f1f0 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Sun, 1 Nov 2026 21:35:15 -0800 Subject: [PATCH 10/13] docs(rfc): revise the GQ 3 grammar after review and spike: today's score columns, having semantics, both triple orders, registry rows kept, grammar fixes --- docs/rfcs/2026-10-28-gq-3-grammar.md | 746 ++++++++++++------ .../2026-10-28-gq-3-grammar-evaluation.md | 155 ++-- 2 files changed, 607 insertions(+), 294 deletions(-) diff --git a/docs/rfcs/2026-10-28-gq-3-grammar.md b/docs/rfcs/2026-10-28-gq-3-grammar.md index 4100f9c03..b9632d78d 100644 --- a/docs/rfcs/2026-10-28-gq-3-grammar.md +++ b/docs/rfcs/2026-10-28-gq-3-grammar.md @@ -7,19 +7,23 @@ implementation: not-started authors: - Ragnor Comerford (@ragnorc) created: 2026-10-28 -updated: 2026-10-29 +updated: 2026-11-01 discussion: "https://github.com/ModernRelay/omnigraph/pull/1202" supersedes: [] superseded_by: [] blocked_on: - - "The in-context competence instrument run on the edition-3 spelling through a parser-only build: first-try validity at or above the current grammar's 37 of 45, and 45 of 45 after one repair" - - "The rewriter turning every `.gqt` case under `crates/omnigraph-gqt/cases/` into edition 3 with no hand edit, the rewritten corpus green, and IR equality on every case" + - "The in-context competence instrument run on the edition-3 spelling through a parser-only build: first-try validity at or above the current grammar's 37 of 45 (RFC 0047 reports 8 of 45 first attempts refused, every one for a missing `()`), and 45 of 45 after one repair" + - "The rewriter turning every `.gqt` case under `crates/omnigraph-gqt/cases/` into edition 3 with no hand edit to query text, the rewritten corpus green, and typed-IR equality on every declaration; an `expect error` code or a result column name that moves is rewritten by the tool from its fixed map, never by hand" + - "PR #805, PR #792 and PR #793 amended to remove the majors and refusals this RFC's decision log names, so that edition 3 is the one scheduled break" --- # RFC: GQ 3 grammar: the one break and the slots it freezes > A term set in ***bold italics*** is defined at that spot; it is plain text -> afterwards. Code references are at `main` `ea895066`. +> afterwards. Code references are at `main` `f728f01e`; `query.pest`, +> `schema.pest`, `GQ_LANGUAGE_VERSION` and `EXPLAIN_VERSION` are unchanged +> since `ea895066`, the commit the evaluation read. The bump base is +> `GQ_LANGUAGE_VERSION` `(2, 3)` and `EXPLAIN_VERSION` `7`. ## Summary @@ -28,26 +32,33 @@ stops making incompatible changes. The release that ships edition 3 lands every syntactic decision that would otherwise force a later break, including the ones whose semantics arrive in later RFCs: those land as ***slots***, forms the parser accepts and the type checker refuses with a stable code naming the -RFC that will serve them. After this release a new capability is a registry -row, a plan node behind existing syntax, or the filling of a slot, and none of -those breaks a stored query. +RFC that will serve them. After this release a new capability never changes +the meaning of a well-typed edition-3 query: it is a registry row, a plan node +behind existing syntax, the filling of a slot, or a new production that starts +at a token edition 3 refuses. The slots buy a committed spelling and a named +diagnostic for the forms the next RFCs already depend on; the compatibility +itself comes from the grammar's shape (ordered choice, contextual keywords, +`$`-variables, the `@` namespace), which is why a form with no slot is still +additive ([What a slot buys](#what-a-slot-buys)). The break is defined by a ***freeze list***. It renames the write declaration -`mutation`; keeps the triple traversal (`$p worksAt $c`, source on the left, -`` for either orientation), rejecting an arrow spelling as decoration; gives -retrieval its own clause (`retrieve { }`) that binds typed score columns, -leaving `order` to sort values; makes every function a generic call resolved -against a registry, so no function remains in the grammar; closes the -reserved-word set at the six logic words; lets `limit` take a parameter; admits -a generic `@name(args)` annotation on any declaration; reserves the -declaration kinds `view` and `batch`, the block `optional { }`, the -clauses `having { }` and `returning { }`, the meta-fields `@type_id`, -`@distance` and `@score`, the catalogue type names the schema catalogue will -use, and the namespaced type reference `a::b::C`; reserves the binding form `$x: Name($arg)` and a block aggregate in -operand position, so that derived relations, optional patterns and block -aggregates as values arrive without a grammar change; and adds an optional -`gq 3;` edition line. Schema declarations and query declarations share one -grammar; a `.pg` file is a GQ file holding only schema declarations. +`mutation`; keeps the triple traversal exactly as edition 2 has it +(`$p worksAt $c` in either written order, `` for either orientation), +rejecting an arrow spelling as decoration; gives retrieval its own clause +(`retrieve { }`) that binds typed score columns, leaving `order` to sort +values; adds `having { }` over aggregates, group keys and score columns; makes +every function a generic call resolved against a registry, so no function +remains in the grammar; closes the reserved-word set at the six logic words; +lets `limit` take a parameter; admits a generic `@name(args)` annotation on +any declaration; reserves the declaration kinds `view` and `batch`, the block +`optional { }`, the clause `returning { }`, the meta-field `@type_id`, the +catalogue type names the schema catalogue will use with their `@`-prefixed +edges, and the namespaced type reference `a::b::C`; reserves the binding form +`$x: Name($arg)` and a block aggregate in operand position, so that derived +relations, optional patterns and block aggregates as values arrive without a +grammar change; and adds an optional `gq 3;` edition line. Schema declarations +and query declarations share one grammar; a `.pg` file is a GQ file holding +only schema declarations. A ***rewriter***, `omnigraph queries migrate`, turns every edition-2 source into edition 3 by byte-range edits that preserve comments and layout, and @@ -56,7 +67,10 @@ logic-test corpus, the user guide, the bundled skill and every stored query on the graphs the maintainers operate are rewritten by that tool before the release. The release that ships edition 3 still compiles edition 2, rewriting a stored source at boot with a warning instead of quarantining its graph; the -release after removes the edition-2 grammar. +release after removes the edition-2 grammar. A spike built the edition-3 front +end and the rewriter against this text and ran the proof over the repository's +2,609 sources ([Evidence](#evidence-and-tests)); its amendments are in this +revision. What does not change: every semantic contract the language has today (static closed typing, three-valued logic with no null literal in reads, pattern @@ -93,9 +107,11 @@ a queue of requests. Its aim is a foundation with six properties, each of which a later section of this document or a named follow-up RFC is responsible for. -- **Evolvable.** Every later capability lands without a break because its - position already exists: a reserved clause, block, declaration kind, call - form or envelope slot ([The freeze list](#the-freeze-list)). +- **Evolvable.** Every later capability lands without a break: the forms the + next RFCs depend on have a reserved clause, block, declaration kind, call + form or envelope slot ([The freeze list](#the-freeze-list)), and every + other form starts at a token edition 3 refuses, so adding it changes no + accepted query ([What a slot buys](#what-a-slot-buys)). - **Maintainable.** One grammar for schema and queries, one registry for functions, one annotation form, one diagnostic contract, so a change has one place to land and one surface to document ([Design](#design)). @@ -116,15 +132,16 @@ responsible for. query, is the agent the evidence below measures as reliable. - **Self-describing at any size.** The schema is data in the same language: the catalogue types `@Type`, `@Property`, `@Edge`, `@Interface`, `@Index`, - `@Function` and `@View`, joined by catalogue edges, are bound, traversed, - searched and counted with the same `match`, `retrieve` and `return` as user - data. A schema of a few dozen types fits in a context window; one of a few + `@Function` and `@View`, joined by catalogue edges, are bound, traversed and + counted with the same `match` and `return` as user data, and joined to it. + A schema of a few dozen types fits in a context window; one of a few thousand (every source system's types, every agent's views) does not, and the agent then finds the type it needs the way it finds a row: by name, by - the text of its `@instruction`, by its neighbourhood. The evaluation's - literature review found the full schema wins only while it fits - (R1 in §9.1); past that, a schema the language cannot query is a schema - the agent cannot use. + its neighbourhood, and, once the catalogue RFC's amendment names a + mechanism that does not open a Lance dataset, by the text of its + `@instruction` ([Traversal](#traversal)). The evaluation's literature + review found the full schema wins only while it fits (R1 in §9.1); past + that, a schema the language cannot query is a schema the agent cannot use. The rest of this section records why the project has not had these properties, and why one early break buys them more cheaply than the breaks @@ -140,10 +157,11 @@ infix `contains` and `starts_with` to become calls through a deprecated rule and two more majors (3.0, then 4.0); the analyzed lexical search RFC (PR #792) schedules the removal of `search`, `match_text` and `fuzzy` at "the next GQ language major"; RFC 0048 (PR #793) deprecates the positional `k` of `rrf()` -for one release; the polymorphic types RFC (PR #884) allocates a minor -version and an explain version of its own. Each is a break with its own -deprecation shim, lint warning, corpus edit, skill edit and fleet sweep, and -each is a release in which agents meet two spellings of one operation. +for one release. Each is a break with its own deprecation shim, lint warning, +corpus edit, skill edit and fleet sweep, and each is a release in which +agents meet two spellings of one operation. (The polymorphic types RFC, PR +#884, allocates a minor and an explain version against a stale base; it is +additive and is not one of these breaks.) Three facts make one early break cheaper than many late ones. @@ -163,9 +181,10 @@ Three facts make one early break cheaper than many late ones. not have. Added one at a time, each is additive only if its position was already reserved; added after a freeze, each is a slot filled. 3. **The agent is the author, and the author's cost is round trips, not - spelling.** Measured on this grammar, a general agent solved 45 of 45 tasks - with 37 of 45 first queries valid, and every first-try failure was one - missing `()` (RFC 0047, Evidence). Published comparisons of query languages + spelling.** Measured on this grammar, a general agent answered 45 of 45 + tasks correctly, with 8 of 45 first attempts refused, every one for a + missing `()` (RFC 0047, Evidence, cited as written; the instrument reports + no direction result). Published comparisons of query languages for LLM generation find a 10 to 45 point zero-shot gap between familiar and unfamiliar spellings that five retrieved worked examples erase (Sivasubramaniam et al., SM3-Text-to-Query, NeurIPS 2024), and that better @@ -251,21 +270,21 @@ names the construct and whose `fix` names the owning RFC; a form marked | Surface | Edition 3 | In this release | Owner of the semantics | |---|---|---|---| -| Declaration kinds | `query` (reads only), `mutation` (writes), `view`, `batch`; the schema kinds `node`, `edge`, `interface` in the same grammar | `query`, `mutation` and the schema kinds work; `view` and `batch` are slots | derived relations RFC (`view`), the agent-first batch (`batch`) | -| Read clause order | `match` → `retrieve`? → `return` → `having`? → `order`? → `limit`? | `match`, `return`, `order`, `limit` work; `retrieve` works with today's rankings (below); `having` works (a Boolean expression over aggregates and group keys, one filter above the aggregate) | this RFC for `having`; RFC 0048 for everything `retrieve` gains later | +| Declaration kinds | `query` (reads only), `mutation` (writes), `view`, `batch`; the schema kinds `node`, `edge`, `interface` in the same grammar | `query`, `mutation` and the schema kinds work; `view` and `batch` are slots; a file mixing schema and query declarations is refused as a slot (the loader and the registry each take one kind) | derived relations RFC (`view`), the agent-first batch (`batch`), the one-file loader for the mixed file | +| Read clause order | `match` → `retrieve`? → `return` → `having`? → `order`? → `limit`? | `match`, `return`, `order`, `limit` work; `retrieve` works with today's rankings ([below](#the-retrieve-clause-in-this-release)); `having` works over aggregates, group keys and score columns under its own type rule | this RFC for `having` and the arms; RFC 0048 for everything `retrieve` gains later | | Mutation clause order | statements → `returning`? | statements work; `returning` is a slot | writes RFC | | Blocks in `match` | `not { }`, `exists { }`, `count|sum|min|max|avg(…) { } op expr`, `optional { }` | the four work; `optional` is a slot | derived relations RFC | | Blocks as values | a block aggregate in operand position, `return { count { $n knows $m } as knows }`, `exists { … } as employed` | parses; a slot | derived relations RFC | -| Bindings | `$x: Type { … }` as today; `$x: Name($arg, …) { … }` binds a declared `view` with arguments; `$x: Interface` binds an abstract type | the first works; a binding with arguments is a slot; an interface name is refused as today (`T1`) until its RFC | derived relations RFC (views), polymorphic types RFC (interfaces) | -| Traversal | unchanged from edition 2: `$a knows $b` (source on the left, the one directed form), `$a $b` (whichever orientation the endpoint types admit), `$a $e:knows $b`, `$a (knows\|likes){1,3} $b`, `$a * $b`, `$a "in" $b` | works; an arrow form (`$a -knows-> $b`, `$b <-knows- $a`, `$a -knows- $b`) is a parse error whose fix prints the triple with the ends in schema order; a triple written against the schema is `T5` with the swapped clause as its fix; `T22` narrows to an orientation the endpoint types leave ambiguous | this RFC | -| Calls | `name(positional…, option: value…)` for every function; the registry holds `count`, `sum`, `avg`, `min`, `max`, `now`, `date`, `datetime`, `nearest`, `bm25`, `rrf`, `search`, `fuzzy`, `match_text`, `contains`, `starts_with`; `count(distinct x)` is the one special form | works; the infix `x contains y` and `x starts_with y` are parse errors whose fix prints the call; `rrf(a, b, 60)` is a type error whose fix prints `k: 60` | the operator-vocabulary amendment for the call rule; the function-registry work for `@Function` and new rows | +| Bindings | `$x: Type { … }` as today; `$x: Name($arg, …) { … }` binds a declared `view` with arguments, the `(` touching the name; `$x: Interface` binds an abstract type | the first works; a binding with arguments is a slot; an interface name is refused as today (`T1`) until its RFC | derived relations RFC (views), polymorphic types RFC (interfaces) | +| Traversal | unchanged from edition 2: `$a knows $b` (the directed form, either end first; the schema fixes the orientation), `$a $b` and `$a $b` (whichever orientation the endpoint types admit), `$a $e:knows $b`, `$a (knows\|likes){1,3} $b`, `$a * $b`, `$a "in" $b` | works, with today's `T5` and `T22`; `queries fmt` writes a directed triple in schema order; an arrow form (`$a -knows-> $b`, `$b <-knows- $a`, `$a -knows- $b`) is a parse error whose fix prints the triple in the written order; `` over an alternation of opposed members resolves per member, which `T22` refuses today (the one semantic change, additive) | this RFC | +| Calls | `name(positional…, option: value…)` for every function; the registry holds `count`, `sum`, `avg`, `min`, `max`, `now`, `date`, `datetime`, `nearest`, `bm25`, `rrf`, `search`, `fuzzy`, `match_text`, `contains`, `starts_with`; `agg(distinct x)` is the one special form | works; the infix `x contains y` and `x starts_with y` are parse errors whose fix prints the call; `rrf(a, b, 60)` and `fuzzy(a, b, 2)` are type errors whose fix prints `k: 60` and `max_edits: 2`; a registry row is never removed | the operator-vocabulary amendment for the call rule; the function-registry work for `@Function` and new rows | | Reserved words | `and`, `or`, `not`, `is`, `null`, `in`; every other keyword is contextual (word-bounded, legal as an identifier in operand and property position) | works; a grammar test pins the set | this RFC | -| Null | no literal in a read; `null` legal as the value of an `insert` or `set` assignment | works | this RFC | +| Null | no literal in a read, a binding's property match included; `null` legal as the value of an `insert` or `set` assignment | works | this RFC | | Parameters | `$x: Type? = default`; `limit $k` with a non-null `I64` parameter; the parameter kinds `Branch` and `Commit` | defaults and `limit $k` work; `Branch` and `Commit` parse and are slots | this RFC; the branch-statements RFC for binding them | -| Annotations | `@name(args)` on any declaration, retained as metadata; `@description`, `@instruction`, `@example`, `@if_commit($p)` are typed; `@budget`, `@approximate`, `@satisfice`, `@brief`, `@phase` are reserved | the typed four work (`@if_commit` binds the commit precondition the request carries today); the reserved five are slots | RFC 0048 and the agent-first batch | -| Meta-fields | `@id`, `@src`, `@dst`, `@type`, `@type_id`, `@distance`, `@score` | the first four work; `@type_id` is a slot; `@distance` and `@score` are the typed fields of a `retrieve` arm's column | polymorphic types RFC for `@type_id` | -| Catalogue names | `@Operator`, `@Pass`, `@Setting`, `@Engine`, `@LogicalKind` exist; `@Type`, `@Property`, `@Edge`, `@Interface`, `@Index`, `@Function`, `@View` reserved; catalogue edges between them spelled with the same prefix, `$t: @Type @property $p: @Property`, `$e: @Edge @from $t`, `$t @implements $i: @Interface`; a catalogue type is a node type for every purpose of the grammar (binding, traversal, a `retrieve` arm over its text fields, aggregation) | the five exist; the seven and the `@`-prefixed edge selector are slots | the engine catalogue RFC's schema extension, which declares the catalogue's own edges and which of its text fields are searchable | -| Type references | `a::b::C`: zero or more lowercase segments then a type name; the grammar admits it everywhere a type name stands | one or more segments is a slot | schema namespaces RFC (`ns::Type`); a federation RFC for an outermost graph segment | +| Annotations | `@name(args)` on any declaration, retained as metadata; arguments positional or `name = value`, the `.pg` spelling; `@description`, `@instruction` typed; `@example`, `@if_commit($p)`, `@budget`, `@approximate`, `@satisfice`, `@brief`, `@phase` reserved | the typed two work; the reserved seven are slots | RFC 0048 and the agent-first batch (`@example`, the five), the writes RFC (`@if_commit`, which must become the one source of the request's precondition) | +| Meta-fields | `@id`, `@src`, `@dst`, `@type`, `@type_id` | the first four work; `@type_id` is a slot | polymorphic types RFC for `@type_id` | +| Catalogue names | `@Operator`, `@Pass`, `@Setting`, `@Engine`, `@LogicalKind` exist; `@Type`, `@Property`, `@Edge`, `@Interface`, `@Index`, `@Function`, `@View` reserved; catalogue edges between them spelled with the same prefix, `$t @property $p`, `$e @from $t`, `$t @implements $i`, the ends bound as `$t: @Type` beside them; a catalogue type is a node type for binding, traversal, filters and aggregation | the five exist; the seven and the `@`-prefixed edge selector are slots; a `retrieve` arm over a catalogue type is refused with them | the engine catalogue RFC's schema extension, which declares the catalogue's edges and, if it lifts the rank-search refusal, names a mechanism that opens no Lance dataset | +| Type references | `a::b::C`: zero or more lowercase segments then a type name; a catalogue name takes no segment; the grammar admits the form everywhere a type name stands | one or more segments is a slot | schema namespaces RFC (`ns::Type`); a federation RFC for an outermost graph segment | | Schema roles | `Vector(n, distance="l2"\|"cosine"\|"dot")`, `@analyzed(…)`, `@valid(from, to)`, `@embed(source, model=…, …)` | `distance` works (RFC 0048's rule: omitted means `l2`); `@analyzed` and `@valid` are accepted as SchemaIR features and refused at use | RFC 0048 (distance, recipes), the lexical RFC (`@analyzed`), the writes RFC (`@valid`) | | Envelope | `graph_commits` (a map keyed by graph id, one entry today) beside `graph_commit_id`; `notifications`, `why_not`, `complete`, `incomplete_reason`, `continuation`, `estimate` | present where the owning stage has shipped, absent otherwise; all additive | the envelope stage RFCs | @@ -273,31 +292,87 @@ Two things are not on the list because they are additive by nature: a new registry row (a function, an annotation name, a catalogue type) and a new plan node behind existing syntax. +### What a slot buys + +The spike wrote sixty later capabilities, from this RFC, its evaluation, PR +#792, #793, #805, #884, #1015 and the Lance primitives the engine does not +yet reach, as the queries they need, and parsed each under the edition-3 +grammar. Sixteen parse into a slot and are refused with its owner; fourteen +are registry rows or call options; five already work; twenty-four need a new +production (pattern-scoped `update` and `delete`, `invalidate`, unions and +`is T`, qualified edge names, `use` and `namespace` blocks, arithmetic, +partial objects, `limit n per`, `at $commit`, `nulls last`, `offset`, +multivectors, new base types); one, `$p: Person at date(…)`, already parses +as a binding and two bare filters and is refused by the type checker. None +changes the meaning of a well-typed edition-3 query, because each new +production starts at a token edition 3 refuses (`Q001`). The compatibility +therefore comes from the grammar's shape, and a slot buys two things only: a +spelling the owning RFC cannot change, and a diagnostic that names that RFC +instead of reporting an unexpected token. Two later productions have a known +collision to design around, recorded here so their RFCs do: arithmetic `$x * +$y` in `match` meets the wildcard traversal `$a * $b` and needs a lookahead; +`at` on a binding meets the bare-filter clause. + ### The `retrieve` clause in this release A ***retrieve arm*** binds a score column to a binding: `$p.lex: bm25($p.text, $q)`. The column's name lives in the binding's namespace beside its properties; a name that collides with a declared property is a type error. In -this release the arms are exactly today's rankings and keep today's rules: +this release the arms are exactly today's rankings and keep today's columns: -| Arm | Column type | Rule carried over | +| Arm | Column | Rule carried over | |---|---|---| -| `$d.vec: nearest($d.embedding, $q)` | `F64?`, the squared L2 distance Lance ranks by; null for a row outside the window | `limit` required (today's `T17`); the window is the query's `limit`, widened by the overfetch ladder as today | -| `$d.lex: bm25($d.text, $q)` | `F64?`, the BM25 score; null for a non-match | the property needs a full-text index (`T27`); eligibility placement as today | +| `$d.vec: nearest($d.embedding, $q)` | `F32`, non-null: the squared L2 distance Lance ranks by, the `_distance` column the ranked scan appends; a row outside the window is absent, never null | `limit` required (today's `T17`); the window is the query's `limit`, widened by the overfetch ladder as today | +| `$d.lex: bm25($d.text, $q)` | `F32`, non-null: the BM25 score, the `_score` column; a non-match is absent | the property needs a full-text index (`T27`); eligibility placement as today | + +The column keeps the type `TypedExpr::score_type` publishes today (`F32` for +these two arms, `F64` for `rrf`), so a later change of Lance's distance type +is a language break only when that owner changes. An arm is a projection alias +over the column the ranked scan appends; projected without an alias, its +result column is `.`, the name the query gave it. `order` must lead with an arm or with `rrf(arm, arm [, k: n])`, exactly the forms `order` accepts today for `nearest(…)`, `bm25(…)` and `rrf(…)`; `asc` on a `bm25` arm or on `rrf` is refused (`T54`), as is any modifier on a `nearest` +arm; a ranking call written in `order` or `return` is `T65`, whose fix is the arm. `return` may project an arm as a value (`$d.lex as score`), which is today's `T33` projection without the repetition rule, since the arm is named -once. A filter in `match` may not read an arm (arms are bound after `match`); -`having` may, and `having { $d.vec < 0.6 }` is the one new capability this -release gives retrieval, because it falls out of scores being values: it is -the same `Filter` the engine already applies when the overfetch ladder drops -rows, now written by the query. Everything else RFC 0048 adds (a `k` on an -arm, more than two `rrf` arms, weights, `norm()`, `exact: true`, a reranker) -is a registry row or an option on a call, not grammar, and lands with that -RFC. +once. Under `rrf` an arm's own column is not a value (`T33`, as today): the +fusion carries only the fused column, and RFC 0048 owns carrying each arm's +column through `RankFuse`. A filter in `match` may not read an arm (arms are +bound after `match`). + +`having` may read an arm, and `having { $d.vec < 0.6 }` is the one new +capability this release gives retrieval. Its meaning is fixed here: the query +returns the top `limit` rows of the population that pass the predicate. That +is what the engine already does when a graph or property filter follows a +ranking: the overfetch ladder widens the window until `limit` rows pass or the +population is exhausted, and for a lone `nearest` arm the result equals what +Lance's `distance_range` computes, which a later planner change may push down. +The spike's `having_filters_groups_and_scores` pins the semantics: a lower +bound `> 0.01` on a six-row graph returns the three passing rows, not the two a +filter after the window would leave. `having` has its own type rule, `T67` +(allocated at implementation): a Boolean over aggregates, the query's group +keys and arm columns, where a group key beside an aggregate is legal, as it is +in SQL's `HAVING`, while `T58` keeps refusing that mix in `return`. + +Three refusals RFC 0048 scheduled for a later major land in this release, +because edition 3 would otherwise accept the forms and refusing them afterwards +would be a break: two arms with the same call over the same property, arms +over two different bindings, and an arm in an aggregate query (today's `T32`, +kept). Each is a type error with a code allocated at implementation and RFC +0048's reason as its text; `queries migrate` reports a stored source that uses +one instead of rewriting it (the corpus holds none). + +Everything else RFC 0048 adds (a `k` on an arm, more than two `rrf` arms, +weights, `norm()`, `exact: true`, a reranker) is a registry row or an option on +a call, not grammar, and lands with that RFC. Plan acceptance in this release +gains the arm as a named score origin (RFC 0047's "score meaning" +requirement), `having` (predicate origin, order, row cut) and `limit $k` (the +bound value, where the validator compares the written limit today). RFC 0047's +exact fragment is a closed, versioned contract: a read with `having` or `limit +$k` is outside it in this release, and RFC 0048, which owns the fragment's +extension, either admits the two with a derivation or keeps them out. ### The rewriter @@ -319,26 +394,51 @@ each source under edition 2, and writes edition 3: | `$x.title contains "g"` | `contains($x.title, "g")` | | `$x.tags contains "rust"` (a list property) | `"rust" in $x.tags` | | `$x.title starts_with "O"` | `starts_with($x.title, "O")` | -| `order { nearest($d.e, $q) }` | `retrieve { $d.nearest: nearest($d.e, $q) } … order { $d.nearest }` | -| `order { bm25($d.t, $q) desc }` | `retrieve { $d.bm25: bm25($d.t, $q) } … order { $d.bm25 desc }` | -| `order { rrf(nearest(…), bm25(…), 60) }` | two arms, `order { rrf($d.nearest, $d.bm25, k: 60) }` | -| `return { …, nearest($d.e, $q) as score }` | `return { …, $d.nearest as score }` | - -Everything not in the table is unchanged text. The generated arm names are -`nearest`, `bm25` and, for a second arm of the same kind, `nearest2`; a name -that collides with a declared property gets a numeric suffix. A human may -rename them afterwards; the rewriter never has to. - -### Operators +| `fuzzy($x.title, $q, 2)` | `fuzzy($x.title, $q, max_edits: 2)` | +| `order { nearest($d.e, $q) }` | `retrieve { $d._distance: nearest($d.e, $q) } … order { $d._distance }` | +| `order { bm25($d.t, $q) desc }` | `retrieve { $d._score: bm25($d.t, $q) } … order { $d._score desc }` | +| `order { rrf(nearest(…), bm25(…), 60) }` | two arms, `order { rrf($d._distance, $d._score, k: 60) }` | +| `return { …, nearest($d.e, $q) as score }` | `return { …, $d._distance as score }` | +| `return { …, nearest($d.e, $q) }` | `return { …, $d._distance }`, the result column `d._distance` as today | + +Everything not in the table is unchanged text; a triple in either written +order, `search`, `match_text` and `fuzzy` with named options, and `` stay +as they are. The generated arm names are `_distance` and `_score`, the names +of the columns the ranked scan appends, and `_distance2` for a second arm of +the same kind, so that every result column an edition-2 query produces keeps +its name and its type and no client reads a renamed column. A human may +rename an arm afterwards; the rewriter never has to. + +The proof compares typed IR, and it passes for rankings because an arm +reference is resolved to its call before IR is built: `$d._distance` in `order` +lowers to the same `IRExpr::Nearest` the edition-2 call lowered to, so +`search_node` finds what it finds today and `Eq` holds. The spike ran this +proof over the repository's 2,609 sources (every GQT `query` and `mutate` +section, the user and developer guides' code blocks, the skill, the `.gq` +fixtures): 1,818 declarations compile to identical typed IR, none differ, 611 +statements are unchanged, 43 documentation blocks with no schema are checked +for parse only, and 2,968 generated reads migrate IR-equal and idempotent. Of +132 `expect error` steps, 129 keep their code and 3 move from `T33` or `T32` +to `T65`, the edition-3 code for a ranking written as a call; the corpus gate +counts query-text edits only, and the tool rewrites those expectation lines +from its fixed code map. + +### The edition window at boot The edition-3 release boots an edition-2 stored source by rewriting it in -memory and logging the edits once per query (a warning with the query's name -and each edit), so no graph is quarantined for this change. `cluster plan` -prints the same edits as a pre-upgrade finding. The release after this one -compiles edition 3 only; an edition-2 source then fails validation with a -diagnostic whose fix is the `queries migrate` command. The bundled skill, the -user guide and the GQT corpus are edition 3 in the release that ships it; the -skill's command index and condensed reference are regenerated, not edited. +memory and logging the edits, once per source per boot (a warning with the +query's name and each edit, repeated on every boot until an operator runs +`queries migrate` on the cluster bundle, since the boot rewrite never writes +the bundle), so no graph is quarantined for a source the proof accepts. A +source the proof refuses is a parse failure for the purposes of boot and +quarantines its graph as one does today, with the construct the proof could +not carry in the diagnostic; the loud outcome is the point (invariant 8). +`cluster plan` prints the same edits as a pre-upgrade finding. The release +after this one compiles edition 3 only; an edition-2 source then fails +validation with a diagnostic whose fix is the `queries migrate` command. The +bundled skill, the user guide and the GQT corpus are edition 3 in the release +that ships it; the skill's command index and condensed reference are +regenerated, not edited. ### Refusals @@ -346,18 +446,27 @@ Every refusal keeps the four-field diagnostic. New codes, allocated at implementation: - `Q00x` unknown edition; `Q00x` an edition line after another statement; -- an arrow traversal (`-e->`, `<-e-`, `-e-`), the infix string operators - and `query` with a mutation body are parse or type errors whose `fix` is - the rewritten text (a `machine_applicable` suggestion); -- `T5` for a triple written against the schema, whose `fix` is the clause - with its ends swapped; `T22` only for an undirected traversal whose - orientation the endpoint types leave ambiguous; -- `L…` lint for `count { } > 0` and `count { } = 0`, whose `fix` is the - `exists` or `not` block; +- an arrow traversal (`-e->`, `<-e-`, `-e-`, with or without a binding and + bounds), the infix string operators, a positional `k` or `max_edits`, a + ranking written as a call in `order` or `return` (`T65`) and `query` with a + mutation body are parse or type errors whose `fix` is the rewritten text (a + `machine_applicable` suggestion computed without the schema); +- `T5` and `T22` keep today's meaning; `T22` no longer fires for an undirected + alternation whose members run in opposed orientations, which resolves per + member; +- a lint for `count { } > 0` and `count { } = 0`, whose `fix` is the `exists` + or `not` block, and one for a directed triple written against schema order, + whose `fix` is the clause in schema order; both are `queries fmt` rewrites. + The query diagnostic catalogue (`crates/omnigraph-compiler/src/query/codes.rs`) + admits `Q`, `T` and `P` today and gains the prefix `L` for lints, whose + rule is that a lint never changes the typed IR; +- `T67` for a `having` expression outside its rule; the three RFC 0048 + refusals named above; - `T6x` reserved construct: `view`, `batch`, `optional { }`, a binding with arguments, a block aggregate as a value, `returning`, - `@type_id`, the reserved annotations and catalogue names, a namespaced type - reference, a `Branch` or `Commit` parameter; `expected` + `@type_id`, the reserved annotations and catalogue names, a `retrieve` arm + over a catalogue type, a namespaced type reference, a `Branch` or `Commit` + parameter, a file mixing schema and query declarations; `expected` names the construct, `fix` names the owning RFC and says "not yet served in this release"; - a `retrieve` arm name colliding with a property; an arm read in `match`; an @@ -368,20 +477,31 @@ implementation: ### One grammar for schema and queries -`query.pest` and `schema.pest` become one grammar with one `file` rule: +`query.pest` and `schema.pest` become one grammar whose start rule is today's +`query_file`, extended by the edition line and the declaration kinds and +otherwise kept, so that every statement the file accepts today (`show`, +`branch`, `commit`, `load`, `explain`, the settings, `statement_trailer`) and +the missing-`()` diagnostic the competence instrument measures stay where +they are: ```pest -file = { SOI ~ edition? ~ setting_stmt* ~ (statement | declaration*) ~ EOI } -edition = { kw_gq ~ integer ~ ";" } -declaration = { schema_decl | query_decl | mutation_decl | view_decl | batch_decl } +query_file = { SOI ~ edition? ~ setting_stmt* ~ (show_stmt ~ statement_trailer? | branch_stmt ~ ";"? ~ statement_trailer? | commit_stmt ~ ";"? ~ statement_trailer? | load_stmt ~ ";"? ~ statement_trailer? | explain_stmt ~ statement_trailer? | declaration* ~ missing_param_list?) ~ EOI } +edition = { kw_gq ~ integer ~ ";" } +declaration = { schema_decl | query_decl | mutation_decl | view_decl | batch_decl } +missing_query_keyword = @{ ("query" | "mutation" | "view") ~ !(ASCII_ALPHANUMERIC | "_") } ``` The schema rules move unchanged except for `type_name`, which becomes a -`type_ref` admitting namespace segments (below), and `vector_type`, which -admits `distance=`. A `.pg` file is a `file` holding only `schema_decl`s; the -loader keeps the extension and `cluster.yaml` keeps its `schema:` key, both -of which now parse with the one grammar. Nothing a `.pg` file accepted today -is refused. +`type_ref` admitting namespace segments (below), `vector_type`, which +admits `distance=`, and `annotation`, which becomes the generic form below. A +`.pg` file is a `query_file` holding only `schema_decl`s; the loader keeps the +extension and `cluster.yaml` keeps its `schema:` key, both of which now parse +with the one grammar. Nothing a `.pg` file accepted today is refused (the +spike parsed the 432 GQT schemas and the repository's five `.pg` files under +the one rule). A file holding both schema and query declarations parses and is +refused as a slot in this release; the loader and the stored-query registry +each take one kind, and the RFC that lets one file carry both says what the +mixed file means. ### One spelling per meaning @@ -391,19 +511,27 @@ cannot forbid. Concretely: - A respelling that adds no meaning is not in the grammar. The arrow traversal (`-e->`, `<-e-`, `-e-`), the infix `contains` and `starts_with`, - the positional `k` of `rrf`, and `query` around a write are each a parse or - type error whose `fix` prints the one form. The diagnostics know every - respelling an agent trained on Cypher, SQL or GraphQL is likely to write. - The rule also cuts the other way: a decoration on the one spelling that - carries no information is not added, which is why the traversal keeps its - triple and gains no arrowhead ([Traversal](#traversal)). + the positional `k` of `rrf` and `max_edits` of `fuzzy`, a ranking written as + a call in `order`, and `query` around a write are each a parse or type error + whose `fix` prints the one form. The diagnostics know every respelling an + agent trained on Cypher, SQL or GraphQL is likely to write. The rule also + cuts the other way: a decoration on the one spelling that carries no + information is not added, which is why the traversal keeps its triple and + gains no arrowhead ([Traversal](#traversal)). - A form that falls out of a general construct and has a canonical instance is rewritten to that instance by `omnigraph queries fmt`, the rewriter's second job (`gofmt -s` is the precedent), and named by lint: `count { P } > - 0` becomes `exists { P }` and `count { P } = 0` becomes `not { P }`; once - the polymorphic types RFC adds `is`, `$x.@type = "Person"` becomes - `$x is Person`. The grammar keeps `exists` because ISO GQL has only - `EXISTS` and the planner's semi-join is the honest plan for it. + 0` becomes `exists { P }` and `count { P } = 0` becomes `not { P }`; a + directed triple written against the schema's order (`$c worksAt $p` under + `edge WorksAt: Person -> Company`), which edition 2 accepts and the type + checker resolves to the incoming orientation, becomes the triple in schema + order; once the polymorphic types RFC adds `is`, `$x.@type = "Person"` + becomes `$x is Person`; once the lexical RFC lands `match_terms`, a `search`, + `match_text` or `fuzzy` call becomes the `match_terms` call it equals, the + three rows staying in the registry for good. The grammar keeps `exists` + because ISO GQL has only `EXISTS` and the planner's semi-join is the honest + plan for it. A formatter rewrite never changes the typed IR; that is what + separates it from the rewriter's break. - Forms that look alike but differ in meaning stay: `return { $a.slug }` (a column) and `return { $a { slug } }` (one object); `$e.@src` (stored orientation) and the pattern's left end (written orientation); `limit 10` @@ -412,8 +540,9 @@ cannot forbid. Concretely: - One argument has one form: a registry function declares each argument positional (required) or named (an option), and a caller cannot choose. - A later RFC's Alternatives section answers this rule for every form it - adds; `@if_commit($p)` and the request's precondition field are the first - case, and the writes RFC must say the field is derived from the + adds; `@if_commit($p)` and the request's `--if-commit` precondition field + are the first case, which is why `@if_commit` is a slot here and not a + typed annotation: the writes RFC must say the field is derived from the annotation, never a second source. ### Declarations @@ -449,129 +578,179 @@ retrieve_clause = { kw_retrieve ~ "{" ~ retrieve_arm+ ~ "}" } retrieve_arm = { variable ~ "." ~ ident ~ ":" ~ call ~ ","? } having_clause = { kw_having ~ "{" ~ expr ~ "}" } limit_clause = { kw_limit ~ (integer | variable) } -clause = { negation | optional_block | subquery_predicate | exists_block | binding | traversal | filter } +clause = { negation | optional_block | subquery_predicate | exists_block | binding | arrow_traversal | traversal | filter } optional_block = { kw_optional ~ "{" ~ clause+ ~ "}" } -binding = { variable ~ ":" ~ type_ref ~ ("(" ~ (expr ~ ("," ~ expr)*)? ~ ")")? ~ ("{" ~ prop_match_list ~ "}")? } +binding = { variable ~ ":" ~ bound_type ~ ("{" ~ prop_match_list ~ "}")? } +bound_type = ${ type_ref ~ view_args? } // the `(` touches the name +view_args = !{ "(" ~ (expr ~ ("," ~ expr)*)? ~ ")" } block_value = { (agg_name ~ ("(" ~ expr ~ ")")? | kw_exists) ~ subquery_block } ``` A `binding` with an argument list names a `view`; the type checker refuses it -in this release. `block_value` joins `operand` (below) so a block aggregate -can stand where a value stands; the type checker refuses it in this release -and the derived relations RFC gives it the `SubqueryAggregate` the engine -already runs for the predicate form. Both are listed so that compositional -closure, the capability the next RFC adds, needs no grammar change: a view -declaration, a view binding, an optional block, a block as a value and -`having` all parse today. +in this release. The argument list must touch the type name (`Contested(`): +with whitespace admitted, `$p: Person` followed by a parenthesised filter on +the next line, legal edition-2 text, would be read as view arguments, a silent +change of meaning the spike's `rfc_binding_rule_swallows_a_parenthesised_filter` +caught. `block_value` joins `operand` (below) so a block aggregate can stand +where a value stands; the type checker refuses it in this release and the +derived relations RFC gives it the `SubqueryAggregate` the engine already runs +for the predicate form. Both are listed so that compositional closure, the +capability the next RFC adds, needs no grammar change: a view declaration, a +view binding, an optional block, a block as a value and `having` all parse +today. An arm's column is a `PropAccess` whose property is the arm name; the type checker records it in the binding's scope with the arm's result type and -refuses a read of it inside `match`. Lowering maps an arm to today's logical -nodes (`Nearest`, `TextSearch`, `RankFuse` in `crates/omnigraph-planner/src/logical.rs`) -and the arm's column to the score column the ranked scan already appends -(`_distance`, `_score`, `_rrf`), under the arm's name. The planner's search -node selection (`optimizer.rs` `search_node`) reads the leading `order` key as -it does today, finding an arm reference or an `rrf` over arm references where -it found a call. `having` lowers to a `Filter` above `Aggregate`, or above the -retrieval when the query has no aggregate; both are nodes the planner has. -Plan acceptance gains the arm as a named score origin (RFC 0047's "score -meaning" requirement already binds a projected score to its producer; the arm -is that binding made explicit). The explain document version advances because -score columns are named by the query. +refuses a read of it inside `match`. Before IR is built, every reference to an +arm is resolved to the arm's call, so `order { $d._distance }` lowers to the +`IRExpr::Nearest` the edition-2 `order { nearest(…) }` lowers to, the +planner's search node selection (`optimizer.rs` `search_node`) reads the +leading `order` key exactly as it does today, and the rewriter's IR-equality +proof holds without a second comparison. Lowering maps the call to today's +logical nodes (`Nearest`, `TextSearch`, `RankFuse` in +`crates/omnigraph-planner/src/logical.rs`) and the arm's column to the score +column the ranked scan already appends (`_distance`, `_score`, `_rrf`), the +arm name being the projection alias. `having` lowers to a `Filter` above +`Aggregate`, or above the ranked scan's `Sort` and below `Limit` when the +query has no aggregate (the spike's `explain_retrieve_having.json`: the +`match` filter in the Lance scan filter, `Nearest k: 2` bound from `$k`, the +scan and search fused physically, `having` a typed `Filter` with its threshold +cast to `F32`); both are nodes the planner has. The explain document and +bound-plan versions advance once because score columns are named by the query +and `having` and `limit $k` are new plan inputs. ### Traversal ```pest traversal = { variable ~ edge_binding? ~ (undirected_edge | edge_sel) ~ traversal_bounds? ~ variable } -undirected_edge = { "<" ~ edge_sel ~ ">" } +undirected_edge = { "<" ~ (edge_wildcard | edge_alternatives | catalogue_edge | edge_ident | string_lit) ~ ">" } // `` as today edge_sel = { edge_alternation | edge_wildcard | catalogue_edge | edge_ident | string_lit } catalogue_edge = @{ "@" ~ ident } // an edge of the catalogue's own schema; a slot -arrow_traversal = { variable ~ ("<-" | "-") ~ edge_binding? ~ edge_sel ~ traversal_bounds? ~ ("->" | "-") ~ variable } // recognised only to print the fix +arrow_traversal = { variable ~ ("<-" | "-") ~ edge_binding? ~ edge_sel ~ traversal_bounds? ~ ("->" | "-") ~ variable } // tried before `traversal` in `clause`; recognised only to print the fix ``` -The traversal keeps edition 2's spelling: a triple, source on the left, -`` for either orientation. This RFC's first draft replaced it with an -arrow (`$p -worksAt-> $c`, `$p -worksAt- $c`) for the reader and for the -model's Cypher prior, and the direction evidence removed both reasons. The -arrowhead carries no information: when the two endpoint types differ the -schema fixes the orientation, and when they are the same +The traversal keeps edition 2's spelling and edition 2's rules: a triple, +`` for either orientation, today's `T5` and `T22`. This RFC's first draft +replaced it with an arrow (`$p -worksAt-> $c`, `$p -worksAt- $c`) for the +reader and for the model's Cypher prior, and the direction evidence removed +both reasons. The arrowhead carries no information: when the two endpoint +types differ the schema fixes the orientation, and when they are the same (`Person knows Person`, `Company subsidiaryOf Company`) the position does, in a triple exactly as in an arrow form. It is a decoration on the one spelling, and the one-spelling rule, applied from the other side, asks what a decoration buys. A reader reads `$p worksAt $c` in source-to-target order, -as a sentence; the one in-context measurement GQ has is on the triple -(RFC 0047, Evidence: 45 of 45 tasks, 37 first tries valid, no direction -error), while CypherBench counts a reversed relation in about one gpt-4o -failure in six on the arrow form; and keeping the triple removes the single -largest rewrite from the break, since every stored query has a traversal, -and the arithmetic-minus rule with it. `arrow_traversal` is in the grammar -only as a recognizer, so that an agent writing `$p -worksAt-> $c`, -`$c <-worksAt- $p` or `$a -knows- $b` from a Cypher habit gets a parse error -whose fix prints the triple with its ends in schema order. - -The type checker's endpoint rule (`T5`) checks the written triple against -the declared endpoints, and when the triple is written against the schema -(`$c: Company worksAt $p`) its `fix` is the swapped clause, -`machine_applicable`. This is the deterministic correction the Cypher -ecosystem bolts on after generation (LangChain's `CypherQueryCorrector`, -built because models "can sometimes mess up the direction of the -relationship") moved into the compiler, with two differences: it is loud, so -the agent learns in one turn, and it never guesses. For a same-type edge the -position is the only carrier of meaning and nothing can check it against the -schema; the edge's `@description` and the skill's schema card say in words -which end is which. +as a sentence; the one in-context measurement GQ has is on the triple (RFC +0047, Evidence: 45 of 45 tasks correct, 8 of 45 first attempts refused, all +for a missing `()`; it reports no direction result), while CypherBench marks a +reversed relation in 9 of 55 gpt-4o error marks on the arrow form; and +keeping the triple removes the single largest rewrite from the break, since +every stored query has a traversal, and the arithmetic-minus rule with it. +`arrow_traversal` is in the grammar only as a recognizer, placed before +`traversal` in the `clause` choice so the parser reaches it, and never +reaching ``, which stays the undirected form. An agent writing +`$p -worksAt-> $c`, `$c <-worksAt- $p` or `$a -knows- $b` from a Cypher +habit gets a parse error whose fix prints the triple in the written order, +computed without the schema (the spike: `Q008`, with a binding and bounds +carried through, meaning preserved). + +Edition 2 accepts a directed triple in either written order and resolves it +against the schema: `$c: Company worksAt $p` under `edge WorksAt: Person -> +Company` is the incoming orientation (`Direction::In` on the member), and +the corpus writes it (`issue_659_compiler_scopes.gqt`, inside `not`, `exists` +and `count` blocks). Edition 3 keeps that. The two orders are one meaning, the +type checker already records which orientation each member takes, and +refusing one order would be a break the first draft's `T5` hid from the +compatibility list; the spike showed the swap keeps typed IR and rows equal, +so either rule is implementable, and the one that breaks nothing wins. +`queries fmt` writes the schema's order, so a stored query reads +source-to-target, and lint names the rewrite. `T5` keeps today's meaning, an +edge whose declared endpoints admit neither order of the written types, with +its fix unchanged. The deterministic correction the Cypher ecosystem bolts +on after generation (LangChain's `CypherQueryCorrector`, built because models +"can sometimes mess up the direction of the relationship") is therefore not +needed on the triple: the compiler resolves the orientation itself, and the +formatter makes the text say it. For a same-type edge the position is the +only carrier of meaning and nothing can check it against the schema; the +edge's `@description` and the skill's schema card say in words which end is +which. A catalogue edge carries the `@` prefix for the reason catalogue types do: a user schema may declare an edge named `property` or `from`, and one spelling per meaning forbids resolving a bare name against two namespaces by -endpoint type. The engine catalogue RFC's schema extension declares the -catalogue's edges and which of its text fields a `retrieve` arm may search, -so that `match { $t: @Type } retrieve { $t.lex: bm25($t.instruction, $q) }` -is an ordinary query over an ordinary table; nothing in the grammar -distinguishes it from a query over user data, which is the point. +endpoint type. Its ends are bound as any traversal's are, by separate +bindings: `$t: @Type $p: @Property $t @property $p`. The engine catalogue +RFC says the five catalogue types are derived at query time, never stored, +that writes, loads and text or rank search over them are refused, and that +catalogue access never calls Lance; this RFC reserves the seven names and the +edge selector as slots and changes none of that. A `retrieve` arm over a +catalogue type (`match { $t: @Type } retrieve { $t.lex: bm25($t.instruction, +$q) }`) therefore parses and is refused with the slot. The amendment that +lifts the refusal names its mechanism, and that mechanism opens no Lance +dataset for rows the catalogue keeps derived (a searchable catalogue table +would be a second schema store, invariant 12); `bm25` stays on user tables, +where the index exists, until then. `` means whichever orientation the endpoint types admit. For a same-type edge that is both orientations, deduplicated, as today. For an alternation -or wildcard whose members run in different orientations between the two -bound types (`$p: Person $d: Doc`; the directed triple -resolves these per member today) it is each member's declared orientation, -and the bound edge's `@src` and `@dst` keep reporting the stored orientation. -`T22` ("undirected traversal needs a same-endpoint-type edge") narrows to the -one case that stays ambiguous: an asymmetric edge whose endpoints are not -typed well enough to fix the orientation. No traversal token collides with -an operator the function-registry work may add: the ordered choice tries -`traversal` before `filter` inside `match`, as today, so `$a $b` is -never read as a comparison. +whose members run in opposed orientations between the two bound types +(`$p: Person $d: Doc $p $d`), the directed triple resolves +the members individually today while `` is refused (`T22`, +`test_undirected_traversal_rejected_on_asymmetric_edge`); edition 3 resolves +`` per member too, and the bound edge's `@src` and `@dst` keep reporting +the stored orientation. This is the one semantic change in the traversal, and +it is additive: text edition 2 refuses gains a meaning, and no accepted text +changes. `T22` narrows to the case that stays ambiguous, an asymmetric edge +whose endpoints are not typed well enough to fix the orientation. The +`<` and `>` of the undirected form are already the comparison operators of +`comp_op`; the ordered choice tries `traversal` before `filter` inside +`match`, as today, so `$a $b` is never read as a comparison, and the +function-registry work adds no operator that changes this. ### Calls and the reserved set ```pest -call = { call_name ~ "(" ~ call_args? ~ ")" } -call_name = @{ !reserved ~ ident } -call_args = { positional ~ ("," ~ positional)* ~ ("," ~ option)* | option ~ ("," ~ option)* } -option = { ident ~ ":" ~ expr } -count_call = { "count" ~ "(" ~ kw_distinct ~ expr ~ ")" } -operand = { "(" ~ expr ~ ")" | block_value | count_call | call | prop_access | meta_field | variable | literal | reserved_property | expr_ident } -reserved = { kw_and | kw_or | kw_not | kw_is | kw_null | kw_in } +call = { call_name ~ "(" ~ call_args? ~ ")" } +call_name = @{ !reserved ~ ident } +call_args = { positional ~ ("," ~ positional)* ~ ("," ~ option)* | option ~ ("," ~ option)* } +positional = { !(ident ~ ":") ~ expr } // so `rrf($a, $b, k: 60)` reaches `option` +option = { ident ~ ":" ~ expr } +agg_distinct = { agg_name ~ "(" ~ kw_distinct ~ expr ~ ")" } +operand = { "(" ~ expr ~ ")" | block_value | agg_distinct | call | prop_access | meta_field | variable | literal | reserved_property | expr_ident } +list_lit = { "[" ~ (list_item ~ ("," ~ list_item)*)? ~ "]" } +list_item = { literal | call } // `[date("…"), …]` stays legal; the call must fold +reserved = { kw_and | kw_or | kw_not | kw_is | kw_null | kw_in } ``` The rules `now_call`, `search_call`, `fuzzy_call`, `match_text_call`, `bm25_call`, `rrf_call`, `rank_expr`, `agg_call`, `agg_func`, `nearest_ordering`, `date_lit` and `datetime_lit` are deleted; `date("…")` and `datetime("…")` are calls the compiler folds to literals as `ir/fold.rs` folds -constants today. A call resolves in the type checker against a ***function -registry***: name, positional parameter types, named options with their types -and ranges, result type rule, kind (`scalar`, `aggregate`, `predicate`, -`ranking`), and the two planner facts `can_fail` and `volatile`. In this -release the registry holds exactly the functions the grammar held, with their -current type rules (`T8`, `T19`, `T20`, `T21`, `T27`, `T33`, `T38` keep their -texts, re-anchored to registry rows), and `rrf` admits `k:` as its one option. -The registry is a Rust table in `omnigraph-compiler`; the catalogue type -`@Function` that exposes it is a reserved name here and lands with the schema -catalogue. Block aggregates (`count { } > n`) keep their keyword form: they -are logic over a pattern, not calls, as the operator-vocabulary amendment -classifies them. `count(distinct x)` keeps its special form because `distinct` -is contextual and the form is the one every comparator spells the same way. +constants today, inside a list literal included (`issue_801_in_predicate_over_a_list.gqt` +writes `[date("…"), …]`). `positional` looks ahead for `ident ~ ":"` because +PEG does not backtrack: without it `k` in `rrf($a, $b, k: 60)` is taken as a +bare-name positional and the option never parses. A call resolves in the type +checker against a ***function registry***: name, positional parameter types, +named options with their types and ranges, result type rule, kind (`scalar`, +`aggregate`, `predicate`, `ranking`), and the two planner facts `can_fail` and +`volatile`. In this release the registry holds exactly the functions the +grammar held, with their current type rules (`T8`, `T19`, `T20`, `T21`, +`T27`, `T33`, `T38` keep their texts, re-anchored to registry rows); `rrf` +admits `k:` as its one option and `fuzzy` admits `max_edits:` where its third +positional stood. A registry row is never removed: `search`, `match_text` and +`fuzzy` stay, and when the lexical RFC lands `match_terms`, `queries fmt` +canonicalises the three to it and lint names the rewrite; that RFC's +scheduled removal of the three names and its `T27` tightening (refusing +`bm25` over `@index` without `@analyzed`, a refusal of text every graph +writes today) are withdrawn by this rule, since either would be a second +break ([Decision log](#decision-log)). The registry is a Rust table in +`omnigraph-compiler`; the catalogue type `@Function` that exposes it is a +reserved name here and lands with the schema catalogue. Block aggregates +(`count { } > n`) keep their keyword form: they are logic over a pattern, not +calls, as the operator-vocabulary amendment classifies them. `agg(distinct +x)` keeps its special form for every aggregate name, because `distinct` is +contextual and the form is the one every comparator spells the same way; +`sum(distinct $x)` stays the type error it is today (`T57`) rather than +becoming a parse error (`issue_954_count_distinct.gqt`). A grammar test pins the reserved set: every keyword outside the six is used as a property name after a dot, as an edge name in a traversal, as a return @@ -580,32 +759,52 @@ alias and as a bare property in a mutation `where`, and parses. ### Annotations, meta-fields, catalogue names, type references ```pest -annotation = { "@" ~ ident ~ ("(" ~ annotation_args ~ ")")? } -meta_field = @{ "@" ~ ident } -type_ref = @{ (ident ~ "::")* ~ "@"? ~ ASCII_ALPHA_UPPER ~ (ASCII_ALPHANUMERIC | "_")* } +annotation = { "@" ~ ident ~ ("(" ~ annotation_args ~ ")")? } +annotation_args = { annotation_arg ~ ("," ~ annotation_arg)* } +annotation_arg = { ident ~ "=" ~ annotation_value | annotation_value } +annotation_value = { literal | variable | meta_field | ident } +meta_field = @{ "@" ~ ident } +type_name = @{ ASCII_ALPHA_UPPER ~ (ASCII_ALPHANUMERIC | "_")* } +type_ref = @{ (ident ~ "::")+ ~ type_name | "@" ~ type_name | type_name } // a catalogue name takes no segment ``` +Annotation arguments are positional values or `name = value`, in any order +and number (`@valid(a, b)`, `@analyzed(analyzer = "…")`, `@embed("text", +model = "…")`, `@if_commit($c)`, `@key(@src, @dst)`), the spelling stored +`.pg` text already uses (`annotation_kwarg` in `schema.pest`); a call's +options use `:` and an annotation's use `=`, because the two are different +things (a call is an expression, an annotation a declaration's metadata) and +the one-spelling rule applies per meaning, not per punctuation mark. The +evaluation's `@budget(rows: 10)` is therefore spelled `@budget(rows = 10)`. Annotations are generic on every declaration and resolve against an ***annotation registry*** of the same shape as the function registry (typed arguments; `scalar` or `declaration` scope); an unknown name is retained as metadata as `.pg` does today. Meta-fields resolve against a closed list in the -type checker; `@type_id` is in the list and refused. Catalogue type names -resolve against the catalogue registry; the seven reserved names are in it and -refused. A `type_ref` with a namespace segment resolves to a refusal until the -schema namespaces RFC lands; the grammar admits it so that RFC adds no rule, -and the federation reservation is the sentence that an outermost segment may -later name a graph. +type checker (`@id`, `@src`, `@dst`, `@type`, `@type_id`); `@type_id` is in +the list and refused. The first draft also reserved `@distance` and `@score` +as fields of an arm's column; they are dropped, because the arm's column is +the one spelling of the score and `prop_access` is one level deep, so the two +names had no position to stand in. Catalogue type names resolve against the +catalogue registry; the seven reserved names are in it and refused. A +`type_ref` with a namespace segment resolves to a refusal until the schema +namespaces RFC lands; the grammar admits it so that RFC adds no rule, and the +federation reservation is the sentence that an outermost segment may later +name a graph. A catalogue name takes no segment (`crm::@Operator` is a parse +error), which is the rule the schema namespaces RFC states. ### Parameters, limit, null `param` gains a default literal and the kinds `Branch` and `Commit` in `type_ref_or_kind`; the binder refuses the two kinds in this release. `limit` takes an `integer` or a `variable` declared `I64` non-null; a null or negative -value is a bind-time refusal. `null` is admitted by the grammar only as the -whole value of an assignment (`mutation_assignment` and `prop_match`), where it -is `Literal::Null` of the property's type, which the write path already -handles for a null constant result; a read filter containing `null` keeps -today's parse error. +value is a bind-time refusal (the spike's `limit_binds_a_parameter`: `k` of +1, 2 and 4 on Lance, a negative value refused at bind, explain showing +`Nearest k: 2` bound from `$k`). `null` is admitted by the grammar only as the +whole value of a mutation assignment (`mutation_assignment`), where it is +`Literal::Null` of the property's type, which the write path already handles +for a null constant result; a binding's property match (`$p: Person { age: +null }`) is a read and keeps today's parse error, as does a read filter +containing `null`. ### Envelope @@ -645,11 +844,20 @@ rewrite. score is a typed column with a named producer; nothing new travels in strings. - **8, integrity failures are loud.** Every slot refuses with a stable code - and a fix; the rewriter refuses to write a rewrite it cannot prove; an - edition it does not know is a refusal, never a guess. + and a fix; the rewriter refuses to write a rewrite it cannot prove, and a + stored source it cannot prove at boot quarantines its graph as a parse + failure does today; an edition it does not know is a refusal, never a + guess. +- **12, one source of truth.** The frozen score column is the type + `score_type` publishes and the column the ranked scan appends; the arm is + an alias over it, not a second column. The catalogue stays derived: a + `retrieve` over it is refused until an amendment names a mechanism that + opens no Lance dataset. - **13, evidence matches the boundary.** The grammar's evidence is the corpus rewritten mechanically and green, plus IR equality; the spelling's evidence - is the competence instrument on the new spelling before the break. + is the competence instrument on the new spelling before the break; the + spike is the evidence that the grammar as written parses its own examples + and today's text. - **3 and 4, one coherent view, one publication.** Untouched; no read or write path changes. - **11, bounded resources.** Untouched; `limit $k` binds to the same `Limit`. @@ -660,16 +868,29 @@ channel; `having` is a `Filter` node, not a string. ## Compatibility and reversibility - **This RFC is the break.** The spellings it removes are `query` for a - write, the infix string operators and the positional - `rrf` constant; every other edition-2 text parses unchanged under edition 3 - (the reserved set does not grow, every new keyword is contextual). + write, the infix string operators, the positional `k` of `rrf` and + `max_edits` of `fuzzy`, and a ranking written as a call in `order` or + `return`; the refusals it adds to accepted text are RFC 0048's three + (duplicate arms, arms over two bindings, an arm in an aggregate query). + Every other edition-2 text parses and typechecks unchanged under edition 3: + the triple in either written order, ``, the edge binding, alternation, + wildcard and quoted edge, `search`, `match_text` and `fuzzy`, every `.pg` + construct (the reserved set does not grow, every new keyword is + contextual). One text edition 2 refuses gains a meaning: `` over an + alternation of opposed members. - **One release of overlap.** The edition-3 release compiles edition 2 and rewrites it at boot with a warning; the next release removes the edition-2 grammar and refuses it with the `queries migrate` fix. `GQ_LANGUAGE_VERSION` - becomes the edition and moves to `(3, 0)`; the compatibility-surfaces hash - file, when it lands, pins the edition-3 rules. -- **Wire.** Additive envelope fields; no HTTP contract bump; explain and - bound-plan versions advance once (named score columns, `having`). + becomes the edition and moves from `(2, 3)` to `(3, 0)`; PR #805's `(3, 0)` + and `(4, 0)` are withdrawn so this is the one major ([Decision + log](#decision-log)); the compatibility-surfaces hash file, when it lands, + pins the edition-3 rules. +- **Wire.** Additive envelope fields; no HTTP contract bump (the spike booted + and served an edition-3 stored read with `retrieve`, `having` and `limit + $k` under today's envelope, `ranking.score_columns` included); explain + moves from `7` and the bound-plan version advances once (named score + columns, `having`, `limit $k`). Result column names and types are + unchanged for every migrated query. - **Storage.** None. - **Stored queries.** Every stored query in the fleet is rewritten by the tool before the release ships (Evidence); a stored source the sweep missed is @@ -689,6 +910,11 @@ channel; `having` is a `Filter` node, not a string. | Additive first, break last: ship every new form beside the old one with a deprecation, remove the old ones in a final major | Every construct built twice, every corpus and skill edit made twice, agents reading two spellings for a release per construct; three open RFCs already schedule four majors this way | | Keep rankings in `order` (RFC 0048's "adds no clause and no stage") | One token selects the population and sorts the rows; a score cannot be a filter; `candidates`, `weight`, `exact` and `k` become grammar on the call; the aggregate and `distinct` cases need a special rule. A named arm makes every one of those an expression or an option | | An arrow spelling for traversals (`$p -worksAt-> $c`, `-e-`), as in Cypher, GQL and SQL/PGQ | This RFC's first draft, for readability and the model's prior; rejected on 2026-10-30: the arrowhead carries nothing the position and the schema do not, the one measurement on the triple shows no direction errors while CypherBench shows arrows do not prevent them, and it was the break's largest rewrite | +| Refuse an incoming triple (`$c worksAt $p` under `Person -> Company`) as `T5` with a swap fix, so every stored query reads source-to-target | The first two drafts; rejected on 2026-11-01: the two orders are one meaning the type checker already resolves, refusing one is a break the compatibility list had hidden, and the formatter gives the reader schema order without one | +| Freeze the score column as `F64?`, null outside the window, named after the arm | The first drafts; rejected: the engine publishes a non-null `F32` (`F64` for `rrf`) and absent rows, and renaming the column changes two result shapes for nothing; the arm is an alias over the column the scan appends | +| Remove `search`, `match_text` and `fuzzy` in edition 3, as the lexical RFC schedules | Their replacement, `match_terms`, is the lexical RFC's and ships after the break, so the removal would make the break wait on that RFC or leave edition 3 without a lexical predicate; the rows stay for good and `queries fmt` canonicalises once `match_terms` exists | +| `@distance` and `@score` as meta-fields of an arm's column | Dropped: the arm's column is the one spelling of the score, and `prop_access` is one level deep, so the two names had no position | +| A `file` rule of `statement \| declaration*` | Dropped the statements the grammar accepts today and the missing-`()` diagnostic the competence instrument measures; today's `query_file` is extended instead | | Keep `query` for writes | Agents trained on GraphQL write `mutation { }` and the skill documents the trap; a keyword lets lint classify before the body | | A separate `where { }` clause for filters | Rejected by the shared expression model for reasons that hold: conditions belong beside the pattern; retrieval is different because an arm is not a condition on one row | | `graph@ns::Type` for the federation segment | A second delimiter where the namespace RFC's `::` already nests; an outermost segment is enough and needs no grammar now | @@ -706,24 +932,47 @@ Existing owners, per [the testing map](../dev/testing.md): - **Compiler parser tests** (`query/parser_tests.rs`, `schema/parser_tests.rs`): one test per freeze-list row; the reserved-set test; the traversal-versus- minus rule as a parser test even though no minus exists yet; the edition - line in every position. + line in every position; the eight grammar defects the spike found, each + pinned by the text that must parse (`rrf($a, $b, k: 60)`, a binding + followed by a parenthesised filter on the next line, ``, + `[date("…")]`, `sum(distinct $x)` as `T57`, `@embed("text", model = "…")`, + the arrow forms reached from `clause`, every example in this RFC). - **Type-check tests**: every slot's refusal with its code and fix; arm collision with a property; an arm read in `match`; `query` with a mutation - body; an arrow traversal's parse error and its triple fix; `T5` with the - swapped fix for a triple against the schema; `` over an alternation - of opposed members (the issue 659 shape) resolving per member; `T22` only - when the endpoints leave the orientation ambiguous; the `count { } > 0` and - `count { } = 0` lint with its fix. + body; an arrow traversal's parse error and its triple fix; an incoming + triple accepted with today's orientation and the formatter's rewrite; `` + over an alternation of opposed members (the issue 659 shape) resolving per + member; `T22` only when the endpoints leave the orientation ambiguous; + `T65` for a ranking call in `order`; `T67` for each shape outside the + `having` rule and the group-key-beside-aggregate shape inside it; RFC + 0048's three refusals; the `count { } > 0` and `count { } = 0` lint with its + fix. - **GQT corpus** (`crates/omnigraph-gqt/cases/`): the whole corpus rewritten by - `queries migrate` with zero hand edits, run green; a hand edit is a rewriter - defect, fixed in the tool. New cases: `having` over an aggregate and over a - retrieval; `limit $k`; `null` in an assignment; one `expect error` case per - slot, written now so the owning RFC flips it. + `queries migrate` with zero hand edits to query text, run green; a hand edit + is a rewriter defect, fixed in the tool; the three `expect error` codes that + move to `T65` are rewritten by the tool's code map and listed in the + change. New cases: `having` over an aggregate, over a group key beside an + aggregate, and over a retrieval with the ladder's semantics (a lower bound + returning the top `limit` passing rows); `limit $k`; `null` in an + assignment; an unaliased arm projection keeping today's result column; one + `expect error` case per slot, written now so the owning RFC flips it. - **Rewriter tests**: IR equality for every corpus case and every fleet stored query; idempotence; comment and layout preservation on a fixture with every - construct; refusal on an unprovable rewrite. + construct; refusal on an unprovable rewrite; the generated arm names + preserving every result column. - **Planner acceptance** (`crates/omnigraph-planner/src/validate/tests.rs`): - the arm as a score origin; `having` placement; explain version. + the arm as a score origin; `having` placement, origin and row cut; `limit + $k` lowering; a `having` or `limit $k` read outside RFC 0047's exact + fragment; explain version. +- **The spike** ([`spike/gq3-1202`](https://github.com/ragnorc/omnigraph/tree/spike/gq3-1202), + commit `f63e4393` from `main` `f728f01e`; report + [`docs/spikes/gq3-1202.md`](https://github.com/ragnorc/omnigraph/blob/spike/gq3-1202/docs/spikes/gq3-1202.md)): + the edition-3 front end, the rewriter and the proof over 2,609 sources, + sixty later capabilities parsed against the grammar, engine tests on Lance + for `retrieve`, `having` and `limit $k`, a served edition-3 stored read. + Throwaway code; its tests are the specification of the ones above, and its + numbers are the ones this RFC cites. It did not run the competence + instrument, build the boot rewrite, or run the workspace gate. - **Server** (`stored_queries`, `boot_settings`): an edition-2 stored source boots with the warning and the rewritten registry entry; `queries validate` and `cluster plan` print the edits; the release-after refusal. @@ -805,3 +1054,36 @@ the release; step 5 is the removal every deprecation ends in. failed the one-spelling rule. The rewriter table and the compatibility section lose the traversal rows; the break no longer touches every stored query. +- 2026-11-01: revised after two reviews and the spike. Contracts the drafts + had frozen wrong: the score column is the `F32` (`F64` for `rrf`), non-null, + absent-outside-the-window column `score_type` publishes, the arm an alias + over it, and the rewriter names its arms `_distance` and `_score` so no + result column is renamed; `having` over a score means the top `limit` rows + of the passing population, through the ladder the engine already runs, with + its own type rule `T67`; the incoming triple stays legal and `queries fmt` + writes schema order (the first drafts' `T5` refusal and swap were a hidden + break); `T22`'s narrowing is recorded as the one semantic change, additive; + RFC 0048's three refusals land in edition 3; a registry row is never + removed, so `search`, `match_text` and `fuzzy` stay and the lexical RFC's + removal and `T27` tightening are withdrawn; `@example` and `@if_commit` are + slots, not typed; `@distance` and `@score` are dropped; a `retrieve` over a + catalogue type is a slot behind the catalogue RFC's refusal; `null` leaves + `prop_match`; a catalogue name takes no namespace segment. Grammar: today's + `query_file` is extended instead of a new `file` rule; `arrow_traversal` + is reached from `clause`; `positional` looks ahead for an option; the view + argument list touches the name; `` is admitted; list items may be + constant calls; `agg(distinct x)` covers every aggregate; `annotation_args` + is defined with `=`. The IR proof is kept, since resolving an arm reference + to its call before IR makes it pass (the spike: 1,818 declarations and + 2,968 generated reads equal). The sentence that every later capability is + a row, a node or a slot is replaced by what the spike measured: twenty-four + of sixty need a new production, each additive at a token edition 3 refuses. + The open drafts must remove, before this RFC is accepted: PR #805's phases + that set `GQ_LANGUAGE_VERSION` to `(3, 0)` and `(4, 0)` (the call rule + lands here, as the one major); PR #793's sentence that retrieval adds no + clause and its deferral of positional `k` and the three refusals to a later + major (all land here); PR #792's removal of `search`, `match_text` and + `fuzzy` at the next major and its `T27` tightening (withdrawn: rows stay, + `fmt` canonicalises); PR #884's own minor and explain allocation against + the stale base `(2, 2)`, explain `4` (it is additive and allocates nothing + here). `supersedes` stays empty while those drafts own their semantics. diff --git a/docs/rfcs/assets/2026-10-28-gq-3-grammar-evaluation.md b/docs/rfcs/assets/2026-10-28-gq-3-grammar-evaluation.md index 71a267dcc..4686d7c22 100644 --- a/docs/rfcs/assets/2026-10-28-gq-3-grammar-evaluation.md +++ b/docs/rfcs/assets/2026-10-28-gq-3-grammar-evaluation.md @@ -13,7 +13,13 @@ grouping, rank profiles). **Status:** a design evaluation, not an RFC. Where it recommends, it ranks. **Outcome:** the [GQ 3 grammar RFC](../2026-10-28-gq-3-grammar.md), which cites this document as its evidence; §12 to §14 were added after that RFC -was drafted and are folded into it. +was drafted and are folded into it. The RFC is the decision: its freeze list, +rewriter table and compatibility list win wherever §9 or §11 differs, and +the passages below that once differed (the arrow spelling, a `contract` +declaration, an `F64?` score column, a swap-fix refusal of an incoming +triple, an HTTP contract bump) have been brought into line or marked as the +first draft's choice. §11.1 to §11.5 record the additive-first order the +project did not take; §11.6 is the order it took. --- @@ -2834,7 +2840,7 @@ canonical result. | `query` for reads and writes alike | `query` for reads, `mutation` for writes, `view` for derived relations, `batch` for several under one snapshot | agents trained on GraphQL write `mutation { }` (the skill's documented trap); lint classifies by keyword, not body | | `$p worksAt $c`, direction from the schema | `$p -worksAt-> $c`, `$c <-worksAt- $p`, `$a -knows- $b` undirected, `-$e:worksAt->` bound, `-(knows|likes){1,3}->` | direction visible to the reviewer and the model; `T5` becomes a reading error, not a schema lookup. **Withdrawn 2026-10-30 (§13.3): the triple stays; the arrowhead carries nothing position and schema do not** | | Ranking functions as the leading `order` key | a `retrieve { }` clause that binds typed score columns; `order` sorts any expression | 2.10; Vespa; Bruch; thresholds, weights, more arms and rerank become expressions | -| `rrf(a, b)` only | `rrf(...)`, `norm(...)`, weighted sums, `rerank(model, text) k` as a stage, `where $a.vec.distance < 0.4` | Bruch (+2–3 nDCG), BEIR (+11% rerank), LanceDB `distance_range` | +| `rrf(a, b)` only | `rrf(...)`, `norm(...)`, weighted sums, `rerank(model, text, k: n)` as a stage, `having { $a.vec < 0.4 }` | Bruch (+2–3 nDCG), BEIR (+11% rerank), LanceDB `distance_range` | | Blocks as predicates only, constant right side | blocks as values (`count { } as n`), any expression on the right | #1050, Cypher `COUNT {}`, Soufflé | | No optional | `optional { }` with pattern-bound filters and nullable bindings | #658, GQL `OPTIONAL MATCH` | | No post-group filter, no collection | `having { }`, `collect($x)` / `collect($x { a, b })` | R3 Rosetta row; Datomic pull; one nesting level | @@ -2887,7 +2893,7 @@ edge Supersedes: Assertion -> Assertion @card(0..1) view Contested($since: DateTime) @description("Assertions attacked since a date") { match { $a: Assertion - $r: Assertion -supports-> $a // reuse: a 'Reason' would be its own type in a real schema + $r: Assertion $r supports $a // reuse: a 'Reason' would be its own type in a real schema $r.confidence < 0.3 $r.asserted_at >= $since } return { $a as assertion, count($r) as attacks } @@ -2895,24 +2901,24 @@ view Contested($since: DateTime) @description("Assertions attacked since a date" // ---- a read: scope by graph, retrieve, fuse, filter on scores, project compactly query evidence($slug: String, $q: String, $k: I64 = 10, $at: Commit?) - @approximate(recall: 0.9) @budget(rows: 200000, ms: 2000) { + @approximate(recall = 0.9) @budget(rows = 200000, ms = 2000) { match { $c: Customer { slug: $slug } - $a: Assertion -about-> $c - not { $a -supersedes-> $_ } // only current claims + $a: Assertion $a about $c + not { $a supersedes $_ } // only current claims optional { $x: Contested(datetime("2026-01-01")) { assertion: $a } } } retrieve { - $a.vec: nearest($a.embedding, $q) k 200 + $a.vec: nearest($a.embedding, $q, k: 200) $a.lex: bm25($a.statement, $q) } return { $a { slug, statement, asserted_at }, $x.attacks, - count { $s -supports-> $a } as support, + count { $s supports $a } as support, 0.7 * norm($a.vec) + 0.3 * norm($a.lex) as score } - having { $a.vec.distance < 0.6 or $a.lex is not null } + having { $a.vec < 0.6 or $a.lex is not null } order { score desc } limit $k } @@ -2921,21 +2927,29 @@ query evidence($slug: String, $q: String, $k: I64 = 10, $at: Commit?) mutation retract($slug: String, $why: String, $head: Commit) @if_commit($head) { update $a: Assertion where { $a: Assertion { slug: $slug } } set { retracted_at: now(), note: $why } - invalidate $e where { $a: Assertion { slug: $slug } -$e:supports-> $_ } + invalidate $e where { $a: Assertion { slug: $slug } $a $e:supports $_ } returning { $a.@id, count($e) as detached } } // ---- speculation: the same statements as today, plus a batch under one snapshot batch explore @brief("which assertion types mention pricing") @phase(explore) { query types() { match { $t: @Type } return { $t.name, $t.instruction } } - query hits($q: String) @satisfice(ms: 300) { + query hits($q: String) @satisfice(ms = 300) { match { $a: Assertion } retrieve { $a.lex: bm25($a.statement, $q) } - return { $a.slug, $a.lex.score as s } order { s desc } limit 20 + return { $a.slug, $a.lex as s } order { s desc } limit 20 } } ``` +(The traversals are the triple the RFC kept; the first draft of this file +wrote them with arrows. Annotation arguments use `=`, the `.pg` spelling the +RFC fixed; call options use `:`. A score is the arm's column itself, +`$a.vec`, not a field of it. Several forms here are later productions, not +edition-3 text: arithmetic, the partial object `$a { … }`, `invalidate`, the +pattern-scoped `update`, `norm`, `k:` on an arm, and the `bm25` arm over +`@Type` in `types()` would be a slot refusal until the catalogue amendment.) + Reading the file against the requirements: R1 is the schema with `@example` and `@instruction` served through `@Type`; R2 is the diagnostic, `why_not` and `N…` channel the envelope carries; R3 is `optional`, the block as a value, @@ -2951,21 +2965,26 @@ line; R10 is the one file. ### 9.5 The retrieval clause, precisely -`retrieve { $b.name: fn(args) [k N] … }` declares one or more *arms*, each -over one binding's indexed property, each binding a typed score column -(`$a.vec.distance: F64?`, `$a.lex.score: F64?`, null when the row is outside -that arm's window). Semantics, unchanged in substance from today's contracts: +`retrieve { $b.name: fn(args [, k: n]) … }` declares one or more *arms*, each +over one binding's indexed property, each binding a typed score column. The +column is the one the ranked scan appends today, with today's type: `F32`, +non-null (`F64` for `rrf`), the squared L2 distance for `nearest` and the +BM25 score for `bm25`; a row outside an arm's window is absent, not null (the +first draft wrote `F64?` with nulls, which the RFC corrected to what the +engine publishes). Semantics, unchanged in substance from today's contracts: the `match` block's filters and traversals are applied before any arm scores (prefilter); a `nearest` arm is a top-`k` window over the eligible rows with the declared recall, a `bm25` arm is the full matching population; the retrieved population is the union of the arms' rows; `return`, `having` and `order` may use the score columns as ordinary values, so `norm`, `rrf`, a -weighted sum and a threshold are expressions, not grammar; `rerank(model, -$a.statement, k 50)` is a stage declared after the arms, run over the fused +weighted sum and a threshold are expressions, not grammar; `having` over a +score means the top `limit` rows of the population passing the predicate, +which is what the overfetch ladder already computes; `rerank(model, +$a.statement, k: 50)` is a stage declared after the arms, run over the fused top-k with a provider named in `cluster.yaml` the way embedders are; a retrieval forces a `limit`. The envelope reports per arm the recall contract and, under filters, the recall achieved, and per row the arm contributions -behind a fused score. A ranking that scopes by traversal (`$a -about-> $c` +behind a fused score. A ranking that scopes by traversal (`$a about $c` above) ranks only the reached rows, as today. ### 9.6 Writes, precisely @@ -2988,7 +3007,9 @@ for literals and parameters, `as commit`, `as unattached branch`. ### 9.7 The envelope, precisely Beside `columns`, `rows` and `graph_commit_id`: `retrievals` and `metrics` -(today's descriptors, plus `recall_achieved` per filtered arm); +(today's descriptors, plus `recall_achieved` per filtered arm, a field the +RFC does not reserve, since a new envelope field is additive and the +retrieval RFC that measures recall owns it); `notifications` (`N…` codes with the same four fields as refusals: a cross join, a destination-ranked scan of a whole type, a wildcard without a budget, a `bm25` arm without a cap); `why_not`, present when zero rows were returned @@ -3070,15 +3091,18 @@ envelope that costs nothing now and is expensive to retrofit. Most of §9.3 is additive and already in §4.3 and §7.7: blocks as values, `optional`, `having`, `collect`, views, the function registry, interface bindings, annotations, catalogue types, paging, `returning`, pattern-scoped -writes, notifications, `why_not`, editions. Only four items are incompatible -with stored text and are the reason to ask the question at all: the arrow -spelling for traversals (withdrawn on 2026-10-30, §13.3), `mutation` as a keyword, `retrieve` replacing ranking -functions in `order`, and `@valid`/`invalidate` as schema and verb. Under an -edition line all four ship as GQ 3 beside a GQ 2 the compiler keeps compiling -for one release, with `queries validate` printing the rewrite; the rewrite from -`$p worksAt $c` to `$p -worksAt-> $c` and from `order { nearest(…) }` to a -`retrieve` arm is mechanical, which is what the `suggestion` field with -`machine_applicable` edits was built for. +writes, notifications, `why_not`, editions. Only three items are incompatible +with stored text and are the reason to ask the question at all: `mutation` as +a keyword, `retrieve` replacing ranking functions in `order`, and the call +form for the infix string operators and the positional `k` (a fourth, the +arrow spelling for traversals, was withdrawn on 2026-10-30, §13.3; `@valid` +and `invalidate` are additive). Under an edition line the three ship as GQ 3 +beside a GQ 2 the compiler keeps compiling for one release, with `queries +migrate` printing the rewrite; the rewrite from `order { nearest(…) }` to a +`retrieve` arm, from `query` to `mutation` and from `x contains y` to +`contains(x, y)` is mechanical, which is what the `suggestion` field with +`machine_applicable` edits was built for, and the RFC proves each rewrite by +typed-IR equality. --- @@ -3483,10 +3507,10 @@ one pay. Sizes are relative (S, M, L), not dates. ├──► N envelope side channel (notifications, why_not, estimate) ────┤ │ ├──► P paging (limit $k, continuation token) ────────────────────────┤ ▼ │ ├──► X edition-3 spelling flip - ├──► C closure (blocks as values, having, collect, optional, views) ─┤ (arrows, mutation, retrieve - │ │ │ default, deprecations run - │ └──► A permissions (contracts as declarations │ out: search/fuzzy/match_text, - │ over views; actor views first scope) ──────────┤ infix contains, rrf positional k) + ├──► C closure (blocks as values, having, collect, optional, views) ─┤ (mutation, retrieve default, + │ │ │ deprecations run out: + │ └──► A permissions (views as the unit of │ infix contains, rrf positional k; + │ authority; actor views first scope) ───────────┤ the first draft also listed arrows) ├──► L lexical (@analyzed, terms, match_terms, bm25_v1) ─────────────┤ ├──► W1 writes (pattern-scoped, returning, @valid/invalidate) ────────┤ │ │ @@ -3519,13 +3543,13 @@ bounded union §7.6 wants. **A** waits for views (contracts bind to them). | **P Paging** (S) | `limit $k` (I64 parameter, non-null); continuation token in the envelope derived from the total order over the pinned snapshot; `--after ` on CLI, `after` on HTTP | total order (exists) | no | token round trip across a write (snapshot pinned, no skip, no repeat); two-key tie cases | stable pages without hand-written keyset predicates | | **F Function registry** (M) | generic `name(args, opt: v)` call rule; registry with signature, nullability rule, `can_fail`, `volatile`, pushdown mapping; the existing aggregates and search calls become rows; `contains()`/`starts_with()` land as the operator amendment specifies (infix kept, deprecated); first scalar library (`lower`, `upper`, `trim`, `length`, `coalesce`, date extraction, `case`); arithmetic with typed overflow refusal and the `can_fail` pushdown rule | E (for the infix deprecation) | no (infix stays one release) | `@Function` rows equal the compiler's registry (a test, not a doc); pushdown refuses to lift a fallible conjunct past its guard; GQT per function | derived values filter and order server-side; the end of client-side arithmetic | | **C Closure** (L) | blocks as values (`count { } as n`, `exists { } as b`), any expression on a block's right side; `having { }`; `collect($x)` / `collect($x { a, b })`; `optional { }` with pattern-bound filters and nullable bindings (`LeftExpand`); `view` declarations inlined at compile time, bindable as `$x: View(args)`, registered and validated with stored queries, listed in `@View`; interface *bindings* `$x: Named` as a union scan with the common projection (from the polymorphism RFC, no storage change); the Felleisen test applied to each as an acceptance criterion in the RFC | S (for `@View`), registry infrastructure | no | GQT per form; plan acceptance updated for `LeftExpand` and the projected block; a view invalidation case under `cluster plan`; the node-page task in M0 goes from N requests to 1 | the largest round-trip reduction in the plan; derived relations agents can author on a branch | -| **L Lexical** (M–L) | `@analyzed` profiles in SchemaIR; `terms()`, `match_terms`; exact scan as oracle; `bm25_v1`; `search`/`fuzzy`/`match_text` compile to `match_terms` with a deprecation under E | E; F (named options) | no (deprecation window) | the RFC's oracle fixtures and the three Lance guards; #747/#748/#749 cases green at every index state | text answers that do not depend on index state | +| **L Lexical** (M–L) | `@analyzed` profiles in SchemaIR; `terms()`, `match_terms`; exact scan as oracle; `bm25_v1`; `search`/`fuzzy`/`match_text` stay as registry rows and `queries fmt` canonicalises them to `match_terms` (the RFC's rule that a registry row is never removed; this file's first draft had them removed) | E | no | the RFC's oracle fixtures and its four Lance guards, re-qualified on the pinned Lance; #747/#748/#749 cases green at every index state | text answers that do not depend on index state | | **R Retrieve** (L) | `retrieve { }` clause binding typed score columns; arms with `k`; `norm()`, `rrf()` with n arms and weights, `rerank(model, text, k)` as a provider-backed stage; score columns usable in `having`/`order`/`return`; `@approximate(recall)` declaration; achieved recall in the envelope; 0048's laws, `exact: true`, `Vector(n, distance=…)` and recipes; rankings in `order` keep compiling for one release and lint prints the rewrite | E, F, 0047 (shipped) | no until X | the fusion and vector oracles 0048 names; GQT for windows, weights, thresholds, rerank, n arms; the discovery task in M0 answered by `rrf` over interface arms | hybrid search with weights, thresholds and a reranker; the end of two-query hybrids | | **W1 Writes** (M) | pattern-scoped `update $a … where { }` and `delete $e where { }` with one target type; `returning { }`; `@valid(from, to)` schema role and `invalidate`; `set { x: null }` in assignment position; bounded pattern-scoped `insert` | read planner under the write door (the shared expression RFC's "later RFC") | no | staging protocol unchanged (D2); GQT per form; affected receipts | invalidation as a verb; one round trip per write instead of read-then-write | | **W2 Computed writes** (S) | `set { visits: visits + 1 }` under `@if_commit` or a key equality; the read-set rule | F (arithmetic), W1 | no | contention case under DST; retry idempotence | the increment without a round trip | -| **X Edition 3** (M) | `mutation` keyword; arrow traversal spelling (`-e->`, `<-e-`, `-e-`; withdrawn 2026-10-30, §13.3); `retrieve` as the only ranking position; removal of `search`/`fuzzy`/`match_text`, infix `contains`/`starts_with`, positional `rrf` `k`; the GQ 2 compiler kept for one more release; `queries validate` prints every rewrite as a machine-applicable suggestion | every deprecation above having lived one release | **yes, under the edition** | every stored query in the fleet rewritten by the suggestion tool and re-validated; the 45-task instrument re-run on the new spelling | one spelling per operation; direction visible; the novelty tax paid once | +| **X Edition 3** (M) | `mutation` keyword; `retrieve` as the only ranking position; removal of infix `contains`/`starts_with` and positional `rrf` `k`; the GQ 2 compiler kept for one more release; `queries migrate` prints every rewrite as a machine-applicable suggestion (the first draft also flipped the traversal to arrows and removed `search`/`fuzzy`/`match_text`; both withdrawn, §13.3 and the RFC's registry rule) | every deprecation above having lived one release | **yes, under the edition** | every stored query in the fleet rewritten by the suggestion tool and re-validated; the 45-task instrument re-run on the new spelling | one spelling per operation; the novelty tax paid once | | **I + Poly** (L, own clock) | integer identity storage upgrade; polymorphic endpoints; unions `A \| B`; type tests; `@type_id`; interface-wide mutations; `rrf` over member arms as the interim cross-member search | storage upgrade RFC pair; C (interface bindings already shipped) | no (schema feature; old binaries refuse) | the RFC's prototype findings as regressions; upgrade journey tests | edge families collapse; discovery across types | -| **A Permissions** (L) | actor views first scope (type and column visibility, per-row placement, reads); `contract` declarations in GQ bound to views and stored queries; Cedar grants on named capabilities; the fragment = kernel minus retrieval | C (views), S, both RFCs amended per §10.5 | no | #951's differential oracle for the first scope; #1197's adversarial matrix | query-only agents; shared graphs with audiences | +| **A Permissions** (L) | actor views first scope (type and column visibility, per-row placement, reads); a permissions declaration in GQ bound to views and stored queries (the first draft called it `contract`; the RFC reserves no permissions declaration, since a file-level keyword is additive whenever the model is accepted); Cedar grants on named capabilities; the fragment #1197 defines (one concrete binding, scalar filters, projections, optional group, count, order and a literal limit) | C (views), S, both RFCs amended per §10.5 | no | #951's differential oracle for the first scope; #1197's adversarial matrix | query-only agents; shared graphs with audiences | | **Fed Federation** (L, later) | graph qualifier in type references; per-graph commit pins in the envelope; views over a foreign graph; change-feed-as-load for shares | E/S reservation; C (views); Poly (one-corpus search) | no | nothing until designed | cross-graph reads; shares kept current | ### 11.3 Three parallel tracks @@ -3614,20 +3638,20 @@ not on it is additive forever after, by construction. | Frozen in the break | Decision | Semantics at break time | |---|---|---| -| Declaration kinds | `query`, `mutation`, `view`, `contract`, `batch`, plus the schema kinds `node`, `edge`, `interface` in the same grammar; one file type, `.pg` accepted as the same grammar | `view`, `contract`, `batch` parse and are refused with a stable code ("reserved; not yet served") until their stage | -| Clause set and order | `match` → `retrieve`? → `return` → `having`? → `order`? → `limit`? → `returning`? (mutations: statements → `returning`?) | `retrieve` carries exactly today's power (`nearest`, `bm25`, two-arm `rrf`); `having` works immediately (one `Filter` over `Aggregate`); `returning` parses, refused until W1 | +| Declaration kinds | `query`, `mutation`, `view`, `batch`, plus the schema kinds `node`, `edge`, `interface` in the same grammar; one file type, `.pg` accepted as the same grammar (the first draft also reserved `contract`; the RFC removed it on 2026-10-29) | `view` and `batch` parse and are refused with a stable code ("reserved; not yet served") until their stage | +| Clause set and order | `match` → `retrieve`? → `return` → `having`? → `order`? → `limit`? (mutations: statements → `returning`?) | `retrieve` carries exactly today's power (`nearest`, `bm25`, two-arm `rrf`) with today's `F32` columns; `having` works immediately over aggregates, group keys and score columns, a score threshold meaning the top `limit` passing rows; `returning` parses, refused until W1 | | Block kinds in `match` | `not`, `exists`, `count`/`sum`/… blocks, `optional` | `optional` parses, refused until C | | Traversal spelling | the triple stays (`$a e $b`, ``); the first draft's arrow forms were withdrawn on 2026-10-30 (§13.3) | today's semantics | -| Call syntax | `name(args, opt: v)`, generic, resolved against a registry; `contains()`, `starts_with()`, `terms()` reserved as names; no function in the grammar | today's functions become registry rows; `terms()` refused until L | +| Call syntax | `name(args, opt: v)`, generic, resolved against a registry; `contains()`, `starts_with()` registry rows; no function in the grammar; a registry row is never removed | today's functions become registry rows (`search`, `fuzzy`, `match_text` included, for good); `terms()` and `match_terms()` are rows L adds, no slot needed | | Reserved words | the logic set only: `and`, `or`, `not`, `is`, `null`, `in`; every clause and declaration keyword contextual (word-bounded, position-based, as today) | closed; the amendment's rule becomes a grammar test | | Null | no literal in reads; `null` legal in assignment position only | today's rules plus the assignment case | -| Parameters | `limit $k`; `$branch: Branch`, `$at: Commit` typed parameter kinds; defaults `($k: I64 = 10)` | `limit $k` works; `Branch`/`Commit` kinds parse and bind | -| Annotations | `@ident(args)` on any declaration, retained as metadata; engine-known names typed (`@budget`, `@approximate`, `@satisfice`, `@if_commit`, `@example`, `@description`, `@instruction`) | known ones that are stubs refuse with a code | -| Meta-fields and catalogue names | `@id`, `@src`, `@dst`, `@type`, `@type_id`, `@distance`, `@score` reserved; `@Type`, `@Property`, `@Edge`, `@Interface`, `@Index`, `@Function`, `@View` names reserved | `@distance`/`@score` are the `retrieve` columns; the catalogue types ship in S | +| Parameters | `limit $k`; `$branch: Branch`, `$at: Commit` typed parameter kinds; defaults `($k: I64 = 10)` | `limit $k` works; `Branch`/`Commit` kinds parse and are slots for the branch-statements RFC | +| Annotations | `@ident(args)` on any declaration, retained as metadata; arguments positional or `name = value`; `@description` and `@instruction` typed; `@budget`, `@approximate`, `@satisfice`, `@if_commit`, `@example`, `@brief`, `@phase` reserved | the reserved ones refuse with a code naming their owner | +| Meta-fields and catalogue names | `@id`, `@src`, `@dst`, `@type`, `@type_id` reserved (the first draft also had `@distance` and `@score`; the arm's column is the score, so they were dropped); `@Type`, `@Property`, `@Edge`, `@Interface`, `@Index`, `@Function`, `@View` names and the `@`-prefixed catalogue edge reserved | the catalogue types ship in S; a `retrieve` over them waits for the catalogue amendment | | Type references | `[graph@]ns::Type` admitted by the grammar | the graph segment refused ("federation reserved") | | Edition line | `gq 3;` optional first line | the compiler is edition 3; edition 2 parses for one release through the rewriter | | Schema roles | `@valid(from, to)`, `Vector(n, distance=…)`, `@analyzed(...)`, `@embed` recipe arguments | `@valid` and `@analyzed` are SchemaIR features whose enforcement arrives with W1 and L; distance ships | -| Envelope | `graph_commit_id` as a map keyed by graph (one entry today), `notifications`, `why_not`, `complete`, `continuation`, `retrievals.recall_achieved`, `estimate` reserved | present and empty until their stage; HTTP contract 0.14 → 0.15, explain and bound-plan versions bumped once | +| Envelope | `graph_commits` as a map keyed by graph (one entry today) beside `graph_commit_id`, `notifications`, `why_not`, `complete`, `incomplete_reason`, `continuation`, `estimate` reserved (`recall_achieved`, in the first draft, is left to the retrieval RFC as an additive field) | present where their stage has shipped, absent otherwise; no HTTP contract bump (the first draft wrote 0.14 → 0.15); explain and bound-plan versions bumped once | Two things are deliberately *not* on the list, because they are additive by nature: new registry rows (functions, catalogue types, annotations) and new @@ -3658,9 +3682,10 @@ and repair turns on the frozen grammar) rather than on taste. deliverables: the grammar and type rules of the freeze list; a **rewriter** (`omnigraph queries migrate`, the machine-applicable `suggestion` machinery turned into a tool) that rewrites every edition-2 source mechanically -(`$p worksAt $c` → `$p -worksAt-> $c`, `order { nearest(…) }` → a `retrieve` -arm, `query` writes → `mutation`, infix `contains` → `contains()`); the GQT -corpus rewritten by that tool and re-run as the proof that the rewrite is +(`order { nearest(…) }` → a `retrieve` arm, `query` writes → `mutation`, +infix `contains` → `contains()`, positional `k` → `k:`; the traversal is +untouched); the GQT corpus rewritten by that tool and re-run, with typed-IR +equality per declaration, as the proof that the rewrite is semantics-preserving; and the fleet's stored queries rewritten under `cluster plan` with the same tool. Edition 2 keeps parsing for that one release so a stored query that escaped the sweep is rewritten at boot with a warning rather @@ -4010,20 +4035,25 @@ one-spelling test of §14 reversed that on 2026-10-30. Four rules remain. `$a -subsidiaryOf-> $b`. It is a decoration on the one spelling, and its two justifications did not survive: `$p worksAt $c` reads in source-to-target order as a sentence does, and the only in-context - measurement GQ has is on the triple (RFC 0047: 45 of 45, 37 first tries - valid, no direction error) while CypherBench shows the arrow form does not - prevent reversals. Dropping it also removes the break's largest rewrite, - since every stored query has a traversal, and the arithmetic-minus rule. - The arrow forms stay in the grammar as a recognizer only: `$p -worksAt-> - $c`, `$c <-worksAt- $p` and `$a -knows- $b` are parse errors whose fix - prints the triple with the ends in schema order. -2. **A triple written against the schema is a typed error with a - machine-applicable swap**, never an empty result and never a silent - rewrite. `$c: Company worksAt $p` is `T5` with `fix` = `$p worksAt $c`. - This is LangChain's corrector moved into the compiler, where it belongs, - with the two differences the evidence asks for: it is loud (the agent - learns in one turn, which Olausson's result in §8.3 says is worth more - than the edit) and it never guesses when both orientations fit. + measurement GQ has is on the triple (RFC 0047: 45 of 45 correct, 8 of 45 + first attempts refused, all for a missing `()`; it reports no direction + result) while CypherBench shows the arrow form does not prevent reversals + (9 of 55 gpt-4o error marks). Dropping it also removes the break's + largest rewrite, since every stored query has a traversal, and the + arithmetic-minus rule. The arrow forms stay in the grammar as a recognizer + only: `$p -worksAt-> $c`, `$c <-worksAt- $p` and `$a -knows- $b` are parse + errors whose fix prints the triple in the written order. +2. **Both written orders of a directed triple stay legal, and the formatter + writes the schema's.** `$c: Company worksAt $p` typechecks today as the + incoming orientation, and the corpus writes it (issue 659's cases). The + first draft of this section made it `T5` with a swap as the fix, + LangChain's corrector moved into the compiler; the RFC's review found that + a break the compatibility list had hidden, and that the type checker + already resolves the orientation, so no correction is needed: `queries + fmt` rewrites the clause to schema order and lint names it, and `T5` keeps + today's meaning (an edge whose declared endpoints fit neither order). + Loud and never guessing, which Olausson's result in §8.3 says matters, + are properties the resolved triple has for free. 3. **Same-type edges are where direction is meaning, and position carries it.** `$a subsidiaryOf $b` means `$a` is the subsidiary. No rule can check that against the schema; what GQ can do is make the schema say which end @@ -4033,7 +4063,7 @@ one-spelling test of §14 reversed that on 2026-10-30. Four rules remain. 4. **`` means "whichever orientation the endpoint types admit".** For a same-type edge that is both orientations, deduplicated, as today. For an alternation or wildcard over edges of differing orientation - (`$p: Person $d: Doc`, the issue 659 case, which the + (`$p: Person $d: Doc $p $d`, the issue 659 case, which the directed triple resolves per member today) it is each member's declared orientation. The only refusal left is the genuinely ambiguous one: an asymmetric edge whose endpoints are not typed well enough to fix the @@ -4090,7 +4120,8 @@ Every pair of forms in GQ 3 that could mean the same thing, and the ruling. | Pair | Same meaning? | Ruling | |---|---|---| -| `$a knows $b` and a reversed arrow `$b <-knows- $a` | Yes, exactly | Grammar: the triple only, source on the left; every arrow form is a parse error with the swapped triple as its fix (§13.3) | +| `$a knows $b` and a reversed arrow `$b <-knows- $a` | Yes, exactly | Grammar: the triple only; every arrow form is a parse error with the triple as its fix (§13.3) | +| `$p worksAt $c` and `$c worksAt $p` under `Person -> Company` | Yes: the schema fixes the orientation, and the type checker resolves both | Both parse and typecheck, as in edition 2; `queries fmt` writes the schema's order and lint names it. Refusing one order was the first draft's `T5` swap, a hidden break | | `$a knows $b` (triple) and `$a -knows-> $b` | Yes, and the arrowhead adds nothing | Grammar: triple only, decided 2026-10-30 (§13.3); the first draft had chosen the arrow | | `x contains y` and `contains(x, y)` | Yes | Grammar: the call only; already in the RFC | | `rrf(a, b, 60)` and `rrf(a, b, k: 60)` | Yes | Grammar: named option only; already in the RFC | From 667642b137d52af72642f50c46d46f5269a0fb11 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Sun, 1 Nov 2026 21:38:04 -0800 Subject: [PATCH 11/13] docs(rfc): pattern-scoped writes are new productions, not slots --- docs/rfcs/2026-10-28-gq-3-grammar.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/rfcs/2026-10-28-gq-3-grammar.md b/docs/rfcs/2026-10-28-gq-3-grammar.md index b9632d78d..f54dbde73 100644 --- a/docs/rfcs/2026-10-28-gq-3-grammar.md +++ b/docs/rfcs/2026-10-28-gq-3-grammar.md @@ -116,9 +116,11 @@ responsible for. functions, one annotation form, one diagnostic contract, so a change has one place to land and one surface to document ([Design](#design)). - **Expressive.** The forms the open gaps need, derived relations, optional - patterns, block aggregates as values, retrieval as typed columns, post-group - filters and pattern-scoped writes, have slots whose semantics the closure, - retrieval and write RFCs fill without touching the grammar. + patterns, block aggregates as values and retrieval as typed columns, have + slots whose semantics the closure and retrieval RFCs fill without touching + the grammar; post-group filters ship in the break; pattern-scoped writes + are new productions the writes RFC adds, additive because they start at a + token edition 3 refuses. - **Extensible.** A registry row or a declaration is something an agent can author against a graph; a new function, view or annotation then extends the language without a compiler release, which is the property that matters as From 174fb239b96a9145c1c6ec223a382511a921a5e6 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Sun, 1 Nov 2026 23:20:48 -0800 Subject: [PATCH 12/13] rfc(gq3): an arm column is a value of its binding for the rest of the read, except under rrf --- docs/rfcs/2026-10-28-gq-3-grammar.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/rfcs/2026-10-28-gq-3-grammar.md b/docs/rfcs/2026-10-28-gq-3-grammar.md index f54dbde73..6863b5ccf 100644 --- a/docs/rfcs/2026-10-28-gq-3-grammar.md +++ b/docs/rfcs/2026-10-28-gq-3-grammar.md @@ -339,10 +339,12 @@ forms `order` accepts today for `nearest(…)`, `bm25(…)` and `rrf(…)`; `asc arm; a ranking call written in `order` or `return` is `T65`, whose fix is the arm. `return` may project an arm as a value (`$d.lex as score`), which is today's `T33` projection without the repetition rule, since the arm is named -once. Under `rrf` an arm's own column is not a value (`T33`, as today): the -fusion carries only the fused column, and RFC 0048 owns carrying each arm's -column through `RankFuse`. A filter in `match` may not read an arm (arms are -bound after `match`). +once. Outside `rrf`, an arm's column is a value of its binding for the rest +of the read: `return`, `having` and `order` may read it, and a later traversal +from that binding does not drop it. Under `rrf` an arm's own column is not a +value (`T33`, as today): the fusion carries only the fused column, and RFC +0048 owns carrying each arm's column through `RankFuse`. A filter in `match` +may not read an arm (arms are bound after `match`). `having` may read an arm, and `having { $d.vec < 0.6 }` is the one new capability this release gives retrieval. Its meaning is fixed here: the query From b48c9a5a70e9f8a726a846ad026a1d960dbc4d37 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Mon, 2 Nov 2026 12:54:30 -0800 Subject: [PATCH 13/13] docs(rfc): the GQ 3 syntax by example, one worked form per freeze-list row --- docs/rfcs/2026-10-28-gq-3-grammar.md | 384 ++++++++++++++++++++++++++- 1 file changed, 383 insertions(+), 1 deletion(-) diff --git a/docs/rfcs/2026-10-28-gq-3-grammar.md b/docs/rfcs/2026-10-28-gq-3-grammar.md index 6863b5ccf..d503ba5c0 100644 --- a/docs/rfcs/2026-10-28-gq-3-grammar.md +++ b/docs/rfcs/2026-10-28-gq-3-grammar.md @@ -378,6 +378,387 @@ exact fragment is a closed, versioned contract: a read with `having` or `limit $k` is outside it in this release, and RFC 0048, which owns the fragment's extension, either admits the two with a derivation or keeps them out. +### The syntax by example + +Every form of edition 3, one example each, against one schema. "Works" +forms run in the edition-3 release with the meaning given; "slot" forms parse +and refuse with their owner; "refused" forms are the respellings the +diagnostics name, with the fix the compiler prints. The examples are the +parser-test fixtures of the Evidence section, so a form that is not here is +not in the grammar. + +#### The file and the schema + +```gq +gq 3; // optional; absent means the compiler's edition +set ann_nprobes = 40; // settings lines come first, as today + +interface Named { name: String } + +node Person implements Named + @instruction("One row per person, keyed by handle") { + handle: String @key // identity; insert upserts on it + name: String @index // full-text index; `bm25` and `search` need it + age: I32? // nullable + tags: [String] // a list of scalars + joined: Date + bio: String? @index + @range(age, 0..150) // body constraint, exact comparison +} + +node Company implements Named { + slug: String @key + name: String + tier: enum(standard, strategic) @index // an enum takes no full-text index +} + +node Doc @description("A document with an embedded body") { + path: String @key + title: String @index + body: String @index + embedding: Vector(1536)? @embed("body", model = "openai/text-embedding-3-large") @index + published: DateTime + bytes: Blob? +} + +node Collection { slug: String @key } + +edge WorksAt: Person -> Company @card(0..1) { // at most one employer per person + since: Date + @unique(@src) +} +edge Employs: Company -> Person // the opposed orientation, kept for an older load +edge Knows: Person -> Person { weight: F64? } +edge Mentors: Person -> Person +edge Authored: Person -> Doc @key(@src, @dst) // identity from the endpoints +edge Cites: Doc -> Doc +edge In: Doc -> Collection // an edge a reserved word names; quoted in a traversal +``` + +One grammar: this file is a `.pg` when it holds only schema declarations +and a `.gq` when it holds only query, mutation, view or batch declarations; +a file holding both is a slot. Annotation arguments are positional or +`name = value`. `@card`, `@key`, `@unique`, `@index`, `@range` and `@check` +keep their meanings and spellings; nothing a `.pg` file accepted is refused. + +#### A read: bindings, filters, operators + +```gq +query adults($min: I32 = 30, $handles: [String], $cutoff: Date?) + @description("People over a given age among the given handles") + @instruction("Pass `$cutoff` to keep only people who joined before it") { + match { + $p: Person { name: "Alice" } // binding with a property match (equality) + $p.age >= $min // comparison: = != < <= > >= + $p.age is not null // null test; a null comparison is null, and a filter keeps true rows + "rust" in $p.tags // membership in a list property + $p.handle in $handles // membership in a list parameter + $p.handle in ["alice", "bob"] // membership in a list literal + $p.joined < $cutoff // a nullable parameter: a null value makes the comparison null, so no row passes + contains($p.name, "Al") // exact substring (a call, not an operator) + starts_with($p.name, "A") // exact prefix + $p.joined >= date("2025-01-01") // a call folded to a literal at compile time + $p.joined <= now() // the request's clock + ($p.age > 60 or $p.age < 20) and not $p.name = "Bob" // `or` loosest, then `and`, then `not` + } + return { $p.handle, $p.name, $p.age as years } + order { years desc, $p.handle } // any projected value or alias; ties by the next key + limit 20 +} +``` + +A parameter is `$name: Type`, nullable with `?`, with an optional literal +default. The six reserved words are `and`, `or`, `not`, `is`, `null`, `in`; +every other keyword is contextual, so `$p.order`, `$p.limit` and an edge +named `match` are legal. There is no `null` literal in a read: `$p.age = +null` is a parse error, `is null` is the test. + +#### Traversals + +```gq +query graph_shapes($h: String) { + match { + $p: Person { handle: $h } + $p worksAt $c // directed, source on the left; `$c` is bound by the edge's endpoint type + $c: Company // the same variable bound again adds a constraint, not a second row + $c worksAt $boss // legal: written against the schema, resolved as the incoming orientation; + // `queries fmt` rewrites it to `$boss worksAt $c` + $p $f // undirected, either orientation, deduplicated + $p $k:knows $g // the edge row bound as `$k`; one row per matching edge + $k.weight > 0.5 // an edge property + $p (knows | mentors) $x // alternation, each member in its declared orientation + $p $firm // undirected alternation; opposed members resolve per member (new in edition 3) + $p *{1,2} $near // any edge type, one or two hops, shortest-path distance + $p knows{2,2} $fof // exactly two hops + $d: Doc $d "in" $coll // a quoted edge name for an edge a reserved word names + } + return { $f.handle, $k.@src, $k.@dst, $k.@type, $near.@id, $p } // meta-fields; a bare node is one object column +} +``` + +Bounds are finite and need a same-type edge above one hop. A bound edge +cannot be projected bare (`T23`), a hop-bounded edge cannot be bound, and an +anonymous `$_` is a fresh node each time. `$e.@src` and `$e.@dst` report the +stored orientation whatever order the triple was written in, which is why +both stay. + +#### Blocks + +```gq +query by_blocks($h: String, $least: I64) { + match { + $p: Person { handle: $h } + not { $p worksAt $_ } // no matching row + exists { $p authored $d $d.published > date("2026-01-01") } // at least one + count { $p knows $f } >= $least // the aggregate of the block's matches, compared with a scalar + sum($k.weight) { $p $k:knows $_ } > 1.5 // sum, avg, min, max over a block property + } + return { $p.handle } +} +``` + +A block runs once per outer row and must read an outer variable. The right +side is a literal, `now()` or a parameter. `count { P } > 0` and `count { P } += 0` are legal and `queries fmt` writes them as `exists { P }` and `not { P }`. + +#### Return, aggregates, distinct, paging + +```gq +query colleagues($k: I64 = 10) { + match { $p: Person $p worksAt $c $p knows $f } + return { + $c.name, // a group key: every non-aggregate projection groups + count($f) as friends, + count(distinct $f.age) as ages, // distinct under `count` only; `sum(distinct …)` is `T57` + avg($f.age) as mean_age, + min($p.joined) as first_joined, + count($f) > 3 as busy // a comparison in `return` needs an alias + } + having { friends >= 2 and $c.name != "Acme" } // over aggregates and group keys; `T58` keeps refusing this mix in `return` + order { friends desc, $c.name } + limit $k // a non-null `I64` parameter; null or negative refuses at bind +} + +query names() { + match { $p: Person } + return distinct { $p.name } // one row per distinct projected tuple, before `order` and `limit` +} +``` + +`sum` over integers accumulates exactly and rounds once to `F64`; an +all-aggregate query over zero rows returns one row. Each result column needs +a distinct name (`T25`). + +#### Retrieval + +```gq +query hybrid($vq: Vector(1536), $tq: String, $slug: String, $k: I64 = 10) { + match { + $d: Doc // the ranked binding + $a: Person $a authored $d // scope first: only this author's documents are candidates + $a worksAt $c $c: Company { slug: $slug } + $d.published > date("2025-01-01") // a filter in `match` is applied before any arm scores + } + retrieve { + $d.vec: nearest($d.embedding, $vq) // F32 squared L2 distance; `$vq` may be a String the provider embeds + $d.lex: bm25($d.body, $tq) // F32 BM25 score; the property needs a full-text index (`T27`) + } + return { $d.path, $d.title, rrf($d.vec, $d.lex, k: 60) as score } // the fused score, F64, projected as today + order { rrf($d.vec, $d.lex, k: 60) } // leads with an arm or `rrf` over arms + limit $k +} + +query nearby($vq: Vector(1536), $k: I64 = 10) { + match { $d: Doc } + retrieve { $d.vec: nearest($d.embedding, $vq) } + return { $d.path, $d.vec as distance } // an arm is a value + having { $d.vec < 0.6 } // the top `limit` rows of the population passing the threshold + order { $d.vec } // a `nearest` arm takes no direction + limit $k +} + +query one_arm($q: String) { + match { $d: Doc } + retrieve { $d.lex: bm25($d.title, $q) } + return { $d.title, $d.lex } // unaliased: the result column is `d.lex` + order { $d.lex desc } // `desc` is accepted on a score; `asc` is `T54` + limit 5 +} + +query text_predicates($q: String) { + match { + $d: Doc + search($d.title, $q) // full-text filter, no ranking, no `limit` needed + fuzzy($d.title, $q, max_edits: 2) // the third argument is a named option + match_text($d.body, $q) + } + return { $d.path } +} +``` + +A `nearest` arm requires a `limit` (`T17`) and its window is widened by the +overfetch ladder as today; a `bm25` arm is the whole matching population; an +`rrf` is the union of its arms. A filter in `match` cannot read an arm; under +`rrf` an arm's own column is not a value in `return` or `having` (`T33`, which +is why `hybrid` projects the fused score and `nearby` filters on its lone +arm); a retrieval under an +aggregate is refused (`T32`); two arms with the same call over one property, +arms over two bindings, and more than two `rrf` arms are refused in this +release (RFC 0048's rules). `search`, `fuzzy` and `match_text` stand alone or +`= true`, joined by `and`; negate one with `not { search(…) }`. + +#### Writes + +```gq +mutation hire($h: String, $slug: String, $since: Date) + @description("Employ a person; upserts on the person's handle") { + insert Person { handle: $h, name: $h, tags: [], joined: $since } // keyed insert is an upsert + insert WorksAt { from: $h, to: $slug, since: $since } // endpoints by key value; `from`/`to` are reserved + update Person set { bio: null } where handle = $h // `null` is legal as an assignment value +} + +mutation retire($h: String) { + delete WorksAt where @src = $h // meta-fields in a mutation predicate + delete Person where handle = $h +} + +mutation propose($h: String) as commit { // a publishable proposal, not a branch write + update Person set { name: "Proposed" } where handle = $h +} + +mutation sketch($h: String, $base: String) as unattached branch on $base { // a nonpublishable snapshot + insert Person { handle: $h, name: $h, tags: [], joined: date("2026-01-01") } +} +``` + +A mutation is constructive (inserts and updates) or destructive (deletes), +never both; an edge cannot be updated (`T16`); a `@key` property cannot be +assigned. These are today's rules under the new keyword. + +#### Statements + +```gq +branch create "review/hire" from main +branch create fix from commit "01J…" +branch merge "review/hire" into main +branch merge "review/hire" into main dry run +branch merge "review/hire" into main as commit +branch merge-base "review/hire" into main +branch diff "review/hire" into main after page "…" limit 100 +branch list +branch delete "review/hire" +commit publish "01J…" +commit drop "01J…" +load csv "s3://imports/person.csv" into node:Person mode merge delimiter "|" header +load csv "s3://imports/knows.csv" into edge:Knows mode append columns (from, to, weight) +explain query adults() { match { $p: Person } return { $p.handle } } +explain load csv "s3://imports/person.csv" into node:Person mode append header +show all; +reset ann_nprobes; +``` + +A file holds one statement or declarations, never both, as today; a statement +file is run with `mutate` (writes) or `query` (reads) and is refused by the +stored-query registry. + +#### The catalogue + +```gq +query operators() { + match { $o: @Operator } // the five engine catalogue types exist and work + return { $o.name, $o.inputs } +} + +query schema_types() { // slot: `@Type` and the `@property` edge + match { $t: @Type $p: @Property $t @property $p } + return { $t.name, $p.name } +} +``` + +A catalogue type is bound, traversed, filtered and counted like a node type; +it is never written, loaded or rank-searched (the catalogue RFC's rule), so +`retrieve { $t.lex: bm25($t.instruction, $q) }` over `@Type` is refused with +the slot until the catalogue amendment names its mechanism. + +#### Slots: forms that parse and refuse with their owner + +```gq +view Contested($since: DateTime) { // `T6x`: derived relations RFC + match { $a: Doc $r: Doc $r cites $a $r.published >= $since } + return { $a as doc, count($r) as attacks } +} + +query uses_slots($h: String, $c: Commit, $b: Branch) { // `Commit` and `Branch` kinds: branch-statements RFC + match { + $p: Person { handle: $h } + optional { $p worksAt $co } // outer join: derived relations RFC + $x: Contested(datetime("2026-01-01")) { doc: $d } // a view binding, `(` touching the name: derived relations RFC + } + return { + $p.handle, + count { $p knows $_ } as knows, // a block as a value: derived relations RFC + $p.@type_id // polymorphic types RFC + } +} + +mutation with_returning($h: String, $c: String) @if_commit($c) { // `@if_commit`: writes RFC + delete Person where handle = $h + returning { @id } // writes RFC +} + +batch explore @brief("which types mention pricing") @phase(explore) { // `batch` and its annotations: agent-first batch + query a() @budget(rows = 1000) { match { $p: Person } return { $p.handle } } + query b() @approximate(recall = 0.9) @satisfice(ms = 300) { match { $p: Person } return { $p.name } } +} + +query namespaced() { + match { $o: crm::Operator } // a namespace segment: schema namespaces RFC + return { $o.name } +} + +node Event @valid(asserted_at, retracted_at) { // `@valid`: writes RFC + id: String @key + asserted_at: DateTime + retracted_at: DateTime? + text: String @analyzed(analyzer = "english") // `@analyzed`: lexical RFC +} +``` + +Each refusal's `expected` names the construct and its `fix` names the owning +RFC ("not yet served in this release"). An `expect error` GQT case per slot +is written at the break; the owning RFC flips it. + +#### Refused respellings, and the fix each prints + +| Written | Code | The fix the diagnostic prints | +|---|---|---| +| `query retract($h: String) { delete Person where handle = $h }` | type error | `mutation retract(…) { … }` | +| `$p.name contains "Al"`, `$p.name starts_with "A"` | parse error | `contains($p.name, "Al")`, `starts_with($p.name, "A")` | +| `rrf($d.vec, $d.lex, 60)`, `fuzzy($d.title, $q, 2)` | type error | `k: 60`, `max_edits: 2` | +| `order { nearest($d.embedding, $q) }`, `return { bm25($d.body, $q) as s }` | `T65` | a `retrieve` arm and its name in `order` or `return` | +| `$p -worksAt-> $c`, `$c <-worksAt- $p`, `$a -knows- $b` | parse error | `$p worksAt $c`, `$c worksAt $p`, `$a $b` (the written order, computed without the schema) | +| `gq 4;` | `Q00x` | "this release compiles editions 2 and 3" | +| `gq 3;` after a declaration | `Q00x` | the line moved to the top of the file | +| `$p.age = null`, `$p: Person { age: null }` | parse error | `$p.age is null` | +| `return { $p.name as in }` | parse error | another alias; the six reserved words name the collision | +| `match { … $d.vec < 0.6 … }` with `$d.vec` an arm | type error | the predicate moved to `having` | +| `having { $p.name }` (not Boolean), `having { $f.age > 3 }` (neither aggregate, key nor arm) | `T67` | the rule's three admitted operands | +| two `nearest` arms over `$d.embedding`; arms on `$d` and `$p`; an arm beside `count($f)` | type error | RFC 0048's reason | +| `$p: Person $c: Company` | parse error | bindings first, then `$p $c` | + +#### Formatter rewrites (`queries fmt`), named by lint + +| Written | Canonical | +|---|---| +| `count { $p knows $_ } > 0` | `exists { $p knows $_ }` | +| `count { $p knows $_ } = 0` | `not { $p knows $_ }` | +| `$c worksAt $p` under `edge WorksAt: Person -> Company` | `$p worksAt $c` | +| `$x.@type = "Person"` (once `is` lands) | `$x is Person` | +| `search($d.title, $q)` (once `match_terms` lands) | the `match_terms` call it equals | + +A formatter rewrite never changes the typed IR; the rewriter's edits above +do, which is the line between the two tools. + ### The rewriter `omnigraph queries migrate` takes `.gq` and `.pg` files or a `--target`, parses @@ -537,7 +918,8 @@ cannot forbid. Concretely: plan for it. A formatter rewrite never changes the typed IR; that is what separates it from the rewriter's break. - Forms that look alike but differ in meaning stay: `return { $a.slug }` - (a column) and `return { $a { slug } }` (one object); `$e.@src` (stored + (a column) and `return { $a }` (one object; the partial object + `$a { slug }` is a later production); `$e.@src` (stored orientation) and the pattern's left end (written orientation); `limit 10` and `limit $k`; `count(distinct $x)` and `return distinct`; `having` and a `match` filter.