feat(core): answer store reads through their index, under the pinned schema's access

[nickruigrok]

Third of five stacked PRs for GRA-243. Security: needs human review before merge. Stacked on #386 and #388; this PR's own changes are the last three commits.

Threat model (written first; also in apps/core/src/store-queries.ts)

Who reads: App code, through core, which resolves the caller and hands the store a ReadScope. The descriptor is whatever App code built and is trusted for nothing.

Threat	How it's closed
A caller reads records it may not see, through any terminal	The read mode comes from the store's own record of the pinned schema (sdk_schemas), never from the request. none refuses. appMembers/appBuilders refuse callers core didn't find to have the role (data.access_denied). owner puts owner_id = principal in the one statement every terminal runs, so counts, aggregates, unique and results cover the caller's records only. The owner leads the index, so other owners' records aren't even scanned. Nothing is filtered after the fact.
A record ID discloses another table's or owner's record	get looks a record up by table and ID under the same predicate and answers null in every case. expectedRevision is checked only on a visible record.
A descriptor smuggles SQL or names what it shouldn't	Names resolve through the pinned manifest. Table IDs and index names come from the store's catalog and planner. Field names reach SQL only through the shared emitter. Values are bound.
A query scans without bound	INDEXED BY the named index, so SQLite fails rather than scans. The range is checked against the index's fields in order. At most 10,000 entries and 16 MiB of documents per terminal, then data.scan_limit, never a partial answer. collect fails past 100 (data.result_limit); unique past one (data.not_unique).
A descriptor overruns SQLite	Filter depth, nodes and lists, plus 64 bound values, are checked at the boundary (#386). The emitter's SQL is flat (#388).
A condition drops a record through SQL NULL	Conditions are two-valued (#388), so NOT keeps null and missing.
A pinned consumer sees fields it doesn't know, or misses defaults	Documents are projected to the pinned schema at every depth: objects keep only declared keys, with defaults; records, arrays and unions project what they hold (spec 32.7).
An aggregate overflows	A non-finite result fails with data.aggregate_overflow.
A read creates state, or a missing index makes it scan	A read refuses a store with no row (data.unknown_schema) without creating one. An index SQLite doesn't hold is data.index_required.
A caller claims a schema or a role	Both come from core (ReadScope). An unknown schema hash is data.unknown_schema.

What this PR trusts, and who answers for it

• ReadScope is taken as given. That covers the principal, roles, schema hash and consumer. Only core holds the store binding, and core must build the scope per invocation from its own records. This is a hard requirement on GRA-244.
• An older schema hash keeps its own access mode. If a table goes from appMembers to owner in a newer schema, a read under the older hash still lets members read it. The security review probed this: v1 appMembers, v2 owner, and a read under v1's hash returns another owner's records. It is closed by two hard requirements: GRA-244 takes the hash from the consumer's own pinned manifest, and GRA-246 retires superseded hashes once nothing is pinned to them. No code change here.

What

• StoreHost.read(storeId, scope, request), exposed as DataStore.read over RPC and OpenStore.read from core, with readScopeSchema in @grasp-os/shared/store-queries.
• apps/core/src/store-queries.ts:
  • access resolution;
  • range-to-index checking;
  • planQuery, one statement with the filter as a computed column, ordered by the index columns past the equality prefix and limited to one entry past the scan bound;
  • a bounded match iterator;
  • every terminal except paginate (PR 4);
  • projection.
• The store caches each pinned schema's manifest and index plan per object; neither changes for a hash.
• New errors: data.unknown_schema, data.unknown_index, data.access_denied, data.scan_limit, data.result_limit, data.not_unique.
• The store doesn't apply defaults on write; that's GRA-244's operation boundary.
• Aggregates take declared numeric fields only, never managed ones (D3).

Boundary

GRA-244 builds ctx.db with databaseReader(schema, (request) => store.read(scope, request)) and resolves the scope's role flags and operationAccess narrowing. PR 5 adds the host-side read set and readConsistently.

Tests

apps/core/test/store-queries.test.ts, run in workerd through OpenStore and the DataStore DO, with the SDK reader building the descriptors:

• Owner-only tables: every terminal (collect, count, aggregates, unique with another owner's match, filters, get) answers from the caller's records only; expectedRevision conflicts only on visible records.
• Index plans: real EXPLAIN QUERY PLAN of the store's own statement (via planQuery) uses sdk_ix_… with the owner and range terms, and the owner default-order index, with no TEMP B-TREE.
• Roles: member and builder modes admit and refuse by role; none refuses everyone, by query and by ID.
• Names: unknown schema, table, another schema's table, unknown index or field, out-of-order ranges, a range without an index, and managed aggregate fields are all refused.
• Injection: quotes in values match literally; injection in a field name and raw SQL keys are refused.
• D3 through the store: lt ranges exclude null and missing, the index orders missing < null < value, NOT keeps missing, neq is value-only.
• Bounds: collect > 100 and unique > 1 are refused, take(100) and take(0) work, and 10,001 entries trip data.scan_limit for count and for a filtered first while an early stop and an index range still answer; 140 × 120 KB documents trip the 16 MiB bound while an unfiltered count reads none.
• Projection: records are projected to an older pinned schema (a newer field omitted) and a newer one (defaults filled in). Undeclared nested keys in objects, arrays of objects and records of objects are left out.
• Since the security review:
  • another owner's 10,005 records cause neither scan_limit nor result_limit for Ada;
  • a get with another table's ID answers null;
  • exactly 10,000 entries are answered;
  • min/max over nothing are null and sum over nothing is 0;
  • 1e308 + 1e308 gives data.aggregate_overflow;
  • a sum on a declared string field is refused;
  • a read creates no store row;
  • a dropped index gives data.index_required;
  • plan tests now explain the host's own statement (StoreHost.explain);
  • a union is projected to the member the stored value validates against, not a smaller earlier one; the first member whose projection validates is used only when none does, and validators are cached per descriptor (tested: second member, first member, and a later key);
  • aggregates read JSON numbers only (numberExpression), never booleans, and a missing index is logged as store.index_missing.

Results: vp check is clean. vp test on store-queries, store-expressions, store-indexes, the data-store suites, the SDK database and schema suites and shared store-queries passes 200 tests. After the review fixes, the store and SDK suites pass 154 tests.

[nickruigrok]

@greptileai review

[greptile-apps]

Confidence Score: 5/5

[Critical impact] The reviewed changes appear safe to merge, subject to the human security review requested by the PR.

Summary

The PR adds store reads through the pinned schema’s access rules and indexes.

• Store reads follow the caller’s pinned schema and access.
• Queries stop when their scan or result bounds are reached.
• Returned records match the caller’s pinned schema.

No new actionable issue was found in the changes since the previous review. This review checked code and tests; it did not run the test suites.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A["Core supplies ReadScope"] --> B["StoreHost.read checks the request"]
  B --> C["Load the pinned schema"]
  C --> D["Check table access"]
  D --> E{"Read kind"}
  E -->|get| F["Find the visible record by table and ID"]
  E -->|query| G["Read through the selected index"]
  G --> H["Enforce scan and result limits"]
  F --> I["Project fields to the pinned schema"]
  H --> I
  I --> J["Return the answer"]

Loading

_{Reviews (2) · Last reviewed commit: "fix(core): choose a union's member by it..." · Reviewed by Greptile}

[nickruigrok]

@greptileai review