Skip to content

Commit adbbc5d

Browse files
fix(driver-sql, driver-turso): SQLite $contains / $notContains / $icontains / $endsWith read the whole stored value, not only up to its first U+0000 (#20038)
Part of #20024 Clause-②: no ## What this changes This takes item 1 of #20024, the stored value's cut. Item 2 (`$like` / `$ilike`) stays open on #20024 for a decision; see "Item 2" below. On the SQLite faces, `$contains`, `$notContains`, `$icontains` and `$endsWith` compiled to `GLOB` for a comparand without U+0000. SQLite's `glob()` reads the stored value as a C string, only up to its first U+0000. So `$contains: 'b'` missed a value stored as `'a'` + U+0000 + `'b'`, `$endsWith: 'a'` returned it, and `$icontains: 'B'` missed `'A'` + U+0000 + `'B'`. Nothing raised. #19999 (PR #20019, `9d81af714f`) had already moved a comparand that HOLDS U+0000 onto length-aware constructs; this sends every `contains` / `ends` comparand there, which is option A of contract review 5821310673 (reconfirmed in 5822054158). | shape | SQLite predicate, every comparand | |---|---| | `contains` (`$contains`, `$notContains`, `$icontains`) | `instr(col, ?) > 0` | | `ends` (`$endsWith`), non-empty comparand | `coalesce(substr(CAST(col AS BLOB), -length(CAST(? AS BLOB))), CAST(col AS BLOB)) = CAST(? AS BLOB)` | | `ends`, empty comparand | `instr(col, ?) > 0` | | `starts` (`$startsWith`), comparand without U+0000 | `col GLOB ?`, unchanged byte for byte | | `starts`, comparand holding U+0000 | `instr(col, ?) = 1`, unchanged from #19999 | - **The empty `$endsWith` comparand** needed its own route. In the suffix construct `-length('')` is `-0`, which `substr` reads as "from the start", so the construct would answer false for `'abc'`. Measured on all three engines: the suffix construct with `''` answers 0 on `'abc'`, while `instr('abc', '')` and `instr('', '')` are both 1 and `instr(NULL, '')` is NULL, which is what `GLOB '*'` answered. An empty string is a suffix of every value, so it takes `instr()`. - **`$startsWith` keeps `GLOB`.** A prefix free of U+0000 lies wholly before the value's first U+0000, or it is not a prefix at all, so the value's cut cannot change that answer (0 of 14 `$startsWith` probe cases differed at base, including values that start with U+0000). It is also the one text construct an index serves; see H3. - The fold is still `lower()` on both sides, ASCII-only. The negation is still `NOT (...)` inside the unchanged NULL-safe `$notContains` wrapper. Nothing is escaped, because `instr()` and the BLOB suffix have no pattern language, so `*`, `?` and `[` are bound as written. - Two emitters change: `textMatchPredicate` / `sqliteLengthAwareTextMatch` in `packages/drivers/driver-sql/src/sql-driver.ts`, and `RemoteTransport.pushLike` in `packages/drivers/driver-turso/src/remote-transport.ts`. `SqliteWasmDriver` and `TursoDriver`'s local mode inherit the first. No refusal door, no `packages/spec` file and no shared conformance table is touched. ## Measured at base `9d81af714` (hypotheses H1, H2, H5) A 292-case probe drove the public `find` of six faces over 17 stored values (U+0000 at start, middle and end, alone and doubled, multi-byte UTF-8 around it, GLOB and LIKE metacharacters behind it, `''` and NULL), on dists built at base: - `SqlDriver` on better-sqlite3 (SQLite 3.53.4); - `SqliteWasmDriver` on sql.js (3.49.1); - `TursoDriver` local; - `TursoDriver` remote over a real `@libsql/client` `:memory:` engine (SQLite 3.45.1); - `InMemoryDriver`, and `@objectstack/formula` `matchesFilterCondition` as the oracle. The cases were 5 text operators times 19 comparands (14 without U+0000, 5 with), each bare and under `$not`; 36 `$like` and 9 `$ilike` patterns, each bare and under `$not`; and 12 `$not` / `$or` / `$and` compositions. | cases (per SQLite face) | differ from `formula` at base | at head | |---|---|---| | text operators, comparand without U+0000, bare | 38 of 70 | 0 of 70 | | the same under `$not` | 38 of 70 | 0 of 70 | | text operators, comparand holding U+0000, bare and under `$not` | 0 of 50 | 0 of 50 | | compositions | 10 of 12 | 3 of 12, each with a `$like` leg | | `$like` / `$ilike`, bare and under `$not` | 62 of 90 | 62 of 90 (item 2, unchanged) | - **H1 holds.** Per operator at base, bare: `$contains` 8 of 14, `$notContains` 8 of 14, `$icontains` 9 of 14, `$endsWith` 13 of 14. The four SQLite faces gave byte-identical answer lists on all 292 cases, at base and at head. - **H2 holds.** `$startsWith` with a comparand without U+0000: 0 of 14 differ, bare and under `$not`, at base and at head. - **H5 holds.** `$icontains: 'É'` returns `'CAFÉ'` + U+0000 and `$icontains: 'é'` does not; the fold stays `lower()` on both sides. - `InMemoryDriver` equals `formula` on 292 of 292 at base and at head (the control). No face raised on any case, at base or at head. The constructs themselves were measured with SQL literals, no bind, over 16 comparands without U+0000 (including `*?[y`, `%`, `_`, U+0001 and multi-byte ones), 23 stored values and both folds: `instr() > 0`, the BLOB suffix, and `NOT` over each, three-valued against JavaScript `includes` / `endsWith` (NULL expected on NULL). That is 2944 cells per engine and 0 disagreements on each of better-sqlite3 3.53.4, sql.js 3.49.1 and libSQL 3.45.1. ## H3: the plans `EXPLAIN QUERY PLAN` of the SQL `SqlDriver` compiles, run over `create table plan_probe (id integer primary key, v text)` with an index on `v`, on each engine: | filter | base (`GLOB`) | head | |---|---|---| | `$contains: 'ab'` | `SCAN plan_probe` | `SCAN plan_probe` (`instr`) | | `$notContains: 'ab'` | `SCAN` | `SCAN` | | `$icontains: 'Ab'` | `SCAN` | `SCAN` | | `$endsWith: 'ab'` | `SCAN` | `SCAN` (BLOB suffix) | | `$startsWith: 'ab'` | `SEARCH plan_probe USING COVERING INDEX plan_probe_v (v>? AND v<?)` | the same | | `$like: 'ab%'` | `SEARCH … USING COVERING INDEX plan_probe_v (v>? AND v<?)` | the same | | `$eq: 'ab'` (control) | `SEARCH … (v=?)` | the same | Identical on better-sqlite3, sql.js and libSQL. H3 holds: the four moved shapes scanned under `GLOB` already, so no index plan is lost. The remote transport's statements were not put through `EXPLAIN`; they are the same constructs, pinned by text in `turso-20024-glob-stored-nul.test.ts`. ## Item 2 (`$like` / `$ilike`) is not in this PR Measured, and left for a decision on #20024: - **There is no NUL-safe pattern primitive to compile to.** `GLOB` and `LIKE` both stop at U+0000. The way around it I tried first, substituting a sentinel for U+0000 in the value and the pattern, is dead: SQLite's `replace()` treats a search string whose first byte is 0 as empty and returns its input unchanged. `hex(replace('a' || char(0) || 'b', char(0), char(1)))` is `610062` on all three engines. Over a grid of 52 patterns, 23 values and both folds (2392 cells per engine), `GLOB` disagrees with `@objectstack/spec` `matchesLikePattern` on 294 cells and the sentinel construct still on 223. `instr()` does not apply, because the pattern has wildcards, and `_` is one character, so a byte-level BLOB program would have to decode UTF-8 lengths. - **The stored-value cut also hits a pattern without U+0000**, which the card's item 2 does not name: at base and at head, 15 of the probe's 23 such `$like` patterns differ from `formula` on every SQLite face. For example, `$like: 'a'` returns `'a'` + U+0000 + `'b'`, and `$like: ''` returns U+0000 + `'z'`. No refusal can reach this half, because it depends on stored data. - **A refusal is a cross-face contract question.** The dangling-escape refusal of a `$like` pattern is one spec predicate (`hasDanglingLikeEscape`) applied at every face's refusal door. The same shape for U+0000 needs `packages/spec` and the refusal doors that #20020 is working in, and what Postgres and MySQL do with such a pattern is not measured here (no live server). A SQLite-only refusal would make SQLite the one face that refuses what `driver-memory` answers. The options, with the measured costs, are in the `os-dev-report` on #20024. ## Compile surfaces (`references/compile-surfaces.md`, re-verified with its grep at this head) | # | face | status | evidence | |---|---|---|---| | 1 | `driver-sql` `textMatchPredicate` SQLite arm (`sql-driver.ts`) | **touched** | `sql-driver-20024-glob-stored-nul.test.ts` (better-sqlite3). `driver-sqlite-wasm` inherits it and is pinned by `sqlite-wasm-20024-glob-stored-nul.test.ts`; `driver-turso` local mode by `turso-20024-glob-stored-nul.test.ts`. | | 1 | the same function's Postgres and MySQL arms | not touched, **NOT MEASURED** | No live server in this container. | | 1 | `driver-sql` `likePatternPredicate` SQLite arm | not touched, **measured still cut** | Item 2, above. | | 2 | turso `RemoteTransport.pushLike` (`remote-transport.ts`) | **touched** | Pinned on `makeLibsqlSqliteStub` AND on a real `@libsql/client` `file::memory:` engine, beside local mode, all three to the JavaScript rows. A real Turso server and its wire encoding of U+0000 are NOT MEASURED. | | 2 | turso `RemoteTransport.pushLikePattern` | not touched, **measured still cut** | On the real libSQL engine, in the probe. | | 3 | service-analytics `compileScopedFilterToSql` | not touched, **NOT MEASURED** | Outside this claim; its SQLite text arm is #20025's. | | 4 | service-analytics `lowerAnalyticsWhere` | not touched, **NOT MEASURED** | Outside this claim, as face 3. | | 5 | `formula` `matchesFilterCondition` | **measured unaffected** | It is the oracle in all three new suites and in the probe. | | half | objectql `applyHaving` | **measured unaffected** | Over 9 stored values, 6 of them holding U+0000, 30 cases (5 operators times 6 comparands without U+0000): 29 equal `formula`; `$icontains: ''` is refused, the pre-existing refusal driver-sql shares. | | thawed | `driver-memory` `checkCondition` | **measured unaffected** | 292 of 292 probe cases equal `formula`, at base and at head. | | thawed | `driver-mongodb` `translateFieldOperators` | not touched, **NOT MEASURED** | No mongod in this container. | ## Tests - New suites, each case pinned literally: `sql-driver-20024-glob-stored-nul.test.ts` (74 tests), `sqlite-wasm-20024-glob-stored-nul.test.ts` (24), `turso-20024-glob-stored-nul.test.ts` (26, three transports per case). The driver-sql and sqlite-wasm suites also check every case against `formula`; the turso cases are a subset of the driver-sql rows. They cover the four moved operators, `$startsWith` (H2), the empty comparand, and `$not` / `$or` / `$and` over each moved shape with the `''` and NULL rows in every answer set. The compile block pins the new SQL and raw bindings, the empty-`$endsWith` route, and `$startsWith`'s `GLOB` plus its escaped pattern byte for byte. - `pnpm --filter @objectstack/driver-sql exec vitest run --maxWorkers=2` at `667eddc47`: 183 files passed, 11 skipped (live-dialect cells); 2811 tests passed, 170 skipped. - `driver-turso` 65 of 65 files, 1471 of 1471 tests, and `driver-sqlite-wasm` 32 of 32 files, 606 of 606 tests, at `cf0590562`, whose code is byte-identical to `667eddc47` (the one later commit adds only the changeset). - `typecheck` (`tsc --noEmit`) exits 0 in all three packages, and `tsc --listFiles` shows each program includes its new suite. - ESLint, narrowed: `--no-inline-config --format json` over the 15 changed TS files reports 15 files, 0 errors, 0 warnings, none ignored (every file resolves a config through `--print-config`). `eslint.config.mjs` never enables type-aware linting (no `parserOptions.project`, as its own comment says), so this diff cannot move the verdict of a file it does not touch. The full `pnpm lint` is CI's. ## Respelled assertions (markers the moved plans broke) | file | old | new | |---|---|---| | `driver-sql/src/sql-driver-19999-glob-nul-comparand.test.ts` | `$contains 'a*b'` compiles to `` `v` GLOB ? `` with `*a[*]b*`; `$endsWith` binds `*ab`; `$icontains` compiles `lower(v) GLOB lower(?)`; `$notContains` matches `NOT GLOB` | only `$startsWith` keeps `` `v` GLOB ? `` with `ab*`; the other four compile without `GLOB` and bind `a*b` raw | | `driver-sql/src/sql-driver-15683-temporal-text-operator-gate.test.ts` | text column `toMatch(/LIKE\|GLOB/)`; negatives without `instr(` | `toMatch(/LIKE\|GLOB\|instr\(/)`; `instr\(` added to both negatives | | `driver-sql/src/sql-driver-17343-multi-valued-boolean-membership.test.ts` | `REAL_PREDICATE.sqlite` `/LIKE\|GLOB\|json_each\(/`; control `toMatch(/LIKE\|GLOB/)`; negative without `instr(` | `/LIKE\|GLOB\|instr\(\|substr\(CAST\(\|json_each\(/`; control `/LIKE\|GLOB\|instr\(/`; `instr\(` added to the negative | | `driver-sql/src/sql-driver-17590-json-column-membership.test.ts` | scalar column `toMatch(/LIKE\|GLOB/)` and `toMatch(/GLOB/)`; rest-of-family `toMatch(/LIKE\|GLOB/)`; "not a pattern match" `not.toMatch(/LIKE\|GLOB/)` | `/LIKE\|GLOB\|instr\(/`, `/instr\(/`, `/LIKE\|GLOB\|instr\(\|substr\(CAST\(/`, and the negative gains `instr\(\|substr\(CAST\(` | | `driver-sql/src/sql-driver-icontains-and-retired-operators.test.ts` | `lower(name) GLOB lower(` and `*acme*`; `name GLOB` | `instr(lower(name), lower('acme')) > 0`; `instr(name, 'acme') > 0`; neither contains `GLOB` | | `driver-sql/src/sql-driver-text-case-conformance.test.ts` | "sqlite: GLOB, case-exact": `$contains` `toMatch(/GLOB/)`, `$icontains` `lower(…) GLOB lower(`; numeric-column control `toMatch(/LIKE\|GLOB/)` | `$contains` `instr(`, `$icontains` `instr(lower(…), lower(`, `$startsWith` `GLOB`; control `/LIKE\|GLOB\|instr\(/`; `instr\(` added to both negatives | | `driver-turso/src/turso-19999-glob-nul-comparand.test.ts` | remote `$contains 'a*b'` sends `"v" GLOB ?` | remote `$startsWith 'a*b'` sends `"v" GLOB ?` | | `driver-turso/src/remote-transport-text-predicates.test.ts` | `lower("name") GLOB lower(?)`; `"name" GLOB ?`; `$contains` `*` / `?` / `[` / `a*b` bind `*[*]*`, `*[?]*`, `*[[]*`, `*a[*]b*` | `instr(lower("name"), lower(?)) > 0`; `instr("name", ?) > 0`; the escape class is pinned on `$startsWith` (`[*]*`, `[?]*`, `[[]*`, `a[*]b*`), plus a new case that `$contains 'a*b'` binds `a*b` raw | | `driver-turso/src/remote-transport-null-comparand-refusal.test.ts` | `$contains: true` binds `*true*` | binds `true` | | `driver-turso/src/remote-transport-undefined-comparand-refusal.test.ts` | `$contains: 'w'` binds `*w*` | binds `w` | ## Before-red and ablations (one-off; no test left behind) Both ran from committed state through `scripts/ablation-replace.mjs` (anchor counts and blob hashes checked on disk, restore proven as blob == HEAD with `git diff HEAD` empty), with `driver-sql` rebuilt and `scripts/ablation-dist-preflight.mjs` confirming the mutation in `dist/` before the suites ran, because `driver-sqlite-wasm` and turso's local mode read that dist. - **Ablation 1, the dispatch reverted in both emitters** (`if (shape !== 'starts' || text.includes(NUL_CHARACTER))` back to base's `if (text.includes(NUL_CHARACTER))`), which is base behaviour. Predicted by simulating `GLOB`'s cut over each suite's cases, then observed exactly: driver-sql 24 failed and 50 passed of 74 (22 row cases plus the moved-shapes and empty-`$endsWith` compile pins; the `$startsWith` pin stayed green); sqlite-wasm 16 failed and 8 passed of 24; turso 18 failed and 8 passed of 26. In the 12 turso row diffs whose three columns printed in full, `local`, `remote` and `remoteLibsql` were all wrong the same way; the base probe had already found local mode and remote-over-libSQL byte-identical to the other SQLite faces on all 292 cases. - **Ablation 2, the empty-comparand route removed** (`shape === 'ends' && text !== ''` to `shape === 'ends'`, both emitters). Predicted and observed: driver-sql 3 failed of 74 (`$endsWith ''`, `$not $endsWith ''`, the empty-`$endsWith` compile pin), sqlite-wasm 2 of 24, turso 3 of 26. - After each: `git status --porcelain` empty, `driver-sql` rebuilt, and preflight reads the guard back in `dist/` with the tree clean. ## Gates (at `667eddc47`) - `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` derived 62 commands from the real diff, the same 62 the PM list named. All 62 were run. - `check:dual-build-cjs-loads`, `check:lean-entry-closure` and `check:type-check-debt` first answered `PREREQUISITE NOT MET` (exit 3). They answered 0 after `turbo run build --filter='./packages/*' --filter='./packages/*/*'` (72 tasks). `check:dts-closure`, `check:published-files` and `check:sourcemap-no-sources-content` were re-run over that build and answered 0. - `--ran`: "62 derived, 62 run, 0 NOT-MEASURED, 0 UNRUN". - `check-changeset-no-major`: "This diff introduces no `major` bump." `check-adr-0087-registration`: "this PR adds no declared-breaking changeset (1 non-breaking changeset(s) seen)." - `node scripts/check-issue-citations.mjs --base 9d81af7`: 12 citations across 2 files, all resolve. - `pnpm check:driver-conformance` reads the same before (on a `git archive` of `9d81af714`) and after: `50 covered cell(s), 0 in the DEBT ledger, 0 exempt`. - The roster gates whose roster sits beside these paths also exit 0: `check-changeset-fixed`, `check:authz-resolver`, `check:error-code-casing`, `check:filter-alias-parity`. - `pnpm check:nul-bytes` passes, and a control-byte scan of the changed files finds none. U+0000 is spelled `String.fromCharCode(0x00)` throughout. ## Acceptance notes (not filed; for the seat) 1. **Item 2 of #20024 is open and needs a decision** (above; options in the `os-dev-report`). 2. **Observation, not filed:** `packages/plugins/plugin-auth/src/auth-contains-filter.test.ts` says in its docblock that the SQL family "emits `GLOB` on the SQLite dialects now". For `$contains` that is `instr()` after this PR. It is prose in a test outside this claim; no assertion reads it. --- _Generated by [Claude Code](https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 246314d commit adbbc5d

16 files changed

Lines changed: 915 additions & 122 deletions
Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
---
2+
"@objectstack/driver-sql": patch
3+
"@objectstack/driver-sqlite-wasm": patch
4+
"@objectstack/driver-turso": patch
5+
---
6+
7+
fix(driver-sql, driver-turso): on SQLite, `$contains` / `$notContains` / `$icontains` / `$endsWith` read the whole stored value, instead of stopping at its first U+0000 (#20024)
8+
9+
Clause-②: no
10+
11+
On the SQLite faces these four operators compiled to `GLOB` for a comparand without U+0000, and SQLite's `glob()` reads the stored value only up to its first U+0000. Nothing raised, and the filter answered a different question. Measured on `SqlDriver` over better-sqlite3 (SQLite 3.53.4), on `SqliteWasmDriver` over sql.js (3.49.1), on `TursoDriver`'s local mode, and on its remote transport over a local libSQL engine (3.45.1). All four answered alike:
12+
13+
- `$contains: 'b'` did not return a value stored as `'a'` + U+0000 + `'b'`;
14+
- `$endsWith: 'a'` returned that value, and `$endsWith: 'b'` did not;
15+
- `$notContains: 'b'` returned it;
16+
- `$icontains: 'B'` did not return `'A'` + U+0000 + `'B'`.
17+
18+
What changes: these four operators now compile to the length-aware comparisons a comparand holding U+0000 already used, for every comparand. `$contains`, `$notContains` and `$icontains` use `instr()`, and `$endsWith` compares the value's trailing bytes over BLOB. An empty `$endsWith` comparand uses `instr()` too, so it still matches every non-NULL value. Such a filter now returns the rows `driver-memory` and `@objectstack/formula` return for it, under `$not`, `$or` and `$and` as well. The comparand is bound as written, so `*`, `?` and `[` in it are literal, as they were before. `$icontains` still folds ASCII letters only, and `$notContains` still returns a row whose value is NULL.
19+
20+
- `@objectstack/driver-sql`: the SQLite arm of `SqlDriver`'s text-operator compiler. `SqliteWasmDriver` and `TursoDriver`'s local mode inherit it.
21+
- `@objectstack/driver-sqlite-wasm`: none of its own code changes. It inherits the fix.
22+
- `@objectstack/driver-turso`: the remote transport's own emitter, changed the same way.
23+
24+
What does not change:
25+
26+
- `$startsWith` with a comparand without U+0000 compiles to the same `GLOB` with the same bound pattern as before. The stored value's cut cannot change its answer.
27+
- No index is lost. The SQL `SqlDriver` compiles, run under `EXPLAIN QUERY PLAN` over an indexed TEXT column on all three engines, scanned the table for these four operators under `GLOB` and still does; `$startsWith` keeps its index search.
28+
- The Postgres and MySQL arms are untouched.
29+
- `$like` and `$ilike` still compile to `GLOB` on SQLite, so they still read a stored value only up to its first U+0000, and a pattern holding U+0000 is still cut there.

‎packages/drivers/driver-sql/src/sql-driver-15683-temporal-text-operator-gate.test.ts‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -246,14 +246,18 @@ describe('[#15683] the per-dialect construct, compiled', () => {
246246
// No pattern operator, and no temporal presentation wrapper either:
247247
// the gate runs BEFORE both emitters, so nothing is built over the
248248
// column at all.
249-
expect(sql, `${op} over ${probe.field}`).not.toMatch(/LIKE|GLOB|lower\(|translate\(|CAST\(|strftime|datetime\(/);
249+
// [#20024] `instr(` is the SQLite substring construct now (`contains`).
250+
expect(sql, `${op} over ${probe.field}`).not.toMatch(
251+
/LIKE|GLOB|instr\(|lower\(|translate\(|CAST\(|strftime|datetime\(/,
252+
);
250253
}
251254
const not = d.compileWhere({ [probe.field]: { $notContains: probe.hit } } as FilterCondition);
252255
expect(not, `$notContains over ${probe.field}`).toMatch(/where 1 = 1/);
253-
expect(not, `$notContains over ${probe.field}`).not.toMatch(/LIKE|GLOB|IS NULL/);
256+
expect(not, `$notContains over ${probe.field}`).not.toMatch(/LIKE|GLOB|instr\(|IS NULL/);
254257
}
255-
// …and the text column beside them still compiles a real pattern match.
256-
expect(d.compileWhere({ label: { $contains: '2026' } })).toMatch(/LIKE|GLOB/);
258+
// …and the text column beside them still compiles a real substring match
259+
// (`LIKE` on Postgres / MySQL, `instr()` on SQLite since #20024).
260+
expect(d.compileWhere({ label: { $contains: '2026' } })).toMatch(/LIKE|GLOB|instr\(/);
257261
});
258262
}
259263

‎packages/drivers/driver-sql/src/sql-driver-17343-multi-valued-boolean-membership.test.ts‎

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -255,10 +255,12 @@ describe('[#17343] the per-dialect construct, compiled — the registerExternalO
255255
* emitter OR the membership construct that replaced it for `$contains`. This
256256
* file's question is whether the declared-type gate fired, so it must accept
257257
* either; asserting one SHAPE here would make it red on the card that changes
258-
* the other, which is exactly what #17590 did to its predecessor.
258+
* the other, which is exactly what #17590 did to its predecessor. [#20024]
259+
* On SQLite the substring emitter spells `contains` as `instr(` and `ends` as
260+
* `substr(CAST(`; only `starts` and the `$like` pair keep `GLOB`.
259261
*/
260262
const REAL_PREDICATE: Record<string, RegExp> = {
261-
sqlite: /LIKE|GLOB|json_each\(/,
263+
sqlite: /LIKE|GLOB|instr\(|substr\(CAST\(|json_each\(/,
262264
postgres: /LIKE|GLOB|::jsonb @> /,
263265
mysql: /LIKE|GLOB|JSON_CONTAINS\(/,
264266
};
@@ -286,7 +288,7 @@ describe('[#17343] the per-dialect construct, compiled — the registerExternalO
286288
for (const op of POSITIVE_OPERATORS) {
287289
const sql = d.compileWhere({ [field]: { [op]: 'true' } } as FilterCondition);
288290
expect(sql, `${op} over ${field}`).toMatch(/where 1 = 0/);
289-
expect(sql, `${op} over ${field}`).not.toMatch(/LIKE|GLOB|lower\(|CAST\(/);
291+
expect(sql, `${op} over ${field}`).not.toMatch(/LIKE|GLOB|instr\(|lower\(|CAST\(/);
290292
}
291293
expect(d.compileWhere({ [field]: { $notContains: 'true' } } as FilterCondition), field)
292294
.toMatch(/where 1 = 1/);
@@ -299,9 +301,10 @@ describe('[#17343] the per-dialect construct, compiled — the registerExternalO
299301
expect(d.compileWhere({ [field]: { $contains: 'red' } } as FilterCondition), field)
300302
.toMatch(REAL_PREDICATE[label]!);
301303
}
302-
// …and the SCALAR string column among them is the one still on the pattern
303-
// emitter, which is what keeps the row above from passing vacuously.
304-
expect(d.compileWhere({ label: { $contains: 'red' } } as FilterCondition)).toMatch(/LIKE|GLOB/);
304+
// …and the SCALAR string column among them is the one still on the
305+
// substring emitter (`instr(` on SQLite since #20024), which is what keeps
306+
// the row above from passing vacuously.
307+
expect(d.compileWhere({ label: { $contains: 'red' } } as FilterCondition)).toMatch(/LIKE|GLOB|instr\(/);
305308
});
306309
}
307310

‎packages/drivers/driver-sql/src/sql-driver-17590-json-column-membership.test.ts‎

Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -257,15 +257,19 @@ describe('[#17590] the per-dialect membership construct, compiled', () => {
257257
for (const field of ['tags_', 'picks', 'nums']) {
258258
const sql = d.compileWhere({ [field]: { $contains: 'red' } } as FilterCondition);
259259
expect(sql, `${field} on ${label}`).toMatch(CONSTRUCT[label]!);
260-
expect(sql, `${field} on ${label} must not be a pattern match`).not.toMatch(/LIKE|GLOB/);
260+
// [#20024] The SQLite substring emitter spells `contains` as `instr(`
261+
// and `ends` as `substr(CAST(`; only `starts` and `$like` keep `GLOB`.
262+
expect(sql, `${field} on ${label} must not be a pattern match`).not.toMatch(
263+
/LIKE|GLOB|instr\(|substr\(CAST\(/,
264+
);
261265
expect(sql, `${field} on ${label} must not be a declared constant`).not.toMatch(/1 = 0|1 = 1/);
262266
}
263267
});
264268

265269
it(`${label}: the SCALAR string column still compiles the PATTERN match`, () => {
266270
const d = new CompilerProbeDriver(config).declare();
267271
const sql = d.compileWhere({ label: { $contains: 'red' } } as FilterCondition);
268-
expect(sql, `label on ${label}`).toMatch(/LIKE|GLOB/);
272+
expect(sql, `label on ${label}`).toMatch(/LIKE|GLOB|instr\(/);
269273
expect(sql, `label on ${label}`).not.toMatch(CONSTRUCT[label]!);
270274
});
271275

@@ -279,14 +283,16 @@ describe('[#17590] the per-dialect membership construct, compiled', () => {
279283

280284
/**
281285
* The other text operators are NOT membership spellings and this card does
282-
* not rule on them — they keep the lowering they had. Pinned so a later
283-
* widening is a deliberate edit here rather than a silent side effect.
286+
* not rule on them — they keep the text emitter's lowering (on SQLite, since
287+
* #20024, `instr(` for `$icontains` and `substr(CAST(` for `$endsWith`).
288+
* Pinned so a later widening is a deliberate edit here rather than a silent
289+
* side effect.
284290
*/
285291
it(`${label}: the rest of the text family is UNMOVED on a JSON column`, () => {
286292
const d = new CompilerProbeDriver(config).declare();
287293
for (const op of ['$startsWith', '$endsWith', '$icontains', '$like', '$ilike']) {
288294
const sql = d.compileWhere({ tags_: { [op]: 'red' } } as FilterCondition);
289-
expect(sql, `${op} on ${label}`).toMatch(/LIKE|GLOB/);
295+
expect(sql, `${op} on ${label}`).toMatch(/LIKE|GLOB|instr\(|substr\(CAST\(/);
290296
expect(sql, `${op} on ${label}`).not.toMatch(CONSTRUCT[label]!);
291297
}
292298
});
@@ -340,7 +346,8 @@ describe('[#17590] the per-dialect membership construct, compiled', () => {
340346
expect(d.compileWhere({ [field]: { $contains: 'x' } } as FilterCondition), field)
341347
.toMatch(CONSTRUCT.sqlite!);
342348
}
343-
// …while the scalar string column of the same sweep still gets the pattern.
344-
expect(d.compileWhere({ one_str: { $contains: 'x' } })).toMatch(/GLOB/);
349+
// …while the scalar string column of the same sweep still gets the
350+
// substring emitter (`instr(` on SQLite since #20024).
351+
expect(d.compileWhere({ one_str: { $contains: 'x' } })).toMatch(/instr\(/);
345352
});
346353
});

‎packages/drivers/driver-sql/src/sql-driver-19999-glob-nul-comparand.test.ts‎

Lines changed: 16 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,10 @@
2727
* well as pinned literally, so the table cannot drift from what the other
2828
* faces answer. `driver-memory` answered the same 50-row probe identically.
2929
*
30-
* A comparand WITHOUT U+0000 keeps `GLOB` and binds exactly what it bound
31-
* before, so no existing plan moves; the second block pins that.
30+
* [#20024] A `$startsWith` comparand WITHOUT U+0000 keeps `GLOB` and binds
31+
* exactly what it bound before; every `contains` / `ends` comparand now reads
32+
* the whole stored value too (`sql-driver-20024-glob-stored-nul.test.ts`). The
33+
* second block pins that split.
3234
*/
3335

3436
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
@@ -213,22 +215,21 @@ describe('[#19999] the SQLite construct, compiled', () => {
213215
});
214216

215217
/**
216-
* A comparand without U+0000 compiles exactly as it did before this change —
217-
* `GLOB`, one bound escaped pattern — so no existing plan moves (a
218-
* `$startsWith` GLOB over an indexed column can still use the index).
218+
* [#20024] Of the comparands without U+0000, only `$startsWith`'s keeps
219+
* `GLOB` and its bound escaped pattern — so a `$startsWith` GLOB over an
220+
* indexed column can still use the index. Every `contains` / `ends` shape
221+
* reads the whole stored value now (`sql-driver-20024-glob-stored-nul.test.ts`).
219222
*/
220-
it('a comparand without U+0000 keeps GLOB and its pattern', () => {
221-
expect(probe.compile({ v: { $contains: 'a*b' } })).toEqual({
223+
it('a comparand without U+0000 keeps GLOB and its pattern for $startsWith only', () => {
224+
expect(probe.compile({ v: { $startsWith: 'ab' } })).toEqual({
222225
sql: 'select * from `nul_text_match` where `v` GLOB ?',
223-
bindings: ['*a[*]b*'],
226+
bindings: ['ab*'],
224227
});
225-
expect(probe.compile({ v: { $startsWith: 'ab' } }).bindings).toEqual(['ab*']);
226-
expect(probe.compile({ v: { $endsWith: 'ab' } }).bindings).toEqual(['*ab']);
227-
expect(probe.compile({ v: { $icontains: 'Ab' } })).toEqual({
228-
sql: 'select * from `nul_text_match` where lower(`v`) GLOB lower(?)',
229-
bindings: ['*Ab*'],
230-
});
231-
expect(probe.compile({ v: { $notContains: 'ab' } }).sql).toMatch(/`v` NOT GLOB \?/);
228+
for (const op of ['$contains', '$notContains', '$icontains', '$endsWith'] as const) {
229+
const { sql, bindings } = probe.compile({ v: { [op]: 'a*b' } } as FilterCondition);
230+
expect(sql, op).not.toMatch(/GLOB/);
231+
for (const b of bindings) expect(b, op).toBe('a*b');
232+
}
232233
});
233234

234235
/**

0 commit comments

Comments
 (0)