Skip to content

Commit 93cfc3f

Browse files
fix(spec): $like / $ilike _ is one Unicode code point on every JS face (#20222)
Fixes #20143 Clause-②: yes ## What was wrong `$like` / `$ilike` answered two different row sets for one filter, depending on the driver behind the object. The SQLite faces (`driver-sql` on better-sqlite3, `driver-sqlite-wasm`, `driver-turso` local and remote) compile `_` to GLOB's `?` through `likePatternToGlobPattern`, and `?` counts **code points**. The JavaScript faces compiled the spec's `likePatternToRegexSource` with **no regex flags**, so `_` → `[\s\S]` read one **UTF-16 code unit**, which is half of an emoji. The spec's text said "`_` matches exactly one character" without saying which unit. Triage set the direction: `_` is one code point. The stop condition (a real semantic reason for the UTF-16 reading) was measured and does not hold; see below. ## The fix - **`@objectstack/spec`** (`packages/spec/src/data/filter.zod.ts`) - New export `likePatternToRegExp(pattern, foldAscii = false): RegExp`. It compiles `likePatternToRegexSource` with the `u` flag and nothing else, so the flag lives in the one shared definition instead of in each consumer. - `matchesLikePattern` now evaluates `likePatternToRegExp`. - `likePatternToRegexSource` is **unchanged** and still exported. Its docblock gains a section saying the source means one code point per `_` only under `u`. - `LIKE_DESCRIPTION` and the GLOB table now say which "character" is meant. - **`@objectstack/driver-memory`** (`src/memory-driver.ts`): both mingo doors (the `$like` filter at `normalizeFilterCondition`, and the AST `like` / `ilike` node at `convertConditionToMongo`) built their own `new RegExp(likePatternToRegexSource(...))` with no flags. They now take `likePatternToRegExp`. **This is a source change outside the claim's expected surface.** Measurement put the flag's choice in these two call sites, not in the spec, so a spec-only change would have left both doors on UTF-16. This is the producer of the `RegExp` moving into the spec, not consumer-side tolerance. - **`@objectstack/formula`**: no source change. `matchesFilterCondition` answers through `matchesLikePattern`. - No `driver-sql` / `driver-turso` / `driver-sqlite-wasm` source change. Those faces are the oracle; they gain test files only, per the card's "pin the family cells on every face". ### Changed `describe()` text, verbatim (`LIKE_DESCRIPTION`, both copies) Before: > "_" matches exactly one character, and a backslash escapes the character after it After: > "_" matches exactly one character, and a character is one Unicode code point — an emoji or any other character outside the Basic Multilingual Plane is ONE "_", never two, as SQL LIKE counts it. A backslash escapes the character after it No refusal text changed. `likePatternToRegExp` throws the same plain `Error` as the source translation on a dangling escape (the backstop behind `hasDanglingLikeEscape`), and that message is pinned. ## Measured: every face, before and after A probe (18 patterns over 9 values holding U+1F600 and U+1D49C) ran each face from its built `dist/` against a code-point oracle (the spec regex compiled with `u`): | tree | spec `matchesLikePattern` | `formula` | memory `$like` door | memory AST door | better-sqlite3 | sql.js | Turso local | Turso remote (real libSQL) | differing cells | |---|---|---|---|---|---|---|---|---|---| | base `e7f69dbb` | 12 differ | 12 differ | 12 differ | 12 differ | 0 | 0 | 0 | 0 | **48** | | head `713d1504` | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | **0** | The driver-memory reference matcher is not exported from `dist/`, so it is pinned in the unit suite instead. ### The family cells, before → after on the JS faces (after = what every SQLite face already answered) Rows: `x`, `😀`, `😀😀`, `a😀b`, `a😀😀b`, `A𝒜B`, `ab`, `axb`, `axyb`. | op · pattern | before (UTF-16) | after (code point) | |---|---|---| | `$like` `_` | `x` | `😀`, `x` | | `$like` `__` | `ab`, `😀` | `ab`, `😀😀` | | `$like` `a_b` | `axb` | `a😀b`, `axb` | | `$like` `a__b` | `a😀b`, `axyb` | `a😀😀b`, `axyb` | | `$like` `___` | `axb` | `a😀b`, `axb`, `A𝒜B` | | `$like` `😀_` / `_😀` | none | `😀😀` | | `$ilike` `_` | `x` | `😀`, `x` | | `$ilike` `A_B` | `axb` | `a😀b`, `axb`, `A𝒜B` | | `$ilike` `A__B` | `a😀b`, `axyb`, `A𝒜B` | `a😀😀b`, `axyb` | The answer set moves both ways, so the changeset's declaration names the measured direction arm, narrowing (BREAKING, graded `minor` in the launch window), with an ADR-0087 `not-required (no-migration-prescription)` disposition. No pattern is newly refused, no refusal is lifted, and values made only of BMP characters answer exactly as before. ### Every consumer of the translation (Zone 2 item 2), before → after | consumer | before | after | |---|---|---| | `spec` `matchesLikePattern` | flagless `new RegExp`: UTF-16 | `likePatternToRegExp`: code point | | `formula` `matchesFilterCondition` | via `matchesLikePattern`: UTF-16 | code point | | `driver-memory` `$like` / `$ilike` door (mingo) | local flagless `new RegExp`: UTF-16 | `likePatternToRegExp`: code point | | `driver-memory` AST `like` / `ilike` door (mingo) | local flagless `new RegExp`: UTF-16 | `likePatternToRegExp`: code point | | `driver-memory` reference matcher | via `matchesLikePattern`: UTF-16 | code point | | `driver-turso` `turso-20024-like-stored-nul` grid oracle | `matchesLikePattern`; its rows are BMP-only | unchanged answers | | `service-analytics`, `@objectstack/lint`, objectql `having`, `driver-mongodb` | no consumer (`git grep` of both symbols) | none | | objectui | `git grep` of both symbols: 0 hits at HEAD `5c61e5241` and at the pinned `.objectui-sha` `f8a9d0fb` (exit 1 on both; a control grep in the same tree hits) | none | No other LIKE-to-regex translation exists in `packages/**/src`. ## `u`-flag side effects (Zone 2 items 3 and 4) - **Escapes.** The translation escapes exactly `\ ^ $ . * + ? ( ) [ ] { } |` and `/`. That is the regex syntax characters plus `/`, exactly the identity escapes `u` accepts. Every other ASCII punctuation mark is emitted bare, where `u` reads it as itself. This is pinned over **all 33** printable ASCII non-alphanumerics, bare and backslash-escaped, with `foldAscii` off and on: each compiles under `u` and matches only itself. The escaped set is pinned to equal that set. A control proves `u` really rejects `\-` and `\:`. - **`[\s\S]`, never `.`.** Both wildcards use `[\s\S]`, so under `u` a `_` is one code point including a newline (pinned: source `^[\s\S]$`, and `a_b` matches `a` newline `b`). - **Surrogate pairs.** A character outside the BMP, literal or escaped, is emitted as its whole pair (pinned: the source for backslash + `😀` is `^😀$`), and `u` reads the pair as one code point. Under `u`, `%` never splits a pair: `%` followed by a lone low surrogate does not match `😀` (pinned). - **`$ilike` / `foldAscii`.** The fold stays in the source as `[Aa]` classes, unaffected by `u`. `likePatternToRegExp(...).flags` is pinned to exactly `u`, with no `i`. The `$ilike` family cells are pinned on every face. ## The stop condition: nothing needs the UTF-16 reading (Zone 2 item 5) - Nothing in `@objectstack/formula` lowers CEL to `$like`: `cel-to-filter` emits no `$like`, and `rls-predicate` leaves SQL `LIKE` unparseable. The only `$like` evaluation is `matches-filter.ts`, via the spec. - CEL's own `size()` in `formula` already counts **code points**: `size('😀')` is 1 and `size('a😀b')` is 3 (pinned in the formula suite). The old `$like` answer disagreed with the same package's string length. ## Pins (every face, literal code-point rows, not parity alone) | face | suite | cases | |---|---|---| | spec | `packages/spec/src/data/filter-like-code-point.test.ts` | 23 | | formula | `packages/formula/src/matches-filter-like-code-point.test.ts` | 12 | | driver-memory, all three doors | `packages/drivers/driver-memory/src/memory-20143-like-code-point.test.ts` | 11 | | driver-sql (better-sqlite3) | `packages/drivers/driver-sql/src/sql-driver-20143-like-code-point.test.ts` | 12 | | driver-sqlite-wasm (sql.js) | `packages/drivers/driver-sqlite-wasm/src/sqlite-wasm-20143-like-code-point.test.ts` | 12 | | driver-turso (local, remote stub, remote real libSQL) | `packages/drivers/driver-turso/src/turso-20143-like-code-point.test.ts` | 12 | Each SQLite face also holds itself to the JS answer (`formula`, or the spec's `matchesLikePattern` where `formula` is not a dependency), which is the cross-face half. ## Ablation (committed fix first; one-shot, no permanent test file) The mutation replaced the `'u'` flag argument with an expression that evaluates to the empty flag string and carries the marker `ABLATION20143`. It went through `scripts/ablation-replace.mjs` under its restore trap, inside a script with its own `trap` restoring from `HEAD`. - **Mutate leg:** on disk, marker count 1 and anchor count 0. `pnpm --filter @objectstack/spec build` exited 0, and `ablation-dist-preflight.mjs @objectstack/spec ABLATION20143` exited 0, so the marker is in `dist/`. The expected direction was written down before running: JS faces red on the family cells; SQLite faces green on their literal cells and red only on the cross-face leg. Observed: spec **13 failed** / 23; formula **11** / 12 (CEL `size()` stays green); driver-memory **10** / 11; driver-sql **1** / 12, driver-sqlite-wasm **1** / 12, driver-turso **1** / 12 (each the cross-face leg only). That is the predicted direction. - **Restore leg:** `git hash-object` equals the `HEAD` blob `838a80720b3d`, `git diff HEAD` is empty, and `git status --porcelain` is clean. Rebuild exited 0, `ablation-dist-preflight.mjs ... --absent` exited 0, and all six suites are green again. ## Verification (at `713d1504`, after merging `origin/main` at `585c9af5`) - Build of the six-package closure: turbo 10/10. `pnpm --filter @objectstack/spec check:generated` passed after the merge. Regenerated here: `api-surface/data.json` (**0 breaking, 1 added**: `likePatternToRegExp`), `export-origins/data.json`, `content/docs/references/data/filter.mdx`. - Full suites: spec 542 files / 15,943 passed (2 todo); formula 38 / 1,039; driver-memory 55 / 1,339; driver-sqlite-wasm 35 / 656; driver-turso 73 / 1,921 (16 skipped); driver-sql 189 files (+11 skipped) / 3,111 passed (170 skipped). - Typecheck, exit 0 on all six: spec, formula, driver-memory, driver-sql, driver-sqlite-wasm, driver-turso. Each new test file is counted inside a compiled program with `--listFiles`: `tsconfig.json` for the four drivers, and `tsconfig.test.json` (via `check:test-typecheck`) for spec and formula. - Gates: 108 derived at `713d1504` (`dispatch-gates.mjs --commands --repo objectstack-ai/objectstack`), reconciled with `--ran`: **101 exit 0, 0 unrun, 7 NOT MEASURED**. All seven are exit 3, PREREQUISITE NOT MET: - `check:doc-formula-expressions`, `check:doc-security-posture` and `check:docs-transcript-drift` need `@objectstack/lint` built. - `check:lean-entry-closure` needs `objectql` built, and `check:skill-examples` needs `client` / `client-react` built. - `check:dual-build-cjs-loads` and `check:type-check-debt` need the whole workspace built. The targeted prerequisite build never acquired the shared verify-lock (two 9-minute queue-timeouts behind other seats' batches), so those seven are declared to CI's Lint & Repo Gates. The seven roster families whose roster sits in a touched directory were run as well, all exit 0: `check:meta-url-spelling`, `check:spec-changes`, `check:authz-resolver`, `check:error-code-casing`, `check:filter-alias-parity`, `check:object-def-param-keys`, `check:tenant-chokepoint`. `check-adr-0087-registration` reads the changeset as `[clause-②-narrowing] not-required (no-migration-prescription)`. ## Acceptance notes - Three hand-written surfaces restate "`_` is exactly one character": `content/docs/api/client-sdk.mdx:553`, `content/docs/deployment/troubleshooting.mdx:148`, and the `packages/client/src/query-builder.ts:88` JSDoc. They stay true under the reading the spec now defines, and are not edited here. Carrier: none. - `likePatternToRegexSource` stays exported, so a future consumer could still compile it without `u`. Its docblock carries the prohibition, and after this PR nothing outside the spec calls it. Retiring the export would be a removal and is not this card. - A value or pattern holding an unpaired surrogate is not measured. UTF-8 SQLite cannot store one, and the JS faces under `u` treat it as its own code point. - The Postgres and MySQL `_` arms are NOT MEASURED, as on the card. SQL `_` is one character there. --- _Generated by [Claude Code](https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 6d2571f commit 93cfc3f

12 files changed

Lines changed: 779 additions & 21 deletions

File tree

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
---
2+
"@objectstack/spec": minor
3+
"@objectstack/formula": minor
4+
"@objectstack/driver-memory": minor
5+
---
6+
7+
`$like` / `$ilike`: `_` matches exactly one Unicode code point on every face, so an emoji or any other character outside the Basic Multilingual Plane is one `_`, as SQL `LIKE` and SQLite `GLOB` count it (#20143).
8+
9+
The SQLite faces (`driver-sql` on better-sqlite3, `driver-sqlite-wasm`, `driver-turso` local and remote) already answered by code points. The JavaScript faces did not: they compiled the spec's `likePatternToRegexSource` with no regular-expression flags, so `_` read one UTF-16 code unit, which is half of an emoji. The same REST filter returned a different row set depending on which driver backed the object. Measured at `e7f69dbb` over values holding `😀` (U+1F600) and `𝒜` (U+1D49C), 48 answer cells on the JS faces differed from the SQLite faces; after this change, none do.
10+
11+
- **`@objectstack/spec`**: a new export, `likePatternToRegExp(pattern, foldAscii?)`, compiles the translation with the `u` flag, the one compilation in which `_` is one code point. `matchesLikePattern` evaluates it. `likePatternToRegexSource` is unchanged and still exported; its source means one code point per `_` only under `u`. The `$like` description now says that a character is one Unicode code point.
12+
- **`@objectstack/formula`**: `matchesFilterCondition` answers `$like` / `$ilike` by code points, through the spec's `matchesLikePattern`. Its own CEL `size()` already counted code points.
13+
- **`@objectstack/driver-memory`**: all three `$like` doors (the `$like` filter and the AST `like` / `ilike` node through mingo, and the reference matcher) answer by code points.
14+
15+
The answer set moves in both directions on those three faces, only for values holding a character outside the BMP:
16+
17+
| pattern | a stored `😀` | `a😀b` | `a😀😀b` |
18+
|---|---|---|---|
19+
| `_` | now matches | — | — |
20+
| `__` | no longer matches | — | — |
21+
| `a_b` | — | now matches | — |
22+
| `a__b` | — | no longer matches | now matches |
23+
24+
`$ilike` moves the same way. No pattern is newly refused and no refusal is lifted. Values made only of characters inside the BMP answer exactly as before.
25+
26+
Clause-②: yes (narrowing)
27+
28+
<!-- adr-0087: not-required (no-migration-prescription) Nothing an author writes is removed or renamed: no spec key, no export and no config field moves, and no stored metadata representation changes, so `objectstack migrate meta` has nothing to rewrite. What moves is the row set a `$like` / `$ilike` query returns on the JavaScript faces, and only for stored values holding a character outside the BMP, where it now equals what every SQLite face already returned. -->

‎content/docs/references/data/filter.mdx‎

Lines changed: 5 additions & 5 deletions
Large diffs are not rendered by default.
Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* [#20143] driver-memory — `$like` / `$ilike` `_` is ONE Unicode code point on
5+
* EVERY door, the answer every SQLite face gives.
6+
*
7+
* This driver answers `$like` on three doors: the `$`-spelling through mingo
8+
* (`normalizeFilterCondition`), the AST comparison node `like` / `ilike`
9+
* through mingo (`convertConditionToMongo`), and the reference matcher. The
10+
* two mingo doors compiled the spec's `likePatternToRegexSource` with a local
11+
* `new RegExp(...)` and no flags, and the matcher evaluated the spec's
12+
* `matchesLikePattern`, which did the same — so all three read one UTF-16 code
13+
* unit per `_`. Measured at base `e7f69dbb`: `$like: '_'` missed a stored
14+
* `😀` that better-sqlite3, sql.js and `TursoDriver` (local, and remote on a
15+
* real libSQL engine) all returned, on both mingo doors. The mingo doors now
16+
* take the spec's `likePatternToRegExp` (the `u` flag), and the matcher
17+
* inherits it through `matchesLikePattern`.
18+
*
19+
* Each door is asserted against the literal code-point rows, not merely
20+
* against the others: parity alone is satisfied by breaking all three alike.
21+
*/
22+
23+
import { describe, it, expect, beforeAll } from 'vitest';
24+
import { InMemoryDriver } from './memory-driver.js';
25+
import { match } from './memory-matcher.js';
26+
27+
const GRIN = String.fromCodePoint(0x1f600);
28+
const SCRIPT_A = String.fromCodePoint(0x1d49c);
29+
const TABLE = 'like_code_point';
30+
31+
const ROWS: Readonly<Record<string, string>> = {
32+
x: 'x',
33+
grin: GRIN,
34+
grin_grin: GRIN + GRIN,
35+
a_grin_b: 'a' + GRIN + 'b',
36+
a_grin_grin_b: 'a' + GRIN + GRIN + 'b',
37+
upper_script_b: 'A' + SCRIPT_A + 'B',
38+
ab: 'ab',
39+
axb: 'axb',
40+
axyb: 'axyb',
41+
};
42+
43+
/** [op, pattern, the code-point rows — the SQLite faces' answer]. */
44+
const FAMILY: ReadonlyArray<readonly ['$like' | '$ilike', string, readonly string[]]> = [
45+
['$like', '_', ['grin', 'x']],
46+
['$like', '__', ['ab', 'grin_grin']],
47+
['$like', 'a_b', ['a_grin_b', 'axb']],
48+
['$like', 'a__b', ['a_grin_grin_b', 'axyb']],
49+
['$like', '___', ['a_grin_b', 'axb', 'upper_script_b']],
50+
['$like', GRIN + '_', ['grin_grin']],
51+
['$like', '_' + GRIN, ['grin_grin']],
52+
['$ilike', '_', ['grin', 'x']],
53+
['$ilike', 'A_B', ['a_grin_b', 'axb', 'upper_script_b']],
54+
['$ilike', 'A__B', ['a_grin_grin_b', 'axyb']],
55+
];
56+
57+
describe('[#20143] driver-memory — `_` is one code point on every door', () => {
58+
let driver: InMemoryDriver;
59+
60+
beforeAll(async () => {
61+
driver = new InMemoryDriver({ persistence: false, logger: { info() {}, warn() {}, error() {}, debug() {} } as never });
62+
await driver.connect();
63+
for (const [label, v] of Object.entries(ROWS)) await driver.create(TABLE, { label, v });
64+
});
65+
66+
const labels = (rows: Array<Record<string, unknown>>): string[] => rows.map((r) => String(r.label)).sort();
67+
68+
it('stored every value whole (the premise)', async () => {
69+
const rows = (await driver.find(TABLE, {})) as Array<{ label: string; v: string }>;
70+
expect(Object.fromEntries(rows.map((r) => [r.label, r.v]))).toEqual(ROWS);
71+
});
72+
73+
for (const [op, pattern, expected] of FAMILY) {
74+
it(`${op} ${JSON.stringify(pattern)} answers the code-point rows on all three doors`, async () => {
75+
const where = { v: { [op]: pattern } };
76+
const ast = { type: 'comparison', field: 'v', operator: op === '$ilike' ? 'ilike' : 'like', value: pattern };
77+
expect({
78+
dollarQuery: labels(await driver.find(TABLE, { where: where as never })),
79+
astQuery: labels(await driver.find(TABLE, { where: ast as never })),
80+
matcher: Object.entries(ROWS).filter(([label, v]) => match({ label, v }, where)).map(([l]) => l).sort(),
81+
}).toEqual({ dollarQuery: [...expected], astQuery: [...expected], matcher: [...expected] });
82+
});
83+
}
84+
});

‎packages/drivers/driver-memory/src/memory-driver.ts‎

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ import { canonicalAstOperator, asciiCaseInsensitiveRegexSource } from '@objectst
99
// [#7536] `$like`/`$ilike`'s pattern language, from the spec's one definition —
1010
// the same translation `formula` evaluates and the same pattern `driver-sql`
1111
// hands to LIKE/GLOB, so this face cannot answer a pattern differently.
12-
import { hasDanglingLikeEscape, hasNulInLikePattern, likePatternToRegexSource } from '@objectstack/spec/data';
12+
import { hasDanglingLikeEscape, hasNulInLikePattern, likePatternToRegExp } from '@objectstack/spec/data';
1313
import type { DriverQuery, IDataDriver } from '@objectstack/spec/contracts';
1414
import { Logger, createLogger, nextUtcCalendarDay } from '@objectstack/core';
1515
import { Query, Aggregator } from 'mingo';
@@ -1393,8 +1393,10 @@ export class InMemoryDriver implements IDataDriver {
13931393
if (hasDanglingLikeEscape(value)) throw danglingLikeEscapeError(field, canonical, value);
13941394
// [#20041] The same U+0000 refusal as the `$`-spelling's shape gate.
13951395
if (hasNulInLikePattern(value)) throw nulLikePatternError(field, canonical, value);
1396+
// [#20143] The spec's compiled RegExp, never a local `new RegExp`: it
1397+
// carries the `u` flag that makes `_` one code point, as on SQL.
13961398
return {
1397-
[field]: { $regex: new RegExp(likePatternToRegexSource(value, canonical === 'ilike')) },
1399+
[field]: { $regex: likePatternToRegExp(value, canonical === 'ilike') },
13981400
};
13991401
}
14001402
case 'notcontains': case 'not_contains':
@@ -1618,16 +1620,18 @@ export class InMemoryDriver implements IDataDriver {
16181620
regexConditions.push({ $regex: new RegExp(asciiCaseInsensitiveRegexSource(val)) });
16191621
break;
16201622
// [#7536] `$like` / `$ilike` — the caller's OWN pattern, anchored to the
1621-
// whole value. `likePatternToRegexSource` is the spec's one translation
1622-
// of that language, the same one `formula` evaluates and the same
1623+
// whole value. `likePatternToRegExp` is the spec's one compilation of
1624+
// that language, the same one `formula` evaluates and the same
16231625
// pattern `driver-sql` hands to `LIKE` / `GLOB`.
16241626
//
16251627
// Two things this arm must NOT copy from its neighbours above. It does
16261628
// not `escapeRegex` the comparand — a `%` here is the caller's wildcard,
16271629
// and escaping it back into a literal IS the wire defect #7536 closed,
1628-
// reproduced one layer down. And it does not pass the `i` flag: the
1629-
// `$ilike` fold is ASCII-only and lives in the pattern source, for the
1630-
// reason the `$icontains` arm above spells out at length.
1630+
// reproduced one layer down. And it does not build its own `RegExp`:
1631+
// the spec's carries the `u` flag that makes `_` one code point
1632+
// (#20143) and never the `i` flag — the `$ilike` fold is ASCII-only and
1633+
// lives in the pattern source, for the reason the `$icontains` arm
1634+
// above spells out at length.
16311635
case '$like':
16321636
case '$ilike':
16331637
// The comparand's shape was settled by `assertFieldConstraintShape`
@@ -1640,7 +1644,7 @@ export class InMemoryDriver implements IDataDriver {
16401644
if (hasDanglingLikeEscape(val)) throw danglingLikeEscapeError(field, op, val, path);
16411645
if (hasNulInLikePattern(val)) throw nulLikePatternError(field, op, val, path);
16421646
regexConditions.push({
1643-
$regex: new RegExp(likePatternToRegexSource(val, op === '$ilike')),
1647+
$regex: likePatternToRegExp(val, op === '$ilike'),
16441648
});
16451649
break;
16461650
case '$between': {
Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* [#20143] `SqlDriver` on better-sqlite3 — `$like` / `$ilike` `_` is ONE
5+
* Unicode code point, and `@objectstack/formula` now gives the same rows.
6+
*
7+
* This face was already right: it compiles `$like` to GLOB over the spec's
8+
* `likePatternToGlobPattern`, and GLOB's `?` counts code points. The JS faces
9+
* compiled `likePatternToRegexSource` with no flags and counted UTF-16 code
10+
* units, so measured at base `e7f69dbb` `$like: '_'` returned a stored `😀`
11+
* here and not in `formula`. The spec's `likePatternToRegExp` now compiles with
12+
* the `u` flag, and the last case holds this face and `formula` to one answer.
13+
*
14+
* Each case asserts the literal code-point rows, not only the parity: parity
15+
* alone is satisfied by breaking both faces the same way. The same cells are
16+
* pinned on `driver-sqlite-wasm`, both `driver-turso` transports,
17+
* `driver-memory`, `formula` and the spec.
18+
*/
19+
20+
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
21+
import type { DriverOptions, FilterCondition } from '@objectstack/spec/data';
22+
import { matchesFilterCondition } from '@objectstack/formula';
23+
import { SqlDriver } from './sql-driver.js';
24+
25+
const GRIN = String.fromCodePoint(0x1f600);
26+
const SCRIPT_A = String.fromCodePoint(0x1d49c);
27+
const BYPASS: DriverOptions = { bypassTenantAudit: true };
28+
const TABLE = 'like_code_point';
29+
30+
const ROWS: Readonly<Record<string, string>> = {
31+
x: 'x',
32+
grin: GRIN,
33+
grin_grin: GRIN + GRIN,
34+
a_grin_b: 'a' + GRIN + 'b',
35+
a_grin_grin_b: 'a' + GRIN + GRIN + 'b',
36+
upper_script_b: 'A' + SCRIPT_A + 'B',
37+
ab: 'ab',
38+
axb: 'axb',
39+
axyb: 'axyb',
40+
};
41+
42+
/** [op, pattern, the code-point rows]. */
43+
const FAMILY: ReadonlyArray<readonly ['$like' | '$ilike', string, readonly string[]]> = [
44+
['$like', '_', ['grin', 'x']],
45+
['$like', '__', ['ab', 'grin_grin']],
46+
['$like', 'a_b', ['a_grin_b', 'axb']],
47+
['$like', 'a__b', ['a_grin_grin_b', 'axyb']],
48+
['$like', '___', ['a_grin_b', 'axb', 'upper_script_b']],
49+
['$like', GRIN + '_', ['grin_grin']],
50+
['$like', '_' + GRIN, ['grin_grin']],
51+
['$ilike', '_', ['grin', 'x']],
52+
['$ilike', 'A_B', ['a_grin_b', 'axb', 'upper_script_b']],
53+
['$ilike', 'A__B', ['a_grin_grin_b', 'axyb']],
54+
];
55+
56+
function formulaAnswer(where: FilterCondition): string[] {
57+
return Object.entries(ROWS)
58+
.filter(([, v]) => matchesFilterCondition({ v }, where))
59+
.map(([label]) => label)
60+
.sort();
61+
}
62+
63+
describe('[#20143] SqlDriver on better-sqlite3 — `_` is one code point', () => {
64+
let driver: SqlDriver;
65+
66+
const labelsWhere = async (where: FilterCondition): Promise<string[]> => {
67+
const rows = (await driver.find(TABLE, { where }, BYPASS)) as Array<{ label: string }>;
68+
return rows.map((r) => r.label).sort();
69+
};
70+
71+
beforeAll(async () => {
72+
driver = new SqlDriver({ client: 'better-sqlite3', connection: { filename: ':memory:' }, useNullAsDefault: true });
73+
await driver.initObjects([{ name: TABLE, fields: { label: { type: 'string' }, v: { type: 'text' } } }]);
74+
for (const [label, v] of Object.entries(ROWS)) await driver.create(TABLE, { label, v }, BYPASS);
75+
});
76+
77+
afterAll(async () => {
78+
await driver.disconnect();
79+
});
80+
81+
it('stored every value whole, as UTF-8 (the premise)', async () => {
82+
const rows = (await driver.execute(`select label, hex(v) as h from ${TABLE}`)) as Array<{ label: string; h: string }>;
83+
const stored = Object.fromEntries(rows.map((r) => [r.label, r.h]));
84+
for (const [label, v] of Object.entries(ROWS)) {
85+
expect(stored[label], label).toBe(Buffer.from(v, 'utf8').toString('hex').toUpperCase());
86+
}
87+
});
88+
89+
for (const [op, pattern, expected] of FAMILY) {
90+
it(`${op} ${JSON.stringify(pattern)} answers the code-point rows`, async () => {
91+
expect(await labelsWhere({ v: { [op]: pattern } } as FilterCondition)).toEqual([...expected]);
92+
});
93+
}
94+
95+
it('formula answers the same rows as this face on every family cell', async () => {
96+
const differing: string[] = [];
97+
for (const [op, pattern] of FAMILY) {
98+
const where = { v: { [op]: pattern } } as FilterCondition;
99+
const sql = await labelsWhere(where);
100+
const js = formulaAnswer(where);
101+
if (JSON.stringify(sql) !== JSON.stringify(js)) {
102+
differing.push(`${op} ${JSON.stringify(pattern)}: sql ${JSON.stringify(sql)} != formula ${JSON.stringify(js)}`);
103+
}
104+
}
105+
expect(differing).toEqual([]);
106+
});
107+
});
Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* [#20143] `driver-sqlite-wasm` (sql.js) — `$like` / `$ilike` `_` is ONE
5+
* Unicode code point, and `@objectstack/formula` now gives the same rows.
6+
*
7+
* This face was already right: it compiles `$like` to GLOB over the spec's
8+
* `likePatternToGlobPattern`, and GLOB's `?` counts code points. The JS faces
9+
* compiled `likePatternToRegexSource` with no flags and counted UTF-16 code
10+
* units, so measured at base `e7f69dbb` `$like: '_'` returned a stored `😀`
11+
* here and not in `formula`. The spec's `likePatternToRegExp` now compiles with
12+
* the `u` flag, and the last case holds this face and `formula` to one answer.
13+
*
14+
* Each case asserts the literal code-point rows, not only the parity: parity
15+
* alone is satisfied by breaking both faces the same way. The same cells are
16+
* pinned on `driver-sql` (better-sqlite3), both `driver-turso` transports,
17+
* `driver-memory`, `formula` and the spec.
18+
*/
19+
20+
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
21+
import type { DriverOptions, FilterCondition } from '@objectstack/spec/data';
22+
import { matchesFilterCondition } from '@objectstack/formula';
23+
import { SqliteWasmDriver } from './index.js';
24+
25+
const GRIN = String.fromCodePoint(0x1f600);
26+
const SCRIPT_A = String.fromCodePoint(0x1d49c);
27+
const BYPASS: DriverOptions = { bypassTenantAudit: true };
28+
const TABLE = 'like_code_point';
29+
30+
const ROWS: Readonly<Record<string, string>> = {
31+
x: 'x',
32+
grin: GRIN,
33+
grin_grin: GRIN + GRIN,
34+
a_grin_b: 'a' + GRIN + 'b',
35+
a_grin_grin_b: 'a' + GRIN + GRIN + 'b',
36+
upper_script_b: 'A' + SCRIPT_A + 'B',
37+
ab: 'ab',
38+
axb: 'axb',
39+
axyb: 'axyb',
40+
};
41+
42+
/** [op, pattern, the code-point rows]. */
43+
const FAMILY: ReadonlyArray<readonly ['$like' | '$ilike', string, readonly string[]]> = [
44+
['$like', '_', ['grin', 'x']],
45+
['$like', '__', ['ab', 'grin_grin']],
46+
['$like', 'a_b', ['a_grin_b', 'axb']],
47+
['$like', 'a__b', ['a_grin_grin_b', 'axyb']],
48+
['$like', '___', ['a_grin_b', 'axb', 'upper_script_b']],
49+
['$like', GRIN + '_', ['grin_grin']],
50+
['$like', '_' + GRIN, ['grin_grin']],
51+
['$ilike', '_', ['grin', 'x']],
52+
['$ilike', 'A_B', ['a_grin_b', 'axb', 'upper_script_b']],
53+
['$ilike', 'A__B', ['a_grin_grin_b', 'axyb']],
54+
];
55+
56+
function formulaAnswer(where: FilterCondition): string[] {
57+
return Object.entries(ROWS)
58+
.filter(([, v]) => matchesFilterCondition({ v }, where))
59+
.map(([label]) => label)
60+
.sort();
61+
}
62+
63+
describe('[#20143] driver-sqlite-wasm (sql.js) — `_` is one code point', () => {
64+
let driver: SqliteWasmDriver;
65+
66+
const labelsWhere = async (where: FilterCondition): Promise<string[]> => {
67+
const rows = (await driver.find(TABLE, { where }, BYPASS)) as Array<{ label: string }>;
68+
return rows.map((r) => r.label).sort();
69+
};
70+
71+
beforeAll(async () => {
72+
driver = new SqliteWasmDriver({ filename: ':memory:' });
73+
await driver.initObjects([{ name: TABLE, fields: { label: { type: 'string' }, v: { type: 'text' } } }]);
74+
for (const [label, v] of Object.entries(ROWS)) await driver.create(TABLE, { label, v }, BYPASS);
75+
}, 60_000);
76+
77+
afterAll(async () => {
78+
await driver.disconnect();
79+
});
80+
81+
it('stored every value whole, as UTF-8 (the premise)', async () => {
82+
const rows = (await driver.execute(`select label, hex(v) as h from ${TABLE}`)) as Array<{ label: string; h: string }>;
83+
const stored = Object.fromEntries(rows.map((r) => [r.label, r.h]));
84+
for (const [label, v] of Object.entries(ROWS)) {
85+
expect(stored[label], label).toBe(Buffer.from(v, 'utf8').toString('hex').toUpperCase());
86+
}
87+
});
88+
89+
for (const [op, pattern, expected] of FAMILY) {
90+
it(`${op} ${JSON.stringify(pattern)} answers the code-point rows`, async () => {
91+
expect(await labelsWhere({ v: { [op]: pattern } } as FilterCondition)).toEqual([...expected]);
92+
});
93+
}
94+
95+
it('formula answers the same rows as this face on every family cell', async () => {
96+
const differing: string[] = [];
97+
for (const [op, pattern] of FAMILY) {
98+
const where = { v: { [op]: pattern } } as FilterCondition;
99+
const sql = await labelsWhere(where);
100+
const js = formulaAnswer(where);
101+
if (JSON.stringify(sql) !== JSON.stringify(js)) {
102+
differing.push(`${op} ${JSON.stringify(pattern)}: sql ${JSON.stringify(sql)} != formula ${JSON.stringify(js)}`);
103+
}
104+
}
105+
expect(differing).toEqual([]);
106+
});
107+
});

0 commit comments

Comments
 (0)