Repository navigation
Commit 93cfc3f
fix(spec): $like / $ilike
Fixes #20143
Clause-②: yes
## What was wrong
`$like` / `$ilike` answered two different row sets for one filter,
depending on the driver behind the object. The SQLite faces
(`driver-sql` on better-sqlite3, `driver-sqlite-wasm`, `driver-turso`
local and remote) compile `_` to GLOB's `?` through
`likePatternToGlobPattern`, and `?` counts **code points**. The
JavaScript faces compiled the spec's `likePatternToRegexSource` with
**no regex flags**, so `_` → `[\s\S]` read one **UTF-16 code unit**,
which is half of an emoji. The spec's text said "`_` matches exactly one
character" without saying which unit.
Triage set the direction: `_` is one code point. The stop condition (a
real semantic reason for the UTF-16 reading) was measured and does not
hold; see below.
## The fix
- **`@objectstack/spec`** (`packages/spec/src/data/filter.zod.ts`)
- New export `likePatternToRegExp(pattern, foldAscii = false): RegExp`.
It compiles `likePatternToRegexSource` with the `u` flag and nothing
else, so the flag lives in the one shared definition instead of in each
consumer.
- `matchesLikePattern` now evaluates `likePatternToRegExp`.
- `likePatternToRegexSource` is **unchanged** and still exported. Its
docblock gains a section saying the source means one code point per `_`
only under `u`.
- `LIKE_DESCRIPTION` and the GLOB table now say which "character" is
meant.
- **`@objectstack/driver-memory`** (`src/memory-driver.ts`): both mingo
doors (the `$like` filter at `normalizeFilterCondition`, and the AST
`like` / `ilike` node at `convertConditionToMongo`) built their own `new
RegExp(likePatternToRegexSource(...))` with no flags. They now take
`likePatternToRegExp`. **This is a source change outside the claim's
expected surface.** Measurement put the flag's choice in these two call
sites, not in the spec, so a spec-only change would have left both doors
on UTF-16. This is the producer of the `RegExp` moving into the spec,
not consumer-side tolerance.
- **`@objectstack/formula`**: no source change. `matchesFilterCondition`
answers through `matchesLikePattern`.
- No `driver-sql` / `driver-turso` / `driver-sqlite-wasm` source change.
Those faces are the oracle; they gain test files only, per the card's
"pin the family cells on every face".
### Changed `describe()` text, verbatim (`LIKE_DESCRIPTION`, both
copies)
Before:
> "_" matches exactly one character, and a backslash escapes the
character after it
After:
> "_" matches exactly one character, and a character is one Unicode code
point — an emoji or any other character outside the Basic Multilingual
Plane is ONE "_", never two, as SQL LIKE counts it. A backslash escapes
the character after it
No refusal text changed. `likePatternToRegExp` throws the same plain
`Error` as the source translation on a dangling escape (the backstop
behind `hasDanglingLikeEscape`), and that message is pinned.
## Measured: every face, before and after
A probe (18 patterns over 9 values holding U+1F600 and U+1D49C) ran each
face from its built `dist/` against a code-point oracle (the spec regex
compiled with `u`):
| tree | spec `matchesLikePattern` | `formula` | memory `$like` door |
memory AST door | better-sqlite3 | sql.js | Turso local | Turso remote
(real libSQL) | differing cells |
|---|---|---|---|---|---|---|---|---|---|
| base `e7f69dbb` | 12 differ | 12 differ | 12 differ | 12 differ | 0 |
0 | 0 | 0 | **48** |
| head `713d1504` | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | **0** |
The driver-memory reference matcher is not exported from `dist/`, so it
is pinned in the unit suite instead.
### The family cells, before → after on the JS faces (after = what every
SQLite face already answered)
Rows: `x`, `😀`, `😀😀`, `a😀b`, `a😀😀b`, `A𝒜B`, `ab`, `axb`, `axyb`.
| op · pattern | before (UTF-16) | after (code point) |
|---|---|---|
| `$like` `_` | `x` | `😀`, `x` |
| `$like` `__` | `ab`, `😀` | `ab`, `😀😀` |
| `$like` `a_b` | `axb` | `a😀b`, `axb` |
| `$like` `a__b` | `a😀b`, `axyb` | `a😀😀b`, `axyb` |
| `$like` `___` | `axb` | `a😀b`, `axb`, `A𝒜B` |
| `$like` `😀_` / `_😀` | none | `😀😀` |
| `$ilike` `_` | `x` | `😀`, `x` |
| `$ilike` `A_B` | `axb` | `a😀b`, `axb`, `A𝒜B` |
| `$ilike` `A__B` | `a😀b`, `axyb`, `A𝒜B` | `a😀😀b`, `axyb` |
The answer set moves both ways, so the changeset's declaration names the
measured direction arm, narrowing (BREAKING, graded `minor` in the
launch window), with an ADR-0087 `not-required
(no-migration-prescription)` disposition. No pattern is newly refused,
no refusal is lifted, and values made only of BMP characters answer
exactly as before.
### Every consumer of the translation (Zone 2 item 2), before → after
| consumer | before | after |
|---|---|---|
| `spec` `matchesLikePattern` | flagless `new RegExp`: UTF-16 |
`likePatternToRegExp`: code point |
| `formula` `matchesFilterCondition` | via `matchesLikePattern`: UTF-16
| code point |
| `driver-memory` `$like` / `$ilike` door (mingo) | local flagless `new
RegExp`: UTF-16 | `likePatternToRegExp`: code point |
| `driver-memory` AST `like` / `ilike` door (mingo) | local flagless
`new RegExp`: UTF-16 | `likePatternToRegExp`: code point |
| `driver-memory` reference matcher | via `matchesLikePattern`: UTF-16 |
code point |
| `driver-turso` `turso-20024-like-stored-nul` grid oracle |
`matchesLikePattern`; its rows are BMP-only | unchanged answers |
| `service-analytics`, `@objectstack/lint`, objectql `having`,
`driver-mongodb` | no consumer (`git grep` of both symbols) | none |
| objectui | `git grep` of both symbols: 0 hits at HEAD `5c61e5241` and
at the pinned `.objectui-sha` `f8a9d0fb` (exit 1 on both; a control grep
in the same tree hits) | none |
No other LIKE-to-regex translation exists in `packages/**/src`.
## `u`-flag side effects (Zone 2 items 3 and 4)
- **Escapes.** The translation escapes exactly `\ ^ $ . * + ? ( ) [ ] {
} |` and `/`. That is the regex syntax characters plus `/`, exactly the
identity escapes `u` accepts. Every other ASCII punctuation mark is
emitted bare, where `u` reads it as itself. This is pinned over **all
33** printable ASCII non-alphanumerics, bare and backslash-escaped, with
`foldAscii` off and on: each compiles under `u` and matches only itself.
The escaped set is pinned to equal that set. A control proves `u` really
rejects `\-` and `\:`.
- **`[\s\S]`, never `.`.** Both wildcards use `[\s\S]`, so under `u` a
`_` is one code point including a newline (pinned: source `^[\s\S]$`,
and `a_b` matches `a` newline `b`).
- **Surrogate pairs.** A character outside the BMP, literal or escaped,
is emitted as its whole pair (pinned: the source for backslash + `😀` is
`^😀$`), and `u` reads the pair as one code point. Under `u`, `%` never
splits a pair: `%` followed by a lone low surrogate does not match `😀`
(pinned).
- **`$ilike` / `foldAscii`.** The fold stays in the source as `[Aa]`
classes, unaffected by `u`. `likePatternToRegExp(...).flags` is pinned
to exactly `u`, with no `i`. The `$ilike` family cells are pinned on
every face.
## The stop condition: nothing needs the UTF-16 reading (Zone 2 item 5)
- Nothing in `@objectstack/formula` lowers CEL to `$like`:
`cel-to-filter` emits no `$like`, and `rls-predicate` leaves SQL `LIKE`
unparseable. The only `$like` evaluation is `matches-filter.ts`, via the
spec.
- CEL's own `size()` in `formula` already counts **code points**:
`size('😀')` is 1 and `size('a😀b')` is 3 (pinned in the formula suite).
The old `$like` answer disagreed with the same package's string length.
## Pins (every face, literal code-point rows, not parity alone)
| face | suite | cases |
|---|---|---|
| spec | `packages/spec/src/data/filter-like-code-point.test.ts` | 23 |
| formula |
`packages/formula/src/matches-filter-like-code-point.test.ts` | 12 |
| driver-memory, all three doors |
`packages/drivers/driver-memory/src/memory-20143-like-code-point.test.ts`
| 11 |
| driver-sql (better-sqlite3) |
`packages/drivers/driver-sql/src/sql-driver-20143-like-code-point.test.ts`
| 12 |
| driver-sqlite-wasm (sql.js) |
`packages/drivers/driver-sqlite-wasm/src/sqlite-wasm-20143-like-code-point.test.ts`
| 12 |
| driver-turso (local, remote stub, remote real libSQL) |
`packages/drivers/driver-turso/src/turso-20143-like-code-point.test.ts`
| 12 |
Each SQLite face also holds itself to the JS answer (`formula`, or the
spec's `matchesLikePattern` where `formula` is not a dependency), which
is the cross-face half.
## Ablation (committed fix first; one-shot, no permanent test file)
The mutation replaced the `'u'` flag argument with an expression that
evaluates to the empty flag string and carries the marker
`ABLATION20143`. It went through `scripts/ablation-replace.mjs` under
its restore trap, inside a script with its own `trap` restoring from
`HEAD`.
- **Mutate leg:** on disk, marker count 1 and anchor count 0. `pnpm
--filter @objectstack/spec build` exited 0, and
`ablation-dist-preflight.mjs @objectstack/spec ABLATION20143` exited 0,
so the marker is in `dist/`. The expected direction was written down
before running: JS faces red on the family cells; SQLite faces green on
their literal cells and red only on the cross-face leg. Observed: spec
**13 failed** / 23; formula **11** / 12 (CEL `size()` stays green);
driver-memory **10** / 11; driver-sql **1** / 12, driver-sqlite-wasm
**1** / 12, driver-turso **1** / 12 (each the cross-face leg only). That
is the predicted direction.
- **Restore leg:** `git hash-object` equals the `HEAD` blob
`838a80720b3d`, `git diff HEAD` is empty, and `git status --porcelain`
is clean. Rebuild exited 0, `ablation-dist-preflight.mjs ... --absent`
exited 0, and all six suites are green again.
## Verification (at `713d1504`, after merging `origin/main` at
`585c9af5`)
- Build of the six-package closure: turbo 10/10. `pnpm --filter
@objectstack/spec check:generated` passed after the merge. Regenerated
here: `api-surface/data.json` (**0 breaking, 1 added**:
`likePatternToRegExp`), `export-origins/data.json`,
`content/docs/references/data/filter.mdx`.
- Full suites: spec 542 files / 15,943 passed (2 todo); formula 38 /
1,039; driver-memory 55 / 1,339; driver-sqlite-wasm 35 / 656;
driver-turso 73 / 1,921 (16 skipped); driver-sql 189 files (+11 skipped)
/ 3,111 passed (170 skipped).
- Typecheck, exit 0 on all six: spec, formula, driver-memory,
driver-sql, driver-sqlite-wasm, driver-turso. Each new test file is
counted inside a compiled program with `--listFiles`: `tsconfig.json`
for the four drivers, and `tsconfig.test.json` (via
`check:test-typecheck`) for spec and formula.
- Gates: 108 derived at `713d1504` (`dispatch-gates.mjs --commands
--repo objectstack-ai/objectstack`), reconciled with `--ran`: **101 exit
0, 0 unrun, 7 NOT MEASURED**. All seven are exit 3, PREREQUISITE NOT
MET:
- `check:doc-formula-expressions`, `check:doc-security-posture` and
`check:docs-transcript-drift` need `@objectstack/lint` built.
- `check:lean-entry-closure` needs `objectql` built, and
`check:skill-examples` needs `client` / `client-react` built.
- `check:dual-build-cjs-loads` and `check:type-check-debt` need the
whole workspace built.
The targeted prerequisite build never acquired the shared verify-lock
(two 9-minute queue-timeouts behind other seats' batches), so those
seven are declared to CI's Lint & Repo Gates. The seven roster families
whose roster sits in a touched directory were run as well, all exit 0:
`check:meta-url-spelling`, `check:spec-changes`, `check:authz-resolver`,
`check:error-code-casing`, `check:filter-alias-parity`,
`check:object-def-param-keys`, `check:tenant-chokepoint`.
`check-adr-0087-registration` reads the changeset as
`[clause-②-narrowing] not-required (no-migration-prescription)`.
## Acceptance notes
- Three hand-written surfaces restate "`_` is exactly one character":
`content/docs/api/client-sdk.mdx:553`,
`content/docs/deployment/troubleshooting.mdx:148`, and the
`packages/client/src/query-builder.ts:88` JSDoc. They stay true under
the reading the spec now defines, and are not edited here. Carrier:
none.
- `likePatternToRegexSource` stays exported, so a future consumer could
still compile it without `u`. Its docblock carries the prohibition, and
after this PR nothing outside the spec calls it. Retiring the export
would be a removal and is not this card.
- A value or pattern holding an unpaired surrogate is not measured.
UTF-8 SQLite cannot store one, and the JS faces under `u` treat it as
its own code point.
- The Postgres and MySQL `_` arms are NOT MEASURED, as on the card. SQL
`_` is one character there.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN)_
---------
Co-authored-by: Claude <noreply@anthropic.com>_ is one Unicode code point on every JS face (#20222)1 parent 6d2571f commit 93cfc3f
12 files changed
Lines changed: 779 additions & 21 deletions
File tree
- .changeset
- content/docs/references/data
- packages
- drivers
- driver-memory/src
- driver-sqlite-wasm/src
- driver-sql/src
- driver-turso/src
- formula/src
- spec
- api-surface
- export-origins
- src/data
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
12 | | - | |
| 12 | + | |
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| |||
1393 | 1393 | | |
1394 | 1394 | | |
1395 | 1395 | | |
| 1396 | + | |
| 1397 | + | |
1396 | 1398 | | |
1397 | | - | |
| 1399 | + | |
1398 | 1400 | | |
1399 | 1401 | | |
1400 | 1402 | | |
| |||
1618 | 1620 | | |
1619 | 1621 | | |
1620 | 1622 | | |
1621 | | - | |
1622 | | - | |
| 1623 | + | |
| 1624 | + | |
1623 | 1625 | | |
1624 | 1626 | | |
1625 | 1627 | | |
1626 | 1628 | | |
1627 | 1629 | | |
1628 | | - | |
1629 | | - | |
1630 | | - | |
| 1630 | + | |
| 1631 | + | |
| 1632 | + | |
| 1633 | + | |
| 1634 | + | |
1631 | 1635 | | |
1632 | 1636 | | |
1633 | 1637 | | |
| |||
1640 | 1644 | | |
1641 | 1645 | | |
1642 | 1646 | | |
1643 | | - | |
| 1647 | + | |
1644 | 1648 | | |
1645 | 1649 | | |
1646 | 1650 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
0 commit comments