Skip to content

feat(locale): CJK calendars, postpositional bounds, and number-word arithmetic fixes - #4

Merged
Aymericr merged 1 commit into
mainfrom
feat/locale-depth-and-perf
Jul 27, 2026
Merged

feat(locale): CJK calendars, postpositional bounds, and number-word arithmetic fixes#4
Aymericr merged 1 commit into
mainfrom
feat/locale-depth-and-perf

Conversation

@Aymericr

Copy link
Copy Markdown
Contributor

Wave 2 of plan 033. Wave 1 taught Chinese and Japanese to read numbers; this teaches them to read calendars, and fixes three silent wrong answers in the Romance number engine.

The review that produced this started from "how good is our multi-language support, really" — so the notable finding is that two of the gaps were silent: input parsed successfully with the wrong value, which for a validation library is worse than failing.

The silent wrong answers

input locale before after
5キロ未満 ja 5 kg (bound dropped) < 5 kg
mille cinq cents metres fr 100500 m 1500 m
mil millones de kg es 1,001,000 kg 1e9 kg
between one thousand and two thousand meters en failed to parse 1000..2000 m
cento e vinte metros pt failed to parse 120 m

5キロ未満 is the one that motivated the tokenization work: 未満 was glued to the unit token, so nothing ever saw the comparator.

Four mechanisms, all pack-data-gated

English pays nothing for any of this — each path is entered only when a pack declares the relevant field.

  1. Glued-grammar splitting. Scripts without word spaces write grammar and content as one token, so tokenization now cuts word tokens at the profile's non-Latin grammar vocabulary. The subtle part: the cut must be suppressed inside a unit alias, because is a range word but 時間 is the hour unit — otherwise 一時間半 splits into a range. That's why splitGluedWords takes an alias-length callback.
  2. Postpositional bounds. GrammarBoundPhrase.suffix declares comparators that follow the quantity (以上, 未満, 以内, ), consumed by parseTrailingBound in both the quantity and range paths.
  3. Suffix-delimited dates and clocks (date/suffix.ts, date/numeral.ts): 2026年3月5日, 3月5日, 午後3時半, 三点一刻, 下午3点, plus date+time written without a space (明天下午3点) split at day-period and clock-suffix anchors. Years spelled digit-by-digit (二〇二六年) read as a digit run — positional reading would give 2+0+2+6.
  4. Number-word arithmetic. Hundreds now multiply only the 1..99 group in front of them; a banked smaller scale multiplies the next one; the and-word after a bare scale links its remainder.

Fixing (4) exposed its mirror image: inside between A and B the and-word belongs to the range, so the existing noAnd flag now threads from range.ts through ValueCtx into the word engine. It deliberately does not suppress the fraction tail — between five and a half and ten kg still reads 5.5..10. (Blanking andWords wholesale looks equivalent and breaks that case; there's a regression test.)

Also in here

  • zh/ja currency defaults — the shared /¥ now resolves per locale (CNY under zh, JPY under ja), plus //人民币 and aliases. Previously ¥100 read as JPY in Chinese text.
  • Romance depth — long scales (milliard/billion, mil millones/billón), ordinal days (le 1er mars), Portuguese cento, and Portuguese locative date fillers so na proxima segunda-feira resolves.
  • Performance — resolved profiles and detection scans are memoized per pack-array identity. Measured on the memoized functions in isolation (fresh array literal reproduces the old cost exactly): resolveLanguageProfile 15.1 → 0.04 µs per call, detectLocale 85 → 2.5 µs per call. Profile resolution previously ran on every parse, allocating ~40 Sets/Records each time.
  • A per-locale benchmark suite, so multi-language throughput is measured rather than assumed: explicit locale with 5 packs loaded is 5.66 µs/op, auto-detect romance 13.98 µs/op, auto-detect CJK 8.10 µs/op.

Two things worth a second opinion

Size budgets were recalibrated (approved before landing, per the D19 escalation rule). The growth is capability, not drift, and ./ai moved purely as a cascade — no /ai code changed:

entry before after budget
main 38.19 39.18 39.4
./core 25.46 26.56 26.8
./date standalone 40.85 43.01 43.3
./date marginal 14.29 15.46 15.7
./ai marginal 17.27 18.39 18.6
zh / ja packs 1.23 / 1.31 1.82 / 1.79 2.0 each

The noAnd threading touches the shared English path. It's guarded by the corpus contract (zero changes to any pinned row) and by explicit guard tests, but it's the one change here that isn't locale-isolated, so it's the place to look hardest.

Verification

  • bun run check green: typecheck, 980 tests, build, size, corpus gate, zero-deps gate.
  • Corpus contracts: zero breaking changes across all six (en/es/fr/pt/zh/ja) — the whole wave is additive: +96 locale rows, +5 English rows.
  • Site typechecks and production-builds; playground examples and the agent-facing llms.txt updated (it previously advertised the CJK clock as deferred).
  • Every code snippet added to the README, llms.txt, and site docs was executed before landing, not written from memory. One caught a real mistake: a toISOString() example that renders March 4 in UTC+2.

Docs

D70 in wiki/decisions.md records the mechanisms, the rejected alternatives, and the measured budget numbers; the two non-obvious invariants (alias-guarded splitting, noAnd sparing the fraction tail) are in wiki/architecture.md under key mechanisms. Plans 031/033 updated, and the wave-1 deferrals this retires are marked in plans/backlog.md alongside five new wave-3 items found along the way (Japanese era years, CJK suffix-date ranges, 3月5日(木) weekday agreement, positional CJK numerals in dates).

Made with Cursor

Wave 2 of plan 033. Chinese and Japanese could parse numbers but not
calendars, and comparators that follow the quantity were silently dropped:
`5キロ未満` parsed as a bare `5 kg`, losing the bound entirely.

Four mechanisms, all pack-data-gated so English pays nothing:

- Tokenization splits word tokens at pack-declared non-Latin grammar
  vocabulary, since scripts without word spaces glue grammar to content.
  Unit aliases stay atomic, so `一時間半` no longer risks splitting at the
  range word `間`.
- `GrammarBoundPhrase.suffix` declares comparators that follow the quantity
  (`以上`, `未満`, `以内`, `超`), read by `parseTrailingBound`.
- `date/suffix.ts` + `date/numeral.ts` parse suffix-delimited dates and
  clocks: `2026年3月5日`, `午後3時半`, `下午3点`, unspaced `明天下午3点`, and
  digit-by-digit years (`二〇二六年`).
- Number-word arithmetic: hundreds bind to the 1..99 group in front of them
  (`mille cinq cents` was 100500, now 1500), a banked smaller scale
  multiplies the next (`mil millones` was 1,001,000, now 1e9), and the
  and-word after a bare scale links its remainder (`cento e vinte`).

Fixing the last one exposed its mirror: inside `between A and B` the
and-word belongs to the range, so the existing `noAnd` flag now threads from
range.ts through ValueCtx into the word engine. It deliberately spares the
fraction tail, so `between five and a half and ten kg` still reads 5.5..10.

Also: zh/ja currency defaults (shared `¥` resolves per locale), Portuguese
`cento` and locative date fillers, memoized profile resolution and detection
scans, and a per-locale benchmark suite.

Every existing corpus contract row is unchanged; the wave is additive
(+96 locale rows, +5 English). Budgets recalibrated once per D19 escalation
with owner approval — see D70.

Co-authored-by: Cursor <cursoragent@cursor.com>
@vercel

vercel Bot commented Jul 27, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
lingo Ready Ready Preview, Comment Jul 27, 2026 11:23am

@github-actions

Copy link
Copy Markdown
Contributor

React Doctor found no new issues. 🎉

Reviewed by React Doctor for commit 71fd784.

@Aymericr
Aymericr merged commit a60be5e into main Jul 27, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant