feat(locale): CJK calendars, postpositional bounds, and number-word arithmetic fixes - #4
Merged
Merged
Conversation
Wave 2 of plan 033. Chinese and Japanese could parse numbers but not calendars, and comparators that follow the quantity were silently dropped: `5キロ未満` parsed as a bare `5 kg`, losing the bound entirely. Four mechanisms, all pack-data-gated so English pays nothing: - Tokenization splits word tokens at pack-declared non-Latin grammar vocabulary, since scripts without word spaces glue grammar to content. Unit aliases stay atomic, so `一時間半` no longer risks splitting at the range word `間`. - `GrammarBoundPhrase.suffix` declares comparators that follow the quantity (`以上`, `未満`, `以内`, `超`), read by `parseTrailingBound`. - `date/suffix.ts` + `date/numeral.ts` parse suffix-delimited dates and clocks: `2026年3月5日`, `午後3時半`, `下午3点`, unspaced `明天下午3点`, and digit-by-digit years (`二〇二六年`). - Number-word arithmetic: hundreds bind to the 1..99 group in front of them (`mille cinq cents` was 100500, now 1500), a banked smaller scale multiplies the next (`mil millones` was 1,001,000, now 1e9), and the and-word after a bare scale links its remainder (`cento e vinte`). Fixing the last one exposed its mirror: inside `between A and B` the and-word belongs to the range, so the existing `noAnd` flag now threads from range.ts through ValueCtx into the word engine. It deliberately spares the fraction tail, so `between five and a half and ten kg` still reads 5.5..10. Also: zh/ja currency defaults (shared `¥` resolves per locale), Portuguese `cento` and locative date fillers, memoized profile resolution and detection scans, and a per-locale benchmark suite. Every existing corpus contract row is unchanged; the wave is additive (+96 locale rows, +5 English). Budgets recalibrated once per D19 escalation with owner approval — see D70. Co-authored-by: Cursor <cursoragent@cursor.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
React Doctor found no new issues. 🎉 Reviewed by React Doctor for commit |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wave 2 of plan 033. Wave 1 taught Chinese and Japanese to read numbers; this teaches them to read calendars, and fixes three silent wrong answers in the Romance number engine.
The review that produced this started from "how good is our multi-language support, really" — so the notable finding is that two of the gaps were silent: input parsed successfully with the wrong value, which for a validation library is worse than failing.
The silent wrong answers
5キロ未満5 kg(bound dropped)< 5 kgmille cinq cents metres100500 m1500 mmil millones de kg1,001,000 kg1e9 kgbetween one thousand and two thousand meters1000..2000 mcento e vinte metros120 m5キロ未満is the one that motivated the tokenization work:未満was glued to the unit token, so nothing ever saw the comparator.Four mechanisms, all pack-data-gated
English pays nothing for any of this — each path is entered only when a pack declares the relevant field.
間is a range word but時間is the hour unit — otherwise一時間半splits into a range. That's whysplitGluedWordstakes an alias-length callback.GrammarBoundPhrase.suffixdeclares comparators that follow the quantity (以上,未満,以内,超), consumed byparseTrailingBoundin both the quantity and range paths.date/suffix.ts,date/numeral.ts):2026年3月5日,3月5日,午後3時半,三点一刻,下午3点, plus date+time written without a space (明天下午3点) split at day-period and clock-suffix anchors. Years spelled digit-by-digit (二〇二六年) read as a digit run — positional reading would give 2+0+2+6.Fixing (4) exposed its mirror image: inside
between A and Bthe and-word belongs to the range, so the existingnoAndflag now threads fromrange.tsthroughValueCtxinto the word engine. It deliberately does not suppress the fraction tail —between five and a half and ten kgstill reads 5.5..10. (BlankingandWordswholesale looks equivalent and breaks that case; there's a regression test.)Also in here
¥/¥now resolves per locale (CNY underzh, JPY underja), plus元/块/人民币and円aliases. Previously¥100read as JPY in Chinese text.milliard/billion,mil millones/billón), ordinal days (le 1er mars), Portuguesecento, and Portuguese locative date fillers sona proxima segunda-feiraresolves.resolveLanguageProfile15.1 → 0.04 µs per call,detectLocale85 → 2.5 µs per call. Profile resolution previously ran on every parse, allocating ~40 Sets/Records each time.Two things worth a second opinion
Size budgets were recalibrated (approved before landing, per the D19 escalation rule). The growth is capability, not drift, and
./aimoved purely as a cascade — no/aicode changed:./core./datestandalone./datemarginal./aimarginalThe
noAndthreading touches the shared English path. It's guarded by the corpus contract (zero changes to any pinned row) and by explicit guard tests, but it's the one change here that isn't locale-isolated, so it's the place to look hardest.Verification
bun run checkgreen: typecheck, 980 tests, build, size, corpus gate, zero-deps gate.llms.txtupdated (it previously advertised the CJK clock as deferred).llms.txt, and site docs was executed before landing, not written from memory. One caught a real mistake: atoISOString()example that renders March 4 in UTC+2.Docs
D70 in
wiki/decisions.mdrecords the mechanisms, the rejected alternatives, and the measured budget numbers; the two non-obvious invariants (alias-guarded splitting,noAndsparing the fraction tail) are inwiki/architecture.mdunder key mechanisms. Plans 031/033 updated, and the wave-1 deferrals this retires are marked inplans/backlog.mdalongside five new wave-3 items found along the way (Japanese era years, CJK suffix-date ranges,3月5日(木)weekday agreement, positional CJK numerals in dates).Made with Cursor