Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 15 additions & 6 deletions apps/site/public/llms-small.txt
Original file line number Diff line number Diff line change
Expand Up @@ -49,24 +49,33 @@ Distinct from success `alternatives`, failure `candidate`, and issue `suggestion
import { createLingo } from "@pascal-app/lingo"
import { es } from "@pascal-app/lingo/locales/es"
import { fr } from "@pascal-app/lingo/locales/fr"
import { ja } from "@pascal-app/lingo/locales/ja"
import { zh } from "@pascal-app/lingo/locales/zh"

const lingo = createLingo({ locales: [es, fr, zh] })
const lingo = createLingo({ locales: [es, fr, ja, zh] })
lingo.parseQuantity("dos kg") // auto-detected as es; result.locale === "es"
lingo.parseRange("entre 5 et 10 kg", { locale: "fr" })
lingo.parseQuantity("5公斤", { locale: "zh" })
lingo.parseQuantity("mille cinq cents metres", { locale: "fr" }) // 1500 m

// Comparators that FOLLOW the quantity (CJK); these return ranges, not quantities.
lingo.parseRange("5キロ未満", { locale: "ja" }) // { min: null, max: 5 kg }
lingo.parseRange("5公斤以上", { locale: "zh" }) // { min: 5 kg, max: null }
lingo.parseQuantity("100元", { locale: "zh" }) // 100 CNY (pack currency default)
```

Packs: `en`, `en-gb`, `es`, `fr`, `pt`, `zh`, `ja`. Omit `locale` to detect among loaded packs plus English; pass `locale` for known fields. English is built in and does not require a pack. `parseDate(text, { locale, localePacks, now })` can use loaded packs for relative date vocabulary.

Pack notes:
- `en`: built-in English quantity/date grammar; wave-1 adds spoken decimals and date idioms such as `two point five kg` and `quarter of five`.
- `en-gb`: English plus day-first numeric dates and UK weekday offsets (`Monday week`, `Tuesday fortnight`).
- `es`: Spanish number/range words, Romance composition, spoken decimals, approximants, and date clock/edge idioms.
- `fr`: French number/range words, exhaustive CLDR-style vigesimal compounds, spoken decimals, approximants, and weekday-offset/clock idioms.
- `pt`: Portuguese number/range words, compound hundreds/tens, spoken decimals, approximants, and clock/day-offset idioms.
- `zh`: Chinese CJK number walker, scale grouping, post-unit half, duration counters, period edges, and day offsets. CJK spoken clock (`下午3点半`) is deferred.
- `ja`: Japanese CJK number walker, scale grouping, post-unit half, duration counters, wave-dash ranges, and double-step period modifiers. CJK spoken clock (`午後3時半`) is deferred.
- `es`: Spanish number/range words, Romance composition, spoken decimals, approximants, and date clock/edge idioms. Long scale: `mil millones` = 1e9, `billón` = 1e12. Ordinal days (`el 1º de marzo`).
- `fr`: French number/range words, exhaustive CLDR-style vigesimal compounds, spoken decimals, approximants, and weekday-offset/clock idioms. Long scale: `milliard` = 1e9, `billion` = 1e12. Ordinal days (`le 1er mars`).
- `pt`: Portuguese number/range words, compound hundreds/tens (`cento e vinte`, `mil e quinhentos`), spoken decimals, approximants, clock/day-offset idioms, `-feira` weekdays, and locative date fillers (`na proxima segunda-feira`).
- `zh`: Chinese CJK number walker, scale grouping, post-unit half, duration counters, period edges, day offsets, postpositional bounds (`5公斤以上`, `5公斤以下`, `5公斤以内`), CNY default plus `元`/`块` aliases, suffix dates (`2026年3月5日`, `3月5日`), weekdays (`星期三`, `周三`), and spoken clocks with day periods (`下午3点半`, `三点一刻`, `明天下午3点`).
- `ja`: Japanese CJK number walker, scale grouping, post-unit half, duration counters, wave-dash ranges, double-step period modifiers, postpositional bounds (`5キロ未満`, `5キロ以上`, `5キロ超`), JPY default plus `円` aliases, suffix dates (`2026年3月5日`), weekdays (`水曜日`), and spoken clocks with day periods (`午後3時半`, `明日午後3時`).

Note: `以上`/`未満`-style comparators produce an open RANGE, so read them with `parseRange`/`rangeField`, not `parseQuantity`. The shared `¥`/`¥` symbol resolves per locale (CNY under `zh`, JPY under `ja`).

Verified locale examples (fixed `now = 2026-07-03 14:30` local time for date examples):

Expand Down
31 changes: 25 additions & 6 deletions apps/site/src/components/site/parse-playground.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,31 @@ const localeLingo = createLingo({ locales: [es, fr, pt, zh, ja, enGb] })

const EXAMPLES: Record<LocaleChoice, readonly string[]> = {
auto: ['72 in to cm', 'dos kg', 'entre 5 et 10 kg', '5公斤', '暑い'],
en: ['2 ft', '72 in to cm', 'between 5 and 10 kg', '5 meters', "it's hot"],
es: ['dos kg', 'entre 5 y 10 kg', '72 pulgadas a cm', 'dos metros y medio', 'al menos 2 m'],
fr: ['deux kg', 'entre 5 et 10 kg', '72 pouces en cm', 'deux metres et demi', 'au moins 2 m'],
pt: ['dois kg', 'entre 5 e 10 kg', '72 polegadas em cm', 'pelo menos 2 m'],
zh: ['5公斤', '3米', '5到10公斤', '很热'],
ja: ['5キロ', '3メートル', '5から10キロ', '暑い'],
en: [
'2 ft',
'72 in to cm',
'between 5 and 10 kg',
'between one thousand and two thousand meters',
"it's hot",
],
es: [
'dos kg',
'entre 5 y 10 kg',
'72 pulgadas a cm',
'ciento veinte kg',
'mil millones de kg',
'al menos 2 m',
],
fr: [
'deux kg',
'entre 5 et 10 kg',
'72 pouces en cm',
'mille cinq cents metres',
'deux metres et demi',
],
pt: ['dois kg', 'entre 5 e 10 kg', 'cento e vinte metros', 'mil e quinhentos metros'],
zh: ['5公斤', '5公斤以上', '5公斤以下', '五公斤左右', '100元', '很热'],
ja: ['5キロ', '5キロ未満', '5キロ以上', '5キロぐらい', '1000円', '暑い'],
'en-gb': ['12 stone', '3 quid', 'roundabout 2 m'],
}

Expand Down
8 changes: 7 additions & 1 deletion apps/site/src/lib/code-snippets.ts
Original file line number Diff line number Diff line change
Expand Up @@ -164,13 +164,19 @@ humanizeDuration(5400, { style: "natural" }) // "an hour and a half"`
export const localeSnippet = `import { createLingo } from "@pascal-app/lingo"
import { es } from "@pascal-app/lingo/locales/es"
import { fr } from "@pascal-app/lingo/locales/fr"
import { ja } from "@pascal-app/lingo/locales/ja"
import { zh } from "@pascal-app/lingo/locales/zh"

const lingo = createLingo({ locales: [es, fr, zh] })
const lingo = createLingo({ locales: [es, fr, ja, zh] })

lingo.parseQuantity("dos kg") // auto-detected as es
lingo.parseRange("entre 5 et 10 kg", { locale: "fr" })
lingo.parseQuantity("5公斤", { locale: "zh" })
lingo.parseQuantity("mille cinq cents metres", { locale: "fr" }) // 1500 m

// Comparators that follow the quantity, as CJK writes them.
lingo.parseRange("5キロ未満", { locale: "ja" }) // < 5 kg
lingo.parseRange("5公斤以上", { locale: "zh" }) // >= 5 kg

// Success results expose the resolved profile.
lingo.parse("dos kg").locale // "es"`
Expand Down
4 changes: 3 additions & 1 deletion apps/site/src/lib/docs.md.ts
Original file line number Diff line number Diff line change
Expand Up @@ -318,7 +318,9 @@ lingo.parse('72 in to cm') // locale: 'en'`,
'',
'Published packs: `@pascal-app/lingo/locales/en`, `en-gb`, `es`, `fr`, `pt`, `zh`, and `ja`. Pack-owned CJK unit aliases and fuzzy words are only registered when their pack is loaded. Passing an explicit locale that was not loaded returns `LOCALE_NOT_LOADED` instead of silently falling back to English.',
'',
'Date parsing accepts the same packs directly: `parseDate("mañana", { now, locale: "es", localePacks: [es] })`.',
'Each pack reads its language the way people write it, not a transliteration of English. Romance packs cover tens/hundreds/scale words (`ciento veinte`, `mille cinq cents`, `mil millones`), ordinal days (`le 1er mars`), and spoken decimals (`dos coma cinco`). The Chinese and Japanese packs cover grouped numerals (`一百五十万`), comparators that follow the quantity (`5キロ未満`, `5公斤以上`), trailing approximants (`五公斤左右`), and per-locale currency defaults — the shared `¥` resolves to CNY under `zh` and JPY under `ja`.',
'',
'Date parsing accepts the same packs directly: `parseDate("mañana", { now, locale: "es", localePacks: [es] })`. Localized date grammar includes spoken clocks (`las tres menos cuarto`, `deux heures et quart`), period edges (`a finales de mes`), weekday offsets (`lundi en huit`), and CJK calendar forms — suffix dates (`2026年3月5日`, `3月5日`), weekdays (`星期三`, `水曜日`), clocks with day periods (`午後3時半`, `下午3点`), and date+time written without a space (`明天下午3点`).',
'',
'## Catalog',
'',
Expand Down
48 changes: 48 additions & 0 deletions packages/lingo/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,56 @@ change**, even if the API is untouched.

## [Unreleased]

### Added

- Chinese and Japanese calendar and clock grammar: numeric dates written with
suffixes (`2026年3月5日`, `3月5日`, and years spelled digit-by-digit as
`二〇二六年`), weekdays (`星期三`, `周三`, `水曜日`), clocks closed by
`点`/`時`/`分`/`秒` with day periods (`下午3点`, `午前9時半`, `3点一刻`), and
date+time compounds written with no separating space (`明天下午3点`,
`明日午後3時`).
- Postpositional bound phrases for scripts that put the comparator after the
quantity: `5キロ未満`, `5キロ以上`, `5公斤以下`, `5公斤以内`, `5キロ超`.
Previously `5キロ未満` parsed as a bare `5 kg`, dropping the bound.
- Chinese and Japanese default currencies, so the shared `¥`/`¥` symbol
resolves per locale (`¥100` is CNY under `zh`, JPY under `ja`), plus the
`元`/`圆`/`块`/`人民币` and `円`/`えん` aliases.
- French long-scale number words (`milliard` = 10^9, `billion` = 10^12),
Spanish `mil millones`/`billón`, Portuguese `cento`, and Romance ordinal
day-of-month forms (`le 1er mars`, `el 1º de marzo`).
- A per-locale benchmark suite (`bun run bench`) so multi-language throughput is
measured rather than assumed.
- Per-locale corpus contracts gained 96 rows and the English contract 5,
covering the grammar above plus the number-word fixes below.

### Fixed

- Number words: hundreds now multiply only the 1..99 group in front of them, so
French `mille cinq cents` is 1500 (was 100500). A banked smaller scale
multiplies the next one, so Spanish `mil millones` is 10^9 (was 1,001,000) and
`dos mil millones` is 2×10^9. Both were silent wrong answers.
- `between A and B` now keeps the and-word for the range when both sides are
spelled scale words: `between one thousand and two thousand meters`,
`entre mille et deux mille`, `entre mil y dos mil` (all previously failed to
parse). `between five and a half and ten kg` still reads 5.5..10 — the
fraction tail binds tighter than the range separator.
- Portuguese `cento e vinte` (120) and `mil e quinhentos` (1500) parse; the
and-word after a bare scale word links it to its remainder.
- Portuguese locative contractions are date fillers, so
`na proxima segunda-feira` resolves.
- Scripts written without word spaces no longer swallow grammar words glued to a
quantity (`三至五天`, `5公斤左右`). Unit aliases stay atomic through the split,
so `一時間半` remains 1.5 hours rather than splitting at the range word `間`.

### Changed

- Resolved language profiles and locale-detection scans are memoized per pack
set, so repeated parses with `locale`/auto-detection no longer re-merge packs
or re-tokenize for detection on every call.
- Size budgets recalibrated once for this wave (see D70 in `wiki/decisions.md`
for the measured numbers and rationale); `scripts/size.mjs` remains the source
of truth. No previously-pinned corpus interpretation changed — the wave is
additive.
- README (npm): repo-relative links (`docs/recipes.md`, `plans/`, `wiki/`) now
point at absolute GitHub URLs so they resolve on npmjs.com, and the docs
site plus the agent `llms.txt` index are linked from the top.
Expand Down
22 changes: 16 additions & 6 deletions packages/lingo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -559,24 +559,34 @@ spanish.ok && spanish.quantity.to('kg').value // 35
const chinese = app.parseQuantity('三十五公斤', { locale: 'zh' })
chinese.ok && chinese.quantity.to('kg').value // 35

// comparators that follow the quantity, as CJK writes them
app.parse('5キロ未満', { locale: 'ja' }) // < 5 kg (open range)
app.parse('5公斤以上', { locale: 'zh' }) // ≥ 5 kg
app.parse('100元', { locale: 'zh' }) // 100 CNY

const clock = parseDate('las tres menos cuarto', {
locale: 'es',
localePacks: [es],
now: new Date(2026, 6, 3, 14, 30),
})
clock.ok && [clock.date.getHours(), clock.date.getMinutes()] // [2, 45]

const cjkDate = parseDate('2026年3月5日', { locale: 'zh', localePacks: [zh] })
cjkDate.ok && [cjkDate.date.getMonth() + 1, cjkDate.date.getDate()] // [3, 5] (local)

// omit `locale` to auto-detect among the loaded packs
```

Packs (`en`, `en-gb`, `es`, `fr`, `pt`, `zh`, `ja`) are additive and tree-shakeable;
they add number words, units, ranges, and relative dates (through
`@pascal-app/lingo/date`) for their language. The first idiom wave includes
Romance tens/hundreds/decimal words, CJK scale/grouped numbers and post-unit
half, localized clock phrases, period edges, weekday offsets, and locale unit
words in date/duration parsing. Requesting an unloaded locale returns
`LOCALE_NOT_LOADED` rather than silently parsing as English; `humanizeDate()` output
stays English for now.
`@pascal-app/lingo/date`) for their language. Coverage includes Romance
tens/hundreds/scale words (`ciento veinte`, `mille cinq cents`, `mil millones`),
CJK scale/grouped numbers and post-unit half, postpositional comparators
(`5キロ未満`, `5公斤以上`), per-locale currency defaults (`¥` is CNY under `zh`,
JPY under `ja`), localized clock phrases and period edges, and CJK calendar
dates and clocks (`2026年3月5日`, `午後3時半`, `明天下午3点`). Requesting an
unloaded locale returns `LOCALE_NOT_LOADED` rather than silently parsing as
English; `humanizeDate()` output stays English for now.

The global `lingo()` is itself a `createLingo()` singleton. One code path,
two tiers of convenience.
Expand Down
21 changes: 15 additions & 6 deletions packages/lingo/llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -49,24 +49,33 @@ Distinct from success `alternatives`, failure `candidate`, and issue `suggestion
import { createLingo } from "@pascal-app/lingo"
import { es } from "@pascal-app/lingo/locales/es"
import { fr } from "@pascal-app/lingo/locales/fr"
import { ja } from "@pascal-app/lingo/locales/ja"
import { zh } from "@pascal-app/lingo/locales/zh"

const lingo = createLingo({ locales: [es, fr, zh] })
const lingo = createLingo({ locales: [es, fr, ja, zh] })
lingo.parseQuantity("dos kg") // auto-detected as es; result.locale === "es"
lingo.parseRange("entre 5 et 10 kg", { locale: "fr" })
lingo.parseQuantity("5公斤", { locale: "zh" })
lingo.parseQuantity("mille cinq cents metres", { locale: "fr" }) // 1500 m

// Comparators that FOLLOW the quantity (CJK); these return ranges, not quantities.
lingo.parseRange("5キロ未満", { locale: "ja" }) // { min: null, max: 5 kg }
lingo.parseRange("5公斤以上", { locale: "zh" }) // { min: 5 kg, max: null }
lingo.parseQuantity("100元", { locale: "zh" }) // 100 CNY (pack currency default)
```

Packs: `en`, `en-gb`, `es`, `fr`, `pt`, `zh`, `ja`. Omit `locale` to detect among loaded packs plus English; pass `locale` for known fields. English is built in and does not require a pack. `parseDate(text, { locale, localePacks, now })` can use loaded packs for relative date vocabulary.

Pack notes:
- `en`: built-in English quantity/date grammar; wave-1 adds spoken decimals and date idioms such as `two point five kg` and `quarter of five`.
- `en-gb`: English plus day-first numeric dates and UK weekday offsets (`Monday week`, `Tuesday fortnight`).
- `es`: Spanish number/range words, Romance composition, spoken decimals, approximants, and date clock/edge idioms.
- `fr`: French number/range words, exhaustive CLDR-style vigesimal compounds, spoken decimals, approximants, and weekday-offset/clock idioms.
- `pt`: Portuguese number/range words, compound hundreds/tens, spoken decimals, approximants, and clock/day-offset idioms.
- `zh`: Chinese CJK number walker, scale grouping, post-unit half, duration counters, period edges, and day offsets. CJK spoken clock (`下午3点半`) is deferred.
- `ja`: Japanese CJK number walker, scale grouping, post-unit half, duration counters, wave-dash ranges, and double-step period modifiers. CJK spoken clock (`午後3時半`) is deferred.
- `es`: Spanish number/range words, Romance composition, spoken decimals, approximants, and date clock/edge idioms. Long scale: `mil millones` = 1e9, `billón` = 1e12. Ordinal days (`el 1º de marzo`).
- `fr`: French number/range words, exhaustive CLDR-style vigesimal compounds, spoken decimals, approximants, and weekday-offset/clock idioms. Long scale: `milliard` = 1e9, `billion` = 1e12. Ordinal days (`le 1er mars`).
- `pt`: Portuguese number/range words, compound hundreds/tens (`cento e vinte`, `mil e quinhentos`), spoken decimals, approximants, clock/day-offset idioms, `-feira` weekdays, and locative date fillers (`na proxima segunda-feira`).
- `zh`: Chinese CJK number walker, scale grouping, post-unit half, duration counters, period edges, day offsets, postpositional bounds (`5公斤以上`, `5公斤以下`, `5公斤以内`), CNY default plus `元`/`块` aliases, suffix dates (`2026年3月5日`, `3月5日`), weekdays (`星期三`, `周三`), and spoken clocks with day periods (`下午3点半`, `三点一刻`, `明天下午3点`).
- `ja`: Japanese CJK number walker, scale grouping, post-unit half, duration counters, wave-dash ranges, double-step period modifiers, postpositional bounds (`5キロ未満`, `5キロ以上`, `5キロ超`), JPY default plus `円` aliases, suffix dates (`2026年3月5日`), weekdays (`水曜日`), and spoken clocks with day periods (`午後3時半`, `明日午後3時`).

Note: `以上`/`未満`-style comparators produce an open RANGE, so read them with `parseRange`/`rangeField`, not `parseQuantity`. The shared `¥`/`¥` symbol resolves per locale (CNY under `zh`, JPY under `ja`).

Verified locale examples (fixed `now = 2026-07-03 14:30` local time for date examples):

Expand Down
Loading
Loading