From 5397e1204467584fb0d6c2bedd641a3f0474e4e4 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 2 Nov 2026 07:22:07 +0000 Subject: [PATCH 1/5] feat(spec): a locale tag written in another case is corrected to its BCP-47 case LocaleSchema now returns each tag in its BCP-47 case (zh-cn -> zh-CN, zh-hans-cn -> zh-Hans-CN, EN -> en) instead of accepting the authored case, so a translation saved as zh-cn is merged into the zh-CN locale a session asks for. The fold is case-only: Intl.getCanonicalLocales also replaces legacy subtags and throws on tags the shape accepts. The two bundle records judge their keys together and refuse two keys that fold to one tag, naming both, rather than letting the later one replace the earlier. The #25822 stored-row conversion folds case too, so a row the save door stored as zh-cn is read back as zh-CN by every rehydration seam. New semantic entry translation-locale-case-canonicalized; the stale "case is not judged" sentences are corrected. Claude-Session: https://claude.ai/code/session_01ADHkmRmpbFs6uQ8AuadsTr Co-authored-by: Claude --- .changeset/28993-locale-case-canonicalized.md | 16 +++ .../docs/protocol/kernel/i18n-standard.mdx | 6 +- packages/spec/src/conversions/registry.ts | 86 ++++++++----- ...8.translation-locale-case-canonicalized.ts | 39 ++++++ .../18.translation-locale-non-tag-refused.ts | 3 +- packages/spec/src/system/translation.test.ts | 101 +++++++++++++++- packages/spec/src/system/translation.zod.ts | 114 +++++++++++++++++- 7 files changed, 324 insertions(+), 41 deletions(-) create mode 100644 .changeset/28993-locale-case-canonicalized.md create mode 100644 packages/spec/src/migrations/entries/semantic/18.translation-locale-case-canonicalized.ts diff --git a/.changeset/28993-locale-case-canonicalized.md b/.changeset/28993-locale-case-canonicalized.md new file mode 100644 index 00000000000..cef0b3b2872 --- /dev/null +++ b/.changeset/28993-locale-case-canonicalized.md @@ -0,0 +1,16 @@ +--- +'@objectstack/spec': minor +--- + +spec(i18n): a locale tag written in another case is corrected to its BCP-47 case — a translation saved as `zh-cn` reaches `zh-CN` sessions + +Clause-②: yes (widening) +Metadata-protocol: semantic + +A translation saved with `locale: 'zh-cn'` answered 200, and a `zh-CN` session kept rendering the stock strings. Case was accepted as written on the premise that the resolvers fold it, but a resolver answers one locale for a request, exact match first, so the stock `zh-CN` bundle won and the `zh-cn` entry was never merged into it. + +- **The parse corrects case.** `LocaleSchema` returns each tag in its BCP-47 case: the language subtag in lower case, a script in title case, a region in upper case, and every subtag after a singleton in lower case. `zh-cn` reads back as `zh-CN`, `zh-hans-cn` as `zh-Hans-CN`, `EN` as `en`. This reaches every reader of the schema: a translation item's `locale`, a `TranslationBundle` / `PlatformTranslationBundle` locale key, `i18n.defaultLocale` / `fallbackLocale` / `supportedLocales`, a seed dataset's `locale` scope and the seed loader's `locale`. A canonical tag is returned unchanged, and nothing that parsed before is refused, with the one exception below. +- **Stored rows are read back canonical.** The metadata save door stores the authored body, so a row saved as `zh-cn` stays `zh-cn` at rest. The stored-row conversion `translation-locale-tag-canonicalized` now folds case as well as `_` and padding, and the runtime-authored translation sync replays it before keying a row by its locale. A row saved as `zh-cn` is served to `zh-CN` sessions from the next sync on, with nothing to re-save. Built artifacts inside the conversion's window are folded the same way. +- **Case only.** The fold never replaces a legacy or macrolanguage subtag the way `Intl.getCanonicalLocales` does: `iw` stays `iw`, `cmn-cn` becomes `cmn-CN`. That keeps a translation on the locale its author named, and gives the same answer in every runtime. +- **A bundle with two keys that fold to one tag is refused.** `{ 'zh-cn': …, 'zh-CN': … }` used to parse, keeping both keys. Folding them would put both entries on one key and silently drop one set of strings, so the parse refuses the bundle and names both keys. Merge them by hand under the canonical key. Two separate bundles in `stack.translations`, one with `zh-cn` and one with `zh-CN`, are still accepted; the runtime merges them as one locale. +- **Upgrade path.** The ADR-0087 D3 semantic entry `translation-locale-case-canonicalized` carries the change to `os migrate meta` and the upgrade guide. No key is added or removed, and the published JSON Schema of a locale is unchanged. diff --git a/content/docs/protocol/kernel/i18n-standard.mdx b/content/docs/protocol/kernel/i18n-standard.mdx index 8e3ae9744ac..2f2e9626339 100644 --- a/content/docs/protocol/kernel/i18n-standard.mdx +++ b/content/docs/protocol/kernel/i18n-standard.mdx @@ -245,7 +245,11 @@ Every locale code (a bundle key, an item's `locale`, `i18n.defaultLocale`, between subtags (`en`, `zh-CN`, `zh-Hans-CN`). `LocaleSchema` refuses anything else at parse — `zh_CN`, a padded value, a language name — and the 18.x conversion `translation-locale-tag-canonicalized` rewrites the `_` and padded -spellings in stored rows and older builds. Case is not judged: the resolvers fold it. +spellings in stored rows and older builds. Case is corrected, never refused: the +parse returns each tag in its BCP-47 case (`zh-cn` → `zh-CN`, `zh-hans-cn` → +`zh-Hans-CN`), and the same conversion folds a stored row's locale, so a +translation saved as `zh-cn` is served to `zh-CN` sessions. A bundle holding two +keys that fold to one tag (`zh-cn` beside `zh-CN`) is refused naming both. ```typescript // One locale of a TranslationBundle (e.g. zh-CN) diff --git a/packages/spec/src/conversions/registry.ts b/packages/spec/src/conversions/registry.ts index 36f5e353794..eb5dc2ce8fc 100644 --- a/packages/spec/src/conversions/registry.ts +++ b/packages/spec/src/conversions/registry.ts @@ -3945,8 +3945,9 @@ const elementFormRemoved: MetadataConversion = { }; /** - * A locale written with `_` separators or surrounding whitespace becomes the - * BCP-47 tag it meant (protocol 18, #25822): `zh_CN` → `zh-CN`, `'en '` → `en`. + * A locale written with `_` separators, surrounding whitespace or a case other + * than BCP-47's becomes the tag it meant (protocol 18, #25822, #28993): + * `zh_CN` → `zh-CN`, `'en '` → `en`, `zh-cn` → `zh-CN`. * * `LocaleSchema` was a bare string, so a translation item, a bundle's locale * key, the stack's `i18n` locales and a seed dataset's `locale` scope all @@ -3955,33 +3956,41 @@ const elementFormRemoved: MetadataConversion = { * asks for: its strings were stored and never rendered. The schema now * refuses anything that is not tag-shaped. * - * **What it rewrites.** Only the two spellings that have exactly one tag they - * can have meant: `_` used as the subtag separator, and whitespace around the - * value. Case is kept as written, because every resolver already folds it. - * The result must pass `LocaleSchema`; a value that still does not (a - * language name, `not a locale!!`) is left as stored and reported as a TODO, - * because only its author knows which locale it was for. + * **What it rewrites.** Only the spellings that have exactly one tag they can + * have meant: `_` used as the subtag separator, whitespace around the value, + * and (#28993) the case of each subtag, which the result takes from the + * `LocaleSchema` parse (language lower, script title, region upper). Case was + * first kept as written on the premise that every resolver folds it; that + * premise was measured false: a resolver answers one locale, exact match + * first, so a stored `zh-cn` row was never merged into the stock `zh-CN` + * bundle and a `zh-CN` session never saw it. The metadata save door stores + * the authored body rather than the parse, so this stored-row pass is what + * carries the fold to a row saved in lower case. A value that is still not + * tag-shaped (a language name, `not a locale!!`) is left as stored and + * reported as a TODO, because only its author knows which locale it was for. * * **A rewrite changes what renders, on purpose.** A `zh_CN` item's strings * start resolving for `zh-CN` sessions, the strings their author wrote them * for. On a bundle a rename that would land on a locale key the entry already * holds is merged only when the two values are identical (`renameKey`); * otherwise both are left as stored and reported, because picking a winner - * would silently discard one translator's strings. + * would silently discard one translator's strings. `TranslationBundleSchema` + * refuses the same pair (`zh-cn` beside `zh-CN`) for the same reason. * * **Reach.** `translations[]`: a bare item's `locale` (the shape stored * `translation` rows replay through), or a bundle entry's locale keys. A - * bundle key is judged only when it carries something other than letters, so - * a legacy item written before `locale` was required, whose top-level keys - * are group names (`globalActions`, `dashboards`), is never misread as a - * bundle with bad locale keys. Also `i18n.defaultLocale`, + * bundle key made only of letters is rewritten only when it is a tag in + * another case (`EN` → `en`) and is otherwise never judged, so a legacy item + * written before `locale` was required, whose top-level keys are group names + * (`globalActions`, `dashboards`), is never misread as a bundle with bad + * locale keys. Also `i18n.defaultLocale`, * `i18n.fallbackLocale`, `i18n.supportedLocales[]` and `data[].locale[]`. * * Retired from the load path from day one: a live author meets the parse's * refusal and its prescription instead of being rewritten. Stored rows * (`applyConversionsToStoredItem`) and a built artifact inside its - * declared-floor window get the rewrite. Idempotent: a tag-shaped value is - * never touched again. + * declared-floor window get the rewrite; the parse folds case for a live + * author itself. Idempotent: a canonical tag is never touched again. */ const translationLocaleTagCanonicalized: MetadataConversion = { id: 'translation-locale-tag-canonicalized', @@ -3992,27 +4001,29 @@ const translationLocaleTagCanonicalized: MetadataConversion = { 'translation.locale / stack.translations[]. / i18n.defaultLocale / i18n.fallbackLocale / ' + 'i18n.supportedLocales[] / data[].locale[]', summary: - "a locale written with '_' separators or surrounding whitespace → the BCP-47 tag it meant " - + "('zh_CN' → 'zh-CN'): no resolver matched it, so its strings never rendered; a value that is " - + 'still not tag-shaped is left as stored and reported', + "a locale written with '_' separators, surrounding whitespace or a case other than BCP-47's → " + + "the tag it meant ('zh_CN' → 'zh-CN', 'zh-cn' → 'zh-CN'): no resolver merged it into the " + + 'canonical locale, so its strings never rendered; a value that is still not tag-shaped is left ' + + 'as stored and reported', apply(stack, emit, context) { - const isTag = (v: string): boolean => LocaleSchema.safeParse(v).success; - /** The one tag `raw` can have meant, or `undefined` when it is a tag already or names none. */ - const canonical = (raw: string): string | undefined => { - if (isTag(raw)) return undefined; - const tag = raw.trim().replace(/_/g, '-'); - return isTag(tag) ? tag : undefined; + /** `v` in its canonical case when it is tag-shaped (the `LocaleSchema` parse), else `undefined`. */ + const asTag = (v: string): string | undefined => { + const parsed = LocaleSchema.safeParse(v); + return parsed.success ? parsed.data : undefined; }; + /** The one canonical tag `raw` can have meant (itself when canonical), or `undefined` when it names none. */ + const canonical = (raw: string): string | undefined => + asTag(raw) ?? asTag(raw.trim().replace(/_/g, '-')); const todo = (path: string, from: string, reason: string): void => context?.reportTodo?.({ path, from: JSON.stringify(from), reason }); const NOT_A_TAG = 'is not a BCP-47 language tag and has no single tag it can have meant. Left as stored, it is ' + 'refused where it is parsed. Rewrite it by hand as the tag its strings were written for ' + '(`en`, `zh-CN`, `zh-Hans-CN`).'; - /** A locale value: rewritten, reported, or (already a tag) returned as is. */ + /** A locale value: rewritten, reported, or (already canonical) returned as is. */ const value = (raw: string, path: string, subject: string): string => { - if (isTag(raw)) return raw; const tag = canonical(raw); + if (tag === raw) return raw; if (tag === undefined) { todo(path, raw, `${subject} ${NOT_A_TAG}`); return raw; @@ -4042,9 +4053,12 @@ const translationLocaleTagCanonicalized: MetadataConversion = { // A bundle entry: locale code → data. let bundle = entry; for (const [key, data] of Object.entries(entry)) { - if (!isDict(data) || /^[A-Za-z]+$/.test(key) || isTag(key)) continue; + if (!isDict(data)) continue; const tag = canonical(key); + if (tag === key) continue; if (tag === undefined) { + // Letters only and not a tag: a legacy item's group name, not a locale key. + if (/^[A-Za-z]+$/.test(key)) continue; todo(`${path}.${key}`, key, `The translation bundle locale key ${NOT_A_TAG}`); continue; } @@ -4089,14 +4103,18 @@ const translationLocaleTagCanonicalized: MetadataConversion = { translations: [ // A stored item under an underscore spelling. { name: 'zh_cn', locale: 'zh_CN', messages: { commonSave: '保存' } }, - // Already a tag, in lower case: resolvers fold case, so it is left alone. + // Tag-shaped but in lower case (#28993): no resolver merged it into + // `pt-BR`, so its case is corrected. { name: 'pt_br', locale: 'pt-br', messages: { commonSave: 'Salvar' } }, // Not a tag and no single tag it can have meant: a TODO, left as stored. { name: 'qa_bad_locale', locale: 'not a locale!!', messages: { commonSave: 'x' } }, - // A bundle entry: one padded key, one underscore key, one tag. + // A bundle entry: one padded key, one underscore key, two keys in + // another case (one of them letters only), one canonical tag. { 'en ': { messages: { commonSave: 'Save' } }, ja_JP: { messages: { commonSave: '保存' } }, + 'ko-kr': { messages: { commonSave: '저장' } }, + IT: { messages: { commonSave: 'Salva' } }, 'zh-CN': { messages: { commonSave: '保存' } }, }, // A legacy item without `locale`: its group keys are not locale keys. @@ -4108,21 +4126,23 @@ const translationLocaleTagCanonicalized: MetadataConversion = { after: { translations: [ { name: 'zh_cn', locale: 'zh-CN', messages: { commonSave: '保存' } }, - { name: 'pt_br', locale: 'pt-br', messages: { commonSave: 'Salvar' } }, + { name: 'pt_br', locale: 'pt-BR', messages: { commonSave: 'Salvar' } }, { name: 'qa_bad_locale', locale: 'not a locale!!', messages: { commonSave: 'x' } }, { 'zh-CN': { messages: { commonSave: '保存' } }, en: { messages: { commonSave: 'Save' } }, 'ja-JP': { messages: { commonSave: '保存' } }, + 'ko-KR': { messages: { commonSave: '저장' } }, + it: { messages: { commonSave: 'Salva' } }, }, { name: 'fr', globalActions: { export_all: { label: 'Tout exporter' } } }, ], i18n: { defaultLocale: 'en-US', supportedLocales: ['en-US', 'zh-CN', 'ja-JP'], fallbackLocale: 'en' }, data: [{ object: 'crm_lead', locale: ['zh-CN'], records: [] }], }, - // One per rewritten value: the item, two bundle keys, the default locale, - // two supported locales and the seed scope. - expectedNotices: 7, + // One per rewritten value: two items, four bundle keys, the default + // locale, two supported locales and the seed scope. + expectedNotices: 10, }, }; diff --git a/packages/spec/src/migrations/entries/semantic/18.translation-locale-case-canonicalized.ts b/packages/spec/src/migrations/entries/semantic/18.translation-locale-case-canonicalized.ts new file mode 100644 index 00000000000..b00925c43a9 --- /dev/null +++ b/packages/spec/src/migrations/entries/semantic/18.translation-locale-case-canonicalized.ts @@ -0,0 +1,39 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import type { SemanticMigration } from '../../types.js'; + +// The case half of the `translation-locale-tag-canonicalized` family: the D2 +// conversion folds a stored or built locale to its BCP-47 case, the parse +// folds a live author's, and this entry states what that changes for a +// consumer that compared locales by their exact spelling. +export const entry: SemanticMigration = { + id: 'translation-locale-case-canonicalized', + surface: + 'a locale tag written in another case than BCP-47 canonical case (LocaleSchema): a translation ' + + 'item locale, a translation bundle locale key, i18n.defaultLocale / fallbackLocale / ' + + 'supportedLocales, a seed dataset locale scope, the seed loader locale', + replacement: + 'nothing to rewrite: the parse returns the tag in its BCP-47 case (language lower case, script ' + + 'title case, region upper case, every subtag after a singleton lower case), so `zh-cn` reads ' + + 'back as `zh-CN`, `zh-hans-cn` as `zh-Hans-CN` and `EN` as `en`. A consumer that matched the ' + + 'authored spelling exactly matches the canonical one. A translation bundle holding two keys that ' + + 'fold to one tag (`zh-cn` and `zh-CN`) is refused naming both: merge them by hand under the ' + + 'canonical key', + reason: + 'Case was accepted as written, on the premise that the resolvers fold it. They answer one locale ' + + 'for a request, exact match first, so a translation saved under `zh-cn` beside the stock `zh-CN` ' + + 'bundle was never merged into it: the save answered 200 and a `zh-CN` session kept rendering the ' + + 'stock strings, with no diagnostic. The parse now corrects the case instead of refusing it, so ' + + 'every reader sees one spelling and no input that parsed before is refused, except a bundle ' + + 'whose two keys would now land on one entry and silently drop one set of strings. The fold is ' + + 'case only: it never replaces a legacy subtag the way Intl.getCanonicalLocales does (`iw` stays ' + + '`iw`).', + acceptanceCriteria: + 'Every locale that parses through LocaleSchema reads back in its BCP-47 case. A stored ' + + '`translation` row under `zh-cn` is read back as `zh-CN` by every stored-row seam, so its ' + + 'strings are merged into the `zh-CN` locale a session asks for. A built artifact inside the ' + + 'conversion window is folded the same way. A canonical tag is never rewritten. A bundle holding ' + + '`zh-cn` beside `zh-CN` fails the parse naming both keys, and the stored-row conversion leaves ' + + 'such a pair as stored and reports it.', + conversionIds: ['translation-locale-tag-canonicalized'], +}; diff --git a/packages/spec/src/migrations/entries/semantic/18.translation-locale-non-tag-refused.ts b/packages/spec/src/migrations/entries/semantic/18.translation-locale-non-tag-refused.ts index a7156fb0984..a301dcf45ed 100644 --- a/packages/spec/src/migrations/entries/semantic/18.translation-locale-non-tag-refused.ts +++ b/packages/spec/src/migrations/entries/semantic/18.translation-locale-non-tag-refused.ts @@ -17,7 +17,8 @@ export const entry: SemanticMigration = { + '1-8 letter-or-digit subtags (`en`, `zh-CN`, `zh-Hans-CN`, `es-419`). Write `zh-CN` for ' + '`zh_CN` and drop surrounding whitespace; the conversion does both for stored rows and built ' + 'artifacts. A language name or any other text has no single tag it can have meant, so rewrite ' - + 'it by hand. Case is not judged: `zh-cn` is accepted, because the resolvers fold case', + + 'it by hand. Case is not refused: the parse corrects it (`zh-cn` → `zh-CN`, see ' + + '`translation-locale-case-canonicalized`)', reason: 'LocaleSchema was a bare string, so a translator or AI author who wrote `zh_CN`, a padded ' + 'value or a language name got a green save, and the strings were stored under a locale no ' diff --git a/packages/spec/src/system/translation.test.ts b/packages/spec/src/system/translation.test.ts index 9733d3675be..f2f9455898a 100644 --- a/packages/spec/src/system/translation.test.ts +++ b/packages/spec/src/system/translation.test.ts @@ -5,6 +5,9 @@ import { TEXT_SLOT_TEMPLATE_REFUSAL } from '../automation/flow-text-slot-templat import { FlowSchema } from '../automation/flow.zod'; import { ScreenConfigSchema, ScreenFieldConfigSchema } from '../automation/builtin-node-config.zod'; import { ALL_CONVERSIONS } from '../conversions/registry'; +import { applyConversionsToStoredItem } from '../conversions/stored'; +import { SeedSchema } from '../data/seed.zod'; +import { SeedLoaderConfigSchema } from '../data/seed-loader.zod'; import { TranslationDataSchema, TranslationBundleSchema, @@ -47,7 +50,7 @@ describe('LocaleSchema', () => { }); }); - it.each(['en', 'zh-CN', 'zh-Hans-CN', 'es-419', 'yue-HK', 'zh-cn'])('accepts the tag %j (case is not judged)', (locale) => { + it.each(['en', 'zh-CN', 'zh-Hans-CN', 'es-419', 'yue-HK', 'zh-cn'])('accepts the tag %j (case is corrected, never refused)', (locale) => { expect(LocaleSchema.safeParse(locale).success).toBe(true); }); @@ -120,6 +123,102 @@ describe('LocaleSchema', () => { }); }); +// #28993: a translation saved under `zh-cn` answered 200 and never reached a +// `zh-CN` session, because a resolver answers one locale, exact match first, +// and the stock `zh-CN` bundle won. The parse now corrects a tag's case. +describe('LocaleSchema corrects a tag to its BCP-47 case (#28993)', () => { + it.each([ + ['zh-cn', 'zh-CN'], + ['ZH-CN', 'zh-CN'], + ['zh-hans-cn', 'zh-Hans-CN'], + ['ZH-HANS-cn', 'zh-Hans-CN'], + ['EN', 'en'], + ['en-us', 'en-US'], + ['yue-hk', 'yue-HK'], + ['sr-LATN-rs', 'sr-Latn-RS'], + ['es-419', 'es-419'], + ['de-1996', 'de-1996'], + // Every subtag after a singleton is lower case (RFC 5646 §2.1.1). + ['en-US-u-CA-Gregory', 'en-US-u-ca-gregory'], + ['en-x-US', 'en-x-us'], + ])('%j parses to %j', (input, canonical) => { + expect(LocaleSchema.parse(input)).toBe(canonical); + }); + + it.each(['en', 'zh-CN', 'zh-Hans-CN', 'es-419', 'pt-BR', 'en-US-u-ca-gregory'])('a canonical tag %j is returned unchanged', (locale) => { + expect(LocaleSchema.parse(locale)).toBe(locale); + }); + + it('folds case only: no legacy-subtag replacement, and no throw on a tag the shape accepts', () => { + // `Intl.getCanonicalLocales` answers `he`, `fil` and `zh` for these, which + // would move a translation to a locale its author never named. + expect(LocaleSchema.parse('iw')).toBe('iw'); + expect(LocaleSchema.parse('TL')).toBe('tl'); + expect(LocaleSchema.parse('cmn-cn')).toBe('cmn-CN'); + // ...and throws on this one, which the shape accepts. + expect(LocaleSchema.parse('zh-hans-hans')).toBe('zh-Hans-Hans'); + }); + + it('every reader of LocaleSchema sees the canonical spelling', () => { + // The item door: `defineTranslation` and the metadata save door's parse. + expect(TranslationItemSchema.parse({ name: 'zh_cn', locale: 'zh-cn', messages: { x: 'y' } }).locale).toBe('zh-CN'); + expect(defineTranslation({ name: 'zh_cn', locale: 'ZH-cn', messages: { x: 'y' } }).locale).toBe('zh-CN'); + // The stack's i18n configuration. + expect(TranslationConfigSchema.parse({ defaultLocale: 'EN', supportedLocales: ['en', 'zh-cn'], fallbackLocale: 'en-us' })) + .toEqual({ defaultLocale: 'en', supportedLocales: ['en', 'zh-CN'], fallbackLocale: 'en-US' }); + // A seed dataset's locale scope and the seed loader's own locale. + expect(SeedSchema.parse({ object: 'crm_lead', locale: ['zh-cn'], records: [] }).locale).toEqual(['zh-CN']); + expect(SeedLoaderConfigSchema.parse({ locale: 'zh-cn' }).locale).toBe('zh-CN'); + }); + + it('a bundle\'s locale keys are folded, in authored order, values untouched', () => { + const data = { messages: { commonSave: '保存' } }; + const parsed = TranslationBundleSchema.parse({ 'zh-cn': data, en: { messages: { commonSave: 'Save' } } }); + expect(Object.keys(parsed)).toEqual(['zh-CN', 'en']); + expect(parsed['zh-CN']).toEqual(data); + expect(Object.keys(defineTranslationBundle({ 'ja-jp': data }))).toEqual(['ja-JP']); + expect(Object.keys(PlatformTranslationBundleSchema.parse({ 'zh-hans-cn': data }))).toEqual(['zh-Hans-CN']); + }); + + it.each([ + ['TranslationBundleSchema', TranslationBundleSchema], + ['PlatformTranslationBundleSchema', PlatformTranslationBundleSchema], + ] as const)('%s refuses two keys that fold to one tag, naming both, instead of dropping one', (_name, schema) => { + const result = schema.safeParse({ + 'zh-cn': { messages: { commonSave: '保存(lower)' } }, + 'zh-CN': { messages: { commonSave: '保存' } }, + }); + expect(result.success).toBe(false); + expect(result.error!.issues.map((i) => [i.code, i.path.join('.')])).toEqual([['custom', 'zh-CN']]); + const message = result.error!.issues[0]!.message; + expect(message).toContain('`zh-cn`'); + expect(message).toContain('`zh-CN`'); + }); + + it('CONTROL: distinct tags in one bundle, and one tag across two bundles, are both accepted', () => { + // Distinct after the fold: nothing collides. + const bundle = { 'zh-CN': { messages: { x: 'a' } }, 'zh-TW': { messages: { x: 'b' } }, 'zh-hans-cn': { messages: { x: 'c' } } }; + expect(Object.keys(TranslationBundleSchema.parse(bundle))).toEqual(['zh-CN', 'zh-TW', 'zh-Hans-CN']); + // Two bundles of `stack.translations` are separate records; the runtime + // deep-merges them per locale, so `zh-cn` in one and `zh-CN` in the other + // is not a collision, it is one locale. + expect(TranslationBundleSchema.safeParse({ 'zh-cn': { messages: { x: 'a' } } }).success).toBe(true); + expect(TranslationBundleSchema.safeParse({ 'zh-CN': { messages: { x: 'b' } } }).success).toBe(true); + }); + + it('data at rest: a stored row under `zh-cn` is read back as `zh-CN` by the stored-row seam', () => { + // `applyConversionsToStoredItem` is what every rehydration seam replays, + // the runtime-authored translation sync included, before it keys the row + // by its locale. The save door stores the authored body, so this is the + // seam that carries the fold to a stored row. + const row = { name: 'zh_cn', locale: 'zh-cn', objects: { project: { label: '项目(lower)' } } }; + expect(applyConversionsToStoredItem('translation', row)).toEqual({ ...row, locale: 'zh-CN' }); + // Canonical rows come back untouched: the pass is idempotent. + const canonical = { name: 'zh_cn', locale: 'zh-CN', messages: { x: 'y' } }; + expect(applyConversionsToStoredItem('translation', canonical)).toEqual(canonical); + }); +}); + describe('TranslationDataSchema', () => { it('should accept empty translation data', () => { const data = TranslationDataSchema.parse({}); diff --git a/packages/spec/src/system/translation.zod.ts b/packages/spec/src/system/translation.zod.ts index 858a780f280..363c41f85ee 100644 --- a/packages/spec/src/system/translation.zod.ts +++ b/packages/spec/src/system/translation.zod.ts @@ -24,8 +24,8 @@ import { textSlotTemplateRefusal } from '../automation/flow-text-slot-template'; * * Shape only, by decision: membership has no registry to check against * (`Intl.getCanonicalLocales('xx-YY')` succeeds, which is why - * `shared/value-domain.zod.ts` dropped a `bcp47_locale` domain), and case is - * not judged because the resolvers fold it (`zh-cn` resolves to `zh-CN`). + * `shared/value-domain.zod.ts` dropped a `bcp47_locale` domain). Case is not + * refused either; it is corrected, see {@link canonicalLocaleCase}. * The primary subtag is held to the 2–3 letters of the ISO 639 codes every * tag in use starts with: BCP-47 reserves 4 letters and registers no 5–8 * letter language, so the only values that wider grammar ever admitted were @@ -42,12 +42,108 @@ const LOCALE_TAG_REFUSAL = + 'is under a locale no session ever asks for, so its strings never render. Separate subtags with ' + '`-`, not `_`, and write the tag, not the name (`zh-CN`, not `Chinese`).'; +const LOCALE_TAG_DESCRIPTION = 'BCP-47 Language Tag (e.g. en-US, zh-CN)'; + +/** + * A tag's BCP-47 case (RFC 5646 §2.1.1): the language subtag lower case, a + * 2-character subtag upper case (the region: `CN`), a 4-character one title + * case (the script: `Hans`), everything else lower case, and every subtag + * after a singleton (`-u-`, `-x-`) lower case. `zh-cn` → `zh-CN`, + * `ZH-HANS-cn` → `zh-Hans-CN`, `EN` → `en`; a canonical tag is returned as is. + * + * Why the parse corrects case instead of leaving it to the resolvers: a + * resolver answers ONE locale for a request, exact match first, so a + * translation stored under `zh-cn` beside the stock `zh-CN` bundle was never + * merged into it; a `zh-CN` session kept rendering the stock strings, with no + * diagnostic (#28993). Folding at the parse gives every reader one spelling, + * and a lowercase author is corrected rather than refused, so nothing that + * parsed before is refused now. + * + * Case only, deliberately not `Intl.getCanonicalLocales`, measured on Node 22: + * that also replaces legacy and macrolanguage subtags (`iw` → `he`, + * `tl` → `fil`, `cmn` → `zh`), which would move a translation to a locale its + * author never named, and it THROWS on tags {@link LOCALE_TAG_SHAPE} accepts + * (`zh-Hans-Hans`, `en-GB-oed`), which would turn a shape refusal into a + * crash. Its output also depends on the runtime's ICU data, where this fold + * is the same string in every runtime the spec ships to. + * + * ⛔ Package-internal: the public answer is a {@link LocaleSchema} parse, and + * an exported fold would be an invitation to fold a second time in a consumer. + */ +function canonicalLocaleCase(tag: string): string { + let afterSingleton = false; + return tag + .split('-') + .map((subtag, index) => { + if (index === 0 || afterSingleton) return subtag.toLowerCase(); + if (subtag.length === 1) { + afterSingleton = true; + return subtag.toLowerCase(); + } + if (subtag.length === 2) return subtag.toUpperCase(); + if (subtag.length === 4) return subtag.charAt(0).toUpperCase() + subtag.slice(1).toLowerCase(); + return subtag.toLowerCase(); + }) + .join('-'); +} + +/** + * The tag shape alone, without the case fold: the KEY of the two bundle + * records below. A record cannot fold its keys one at a time, because two + * keys that fold to one tag would land on one entry and the later one would + * silently replace the earlier; the records judge their keys together + * ({@link refuseLocaleKeysFoldingTogether}) and then fold them + * ({@link foldLocaleKeys}). Same pattern, same refusal and description as + * {@link LocaleSchema}, so the published shape of a bundle key does not move. + */ +const LocaleTagKeySchema = lazySchema(() => z + .string() + .regex(LOCALE_TAG_SHAPE, LOCALE_TAG_REFUSAL) + .describe(LOCALE_TAG_DESCRIPTION)); + export const LocaleSchema = lazySchema(() => z .string() .regex(LOCALE_TAG_SHAPE, LOCALE_TAG_REFUSAL) - .describe('BCP-47 Language Tag (e.g. en-US, zh-CN)')); + .overwrite(canonicalLocaleCase) + .describe(LOCALE_TAG_DESCRIPTION)); export type Locale = z.input; +/** + * Refuses a bundle holding two locale keys that fold to one tag (`zh-cn` and + * `zh-CN`), naming both. Picking a winner would discard one translator's + * strings without a word, which is the policy the stored-row conversion + * (`translation-locale-tag-canonicalized`) already follows for the same + * collision. Runs before {@link foldLocaleKeys}, so it sees the keys as written. + */ +function refuseLocaleKeysFoldingTogether(bundle: Record, ctx: z.RefinementCtx): void { + const firstByTag = new Map(); + for (const key of Object.keys(bundle)) { + const tag = canonicalLocaleCase(key); + const first = firstByTag.get(tag); + if (first === undefined) { + firstByTag.set(tag, key); + continue; + } + ctx.addIssue({ + code: 'custom', + path: [key], + message: + `locale keys \`${first}\` and \`${key}\` are one locale, \`${tag}\`: a locale tag's case is ` + + 'corrected on parse, so both entries would land on one key and one set of strings would be ' + + `dropped. Merge the two by hand and keep one key, \`${tag}\`.`, + }); + } +} + +/** Every locale key in its BCP-47 case; the input itself when none moves. */ +function foldLocaleKeys(bundle: Record): Record { + const keys = Object.keys(bundle); + if (keys.every((key) => canonicalLocaleCase(key) === key)) return bundle; + const folded: Record = {}; + for (const key of keys) folded[canonicalLocaleCase(key)] = bundle[key] as T; + return folded; +} + /** * Shared history sentence for every shape in this file. * @@ -2364,7 +2460,11 @@ export type ServedTranslationData = z.input; // Translation Bundle (all locales) // ──────────────────────────────────────────────────────────────────────────── -export const TranslationBundleSchema = lazySchema(() => z.record(LocaleSchema, TranslationDataSchema).describe('Map of locale codes to per-app translation data')); +export const TranslationBundleSchema = lazySchema(() => z + .record(LocaleTagKeySchema, TranslationDataSchema) + .superRefine(refuseLocaleKeysFoldingTogether) + .overwrite(foldLocaleKeys) + .describe('Map of locale codes to per-app translation data')); export type TranslationBundle = z.input; @@ -2374,7 +2474,11 @@ export type TranslationBundle = z.input; * the bundle that needs it; a platform package authoring only groups the * per-app face also declares can keep the narrower {@link TranslationBundle}. */ -export const PlatformTranslationBundleSchema = lazySchema(() => z.record(LocaleSchema, PlatformTranslationDataSchema).describe('Map of locale codes to platform translation data')); +export const PlatformTranslationBundleSchema = lazySchema(() => z + .record(LocaleTagKeySchema, PlatformTranslationDataSchema) + .superRefine(refuseLocaleKeysFoldingTogether) + .overwrite(foldLocaleKeys) + .describe('Map of locale codes to platform translation data')); export type PlatformTranslationBundle = z.input; From 102f342e3b643b0c25f5e56584fd3b44b720e98d Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 2 Nov 2026 07:28:34 +0000 Subject: [PATCH 2/5] chore(spec): record the bundle-key collision refusal in the dropped-refinements ledger The refusal of two locale keys that fold to one tag is a superRefine with no JSON Schema form, so the published TranslationBundle / PlatformTranslationBundle (and the four installed-package envelopes that embed manifest.translations) gain one dropped-refinement site each. Sites as the build-schemas gate printed them; measured totals recounted. Claude-Session: https://claude.ai/code/session_01ADHkmRmpbFs6uQ8AuadsTr Co-authored-by: Claude --- .../spec/dropped-refinements.baseline.json | 20 ++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/packages/spec/dropped-refinements.baseline.json b/packages/spec/dropped-refinements.baseline.json index a91dce7c42b..b080a9f2a8e 100644 --- a/packages/spec/dropped-refinements.baseline.json +++ b/packages/spec/dropped-refinements.baseline.json @@ -3,7 +3,7 @@ "measured": { "zod": "4.4.3", "publishedSchemasWithDroppedRefinements": 248, - "droppedRefinementSites": 983, + "droppedRefinementSites": 989, "refinementSitesThatDidProject": 369, "refinementSitesWithNoJsonFormToCompare": 0 }, @@ -125,8 +125,9 @@ "manifest.reports.element.runtimeFilter.lazy", "manifest.sharingRules.element.sharedWith", "manifest.skills.element.triggerConditions.element", - "manifest.translations.element.valueType.flows.valueType.notices.valueType.message", + "manifest.translations.element", "manifest.translations.element.valueType.flows.valueType.errorMessage", + "manifest.translations.element.valueType.flows.valueType.notices.valueType.message", "manifest.translations.element.valueType.flows.valueType.refusals.valueType.message", "manifest.translations.element.valueType.flows.valueType.screens.valueType.description", "manifest.translations.element.valueType.flows.valueType.screens.valueType.title", @@ -268,6 +269,7 @@ "data.options[1].manifest.reports.element.runtimeFilter.lazy", "data.options[1].manifest.sharingRules.element.sharedWith", "data.options[1].manifest.skills.element.triggerConditions.element", + "data.options[1].manifest.translations.element", "data.options[1].manifest.translations.element.valueType.flows.valueType.errorMessage", "data.options[1].manifest.translations.element.valueType.flows.valueType.successMessage", "data.options[1].manifest.views.element.form", @@ -406,8 +408,9 @@ "options[1].manifest.reports.element.runtimeFilter.lazy", "options[1].manifest.sharingRules.element.sharedWith", "options[1].manifest.skills.element.triggerConditions.element", - "options[1].manifest.translations.element.valueType.flows.valueType.notices.valueType.message", + "options[1].manifest.translations.element", "options[1].manifest.translations.element.valueType.flows.valueType.errorMessage", + "options[1].manifest.translations.element.valueType.flows.valueType.notices.valueType.message", "options[1].manifest.translations.element.valueType.flows.valueType.refusals.valueType.message", "options[1].manifest.translations.element.valueType.flows.valueType.screens.valueType.description", "options[1].manifest.translations.element.valueType.flows.valueType.screens.valueType.title", @@ -484,6 +487,7 @@ "data.packages.element.options[1].manifest.reports.element.runtimeFilter.lazy", "data.packages.element.options[1].manifest.sharingRules.element.sharedWith", "data.packages.element.options[1].manifest.skills.element.triggerConditions.element", + "data.packages.element.options[1].manifest.translations.element", "data.packages.element.options[1].manifest.translations.element.valueType.flows.valueType.errorMessage", "data.packages.element.options[1].manifest.translations.element.valueType.flows.valueType.successMessage", "data.packages.element.options[1].manifest.views.element.form", @@ -1429,12 +1433,13 @@ }, "system/PlatformTranslationBundle": { "sites": [ + "", "valueType.flows.valueType.errorMessage", + "valueType.flows.valueType.notices.valueType.message", "valueType.flows.valueType.refusals.valueType.message", "valueType.flows.valueType.screens.valueType.description", "valueType.flows.valueType.screens.valueType.title", - "valueType.flows.valueType.successMessage", - "valueType.flows.valueType.notices.valueType.message" + "valueType.flows.valueType.successMessage" ] }, "system/PlatformTranslationData": { @@ -1499,12 +1504,13 @@ }, "system/TranslationBundle": { "sites": [ + "", "valueType.flows.valueType.errorMessage", + "valueType.flows.valueType.notices.valueType.message", "valueType.flows.valueType.refusals.valueType.message", "valueType.flows.valueType.screens.valueType.description", "valueType.flows.valueType.screens.valueType.title", - "valueType.flows.valueType.successMessage", - "valueType.flows.valueType.notices.valueType.message" + "valueType.flows.valueType.successMessage" ] }, "system/TranslationData": { From 3b3284c9b72e83e349eb79d56856fa7295899234 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 2 Nov 2026 09:08:50 +0000 Subject: [PATCH 3/5] chore(changeset): declare the bundle-key collision refusal as a narrowing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The refusal of a translation bundle whose two keys fold to one tag refuses an input that parsed before, so the changeset now carries the amended declaration lines (Clause-② no (narrowing), Metadata-protocol narrowed), a BREAKING banner with its FROM -> TO migration, and the ADR-0087 disposition for the new D3 entry. Claude-Session: https://claude.ai/code/session_01ADHkmRmpbFs6uQ8AuadsTr Co-authored-by: Claude --- .changeset/28993-locale-case-canonicalized.md | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/.changeset/28993-locale-case-canonicalized.md b/.changeset/28993-locale-case-canonicalized.md index cef0b3b2872..80a52fff176 100644 --- a/.changeset/28993-locale-case-canonicalized.md +++ b/.changeset/28993-locale-case-canonicalized.md @@ -2,15 +2,22 @@ '@objectstack/spec': minor --- -spec(i18n): a locale tag written in another case is corrected to its BCP-47 case — a translation saved as `zh-cn` reaches `zh-CN` sessions +feat(spec)!: a locale tag written in another case is corrected to its BCP-47 case — a translation saved as `zh-cn` reaches `zh-CN` sessions — and a translation bundle with two keys that fold to one tag is refused -Clause-②: yes (widening) -Metadata-protocol: semantic +Clause-②: no (narrowing) +Metadata-protocol: narrowed A translation saved with `locale: 'zh-cn'` answered 200, and a `zh-CN` session kept rendering the stock strings. Case was accepted as written on the premise that the resolvers fold it, but a resolver answers one locale for a request, exact match first, so the stock `zh-CN` bundle won and the `zh-cn` entry was never merged into it. - **The parse corrects case.** `LocaleSchema` returns each tag in its BCP-47 case: the language subtag in lower case, a script in title case, a region in upper case, and every subtag after a singleton in lower case. `zh-cn` reads back as `zh-CN`, `zh-hans-cn` as `zh-Hans-CN`, `EN` as `en`. This reaches every reader of the schema: a translation item's `locale`, a `TranslationBundle` / `PlatformTranslationBundle` locale key, `i18n.defaultLocale` / `fallbackLocale` / `supportedLocales`, a seed dataset's `locale` scope and the seed loader's `locale`. A canonical tag is returned unchanged, and nothing that parsed before is refused, with the one exception below. - **Stored rows are read back canonical.** The metadata save door stores the authored body, so a row saved as `zh-cn` stays `zh-cn` at rest. The stored-row conversion `translation-locale-tag-canonicalized` now folds case as well as `_` and padding, and the runtime-authored translation sync replays it before keying a row by its locale. A row saved as `zh-cn` is served to `zh-CN` sessions from the next sync on, with nothing to re-save. Built artifacts inside the conversion's window are folded the same way. - **Case only.** The fold never replaces a legacy or macrolanguage subtag the way `Intl.getCanonicalLocales` does: `iw` stays `iw`, `cmn-cn` becomes `cmn-CN`. That keeps a translation on the locale its author named, and gives the same answer in every runtime. -- **A bundle with two keys that fold to one tag is refused.** `{ 'zh-cn': …, 'zh-CN': … }` used to parse, keeping both keys. Folding them would put both entries on one key and silently drop one set of strings, so the parse refuses the bundle and names both keys. Merge them by hand under the canonical key. Two separate bundles in `stack.translations`, one with `zh-cn` and one with `zh-CN`, are still accepted; the runtime merges them as one locale. -- **Upgrade path.** The ADR-0087 D3 semantic entry `translation-locale-case-canonicalized` carries the change to `os migrate meta` and the upgrade guide. No key is added or removed, and the published JSON Schema of a locale is unchanged. +- **Upgrade path.** The ADR-0087 D3 semantic entry `translation-locale-case-canonicalized` carries both halves to `os migrate meta` and the upgrade guide. No key is added or removed, and the published JSON Schema of a locale is unchanged. + + + +**BREAKING** for an author whose one translation bundle spells a locale twice, in two cases. + +- **Before.** `{ 'zh-cn': { messages: { save: '保存' } }, 'zh-CN': { messages: { cancel: '取消' } } }` parsed as a `TranslationBundle` (or `PlatformTranslationBundle`) with both keys kept, and the `zh-cn` strings never reached a `zh-CN` session. +- **Now.** Both keys fold to `zh-CN`, so keeping both would put two entries on one key and silently drop one set of strings. The parse refuses the bundle at the second key, naming both keys and the tag they fold to. Two separate bundles in `stack.translations`, one with `zh-cn` and one with `zh-CN`, are still accepted; the runtime merges them as one locale. +- **Fix.** FROM a bundle holding both spellings, `{ 'zh-cn': A, 'zh-CN': B }`, TO one canonical key holding both sets of strings, `{ 'zh-CN': A and B merged }`, merged by hand: where both set the same string, keep the one you mean. No bundle in this repository has such a pair. A stored row with such a pair and different strings under the two keys is left as stored by the conversion and reported as a TODO. From 713814b9113bdd1c2c4848ce8f9c3bdee40d8792 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 2 Nov 2026 11:12:33 +0000 Subject: [PATCH 4/5] feat(spec): fold a stored locale's case in its own conversion, retired after 18.2.0 The case half of `translation-locale-tag-canonicalized` rode that entry's retiredAfter 18.1.0, so an artifact the released 18.2.0 spec built with a `zh-cn` key fell outside the ingestion window. The case fold is now the D2 entry `translation-locale-tag-case-folded`, stamped retiredAfter 18.2.0 and ordered right after the underscore entry (`zh_cn` -> `zh-cn` -> `zh-CN`). The underscore entry is back to its released code; its docblock points at the new entry and its fixture keeps to its own spellings. The D3 entry and the changeset now also name the two exact matchers that compare a folded i18n locale with a row tagged as written (the email-template floor warning and the messaging template ladder). Claude-Session: https://claude.ai/code/session_01ADHkmRmpbFs6uQ8AuadsTr Co-authored-by: Claude --- .changeset/28993-locale-case-canonicalized.md | 3 +- .../docs/protocol/kernel/i18n-standard.mdx | 3 +- packages/spec/src/conversions/registry.ts | 265 ++++++++++++++---- ...8.translation-locale-case-canonicalized.ts | 31 +- packages/spec/src/system/translation.test.ts | 23 ++ packages/spec/src/system/translation.zod.ts | 2 +- 6 files changed, 261 insertions(+), 66 deletions(-) diff --git a/.changeset/28993-locale-case-canonicalized.md b/.changeset/28993-locale-case-canonicalized.md index 80a52fff176..213d329a0db 100644 --- a/.changeset/28993-locale-case-canonicalized.md +++ b/.changeset/28993-locale-case-canonicalized.md @@ -10,9 +10,10 @@ Metadata-protocol: narrowed A translation saved with `locale: 'zh-cn'` answered 200, and a `zh-CN` session kept rendering the stock strings. Case was accepted as written on the premise that the resolvers fold it, but a resolver answers one locale for a request, exact match first, so the stock `zh-CN` bundle won and the `zh-cn` entry was never merged into it. - **The parse corrects case.** `LocaleSchema` returns each tag in its BCP-47 case: the language subtag in lower case, a script in title case, a region in upper case, and every subtag after a singleton in lower case. `zh-cn` reads back as `zh-CN`, `zh-hans-cn` as `zh-Hans-CN`, `EN` as `en`. This reaches every reader of the schema: a translation item's `locale`, a `TranslationBundle` / `PlatformTranslationBundle` locale key, `i18n.defaultLocale` / `fallbackLocale` / `supportedLocales`, a seed dataset's `locale` scope and the seed loader's `locale`. A canonical tag is returned unchanged, and nothing that parsed before is refused, with the one exception below. -- **Stored rows are read back canonical.** The metadata save door stores the authored body, so a row saved as `zh-cn` stays `zh-cn` at rest. The stored-row conversion `translation-locale-tag-canonicalized` now folds case as well as `_` and padding, and the runtime-authored translation sync replays it before keying a row by its locale. A row saved as `zh-cn` is served to `zh-CN` sessions from the next sync on, with nothing to re-save. Built artifacts inside the conversion's window are folded the same way. +- **Stored rows are read back canonical.** The metadata save door stores the authored body, so a row saved as `zh-cn` stays `zh-cn` at rest. A new stored-row conversion, `translation-locale-tag-case-folded`, folds case right after `translation-locale-tag-canonicalized` rewrites `_` and padding, and the runtime-authored translation sync replays both before keying a row by its locale. A row saved as `zh-cn` is served to `zh-CN` sessions from the next sync on, with nothing to re-save. An artifact built by `@objectstack/spec` 18.2.0 or earlier is folded the same way. - **Case only.** The fold never replaces a legacy or macrolanguage subtag the way `Intl.getCanonicalLocales` does: `iw` stays `iw`, `cmn-cn` becomes `cmn-CN`. That keeps a translation on the locale its author named, and gives the same answer in every runtime. - **Upgrade path.** The ADR-0087 D3 semantic entry `translation-locale-case-canonicalized` carries both halves to `os migrate meta` and the upgrade guide. No key is added or removed, and the published JSON Schema of a locale is unchanged. +- **Exact matchers that read a folded locale against a row tagged as written.** The email-template floor warning compares `i18n.supportedLocales` with `emailTemplates[].locale`, and the messaging template ladder compares the default locale with `sys_notification_template.locale`. A row tagged `zh-cn` no longer matches a supported or default locale authored as `zh-cn`, which now reads `zh-CN`: re-tag the row in its canonical case. diff --git a/content/docs/protocol/kernel/i18n-standard.mdx b/content/docs/protocol/kernel/i18n-standard.mdx index 2f2e9626339..eed34c7ecbb 100644 --- a/content/docs/protocol/kernel/i18n-standard.mdx +++ b/content/docs/protocol/kernel/i18n-standard.mdx @@ -247,7 +247,8 @@ else at parse — `zh_CN`, a padded value, a language name — and the 18.x conversion `translation-locale-tag-canonicalized` rewrites the `_` and padded spellings in stored rows and older builds. Case is corrected, never refused: the parse returns each tag in its BCP-47 case (`zh-cn` → `zh-CN`, `zh-hans-cn` → -`zh-Hans-CN`), and the same conversion folds a stored row's locale, so a +`zh-Hans-CN`), and the conversion `translation-locale-tag-case-folded` folds a +stored row's locale and an artifact built by 18.2.0 or earlier, so a translation saved as `zh-cn` is served to `zh-CN` sessions. A bundle holding two keys that fold to one tag (`zh-cn` beside `zh-CN`) is refused naming both. diff --git a/packages/spec/src/conversions/registry.ts b/packages/spec/src/conversions/registry.ts index eb5dc2ce8fc..cca7011745b 100644 --- a/packages/spec/src/conversions/registry.ts +++ b/packages/spec/src/conversions/registry.ts @@ -3945,9 +3945,8 @@ const elementFormRemoved: MetadataConversion = { }; /** - * A locale written with `_` separators, surrounding whitespace or a case other - * than BCP-47's becomes the tag it meant (protocol 18, #25822, #28993): - * `zh_CN` → `zh-CN`, `'en '` → `en`, `zh-cn` → `zh-CN`. + * A locale written with `_` separators or surrounding whitespace becomes the + * BCP-47 tag it meant (protocol 18, #25822): `zh_CN` → `zh-CN`, `'en '` → `en`. * * `LocaleSchema` was a bare string, so a translation item, a bundle's locale * key, the stack's `i18n` locales and a seed dataset's `locale` scope all @@ -3956,41 +3955,35 @@ const elementFormRemoved: MetadataConversion = { * asks for: its strings were stored and never rendered. The schema now * refuses anything that is not tag-shaped. * - * **What it rewrites.** Only the spellings that have exactly one tag they can - * have meant: `_` used as the subtag separator, whitespace around the value, - * and (#28993) the case of each subtag, which the result takes from the - * `LocaleSchema` parse (language lower, script title, region upper). Case was - * first kept as written on the premise that every resolver folds it; that - * premise was measured false: a resolver answers one locale, exact match - * first, so a stored `zh-cn` row was never merged into the stock `zh-CN` - * bundle and a `zh-CN` session never saw it. The metadata save door stores - * the authored body rather than the parse, so this stored-row pass is what - * carries the fold to a row saved in lower case. A value that is still not - * tag-shaped (a language name, `not a locale!!`) is left as stored and - * reported as a TODO, because only its author knows which locale it was for. + * **What it rewrites.** Only the two spellings that have exactly one tag they + * can have meant: `_` used as the subtag separator, and whitespace around the + * value. Case is kept as written here; its fold is the entry after this one, + * {@link translationLocaleTagCaseFolded}, retired a release later, because the + * premise that every resolver folds case was measured false (#28993). The + * result must pass `LocaleSchema`; a value that still does not (a + * language name, `not a locale!!`) is left as stored and reported as a TODO, + * because only its author knows which locale it was for. * * **A rewrite changes what renders, on purpose.** A `zh_CN` item's strings * start resolving for `zh-CN` sessions, the strings their author wrote them * for. On a bundle a rename that would land on a locale key the entry already * holds is merged only when the two values are identical (`renameKey`); * otherwise both are left as stored and reported, because picking a winner - * would silently discard one translator's strings. `TranslationBundleSchema` - * refuses the same pair (`zh-cn` beside `zh-CN`) for the same reason. + * would silently discard one translator's strings. * * **Reach.** `translations[]`: a bare item's `locale` (the shape stored * `translation` rows replay through), or a bundle entry's locale keys. A - * bundle key made only of letters is rewritten only when it is a tag in - * another case (`EN` → `en`) and is otherwise never judged, so a legacy item - * written before `locale` was required, whose top-level keys are group names - * (`globalActions`, `dashboards`), is never misread as a bundle with bad - * locale keys. Also `i18n.defaultLocale`, + * bundle key is judged only when it carries something other than letters, so + * a legacy item written before `locale` was required, whose top-level keys + * are group names (`globalActions`, `dashboards`), is never misread as a + * bundle with bad locale keys. Also `i18n.defaultLocale`, * `i18n.fallbackLocale`, `i18n.supportedLocales[]` and `data[].locale[]`. * * Retired from the load path from day one: a live author meets the parse's * refusal and its prescription instead of being rewritten. Stored rows * (`applyConversionsToStoredItem`) and a built artifact inside its - * declared-floor window get the rewrite; the parse folds case for a live - * author itself. Idempotent: a canonical tag is never touched again. + * declared-floor window get the rewrite. Idempotent: a tag-shaped value is + * never touched again. */ const translationLocaleTagCanonicalized: MetadataConversion = { id: 'translation-locale-tag-canonicalized', @@ -4001,29 +3994,27 @@ const translationLocaleTagCanonicalized: MetadataConversion = { 'translation.locale / stack.translations[]. / i18n.defaultLocale / i18n.fallbackLocale / ' + 'i18n.supportedLocales[] / data[].locale[]', summary: - "a locale written with '_' separators, surrounding whitespace or a case other than BCP-47's → " - + "the tag it meant ('zh_CN' → 'zh-CN', 'zh-cn' → 'zh-CN'): no resolver merged it into the " - + 'canonical locale, so its strings never rendered; a value that is still not tag-shaped is left ' - + 'as stored and reported', + "a locale written with '_' separators or surrounding whitespace → the BCP-47 tag it meant " + + "('zh_CN' → 'zh-CN'): no resolver matched it, so its strings never rendered; a value that is " + + 'still not tag-shaped is left as stored and reported', apply(stack, emit, context) { - /** `v` in its canonical case when it is tag-shaped (the `LocaleSchema` parse), else `undefined`. */ - const asTag = (v: string): string | undefined => { - const parsed = LocaleSchema.safeParse(v); - return parsed.success ? parsed.data : undefined; + const isTag = (v: string): boolean => LocaleSchema.safeParse(v).success; + /** The one tag `raw` can have meant, or `undefined` when it is a tag already or names none. */ + const canonical = (raw: string): string | undefined => { + if (isTag(raw)) return undefined; + const tag = raw.trim().replace(/_/g, '-'); + return isTag(tag) ? tag : undefined; }; - /** The one canonical tag `raw` can have meant (itself when canonical), or `undefined` when it names none. */ - const canonical = (raw: string): string | undefined => - asTag(raw) ?? asTag(raw.trim().replace(/_/g, '-')); const todo = (path: string, from: string, reason: string): void => context?.reportTodo?.({ path, from: JSON.stringify(from), reason }); const NOT_A_TAG = 'is not a BCP-47 language tag and has no single tag it can have meant. Left as stored, it is ' + 'refused where it is parsed. Rewrite it by hand as the tag its strings were written for ' + '(`en`, `zh-CN`, `zh-Hans-CN`).'; - /** A locale value: rewritten, reported, or (already canonical) returned as is. */ + /** A locale value: rewritten, reported, or (already a tag) returned as is. */ const value = (raw: string, path: string, subject: string): string => { + if (isTag(raw)) return raw; const tag = canonical(raw); - if (tag === raw) return raw; if (tag === undefined) { todo(path, raw, `${subject} ${NOT_A_TAG}`); return raw; @@ -4053,12 +4044,9 @@ const translationLocaleTagCanonicalized: MetadataConversion = { // A bundle entry: locale code → data. let bundle = entry; for (const [key, data] of Object.entries(entry)) { - if (!isDict(data)) continue; + if (!isDict(data) || /^[A-Za-z]+$/.test(key) || isTag(key)) continue; const tag = canonical(key); - if (tag === key) continue; if (tag === undefined) { - // Letters only and not a tag: a legacy item's group name, not a locale key. - if (/^[A-Za-z]+$/.test(key)) continue; todo(`${path}.${key}`, key, `The translation bundle locale key ${NOT_A_TAG}`); continue; } @@ -4103,18 +4091,15 @@ const translationLocaleTagCanonicalized: MetadataConversion = { translations: [ // A stored item under an underscore spelling. { name: 'zh_cn', locale: 'zh_CN', messages: { commonSave: '保存' } }, - // Tag-shaped but in lower case (#28993): no resolver merged it into - // `pt-BR`, so its case is corrected. - { name: 'pt_br', locale: 'pt-br', messages: { commonSave: 'Salvar' } }, + // Already a tag: left alone. (A tag in another case is the next + // entry's, `translation-locale-tag-case-folded`; fixtures stay disjoint.) + { name: 'pt_br', locale: 'pt-BR', messages: { commonSave: 'Salvar' } }, // Not a tag and no single tag it can have meant: a TODO, left as stored. { name: 'qa_bad_locale', locale: 'not a locale!!', messages: { commonSave: 'x' } }, - // A bundle entry: one padded key, one underscore key, two keys in - // another case (one of them letters only), one canonical tag. + // A bundle entry: one padded key, one underscore key, one tag. { 'en ': { messages: { commonSave: 'Save' } }, ja_JP: { messages: { commonSave: '保存' } }, - 'ko-kr': { messages: { commonSave: '저장' } }, - IT: { messages: { commonSave: 'Salva' } }, 'zh-CN': { messages: { commonSave: '保存' } }, }, // A legacy item without `locale`: its group keys are not locale keys. @@ -4132,17 +4117,190 @@ const translationLocaleTagCanonicalized: MetadataConversion = { 'zh-CN': { messages: { commonSave: '保存' } }, en: { messages: { commonSave: 'Save' } }, 'ja-JP': { messages: { commonSave: '保存' } }, + }, + { name: 'fr', globalActions: { export_all: { label: 'Tout exporter' } } }, + ], + i18n: { defaultLocale: 'en-US', supportedLocales: ['en-US', 'zh-CN', 'ja-JP'], fallbackLocale: 'en' }, + data: [{ object: 'crm_lead', locale: ['zh-CN'], records: [] }], + }, + // One per rewritten value: the item, two bundle keys, the default locale, + // two supported locales and the seed scope. + expectedNotices: 7, + }, +}; + +/** + * A locale tag written in a case other than BCP-47's becomes the same tag in + * its canonical case (protocol 18, #28993): `zh-cn` → `zh-CN`, `zh-hans-cn` → + * `zh-Hans-CN`, `EN` → `en`. + * + * The case half of {@link translationLocaleTagCanonicalized}, kept as an entry + * of its own because it retired a release later. That entry left case as + * written, on the premise that every resolver folds it. The premise was + * measured false: a resolver answers one locale for a request, exact match + * first, so a `zh-cn` translation beside the stock `zh-CN` bundle was never + * merged into it, and a `zh-CN` session never saw its strings. `LocaleSchema` + * now folds case on parse, so a live author is corrected there. The metadata + * save door stores the authored body rather than the parse, so this stored-row + * pass is what carries the fold to a row saved in lower case, and to an + * artifact built by 18.2.0 or earlier. + * + * **What it rewrites.** A value that is already tag-shaped and differs from + * its `LocaleSchema` parse only in case. The fold is case only: the language + * subtag lower case, a 4-letter script title case, a 2-letter region upper + * case, every subtag after a singleton lower case. It never replaces a legacy + * subtag (`iw` stays `iw`). A value that is not tag-shaped is not judged here: + * {@link translationLocaleTagCanonicalized} runs first, rewrites the `_` and + * padded spellings into tags and reports the rest. + * + * **A rewrite changes what renders, on purpose.** A `zh-cn` item's strings + * start resolving for `zh-CN` sessions. On a bundle, a key that folds onto a + * key the entry already holds is merged only when the two values are identical + * (`renameKey`); otherwise both are left as stored and reported, because + * picking a winner would silently discard one translator's strings. + * `TranslationBundleSchema` refuses the same pair (`zh-cn` beside `zh-CN`) for + * the same reason. + * + * **Reach.** The surface of {@link translationLocaleTagCanonicalized}: a bare + * item's `locale`, a bundle entry's locale keys, `i18n.defaultLocale`, + * `i18n.fallbackLocale`, `i18n.supportedLocales[]` and `data[].locale[]`. A + * bundle key is rewritten only when it is a tag in another case, so a legacy + * item's group names (`globalActions`, `dashboards`), which are not tags, are + * never touched. + * + * Retired from the load path from day one: the parse folds case for a live + * author itself. Stored rows (`applyConversionsToStoredItem`) and a built + * artifact inside its declared-floor window get the rewrite. Idempotent: a + * canonical tag is never touched again. + */ +const translationLocaleTagCaseFolded: MetadataConversion = { + id: 'translation-locale-tag-case-folded', + toMajor: 18, + retiredFromLoadPath: true, + retiredAfter: '18.2.0', + surface: + 'translation.locale / stack.translations[]. / i18n.defaultLocale / i18n.fallbackLocale / ' + + 'i18n.supportedLocales[] / data[].locale[]', + summary: + "a locale tag written in another case than BCP-47's → the same tag in its canonical case " + + "('zh-cn' → 'zh-CN'): no resolver merged it into the canonical locale, so its strings never " + + 'rendered; a bundle key that would land on a key holding different strings is left as stored ' + + 'and reported', + apply(stack, emit, context) { + /** `v` in its canonical case when `v` is tag-shaped, else `undefined` (left to the entry before). */ + const folded = (v: string): string | undefined => { + const parsed = LocaleSchema.safeParse(v); + return parsed.success ? parsed.data : undefined; + }; + /** A locale value: folded, or returned as is when canonical or not a tag. */ + const value = (raw: string, path: string): string => { + const tag = folded(raw); + if (tag === undefined || tag === raw) return raw; + emit({ from: JSON.stringify(raw), to: JSON.stringify(tag), path }); + return tag; + }; + const list = (raw: unknown, path: string): unknown => { + if (!Array.isArray(raw)) return raw; + let changed = false; + const next = raw.map((v, i) => { + if (typeof v !== 'string') return v; + const mapped = value(v, `${path}[${i}]`); + if (mapped !== v) changed = true; + return mapped; + }); + return changed ? next : raw; + }; + + let next = mapCollection(stack, 'translations', (entry, path) => { + // A bare `translation` ITEM: its own `locale`. + if ('locale' in entry) { + if (typeof entry.locale !== 'string') return entry; + const locale = value(entry.locale, `${path}.locale`); + return locale === entry.locale ? entry : { ...entry, locale }; + } + // A bundle entry: locale code → data. + let bundle = entry; + for (const [key, data] of Object.entries(entry)) { + if (!isDict(data)) continue; + const tag = folded(key); + if (tag === undefined || tag === key) continue; + const renamed = renameKey(bundle, key, tag); + if (renamed === null) { + context?.reportTodo?.({ + path: `${path}.${key}`, + from: JSON.stringify(key), + reason: + `The translation bundle locale key is \`${tag}\` in another case, and the entry already holds ` + + `different strings under \`${tag}\`. Left as stored, it is refused where it is parsed. Merge ` + + 'the two by hand and keep one key.', + }); + continue; + } + emit({ from: JSON.stringify(key), to: JSON.stringify(tag), path: `${path}.${key}` }); + bundle = renamed; + } + return bundle; + }); + + const i18n = next.i18n; + if (isDict(i18n)) { + let config = i18n; + for (const key of ['defaultLocale', 'fallbackLocale'] as const) { + const raw = config[key]; + if (typeof raw !== 'string') continue; + const mapped = value(raw, `i18n.${key}`); + if (mapped !== raw) config = { ...config, [key]: mapped }; + } + const supported = list(config.supportedLocales, 'i18n.supportedLocales'); + if (supported !== config.supportedLocales) config = { ...config, supportedLocales: supported }; + if (config !== i18n) next = { ...next, i18n: config }; + } + + return mapCollection(next, 'data', (dataset, path) => { + const scoped = list(dataset.locale, `${path}.locale`); + return scoped === dataset.locale ? dataset : { ...dataset, locale: scoped }; + }); + }, + fixture: { + before: { + translations: [ + // A stored item saved in lower case. + { name: 'pt_br', locale: 'pt-br', messages: { commonSave: 'Salvar' } }, + // Already canonical: left alone. + { name: 'zh_cn', locale: 'zh-CN', messages: { commonSave: '保存' } }, + // Not a tag: not judged here (the entry before reports it). + { name: 'qa_bad_locale', locale: 'not a locale!!', messages: { commonSave: 'x' } }, + // A bundle entry: two keys in another case (one of them letters only), + // one canonical tag. + { + 'ko-kr': { messages: { commonSave: '저장' } }, + IT: { messages: { commonSave: 'Salva' } }, + 'zh-CN': { messages: { commonSave: '保存' } }, + }, + // A legacy item without `locale`: its group keys are not tags. + { name: 'fr', globalActions: { export_all: { label: 'Tout exporter' } } }, + ], + i18n: { defaultLocale: 'en-us', supportedLocales: ['en-us', 'zh-CN', 'zh-hans-cn'], fallbackLocale: 'en' }, + data: [{ object: 'crm_lead', locale: ['zh-cn'], records: [] }], + }, + after: { + translations: [ + { name: 'pt_br', locale: 'pt-BR', messages: { commonSave: 'Salvar' } }, + { name: 'zh_cn', locale: 'zh-CN', messages: { commonSave: '保存' } }, + { name: 'qa_bad_locale', locale: 'not a locale!!', messages: { commonSave: 'x' } }, + { + 'zh-CN': { messages: { commonSave: '保存' } }, 'ko-KR': { messages: { commonSave: '저장' } }, it: { messages: { commonSave: 'Salva' } }, }, { name: 'fr', globalActions: { export_all: { label: 'Tout exporter' } } }, ], - i18n: { defaultLocale: 'en-US', supportedLocales: ['en-US', 'zh-CN', 'ja-JP'], fallbackLocale: 'en' }, + i18n: { defaultLocale: 'en-US', supportedLocales: ['en-US', 'zh-CN', 'zh-Hans-CN'], fallbackLocale: 'en' }, data: [{ object: 'crm_lead', locale: ['zh-CN'], records: [] }], }, - // One per rewritten value: two items, four bundle keys, the default - // locale, two supported locales and the seed scope. - expectedNotices: 10, + // One per rewritten value: the item, two bundle keys, the default locale, + // two supported locales and the seed scope. + expectedNotices: 7, }, }; @@ -15476,6 +15634,9 @@ const MAJOR_18_CONVERSIONS: readonly OrderedConversion[] = [ { conversion: timeDefaultUtcSuffixDropped, order: 48 }, { conversion: translationComponentSubmitLabelRemoved, order: 12 }, { conversion: translationLocaleTagCanonicalized, order: 76 }, + // AFTER `translationLocaleTagCanonicalized`: an `_` or padded spelling is made a + // tag there, and only a tag's case is folded here (`zh_cn` → `zh-cn` → `zh-CN`). + { conversion: translationLocaleTagCaseFolded, order: 76.5 }, { conversion: translationPerAppSettingsRemoved, order: 34 }, { conversion: translationWidgetSubCaptionRemoved, order: 55 }, { conversion: tursoConfigEncryptionKeyRemoved, order: 74 }, diff --git a/packages/spec/src/migrations/entries/semantic/18.translation-locale-case-canonicalized.ts b/packages/spec/src/migrations/entries/semantic/18.translation-locale-case-canonicalized.ts index b00925c43a9..ccc337e6fd9 100644 --- a/packages/spec/src/migrations/entries/semantic/18.translation-locale-case-canonicalized.ts +++ b/packages/spec/src/migrations/entries/semantic/18.translation-locale-case-canonicalized.ts @@ -2,10 +2,10 @@ import type { SemanticMigration } from '../../types.js'; -// The case half of the `translation-locale-tag-canonicalized` family: the D2 -// conversion folds a stored or built locale to its BCP-47 case, the parse -// folds a live author's, and this entry states what that changes for a -// consumer that compared locales by their exact spelling. +// The case half of the locale-tag family: the D2 conversion +// `translation-locale-tag-case-folded` folds a stored or built locale to its +// BCP-47 case, the parse folds a live author's, and this entry states what +// that changes for a consumer that compared locales by their exact spelling. export const entry: SemanticMigration = { id: 'translation-locale-case-canonicalized', surface: @@ -16,9 +16,16 @@ export const entry: SemanticMigration = { 'nothing to rewrite: the parse returns the tag in its BCP-47 case (language lower case, script ' + 'title case, region upper case, every subtag after a singleton lower case), so `zh-cn` reads ' + 'back as `zh-CN`, `zh-hans-cn` as `zh-Hans-CN` and `EN` as `en`. A consumer that matched the ' - + 'authored spelling exactly matches the canonical one. A translation bundle holding two keys that ' - + 'fold to one tag (`zh-cn` and `zh-CN`) is refused naming both: merge them by hand under the ' - + 'canonical key', + + 'authored spelling exactly matches the canonical one. Two exact matchers compare a folded ' + + 'i18n locale with a row tagged as written, and neither row is folded: the email-template floor ' + + 'warning (warnEmailTemplateLocaleFloor in stack.zod.ts) compares i18n.supportedLocales with ' + + 'emailTemplates[].locale, and the messaging template ladder compares ' + + 'localeCandidates(i18n.getDefaultLocale()) with sys_notification_template.locale. A row tagged ' + + '`zh-cn` no longer matches a supported or default locale authored as `zh-cn`, which now reads ' + + '`zh-CN`: re-tag the row in its canonical case, or fold case in the consumer. The row-side fold ' + + 'lands with the save-door change that stores the parsed locale. A translation bundle holding two ' + + 'keys that fold to one tag (`zh-cn` and `zh-CN`) is refused naming both: merge them by hand ' + + 'under the canonical key', reason: 'Case was accepted as written, on the premise that the resolvers fold it. They answer one locale ' + 'for a request, exact match first, so a translation saved under `zh-cn` beside the stock `zh-CN` ' @@ -32,8 +39,10 @@ export const entry: SemanticMigration = { 'Every locale that parses through LocaleSchema reads back in its BCP-47 case. A stored ' + '`translation` row under `zh-cn` is read back as `zh-CN` by every stored-row seam, so its ' + 'strings are merged into the `zh-CN` locale a session asks for. A built artifact inside the ' - + 'conversion window is folded the same way. A canonical tag is never rewritten. A bundle holding ' - + '`zh-cn` beside `zh-CN` fails the parse naming both keys, and the stored-row conversion leaves ' - + 'such a pair as stored and reports it.', - conversionIds: ['translation-locale-tag-canonicalized'], + + 'conversion window (built by @objectstack/spec 18.2.0 or earlier) is folded the same way. A ' + + 'canonical tag is never rewritten. An email-template or notification-template row tagged in ' + + 'another case is matched by its consumer only once the row is re-tagged or the consumer folds ' + + 'case. A bundle holding `zh-cn` beside `zh-CN` fails the parse naming both keys, and the ' + + 'stored-row conversion leaves such a pair as stored and reports it.', + conversionIds: ['translation-locale-tag-case-folded'], }; diff --git a/packages/spec/src/system/translation.test.ts b/packages/spec/src/system/translation.test.ts index f2f9455898a..700cd3450fd 100644 --- a/packages/spec/src/system/translation.test.ts +++ b/packages/spec/src/system/translation.test.ts @@ -216,6 +216,29 @@ describe('LocaleSchema corrects a tag to its BCP-47 case (#28993)', () => { // Canonical rows come back untouched: the pass is idempotent. const canonical = { name: 'zh_cn', locale: 'zh-CN', messages: { x: 'y' } }; expect(applyConversionsToStoredItem('translation', canonical)).toEqual(canonical); + // An underscore spelling is made a tag by the entry before and folded by + // the case entry after it: the order is what takes `zh_cn` to `zh-CN`. + const underscored = { name: 'zh_cn', locale: 'zh_cn', messages: { x: 'y' } }; + expect(applyConversionsToStoredItem('translation', underscored)).toEqual({ ...underscored, locale: 'zh-CN' }); + }); + + it('data at rest: the case entry reports a case collision instead of guessing, and merges an identical twin', () => { + const conversion = ALL_CONVERSIONS.find((c) => c.id === 'translation-locale-tag-case-folded')!; + expect(conversion.retiredAfter).toBe('18.2.0'); + const todos: { path: string }[] = []; + const notices: { path: string }[] = []; + const stack = { + translations: [ + // `zh-cn` is `zh-CN` in another case, which the entry already holds with other strings. + { 'zh-cn': { messages: { x: 'a' } }, 'zh-CN': { messages: { x: 'b' } } }, + // An identical twin is merged: nothing is lost. + { 'ja-jp': { messages: { x: 'c' } }, 'ja-JP': { messages: { x: 'c' } } }, + ], + }; + const out = conversion.apply(stack, (n) => notices.push(n), { reportTodo: (t) => todos.push(t) }); + expect(todos.map((t) => t.path)).toEqual(['translations[0].zh-cn']); + expect(notices.map((n) => n.path)).toEqual(['translations[1].ja-jp']); + expect(out.translations).toEqual([stack.translations[0], { 'ja-JP': { messages: { x: 'c' } } }]); }); }); diff --git a/packages/spec/src/system/translation.zod.ts b/packages/spec/src/system/translation.zod.ts index 363c41f85ee..0268376e34d 100644 --- a/packages/spec/src/system/translation.zod.ts +++ b/packages/spec/src/system/translation.zod.ts @@ -112,7 +112,7 @@ export type Locale = z.input; * Refuses a bundle holding two locale keys that fold to one tag (`zh-cn` and * `zh-CN`), naming both. Picking a winner would discard one translator's * strings without a word, which is the policy the stored-row conversion - * (`translation-locale-tag-canonicalized`) already follows for the same + * (`translation-locale-tag-case-folded`) already follows for the same * collision. Runs before {@link foldLocaleKeys}, so it sees the keys as written. */ function refuseLocaleKeysFoldingTogether(bundle: Record, ctx: z.RefinementCtx): void { From 263f6c891eac65f49418adc1f13aa7004bf6b9be Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 2 Nov 2026 11:25:08 +0000 Subject: [PATCH 5/5] test(spec): join the case-fold conversion to the release its prescriptions name `translation-locale-tag-case-folded` is retired after 18.2.0, the newest published stable release, so the census pin wants a JOIN row with the prescriptions an author meets for its surface. A lone tag is corrected, not refused, so those are the collision two keys of one bundle make once folded: the bundle refusal at the parse and the conversion's TODO for a stored pair. Both now name the release (`@objectstack/spec 18`). The pin gains a todo row kind for a data-at-rest prescription with no parse refusal to read. Claude-Session: https://claude.ai/code/session_01ADHkmRmpbFs6uQ8AuadsTr Co-authored-by: Claude --- packages/spec/src/conversions/registry.ts | 4 +-- .../retired-key-removal-version.pin.test.ts | 33 +++++++++++++++++-- packages/spec/src/system/translation.zod.ts | 7 ++-- 3 files changed, 37 insertions(+), 7 deletions(-) diff --git a/packages/spec/src/conversions/registry.ts b/packages/spec/src/conversions/registry.ts index cca7011745b..966e69e5382 100644 --- a/packages/spec/src/conversions/registry.ts +++ b/packages/spec/src/conversions/registry.ts @@ -4231,8 +4231,8 @@ const translationLocaleTagCaseFolded: MetadataConversion = { from: JSON.stringify(key), reason: `The translation bundle locale key is \`${tag}\` in another case, and the entry already holds ` - + `different strings under \`${tag}\`. Left as stored, it is refused where it is parsed. Merge ` - + 'the two by hand and keep one key.', + + `different strings under \`${tag}\`. Left as stored, it is refused where it is parsed, since ` + + '@objectstack/spec 18. Merge the two by hand and keep one key.', }); continue; } diff --git a/packages/spec/src/shared/retired-key-removal-version.pin.test.ts b/packages/spec/src/shared/retired-key-removal-version.pin.test.ts index 569e74795d0..dfc27248f10 100644 --- a/packages/spec/src/shared/retired-key-removal-version.pin.test.ts +++ b/packages/spec/src/shared/retired-key-removal-version.pin.test.ts @@ -48,7 +48,9 @@ * its surface. A tombstone row reads the key's parse refusal and its * `[REMOVED]` description, which are the two channels. A refusal row reads the * refusal the surface answers, for a value or union-arm retirement that has no - * key to tombstone. Two guards keep the join honest: + * key to tombstone. A todo row reads the TODO a data-at-rest conversion reports + * for a stored shape it leaves as stored, which `os migrate meta` prints to the + * owner. Two guards keep the join honest: * * - every retired entry whose `retiredAfter` is the newest published stable * release must have a row, so a future retirement cannot ship its @@ -95,7 +97,7 @@ import { InterfacePageConfigSchema, PageComponentSchema, PageSchema } from '../u import { AddRecordConfigSchema, FormFieldSchema, FormSectionSchema, FormViewSchema, ListViewSchema, NavigationModeSchema, ObjectListViewSchema, VIEW_METADATA_MEMBERS } from '../ui/view.zod'; import { FlowSchema } from '../automation/flow.zod'; import { JobSchema } from '../system/job.zod'; -import { TranslationItemSchema } from '../system/translation.zod'; +import { TranslationBundleSchema, TranslationItemSchema } from '../system/translation.zod'; import { PROTOCOL_MAJOR } from '../kernel/protocol-version'; import { EmailTemplateDefinitionSchema } from '../system/email-template.zod'; import { @@ -220,6 +222,20 @@ const refusal = (where: string, parse: () => Parse, at?: PropertyKey): Renderer return issues.map((issue) => ({ where, text: issue.message })); }; +/** + * The TODO a retired conversion reports for a stored shape it cannot rewrite + * without a guess, run on `stack`: the prescription the owner of a stored row + * meets in `os migrate meta`, which has no parse refusal to read. + */ +const todo = (where: string, id: string, stack: Record): Renderer => () => { + const conversion = ALL_CONVERSIONS.find((c) => c.id === id); + if (!conversion) throw new Error(`${where}: no conversion \`${id}\``); + const reasons: string[] = []; + conversion.apply(stack, () => {}, { reportTodo: (detail) => reasons.push(detail.reason) }); + expect(reasons.length, `${where} must be reported`).toBeGreaterThan(0); + return reasons.map((text) => ({ where, text })); +}; + const ELEMENTS: readonly (readonly [unknown, string, readonly string[]])[] = [ [ElementNumberPropsSchema, 'element:number', ['object', 'filter']], [ElementRecordPickerPropsSchema, 'element:record_picker', ['object', 'filter', 'sort', 'limit']], @@ -280,6 +296,19 @@ const JOIN: Readonly> = { 'locale', ), ], + // #28993: the case fold corrects a lone tag, so what an author meets is the + // collision two keys of one bundle make once folded, at the parse and at rest. + 'translation-locale-tag-case-folded': [ + refusal( + 'a translation bundle holding `zh-cn` beside `zh-CN`', + () => TranslationBundleSchema.safeParse({ 'zh-cn': { messages: { x: 'a' } }, 'zh-CN': { messages: { x: 'b' } } }), + ), + todo( + 'a stored translation bundle holding `zh-cn` beside `zh-CN`', + 'translation-locale-tag-case-folded', + { translations: [{ 'zh-cn': { messages: { x: 'a' } }, 'zh-CN': { messages: { x: 'b' } } }] }, + ), + ], 'manifest-permissions-string-list-removed': [ refusal('manifest.permissions as a list of strings', () => ManifestPermissionsSchema.safeParse(['system.user.read'])), ], diff --git a/packages/spec/src/system/translation.zod.ts b/packages/spec/src/system/translation.zod.ts index 0268376e34d..1d5abd75e4a 100644 --- a/packages/spec/src/system/translation.zod.ts +++ b/packages/spec/src/system/translation.zod.ts @@ -128,9 +128,10 @@ function refuseLocaleKeysFoldingTogether(bundle: Record, ctx: z code: 'custom', path: [key], message: - `locale keys \`${first}\` and \`${key}\` are one locale, \`${tag}\`: a locale tag's case is ` - + 'corrected on parse, so both entries would land on one key and one set of strings would be ' - + `dropped. Merge the two by hand and keep one key, \`${tag}\`.`, + `locale keys \`${first}\` and \`${key}\` are one locale, \`${tag}\`. Refused since ` + + "@objectstack/spec 18, because a locale tag's case is corrected on parse, so both entries would " + + 'land on one key and one set of strings would be dropped. Merge the two by hand and keep one ' + + `key, \`${tag}\`.`, }); } }