Fix canonical IDs with language-aware ASCII transliteration

[ziodave]

non-ASCII names previously retained Unicode, while the proposed IRI_UTF_fix stripped unsupported letters and could merge unrelated entities into /web-pages/thing. this change generates ASCII IDs using account.language, with German umlaut rules and explicit ICU routes for supported scripts

• share normalization between the allocator and both generator strategies
• append a hash of the normalized original non-ASCII name when no URL hash is available; keep distinct and repeated multilingual siblings stable across runs
• carry account language through cloud callbacks and one-shot/persistent workers without cross-job state
• preserve authoritative root lookup mappings and existing URL hashing

Compatibility and setup

kg-build and all now require PyICU 2.16.2 and native ICU 74.2. non-ASCII generation rejects another linked ICU version to keep identifiers reproducible. unsupported readings, including Japanese kanji, use deterministic fallback IDs. regenerated IDs may change; this PR does not migrate or delete stored entities

Validation

• 328 tests/kg_build tests passed on Python 3.14
• shared slug module coverage: 100%
• kg-build and all smoke checks passed
• isolated core installation and missing-extra hint checks passed without PyICU
• Ruff, commit hooks, Poetry lock checks, and wheel/sdist builds passed
• existing Pydantic v1/Python 3.14 and RDFLib deprecation warnings remain

Specialist review

independent identity and Unicode/packaging reviews found 2 issues, both fixed in follow-up commit 169d361: repeated duplicate allocation could advance its suffix, and Mandarin private-use subtags could disable transliteration. regression tests cover both

Closes #34

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.