diff --git a/CLAUDE.md b/CLAUDE.md index 783273e189..a9b53d30b4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -143,9 +143,24 @@ post_install do |installer| end ``` -### Localization files are UTF-16LE -iOS `*.lproj/Localizable.strings` files are UTF-16 little-endian, so plain `grep` fails ("Binary file matches"). Convert before searching: +### Localization file encoding differs by target — check before converting +Encoding is **not** uniform across the repo, so never assume: + +- **`DashWallet/*.lproj/Localizable.strings` and `Localizable.stringsdict` (main app) — UTF-8, no BOM.** All 43 locales, both file types. Plain `grep` works. Do **not** run `iconv` on these and do **not** "restore" them to UTF-16 — that would break the plain-text tooling (bartycrouch, `plutil`, review diffs) the repo relies on. + ```bash + grep '"Spend"' DashWallet/de.lproj/Localizable.strings + ``` +- **`WatchApp/*.lproj/Interface.strings` — UTF-16LE with BOM** for 40 of 43 locales (`en`, `zh-Hans`, `zh-Hant-TW` are UTF-8). These are the files where `grep` reports "Binary file matches". They are storyboard-derived, so the keys are ObjectIDs rather than English text: + ```bash + iconv -f UTF-16LE -t UTF-8 WatchApp/de.lproj/Interface.strings | grep '.title' + ``` + +Check first rather than trusting this list — and note `file` reports pure-ASCII UTF-8 as `us-ascii`, which is still UTF-8: ```bash -iconv -f UTF-16LE -t UTF-8 DashWallet/de.lproj/Localizable.strings | grep '"Spend"' +head -c 4 DashWallet/de.lproj/Localizable.strings | xxd # 2f2a 204e -> "/* N" = UTF-8 +head -c 4 WatchApp/de.lproj/Interface.strings | xxd # fffe ... = UTF-16LE BOM ``` -Translations sync via Transifex: `tx push -s` (push source) / `tx pull -a` (pull all). Let Xcode and BartyCrouch manage the files; keep them UTF-16LE. + +Translations sync via Transifex: `tx push -s` (push source) / `tx pull -a` (pull all). Let Xcode and BartyCrouch manage the files, and keep each file in whatever encoding it already has. + +A Transifex pull can hand back UTF-16 or BOM-prefixed catalogs. `./scripts/convert_strings_to_utf8.sh` normalises them back — it is scoped to `DashWallet/*.lproj` on purpose and is idempotent, so it leaves already-correct files byte-identical. Keep that scope: a bare `find . -name '*.strings'` would also rewrite the UTF-16LE WatchApp catalogs and anything under `Pods/`. diff --git a/scripts/convert_strings_to_utf8.sh b/scripts/convert_strings_to_utf8.sh index a18aa8d52d..4176ad4956 100755 --- a/scripts/convert_strings_to_utf8.sh +++ b/scripts/convert_strings_to_utf8.sh @@ -1,8 +1,56 @@ #!/usr/bin/env bash +# +# Normalises the main-app localization catalogs to UTF-8 without a BOM after +# pulling from Transifex, which sometimes returns UTF-16 or BOM-prefixed files. +# +# Scope is deliberate and must stay narrow: ONLY DashWallet/*.lproj catalogs. +# - WatchApp/*.lproj/Interface.strings are UTF-16LE by design (40 of 43 +# locales). Converting them here would silently change their encoding. +# - A bare `find . -name '*.strings'` also descends into Pods/, rewriting +# third-party resources. +# See the "Localization file encoding" section of CLAUDE.md. +# +# Idempotent: files already UTF-8 without a BOM are left byte-identical. +# +# Usage: run from anywhere: ./scripts/convert_strings_to_utf8.sh -## This script fixes wrong Localizable.strings encoding after pulling from Transifex -## Usage: run from DashWallet root directory: ./scripts/convert_strings_to_utf8.sh +set -euo pipefail +cd "$(dirname "$0")/.." -for f in $(find `pwd` -name '*.strings'); do - vim +"set nobomb | set fenc=utf8 | x" $f +shopt -s nullglob +converted=0 +checked=0 + +strip_utf8_bom() { + perl -i -pe 's/^\x{ef}\x{bb}\x{bf}// if $. == 1' "$1" +} + +for f in DashWallet/*.lproj/Localizable.strings DashWallet/*.lproj/Localizable.stringsdict; do + checked=$((checked + 1)) + case "$(head -c 3 "$f" | xxd -p)" in + fffe*) + tmp="$(mktemp)" + iconv -f UTF-16LE -t UTF-8 "$f" >"$tmp" + strip_utf8_bom "$tmp" + mv "$tmp" "$f" + echo "UTF-16LE -> UTF-8: $f" + converted=$((converted + 1)) + ;; + feff*) + tmp="$(mktemp)" + iconv -f UTF-16BE -t UTF-8 "$f" >"$tmp" + strip_utf8_bom "$tmp" + mv "$tmp" "$f" + echo "UTF-16BE -> UTF-8: $f" + converted=$((converted + 1)) + ;; + efbbbf) + strip_utf8_bom "$f" + echo "stripped UTF-8 BOM: $f" + converted=$((converted + 1)) + ;; + esac done + +echo "checked $checked main-app catalog(s), normalised $converted." +echo "WatchApp/*.lproj/Interface.strings left alone on purpose (UTF-16LE by design)."