From 36bcf84e553351b08add6ce51fb05ef4d08c73f1 Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Sun, 9 Aug 2026 14:05:52 +0700 Subject: [PATCH 1/2] docs: correct the stale UTF-16LE claim about localization files MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CLAUDE.md said `*.lproj/Localizable.strings` files are UTF-16LE and told readers to keep them that way. That is inverted for the main app: all 43 locales of `DashWallet/*.lproj/Localizable.strings` and `Localizable.stringsdict` are UTF-8 with no BOM, and plain `grep` works on them. The files that actually are UTF-16LE are `WatchApp/*.lproj/Interface.strings` (40 of 43; en, zh-Hans and zh-Hant-TW are UTF-8). The old wording was actively harmful — acting on it means running iconv over UTF-8 files or "restoring" them to UTF-16, which breaks bartycrouch, plutil and reviewable diffs. It came up in review on #945, where a reviewer cited the guideline to request exactly that conversion. Also notes that `file` reports pure-ASCII UTF-8 as `us-ascii`, and every example command is runnable as written. Co-Authored-By: Claude Opus 5 --- CLAUDE.md | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 783273e189..61c997ea06 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -143,9 +143,22 @@ post_install do |installer| end ``` -### Localization files are UTF-16LE -iOS `*.lproj/Localizable.strings` files are UTF-16 little-endian, so plain `grep` fails ("Binary file matches"). Convert before searching: +### Localization file encoding differs by target — check before converting +Encoding is **not** uniform across the repo, so never assume: + +- **`DashWallet/*.lproj/Localizable.strings` and `Localizable.stringsdict` (main app) — UTF-8, no BOM.** All 43 locales, both file types. Plain `grep` works. Do **not** run `iconv` on these and do **not** "restore" them to UTF-16 — that would break the plain-text tooling (bartycrouch, `plutil`, review diffs) the repo relies on. + ```bash + grep '"Spend"' DashWallet/de.lproj/Localizable.strings + ``` +- **`WatchApp/*.lproj/Interface.strings` — UTF-16LE with BOM** for 40 of 43 locales (`en`, `zh-Hans`, `zh-Hant-TW` are UTF-8). These are the files where `grep` reports "Binary file matches". They are storyboard-derived, so the keys are ObjectIDs rather than English text: + ```bash + iconv -f UTF-16LE -t UTF-8 WatchApp/de.lproj/Interface.strings | grep '.title' + ``` + +Check first rather than trusting this list — and note `file` reports pure-ASCII UTF-8 as `us-ascii`, which is still UTF-8: ```bash -iconv -f UTF-16LE -t UTF-8 DashWallet/de.lproj/Localizable.strings | grep '"Spend"' +head -c 4 DashWallet/de.lproj/Localizable.strings | xxd # 2f2a 204e -> "/* N" = UTF-8 +head -c 4 WatchApp/de.lproj/Interface.strings | xxd # fffe ... = UTF-16LE BOM ``` -Translations sync via Transifex: `tx push -s` (push source) / `tx pull -a` (pull all). Let Xcode and BartyCrouch manage the files; keep them UTF-16LE. + +Translations sync via Transifex: `tx push -s` (push source) / `tx pull -a` (pull all). Let Xcode and BartyCrouch manage the files, and keep each file in whatever encoding it already has. From f3ff8c5d3740b1917c4f5b0b8f794fa382eff062 Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Sun, 9 Aug 2026 14:22:45 +0700 Subject: [PATCH 2/2] fix(ci): scope the Transifex encoding-fixup script to the main-app catalogs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit scripts/convert_strings_to_utf8.sh ran `find $(pwd) -name '*.strings'` and rewrote every match as UTF-8 without a BOM. That swept in the 40 UTF-16LE WatchApp/*.lproj/Interface.strings files — which are UTF-16LE by design — and anything under Pods/. Following the documented post-pull workflow therefore silently changed encodings it should not touch. Now scoped to DashWallet/*.lproj/Localizable.strings and .stringsdict, and it detects the BOM instead of unconditionally rewriting, so it is idempotent: on a clean tree it reports 'checked 86, normalised 0' and leaves every file byte-identical. Also drops the vim dependency. Verified both directions: no-op on the current tree, and a file forced to UTF-16LE converts back byte-identical to the original. Reported by CodeRabbit on #948. Co-Authored-By: Claude Opus 5 --- CLAUDE.md | 2 ++ scripts/convert_strings_to_utf8.sh | 56 +++++++++++++++++++++++++++--- 2 files changed, 54 insertions(+), 4 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 61c997ea06..a9b53d30b4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -162,3 +162,5 @@ head -c 4 WatchApp/de.lproj/Interface.strings | xxd # fffe ... = UTF-16LE ``` Translations sync via Transifex: `tx push -s` (push source) / `tx pull -a` (pull all). Let Xcode and BartyCrouch manage the files, and keep each file in whatever encoding it already has. + +A Transifex pull can hand back UTF-16 or BOM-prefixed catalogs. `./scripts/convert_strings_to_utf8.sh` normalises them back — it is scoped to `DashWallet/*.lproj` on purpose and is idempotent, so it leaves already-correct files byte-identical. Keep that scope: a bare `find . -name '*.strings'` would also rewrite the UTF-16LE WatchApp catalogs and anything under `Pods/`. diff --git a/scripts/convert_strings_to_utf8.sh b/scripts/convert_strings_to_utf8.sh index a18aa8d52d..4176ad4956 100755 --- a/scripts/convert_strings_to_utf8.sh +++ b/scripts/convert_strings_to_utf8.sh @@ -1,8 +1,56 @@ #!/usr/bin/env bash +# +# Normalises the main-app localization catalogs to UTF-8 without a BOM after +# pulling from Transifex, which sometimes returns UTF-16 or BOM-prefixed files. +# +# Scope is deliberate and must stay narrow: ONLY DashWallet/*.lproj catalogs. +# - WatchApp/*.lproj/Interface.strings are UTF-16LE by design (40 of 43 +# locales). Converting them here would silently change their encoding. +# - A bare `find . -name '*.strings'` also descends into Pods/, rewriting +# third-party resources. +# See the "Localization file encoding" section of CLAUDE.md. +# +# Idempotent: files already UTF-8 without a BOM are left byte-identical. +# +# Usage: run from anywhere: ./scripts/convert_strings_to_utf8.sh -## This script fixes wrong Localizable.strings encoding after pulling from Transifex -## Usage: run from DashWallet root directory: ./scripts/convert_strings_to_utf8.sh +set -euo pipefail +cd "$(dirname "$0")/.." -for f in $(find `pwd` -name '*.strings'); do - vim +"set nobomb | set fenc=utf8 | x" $f +shopt -s nullglob +converted=0 +checked=0 + +strip_utf8_bom() { + perl -i -pe 's/^\x{ef}\x{bb}\x{bf}// if $. == 1' "$1" +} + +for f in DashWallet/*.lproj/Localizable.strings DashWallet/*.lproj/Localizable.stringsdict; do + checked=$((checked + 1)) + case "$(head -c 3 "$f" | xxd -p)" in + fffe*) + tmp="$(mktemp)" + iconv -f UTF-16LE -t UTF-8 "$f" >"$tmp" + strip_utf8_bom "$tmp" + mv "$tmp" "$f" + echo "UTF-16LE -> UTF-8: $f" + converted=$((converted + 1)) + ;; + feff*) + tmp="$(mktemp)" + iconv -f UTF-16BE -t UTF-8 "$f" >"$tmp" + strip_utf8_bom "$tmp" + mv "$tmp" "$f" + echo "UTF-16BE -> UTF-8: $f" + converted=$((converted + 1)) + ;; + efbbbf) + strip_utf8_bom "$f" + echo "stripped UTF-8 BOM: $f" + converted=$((converted + 1)) + ;; + esac done + +echo "checked $checked main-app catalog(s), normalised $converted." +echo "WatchApp/*.lproj/Interface.strings left alone on purpose (UTF-16LE by design)."