fix(catalog): select international catalogs by resource version

docs/commands.md

@@ -4,7 +4,7 @@
 nnnotes [global options] <command> [command options]
 ```
 
-Global options (before the command): `--config`, `--region`, `--language`, `--catalog`, `--cache`, `--master`,
+Global options (before the command): `--config`, `--region`, `--language`, `--catalog`, `--catalog-release`, `--cache`, `--master`,
 `--apk`, `--ffmpeg`, `--vgmstream`, `--node`, plus `--version` and `--help`. They set the settings
 described in [configuration.md](configuration.md), which also lists the settings each command needs.
 `nnnotes <command> --help` prints the options of a command. The asset export commands (`export`, `plan`,
@@ -42,16 +42,23 @@ setting's value. Details in [configuration.md](configuration.md#writing-the-conf
 `pull` and every extractor read bundles through the cache (`[paths] cache`):
 
 ```
-<cache>/catalog_main_<language>.bin     the catalog of the language, the same for every region (downloaded on first
-                                        use unless [paths] catalog is set)
+<cache>/catalog_main_<language>.bin     the legacy main catalog (explicit main or no API/version pin)
 <cache>/bundles/<bundle file name>      bundles of the dependency closures, decrypted (UnityFS)
 <cache>/raw/<path>                      raw CDN files stored as they are (e.g. CRI cue sheet data)
 <cache>/catalogs/<region>/              catalogs downloaded by `browse`
+<cache>/international/<CDN SHA256>/<resource version>/
+                                       versioned catalog, bundles and raw files; isolated from old main caches
 ```
 
 Files already in the cache are not downloaded again. When `[paths] apk` is set, the APK's own catalog is merged with
 the region's and bundles that ship inside the APK are read from it.
 
+For international regions with a configured API, catalog-based commands discover `resource_version` first and
+download `catalog_<resource_version>_<language>.bin`. `catalogs fetch` labels the bytes with the version used to
+select that file, rather than querying a newer label after downloading main. Use `--catalog-release VERSION` or
+the [version settings](configuration.md) to pin builds and use cached data offline. An explicit `--catalog FILE`
+bypasses discovery. JP keeps its existing Version-driven path.
+
 ## catalog
 
 ```

docs/configuration.md

@@ -87,19 +87,21 @@ exist when they are given.
 | `[master] key` | `NNNOTES_MASTER_KEY` | — | Rijndael-256 key of the master data files: 32 bytes as 64 hex digits |
 | `[master] iv` | `NNNOTES_MASTER_IV` | — | Rijndael-256 CBC initialization vector: 32 bytes as 64 hex digits |
 | `[catalog] region` | `NNNOTES_CATALOG_REGION` | `--region` | name of one `[servers.<region>]` table |
-| `[catalog] language` | `NNNOTES_CATALOG_LANGUAGE` | `--language` | catalog language: the `<language>` of `catalog_main_<language>.bin`: `ja`, `en`, `zh-Hant`, `zh-Hans` or `ko`; also the client language of `live`, `story` and `web` (the text field, fonts and line spacing of their UI) and of the model labels of `web --live2d` |
+| `[catalog] language` | `NNNOTES_CATALOG_LANGUAGE` | `--language` | catalog language: the `<language>` of `catalog_<version>_<language>.bin`: `ja`, `en`, `zh-Hant`, `zh-Hans` or `ko`; also the client language of `live`, `story` and `web` (the text field, fonts and line spacing of their UI) and of the model labels of `web --live2d` |
+| `[catalog] version` | `NNNOTES_CATALOG_VERSION` | `--catalog-release` | optional international resource version; unset: query the configured region API, or use legacy `main` when no API is configured; JP uses its own discovery |
 | `[servers.<region>] name` | `NNNOTES_SERVERS_<REGION>_NAME` | — | label of the region in `browse` (default: the region name) |
 | `[servers.<region>] provider` | `NNNOTES_SERVERS_<REGION>_PROVIDER` | — | `international` or `jp`; empty selects `jp` for the region named jp, international otherwise |
 | `[servers.<region>] client_version` | `NNNOTES_SERVERS_<REGION>_CLIENT_VERSION` | — | per-region API client version; overrides `[client] version`, then falls back to the region's APK versionName |
 | `[servers.<region>] apk` | `NNNOTES_SERVERS_<REGION>_APK` | `--apk` | per-region APK, APKS/XAPK or directory with base.apk; adjacent splits of base.apk are read automatically; the flag overrides it |
 | `[servers.<region>] catalog` | `NNNOTES_SERVERS_<REGION>_CATALOG` | `--catalog` | per-region catalog file; JP needs the matching `<file>.source.json`; the flag overrides it |
+| `[servers.<region>] catalog_version` | `NNNOTES_SERVERS_<REGION>_CATALOG_VERSION` | `--catalog-release` | per-region international version pin; overrides `[catalog] version`, but not the command-line flag |
 | `[servers.<region>] cdn` | `NNNOTES_SERVERS_<REGION>_CDN` | — | CDN base URL of the region (a trailing `/` is ignored) |
 | `[servers.<region>] languages` | `NNNOTES_SERVERS_<REGION>_LANGUAGES` | — | catalog languages `browse` lists: a TOML array of strings; comma-separated in the environment |
 | `[servers.<region>] api` | `NNNOTES_SERVERS_<REGION>_API` | — | API root of the region: `https://host[:port]` (TLS, port 443 by default), `host[:port]`, or `http://host[:port]` for a plain-text local server; no path |
 | `[servers.<region>] master` | `NNNOTES_SERVERS_<REGION>_MASTER` | — | decoded master data directory of the region (default: `[paths] master`) |
 | `[bootstrap] api` | `NNNOTES_BOOTSTRAP_API` | — | API root that serves the server list (`nnnotes servers`), same format |
 | `[client] version` | `NNNOTES_CLIENT_VERSION` | — | client version sent to the game's API, e.g. `1.0.1`; unset: the `versionName` of `[paths] apk` |
-| `[paths] catalog` | `NNNOTES_PATHS_CATALOG` | `--catalog` | a catalog `.bin` file to read instead of downloading `catalog_main_<language>.bin` |
+| `[paths] catalog` | `NNNOTES_PATHS_CATALOG` | `--catalog` | a catalog `.bin` file to read instead of discovering/downloading a catalog; explicit files stay usable offline |
 | `[paths] cache` | `NNNOTES_PATHS_CACHE` | `--cache` | cache directory (created when missing) |
 | `[paths] store` | `NNNOTES_PATHS_STORE` | `--store` (of `export`, `plan`, `run-stage`, `catalogs`, `store`) | store directory of the asset export ([assets.md](assets.md)); unset: `<[paths] cache>/store` |
 | `[paths] master` | `NNNOTES_PATHS_MASTER` | `--master` | decoded master data directory, one `<Table>.json` per table; the flag overrides `[servers.<region>] master` |
@@ -121,9 +123,24 @@ for several of them. The other commands use the one region named by `[catalog] r
 table per region.
 
 Master data per region: a command that reads master data for region `<r>` takes the directory of the `--master` flag,
-else `[servers.<r>] master`, else `[paths] master`. The regions serve the same catalog for a language, so the
-catalog settings and the cached `catalog_main_<language>.bin` serve every region; bundles are fetched from the CDN of
-the region in use.
+else `[servers.<r>] master`, else `[paths] master`. International catalog selection uses `resource_version`, not
+the master version or a change to the CDN root. `catalog_main` is a separate catalog that may remain old.
+When the region has an API root, catalog-based commands query Version before selecting a download; discovery
+failure stops rather than treating main as current. Without an API or version pin, the old main/offline behavior
+is retained. JP's version/hash directory and authentication remain unchanged.
+
+For repeatable builds, pin the resource version belonging to your master snapshot:
+
+```sh
+nnnotes --region tw --catalog-release 1.0.0.201 web site --story 10948
+nnnotes --region tw --catalog-release 1.0.0.201 live2d MODEL_ID -o out/model
+```
+
+Alternatively set `NNNOTES_SERVERS_TW_CATALOG_VERSION` in the build environment; workers inherit this pin.
+`--catalog-release` selects a remote resource release. The asset commands' existing `--catalog-version LABEL|SHA`
+still selects an already imported store record. To inspect the historical main catalog, explicitly pin `main`.
+Versioned caches are isolated by CDN root and resource version; a cached pin needs the CDN setting to identify
+its directory but performs no API/catalog download. A missing versioned file never falls back to main.
 
 ## What each command needs
 

docs/music-data.md

@@ -90,7 +90,7 @@ A **text** is an object with one string per language of `languages` (`{"ja": ...
 |---|---|
 | `region` | the region whose master data this is (a configured region name), or `embedded` for the APK's master data |
 | `client.versionName`, `client.versionCode` | the APK's version name and code (null without `[paths] apk`) |
-| `catalog.resourceVersion` | the resource version recorded for the catalog in the catalog store (`nnnotes catalogs fetch` / `import`), null when none is recorded |
+| `catalog.resourceVersion` | the resource version used to select the downloaded catalog, or recorded for an explicit file in the catalog store (`nnnotes catalogs fetch` / `import`); null when unknown |
 | `catalog.sha256` | SHA-256 of the remote catalog file the charts were read with |
 | `master.source` | `api` (`--master-files`, `--decoded-master`: the region's files) or `embedded` (`--apk-master`) |
 | `master.version` | the `version` of the master data manifest |

src/nnnotes/addressables.py

@@ -362,13 +362,10 @@ def catalog(self, region: Region, language: str):
                 self.server.catalogs[key] = browse(cat.entries)
                 self.server.jp_catalogs[key] = cat
                 return self.server.catalogs[key]
-            catalog = self.server.cache / "catalogs" / region.name / f"catalog_main_{language}.bin"
-            if not catalog.exists():
-                with urlopen(region.cdn + f"/asset/Android/{catalog.name}", timeout=60) as response:
-                    raw = response.read()
-                catalog.parent.mkdir(parents=True, exist_ok=True)
-                catalog.write_bytes(raw)
-            self.server.catalogs[key] = browse(parse(catalog.read_bytes()))
+            from .catalog import Catalog
+            version = region.config.catalog_version(region.name) if region.config is not None else "main"
+            cat = Catalog.load(language, self.server.cache / "catalogs" / region.name, cdn=region.cdn, version=version)
+            self.server.catalogs[key] = browse(cat.entries)
         return self.server.catalogs[key]
 
     def do_GET(self):

src/nnnotes/catalog.py

@@ -18,7 +18,7 @@
 from .apkset import ApkSet
 from .addressables import REMOTE_PREFIX, BundleKey, decrypt, parse, parse_locations, remote_path
 from .cache import write_atomic as _write_atomic
-from .config import ConfigError, apk_missing
+from .config import ConfigError, apk_missing, check_catalog_version
 
 LOCAL_PREFIX = "{UnityEngine.AddressableAssets.Addressables.RuntimePath}"
 APK_AA_DIR = "assets/aa/"          # + "Android/<bundle>"
@@ -80,12 +80,13 @@ class Catalog:
     """
 
     def __init__(self, catalog_bytes: bytes, cache_dir: Path, *, cdn=None, bundle_key=None, apk: Path | None = None,
-                 source=None, session=None, apk_catalog: bytes | None = None):
+                 source=None, session=None, apk_catalog: bytes | None = None, resource_version: str | None = None):
         self._settings = {"cdn": cdn, "bundle_key": bundle_key}
         self.cache_dir = Path(cache_dir)
         self.cache_dir.mkdir(parents=True, exist_ok=True)
         self.apk = Path(apk) if apk else None
         self.source, self.session = source, session
+        self.resource_version = resource_version
         self._sources = {"remote": catalog_bytes}
         self._locations: list[dict] | None = None
         self._parsed: tuple | None = None
@@ -142,18 +143,28 @@ def _by_key(self) -> dict[str, list[dict]]:
     # --- construction ------------------------------------------------------
     @classmethod
     def load(cls, language: str, cache_dir: Path, *, cdn=None, bundle_key=None,
-             apk: Path | None = None) -> "Catalog":
+             apk: Path | None = None, version: str = "main") -> "Catalog":
         """The remote catalog of `language` from the cache, downloaded from the CDN on first use (`cdn` /
-        `bundle_key`: as for Catalog)."""
+        `bundle_key`: as for Catalog). A versioned catalog is isolated by CDN root and version; main retains
+        its legacy offline cache path. Missing versioned files never fall back to main."""
         cache_dir = Path(cache_dir)
-        cat = cls.cache_file(language, cache_dir)
+        filename = cls.cache_file(language, Path("."), version=version).name
+        if version != "main":
+            cdn = _setting(cdn, filename)
+            if not cdn:
+                raise ConfigError("a versioned catalog needs its CDN root to identify the cache")
+            cache_dir = cache_dir / "international" / hashlib.sha256(cdn.rstrip("/").encode()).hexdigest() / version
+        cat = cache_dir / filename
         if not cat.exists():
             cdn = _setting(cdn, cat.name)
             if not cdn:
                 raise FileNotFoundError(f"{cat} not cached and no CDN base given")
             cat.parent.mkdir(parents=True, exist_ok=True)
-            _write_atomic(cat, download(cdn.rstrip("/") + f"/asset/Android/{cat.name}"))
-        return cls(cat.read_bytes(), cache_dir, cdn=cdn, bundle_key=bundle_key, apk=apk)
+            data = download(cdn.rstrip("/") + f"/asset/Android/{cat.name}")
+            parse(data)  # Never install a corrupt/error response as a reusable catalog.
+            _write_atomic(cat, data)
+        return cls(cat.read_bytes(), cache_dir, cdn=cdn, bundle_key=bundle_key, apk=apk,
+                   resource_version=version if version != "main" else None)
 
     def _setting(self, name: str, needed_by: str):
         """The `cdn` / `bundle_key` given to the catalog, its function called (once) now that `needed_by` needs it."""
@@ -167,9 +178,11 @@ def cdn(self) -> str | None:
         return None if callable(v) or not v else v.rstrip("/")
 
     @staticmethod
-    def cache_file(language: str, cache_dir: Path) -> Path:
+    def cache_file(language: str, cache_dir: Path, *, version: str = "main") -> Path:
         """Where the remote catalog of `language` is cached."""
-        return Path(cache_dir) / f"catalog_main_{language}.bin"
+        check_catalog_version(version)
+        check_catalog_version(language)
+        return Path(cache_dir) / f"catalog_{version}_{language}.bin"
 
     # --- lookup ------------------------------------------------------------
     def keys(self, prefix: str = "") -> list[str]:

src/nnnotes/catalogdb.py

@@ -315,9 +315,9 @@ def apk_catalog(apk) -> bytes:
         return z.read(APK_CATALOG)
 
 
-def fetch(cdn: str, language: str, timeout: float = 120) -> tuple[bytes, str | None]:
+def fetch(cdn: str, language: str, timeout: float = 120, *, version: str = "main") -> tuple[bytes, str | None]:
     """The remote catalog of `language` from a CDN base, and the text of its `.hash` file (None when not served)."""
-    base = cdn.rstrip("/") + "/asset/Android/" + Catalog.cache_file(language, Path(".")).name
+    base = cdn.rstrip("/") + "/asset/Android/" + Catalog.cache_file(language, Path("."), version=version).name
     with urllib.request.urlopen(base, timeout=timeout) as r:
         data = r.read()
     try:

src/nnnotes/cli.py

@@ -72,6 +72,7 @@
 FLAG_SETTINGS = {
     ("catalog", "region"): ("region", "--region"),
     ("catalog", "language"): ("language", "--language"),
+    ("catalog", "version"): ("catalog_release", "--catalog-release"),
     ("paths", "catalog"): ("catalog", "--catalog"),
     ("paths", "cache"): ("cache", "--cache"),
     ("paths", "master"): ("master", "--master"),
@@ -111,10 +112,10 @@ def _existing(cfg: Config, section: str, key: str, kind: str = "file") -> Path |
 
 def open_catalog(cfg: Config, bundles: bool = True, region: str | None = None) -> Catalog:
     """The catalog of [catalog] language (merged with the APK's when [paths] apk is set), fetching from the CDN of
-    `region` (default: [catalog] region); the regions serve the same catalog for a language, so one cached file
-    serves them all. `bundles`: bundles will be fetched; else only the catalog is read. The region, its CDN base
-    and the bundle key are read from the settings only when something must be downloaded (every file in the cache:
-    none of them is needed), then a missing one is a ConfigError naming the setting."""
+    `region` (default: [catalog] region). International versioned catalogs are isolated by CDN root and resource
+    version. `bundles`: bundles will be fetched; else only the catalog is read. Explicit files bypass discovery.
+    Version pins bypass the API but need the CDN root to identify the cache; legacy main keeps lazy CDN lookup.
+    The bundle key is only read for a missing encrypted bundle."""
     if region:
         cfg = cfg.for_region(region)
     cache = cfg.require_path("paths", "cache")
@@ -129,7 +130,7 @@ def open_catalog(cfg: Config, bundles: bool = True, region: str | None = None) -
     key = (lambda: bundle_key(cfg)) if bundles else None
     if catbin is not None:
         return Catalog(catbin.read_bytes(), cache, cdn=cdn, bundle_key=key, apk=apk)
-    return Catalog.load(language, cache, cdn=cdn, bundle_key=key, apk=apk)
+    return Catalog.load(language, cache, cdn=cdn, bundle_key=key, apk=apk, version=cfg.catalog_version())
 
 
 def master_dir(cfg: Config, region: str | None = None) -> Path:
@@ -771,6 +772,7 @@ def build_parser() -> argparse.ArgumentParser:
     p.add_argument("--language", help="catalog and client language: ja, en, zh-Hant, zh-Hans or ko "
                                       "([catalog] language)")
     p.add_argument("--catalog", help="catalog .bin file ([paths] catalog; else downloaded into the cache)")
+    p.add_argument("--catalog-release", help="pin an international resource version ([catalog] version; else API discovery when configured, otherwise main)")
     p.add_argument("--cache", help="cache directory ([paths] cache)")
     p.add_argument("--master", help="decoded master data directory ([paths] master)")
     p.add_argument("--apk", help="base.apk ([paths] apk)")