Skip to content

feat(music-data): read decoded master data with its manifest (--decoded-master) - #6

Merged
nichinichisou0609 merged 1 commit into
mainfrom
feat/music-data-decoded-master
Sep 29, 2026
Merged

nichinichisou0609 merged 1 commit into
mainfrom
feat/music-data-decoded-master

Conversation

@nichinichisou0609

Copy link
Copy Markdown
Collaborator

Summary

nnnotes music-data --decoded-master builds the music data file from master data decoded elsewhere: the directory the other commands already read as decoded master data ([paths] master, --master, [servers.<region>] master), with one <Table>.json per table plus the MasterManifest.json of the .bin files they were decoded from. No master key is needed.

This is the entry the StarMoe CI needs: its story site workflow already reads moenotes-masterdata-sync's decoded snapshots (metadata.bdon.moe, index.json with every file's SHA-256, MasterManifest.json included) as NNNOTES_PATHS_MASTER, and the music data workflow can do the same instead of adding the master key and IV as repository secrets.

  • deckdata.decoded_master(dir): a MasterSource with decoded=True (a new dataclass field, default False); read_master reads <Table>.json for it, skips the decryption, and takes each table's SHA-256 from the manifest. key may be None there.
  • Provenance: master.source stays api (the region's files), master.version and master.tables.<T>.sha256 are what the manifest lists, the same values --master-files records after checking the files. The output is byte for byte the --master-files file of the same files (tested). The schema enum is unchanged; only two descriptions say it.
  • Errors as before, now also for a decoded directory: no manifest, a table the manifest does not list, a table without a SHA-256 in the manifest, a missing, unreadable or row-less <Table>.json.
  • musicdata.py is not touched: export passes the source and the key through.
  • Docs: docs/music-data.md, docs/commands.md, docs/configuration.md, both READMEs, the schema's two descriptions.

Test plan

  • python -m pytest: 1127 passed, 6 skipped (the built nnnotes._deck test, goldens of other library versions, freetype), on a worktree without the extension module; the new tests use the stand-in deck model.
    • test_decoded_master: the decoded tables of a synthetic download directory (master decode) with its manifest give the same rows and SHA-256 as master_files with the key, and the same deck input bytes.
    • test_decoded_master_checks: the error cases above.
    • test_command_decoded_master: --decoded-master without [paths] master is a usage error naming paths.master; with --master DIR and no master key set it writes the file, byte-identical to --master-files with the key.
  • python -m pyflakes src tests: clean.
  • Real data, light: the 65 tables the file reads from metadata.bdon.moe's TW snapshot 0b21c9f4... (SHA-256 checked against index.json) through decoded_master / read_master give master.version and all 65 master.tables SHA-256 equal to the provenance of the --master-files run of the same version, and the deck input tables build (60 tables, 8854 rows). No full music data run was made on this machine.

Generated with Claude Code

…ed-master)

`nnnotes music-data --decoded-master` reads master data decoded elsewhere, the directory the
other commands read (`[paths] master`, `--master`, `[servers.<region>] master`): one
`<Table>.json` per table and the `MasterManifest.json` of the files they were decoded from, as a
master data snapshot published with its manifest carries it. No master key is needed, so a
workflow can build the file from such a snapshot without the key.

The master data version and each table's SHA-256 are the manifest's (the decoded tables cannot
be checked against the files as served); `master.source` stays `api`, and the file is byte for
byte the one `--master-files` writes from the same files. A missing manifest, an unlisted table,
a table without a SHA-256 in the manifest, a missing or unreadable table stop the command as
before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nichinichisou0609
nichinichisou0609 merged commit 12df2a6 into main Sep 29, 2026
6 checks passed
@nichinichisou0609
nichinichisou0609 deleted the feat/music-data-decoded-master branch September 29, 2026 18:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant