feat(music-data): read decoded master data with its manifest (--decoded-master) - #6
Merged
Merged
Conversation
…ed-master) `nnnotes music-data --decoded-master` reads master data decoded elsewhere, the directory the other commands read (`[paths] master`, `--master`, `[servers.<region>] master`): one `<Table>.json` per table and the `MasterManifest.json` of the files they were decoded from, as a master data snapshot published with its manifest carries it. No master key is needed, so a workflow can build the file from such a snapshot without the key. The master data version and each table's SHA-256 are the manifest's (the decoded tables cannot be checked against the files as served); `master.source` stays `api`, and the file is byte for byte the one `--master-files` writes from the same files. A missing manifest, an unlisted table, a table without a SHA-256 in the manifest, a missing or unreadable table stop the command as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
nnnotes music-data --decoded-masterbuilds the music data file from master data decoded elsewhere: the directory the other commands already read as decoded master data ([paths] master,--master,[servers.<region>] master), with one<Table>.jsonper table plus theMasterManifest.jsonof the.binfiles they were decoded from. No master key is needed.This is the entry the StarMoe CI needs: its story site workflow already reads moenotes-masterdata-sync's decoded snapshots (metadata.bdon.moe,
index.jsonwith every file's SHA-256,MasterManifest.jsonincluded) asNNNOTES_PATHS_MASTER, and the music data workflow can do the same instead of adding the master key and IV as repository secrets.deckdata.decoded_master(dir): aMasterSourcewithdecoded=True(a new dataclass field, default False);read_masterreads<Table>.jsonfor it, skips the decryption, and takes each table's SHA-256 from the manifest.keymay be None there.master.sourcestaysapi(the region's files),master.versionandmaster.tables.<T>.sha256are what the manifest lists, the same values--master-filesrecords after checking the files. The output is byte for byte the--master-filesfile of the same files (tested). The schema enum is unchanged; only two descriptions say it.<Table>.json.musicdata.pyis not touched:exportpasses the source and the key through.docs/music-data.md,docs/commands.md,docs/configuration.md, both READMEs, the schema's two descriptions.Test plan
python -m pytest: 1127 passed, 6 skipped (the builtnnnotes._decktest, goldens of other library versions, freetype), on a worktree without the extension module; the new tests use the stand-in deck model.test_decoded_master: the decoded tables of a synthetic download directory (master decode) with its manifest give the same rows and SHA-256 asmaster_fileswith the key, and the same deck input bytes.test_decoded_master_checks: the error cases above.test_command_decoded_master:--decoded-masterwithout[paths] masteris a usage error namingpaths.master; with--master DIRand no master key set it writes the file, byte-identical to--master-fileswith the key.python -m pyflakes src tests: clean.0b21c9f4...(SHA-256 checked againstindex.json) throughdecoded_master/read_mastergivemaster.versionand all 65master.tablesSHA-256 equal to the provenance of the--master-filesrun of the same version, and the deck input tables build (60 tables, 8854 rows). No full music data run was made on this machine.Generated with Claude Code