ci: build, check and publish the music data file on new master data

[nichinichisou0609]

Summary

A workflow that keeps the chart data page's music-data.json current: on new master data it runs nnnotes music-data on moenotes-masterdata-sync's decoded master data, checks the file with quality gates and publishes it into the story site's bucket under music-data/. It follows story-site.yml step for step. Docs: .github/MUSIC_DATA.md.

Published layout (https://storage.bdon.moe/moenotes/music-data/): music-data.json (no-cache), jackets/<jacket>.webp, archive/<master version>/<sha256>.json (immutable, every published file), build.json (the build marker: SHA-256, counts, inputs, gate results, run). Nothing is deleted.

Files (all under .github/, so the fork still syncs with upstream as before):

• workflows/music-data.yml: jobs plan → build
• scripts/music_data.py: plan, master, build, check (the gates), publish
• scripts/music_data_smoke.mjs: the page smoke test (ournotes-player examples/songs/catalog.js + ranking.js in Node.js)
• scripts/songs_page.sh: sparse checkout of examples/songs at MUSIC_DATA_PLAYER_REF (nothing built)
• scripts/test_music_data.py: gate self-test (runs in every build; locally in ~2 s)
• MUSIC_DATA.md

What is reused from the story site

story-site	music-data
repository_dispatch: masterdata-updated + daily schedule + workflow_dispatch	same (schedule 03:41 UTC); inputs force, dry_run
concurrency: story-site, no cancel	concurrency: music-data, no cancel
plan job (cheap) → build job only when there is work	plan compares master version / resource version / client version / deck commit (rust/Cargo.lock) / last nnnotes commit of src rust pyproject.toml / recipe with the published build.json; nothing changed → the run ends
vars.X || 'default' env block	same; the bucket is the story site's STORY_S3_* variables, plus MUSIC_DATA_*
MASTERDATA_BASE_URL decoded master, SHA-256 against index.json	same (story_site.master_index / fetch_table), MasterManifest.json included; no master key
playfetch APK (apk.sh)	same script: provenance.client
STORY_S3_* + boto3 (story_site.Bucket)	same class, key prefix MUSIC_DATA_S3_PREFIX
assets first, indexes last	jackets and archive copy → music-data.json → build.json, each read back by SHA-256
uv pip install --system -e . from this checkout	same, .[test] (jsonschema, pytest); Swatinem/rust-cache@v2 as upstream CI (the install builds nnnotes._deck)
published files read over plain HTTP	same (build.json, the published music-data.json as the gates' baseline)

Not in the workflow: the Cloudflare Pages preview (no Cloudflare secret).

Gates (any failure: nothing is published)

nnnotes' own cross-checks (the command writes no file otherwise); JSON Schema (docs/schema/music-data.schema.json); provenance against the snapshot and its manifest (version, every table's SHA-256, every decoded table read, deck commit = rust/Cargo.lock, exporter = installed nnnotes, APK version); songs and charts not fewer than published; deck statistics complete; play scenario fields complete (offSeeds exactly one, rankBonusPercents five ints, scorePerfect / rangeWeights / rankCheck on every seed, rangeScorePerfect on every seed range; null rangeWeights or kinds are warnings); no NaN / infinity (outside master rows); references and texts; BGM lengths; size 0.8–2× the published file; the page smoke test; SHA-256 read-back after upload.

Checked locally (no build was run here): the self-test passes (60 tests, 3.8 s with the real files below); check on a real TW file with the scenario fields (master 0b21c9f4…, deck d4abb52) against the real decoded snapshot passes every gate; the earlier file without the scenario fields is stopped (scenarios: 2736 failures, page: 3, provenance: deck commit ≠ pin).

Settings

Secrets (existing, the story site's; none new): NNNOTES_BUNDLE_KEY, NNNOTES_BUNDLE_NONCE_SEED, NNNOTES_SERVERS_TW_CDN, PLAYFETCH_CREDENTIALS, STORY_S3_ACCESS_KEY, STORY_S3_SECRET_KEY.

Variables: MUSIC_DATA_PLAYER_REF (required, no default yet), optional MUSIC_DATA_PLAYER_REPOSITORY (empty-sekai/ournotes-player), MUSIC_DATA_S3_PREFIX (music-data), MUSIC_DATA_MASTERDATA_REGION (hk-tw-mo); shared with the story site: STORY_S3_ENDPOINT, STORY_S3_BUCKET, MASTERDATA_BASE_URL, PLAYFETCH_VERSION, STORY_APK_PACKAGE.

Publishing switch

Nothing is uploaded unless the repository variable MUSIC_DATA_PUBLISH is true (unset: off). Off, every run (schedule, masterdata-updated, workflow_dispatch with or without dry_run) is a dry run: it builds, runs every gate and the smoke test and lists what it would upload; the publish step gets no bucket key and music_data.py publish refuses to upload by itself. The variable stays unset for now.

Preconditions

1. Upstream play scenarios feat(music-data): play scenarios: Gekisou off, every rank, the Perfect play MetaSekaiLab/nnnotes#5 (a03591e, deck feat: chart-stats scenarios: Gekisou off, every rank, the Perfect play empty-sekai/ournotes-deck#1 d4abb52): merged.
2. Upstream --decoded-master feat(music-data): read decoded master data with its manifest (--decoded-master) MetaSekaiLab/nnnotes#6 (12df2a6): merged.
3. This fork synced with upstream 12df2a6: chore: sync with upstream nnnotes main (12df2a6) #2 (merge commit fdb7073); Deck model workflow disabled in this fork.
4. MUSIC_DATA_PLAYER_REF = 0107ae769e8f8ecb938af4084c4487abb6be638e (ournotes-player main): set.
5. MUSIC_DATA_PUBLISH: not set (dry runs only).

Generated with Claude Code