Skip to content

fix(music-data): publish verified gzip snapshots and serialize song pages - #10

Merged
nichinichisou0609 merged 9 commits into
devfrom
fix/jp-music-snapshot-publication
Oct 1, 2026
Merged

nichinichisou0609 merged 9 commits into
devfrom
fix/jp-music-snapshot-publication

Conversation

@nichinichisou0609

@nichinichisou0609 nichinichisou0609 commented Oct 1, 2026 •

Copy link
Copy Markdown

The TW export could build and pass every gate but could not publish because the publisher used a nonexistent Bucket.keys API. JSON was also uploaded uncompressed. The publisher now uses exact object HEADs and serves deterministic gzip with application/json, Content-Encoding, and separate encoded/decoded SHA and size metadata. Existing archives are verified before reuse; same-size corruption is repaired before pointers advance.

Every runtime lives under replay// with relative resources confined to that directory. Publication verifies payloads in parallel before serial manifest/main/build-marker barriers. The reencode_only path verifies the current source, marker and all 13 same-source Snap tables, probes the actual store/CDN, and re-encodes JSON without Rust, new scoring or JS/WASM writes. A failed source/marker freshness check stops it. Prebuilt ingestion verifies authentic source bytes for canonical Linux or explicitly observed CRLF checkouts and binds the producer to the publishing repository.

Dedicated songs page writers share one global lock before prebuilt's region lock; story publication excludes their closure. UI-only validation freezes the current marker and downloads its immutable archive/manifest, decompresses before all existing SHA/size checks, validates deck/13-label/model/engine identities, then checks the marker again. JP authentication preflight runs before Rust/WASM with the actual APK client version and mandatory exact source matching. Validated outputs and sanitized reports survive later publication failures.

Real publication exposed 1,391 signed-SDK ValueErrors across 348 objects in run 36860490585; four retries per object exhausted the 30-minute limit. The verified public HTTP response is now checked first against the identical encoding/type/metadata/encoded-SHA/decoded-SHA contract, with strict SDK fallback retained. No assumption is made about which SDK header caused those errors.

Validation: 276 Python tests passed, 5 optional player/sample cases skipped; pyflakes, actionlint and CI pass. A genuine CNB export used 64 workers for 86 songs / 344 charts in 328.47 seconds (59.32 cores mean, 63.61 peak), unchanged before/after source, zero unmet targets, all 14 original gates passing without warnings. The ZIP was independently SHA-verified and freshly unpacked/verified against a clean producer checkout. The production retry 36863987470 succeeded in about six minutes. Independent public verification passed all 344 chart hashes, 13 table hashes, source/model/engine identities, actual WASM transport smoke and unchanged final marker. Seven critical JSON responses additionally passed raw stored-byte/decoded-byte SHA and metadata checks. Main data is 13,188,423 decoded bytes / 1,545,841 encoded bytes (88.3% reduction); the corrected solo SS thresholds are 7,577,522 for 青春コンプレックス and 7,778,519 for Ave Mujica.

The live reencode_only production run 36865051440 succeeded: all 351 JSON objects total 39,445,574 decoded bytes / 5,980,654 encoded bytes. Independent before/after verification found all 352 unique referenced resources, seven critical encoded JSON identities, JS/WASM, the build marker and actual replay results unchanged. JP cloud Version preflight still returns PERMISSION_DENIED with client 1.0.4; a local read using the same actual client succeeded. Authentication is not bypassed and latest JP native certification/publication is not claimed.

Targets dev; existing dev-to-main promotion is #9. Pipeline merge is authorized after these actual checks. Separate UI work remains Draft.

@nichinichisou0609 nichinichisou0609 changed the title fix(music-data): serve verified gzip JSON and preflight JP authentication fix(music-data): publish verified gzip snapshots and serialize song pages Oct 1, 2026
@nichinichisou0609
nichinichisou0609 marked this pull request as ready for review October 1, 2026 13:21
@nichinichisou0609
nichinichisou0609 merged commit c97e293 into dev Oct 1, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant