Repository navigation
fix(music-data): publish verified gzip snapshots and serialize song pages - #10
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The TW export could build and pass every gate but could not publish because the publisher used a nonexistent Bucket.keys API. JSON was also uploaded uncompressed. The publisher now uses exact object HEADs and serves deterministic gzip with application/json, Content-Encoding, and separate encoded/decoded SHA and size metadata. Existing archives are verified before reuse; same-size corruption is repaired before pointers advance.
Every runtime lives under replay// with relative resources confined to that directory. Publication verifies payloads in parallel before serial manifest/main/build-marker barriers. The reencode_only path verifies the current source, marker and all 13 same-source Snap tables, probes the actual store/CDN, and re-encodes JSON without Rust, new scoring or JS/WASM writes. A failed source/marker freshness check stops it. Prebuilt ingestion verifies authentic source bytes for canonical Linux or explicitly observed CRLF checkouts and binds the producer to the publishing repository.
Dedicated songs page writers share one global lock before prebuilt's region lock; story publication excludes their closure. UI-only validation freezes the current marker and downloads its immutable archive/manifest, decompresses before all existing SHA/size checks, validates deck/13-label/model/engine identities, then checks the marker again. JP authentication preflight runs before Rust/WASM with the actual APK client version and mandatory exact source matching. Validated outputs and sanitized reports survive later publication failures.
Real publication exposed 1,391 signed-SDK ValueErrors across 348 objects in run 36860490585; four retries per object exhausted the 30-minute limit. The verified public HTTP response is now checked first against the identical encoding/type/metadata/encoded-SHA/decoded-SHA contract, with strict SDK fallback retained. No assumption is made about which SDK header caused those errors.
Validation: 276 Python tests passed, 5 optional player/sample cases skipped; pyflakes, actionlint and CI pass. A genuine CNB export used 64 workers for 86 songs / 344 charts in 328.47 seconds (59.32 cores mean, 63.61 peak), unchanged before/after source, zero unmet targets, all 14 original gates passing without warnings. The ZIP was independently SHA-verified and freshly unpacked/verified against a clean producer checkout. The production retry 36863987470 succeeded in about six minutes. Independent public verification passed all 344 chart hashes, 13 table hashes, source/model/engine identities, actual WASM transport smoke and unchanged final marker. Seven critical JSON responses additionally passed raw stored-byte/decoded-byte SHA and metadata checks. Main data is 13,188,423 decoded bytes / 1,545,841 encoded bytes (88.3% reduction); the corrected solo SS thresholds are 7,577,522 for 青春コンプレックス and 7,778,519 for Ave Mujica.
The live reencode_only production run 36865051440 succeeded: all 351 JSON objects total 39,445,574 decoded bytes / 5,980,654 encoded bytes. Independent before/after verification found all 352 unique referenced resources, seven critical encoded JSON identities, JS/WASM, the build marker and actual replay results unchanged. JP cloud Version preflight still returns PERMISSION_DENIED with client 1.0.4; a local read using the same actual client succeeded. Authentication is not bypassed and latest JP native certification/publication is not claimed.
Targets dev; existing dev-to-main promotion is #9. Pipeline merge is authorized after these actual checks. Separate UI work remains Draft.