Make release aars reproducible - #207
Open
AcideFluorhydrique wants to merge 4 commits into
Open
AcideFluorhydrique wants to merge 4 commits into
AcideFluorhydrique wants to merge 4 commits into
Conversation
main.yml installed gomobile@latest, so rebuilding a tag later picks up whatever golang.org/x/mobile has published since, and the generated binding code can change underneath an unchanged source tree. go.mod already requires a specific golang.org/x/mobile, kept current by the go get in tidy.yml; install that one.
gomobile bind compiles the generated Java bindings into the aar's classes.jar with javac, so the JDK is an input to the aar. main.yml set none up, leaving javac to whatever the runner image defaults to, which changes when GitHub updates the image. Pin Temurin 17.0.20.1+1, the ubuntu-latest default today, so the JDK stays what it currently is. It has to be written as the Adoptium semver 17.0.20+101: setup-java checks preinstalled JDKs first, and the four-part 17.0.20.1 only matches the preinstalled folder by accident of its name, then matches nothing on Adoptium once the image moves on. Checked by running setup-java v6.0.1's version matching against the live Adoptium release list.
gen_assets.sh downloads geoip.dat and geosite.dat from the v2ray-rules-dat releases/latest redirect and geoip-only-cn-private.dat from the tip of the geoip release branch. Both move daily, and nothing records which data a given release actually embedded, so no release can be rebuilt later with the same assets and compared against the published aar. Keep downloading the newest data, so releases and the tidy.yml automation behave exactly as before, but resolve both sources to an exact release tag and commit first, download from those, and write them with the SHA-256 of each file to data/geo-assets.lock. main.yml publishes that, plus the exact Go release setup-go picked, as libv2ray-build.lock beside the aar. The lock is not copied into assets/, so the aar's contents are unchanged. A new `gen_assets.sh pinned <lock>` downloads exactly what a lock names and fails on any checksum mismatch; the README describes rebuilding a release with it. Downloads now use curl -f, so an HTTP error fails the build instead of being saved as a .dat file.
Comparing a rebuild against a release meant downloading the release assets first just to hash them. Print the SHA-256 of libv2ray.aar and the sources jar at the end of the build, next to the build lock, so both the inputs and the outputs of a run can be read straight from its log.
Author
|
Companion PR: 2dust/v2rayNG#6219 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
v2rayNG's release APKs embed the
libv2ray.aarpublished here. At the moment a published aar cannot be rebuilt from its tag with the same bytes, because some inputs float and are not recorded:@latest, so the binding generator can change after a tag is cut.gomobile bindcompiles the Java bindings intoclasses.jarwithjavac, andmain.ymlsets no Java up, so it uses whatever JDK the runner image defaults to.gen_assets.shdownloads the newest files, and nothing records which ones a release embedded. (The exact Go patch release, picked bygo-version-file, is not recorded either.)A reproducible build lets anyone confirm that a published binary really comes from the published source, and it is what IzzyOnDroid and F-Droid's reproducible-builds verification needs.
What changes
golang.org/x/mobileversiongo.modalready requires (whichtidy.ymlkeeps current) instead of@latest.ubuntu-latestuses by default today. The JDK stays the same; it just stops changing with the runner image. (Written as17.0.20+101, the formsetup-javamatches against Adoptium.)tidy.ymlautomation behave exactly as before. The sources are first resolved to an exact release tag and commit, then downloaded from those, and recorded with their SHA-256 inlibv2ray-build.lock, which is published next to the aar together with the exact Go version.bash gen_assets.sh pinned libv2ray-build.lockdownloads exactly that data again and fails on any mismatch. The README describes how to rebuild a release.curl -f, so an HTTP error fails the build instead of being saved as a.datfile.No new dependencies, no manual step in the release process, and the lock file is not copied into
assets/, so the aar contents are unchanged.Validation
main.ymlfrom this branch, run three times in a fork on separate runners (attempt 1, 2, 3). Attempt 1 resolved v2ray-rules-dat202609092346; Loyalsoldier then published202609102337, which attempts 2 and 3 both resolved. Attempts 2 and 3 produced identical locks and an identical aar (f4d3ddd50068…); attempt 1's aar differs, and its lock shows exactly why.libv2ray-sources.jarwas identical in all three.gen_assets.shwas also exercised locally indownloadmode, inpinnedmode from a published lock, with a tampered checksum (aborts and leaves the lock untouched), with no lock file, and with an unknown action.67bff2374963…with the runner's default JDK, and the identical67bff2374963…after switching to the pinned Temurin 17.0.20.1+1, on a newer runner image: 34494546647.A companion PR to v2rayNG, linked below once it is open, pins the NDK and JDK there and makes sure the downloaded aar matches the submodule.