Skip to content

chore: untrack generated Rust and iOS outputs, generate on demand - #551

Merged
valentinfernandez1 merged 10 commits into
mainfrom
untrack-generated-files
Sep 7, 2026
Merged

chore: untrack generated Rust and iOS outputs, generate on demand#551
valentinfernandez1 merged 10 commits into
mainfrom
untrack-generated-files

Conversation

@pgherveou

@pgherveou pgherveou commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

Generated outputs are not tracked. The Rust dispatcher, wire table and wasm bridge, the UniFFI Swift bindings, the FFI headers and modulemaps, and the lockdown container bundle are git-ignored build outputs produced from the Rust crates on demand.

Producing the outputs

scripts/codegen.sh emits the Rust outputs, including the mod.rs that truapi-server declares with pub mod generated;. make setup runs it, make codegen regenerates on its own. Every Make target that compiles truapi-server depends on check-generated, which names the missing file and points at make codegen instead of letting the failure surface as error[E0583]: file not found for module 'generated'.

In CI the Codegen job produces them once and uploads the codegen-output artifact; every job that compiles truapi-server depends on that job and downloads it. ci-android, release-android and release-ios are separate workflows with no such job, so they run scripts/codegen.sh in place with TRUAPI_SKIP_PACKAGE_BUILD=1.

The Codegen job also gates that nothing generated is committed, with git ls-files -ci --exclude-standard over the whole repo.

How iOS consumers get the sources

SwiftPM resolves a package's source targets from the git checkout and has no way to fetch them from a release asset: binaryTarget accepts only .xcframework and .artifactbundle. So an iOS release is two artifacts.

artifact carries consumer sees it as
@parity/ios-host@<version> release asset truapi_server.xcframework.zip publishedBinaryURL + checksum
plain semver tag <version> generated bindings, FFI headers, container bundle, manifest pointing at the asset the thing they pin

Consumers pin the semver tag:

.package(url: "https://github.com/paritytech/host-rust-core", exact: "0.14.0")

ios/truapi-host/scripts/tag-release.sh builds that commit. It reads the required paths out of Package.swift rather than repeating them, so the file set cannot drift from what SwiftPM looks for, and it refuses to tag an xcframework into git. When the tag already exists recording the same tree — generated sources included, not just the manifest — it exits successfully, so a rerun of the release pushes the existing tag as a no-op; a tag recording anything different is an error that names the paths which differ. Deciding that on the manifest alone would let bindings regenerated at the same version pass as a no-op and leave consumers pinned to the old ones. Since those sources are git-ignored and git will not diff them until they are staged, the comparison stages them into a scratch index seeded from the tag, which leaves the caller's own index untouched.

release-ios.yml publishes the asset and the tag. release.yml calls it for a release: @parity/ios-host <version> subject, and it is also dispatchable on its own, matching release-android.yml. Dispatching it from a branch with a version like 0.14.0-beta.1 cuts an asset and a tag an app can pin, which is how an unmerged host change gets tested in the app. A dispatched run passes no manifest_branch and touches no branch.

When called from the release flow, the job opens a pull request on release/ios-host-<version> that points the release branch's Package.swift at the new asset, and dispatches CI for it explicitly because events created with GITHUB_TOKEN do not start another workflow.

The job clones the tag and compiles it against the published asset before pushing, so a tag that cannot be resolved is never published.

What CI proves

iOS package (swift compile) builds the tag rather than the working tree: it runs tag-release.sh, clones the result, diffs the cloned file set against the working tree under every declared path, and compiles the package and its test target from that clean clone. Compiling in place would pass on generated files the tag does not carry, which is the exact failure being guarded.

iOS bindings (uniffi) regenerates the Swift and provider bindings. With nothing committed to diff against, what it gates is that bindgen still produces a binding for every UniFFI-exposed type.

ios-swift's path filter names every crate the bindings are generated from (truapi, truapi-platform, truapi-server, truapi-provider) plus js/container/, because a protocol change leaves no ios/ diff to key on.

Notes

  • tag-release.sh makes the release commit with -c commit.gpgsign=false. CI configures no signing, but the manual fallback runs on developer machines where it is often on globally, and a release commit's provenance is the tag rather than a signature.
  • Package.swift declares TrUAPIProvider as a source target, so its bindings have to exist even for a consumer that only imports TrUAPIHost. ios/truapi-provider/scripts/sync-bindings.sh produces them without Xcode or the iOS targets, and rebuild.sh delegates to it.
  • .gitignore lists the three generated Swift filenames rather than truapi*.swift. Under core.ignorecase, which git sets by default on macOS, the glob also matches the hand-written TrUAPIHost.swift.
  • The mod.rs emitted alongside dispatcher.rs and wire_table.rs has a golden fixture. It is the file that makes truapi-server parse, so losing it fails a test rather than the next build.
  • tag-release.sh and the manifest step both assert that Package.swift reads TRUAPI_USE_LOCAL_BINARY from the environment, so a published manifest can never pin the local binary.

Verifying the release path

GitHub only exposes workflow_dispatch for workflows present on the default branch, so release-ios.yml cannot be dispatched from this branch. The tag's content and buildability are covered on the PR by the iOS package (swift compile) job above. Cutting a pre-release to exercise the publish half (asset upload, tag push, resolution from GitHub by the app) needs this merged first.

pgherveou and others added 8 commits September 2, 2026 12:43
…pi-server

The untracking commits fixed only ci.yml's rust job. truapi-server declares
`pub mod generated;` unconditionally, so every other job that builds it fails
at parse time: ios-bindings, ios-swift, ci-android, release-android and
release.yml's publish-ios.

Also:
- ios-swift never ran sync-bindings.sh or the js/container build, so the
  package had no Swift sources and no .copy resource to resolve.
- the ios-changes filter keyed on an ios/ diff that generated bindings no
  longer produce; it now names the crates the bindings come from.
- release.yml's post-rebuild `git diff --exit-code` could no longer fail.
- .gitignore's truapi*.swift glob matched the hand-written TrUAPIHost.swift
  under core.ignorecase, which git sets by default on macOS.
- check-generated is a prerequisite of every target that compiles the crate,
  names the missing file, and covers mod.rs and wire_table.rs.
- sync-bindings.sh --check and provider-swift-check compared against files
  that are no longer committed.
SwiftPM resolves source targets from the git checkout, so a branch that
ignores the generated bindings cannot be consumed. A release is now two
artifacts: the xcframework as a release asset, and a plain semver tag whose
commit carries the generated sources plus a manifest pointing at that asset.
Consumers keep the same URL and the same exact() pin.

publish-ios moves into release-ios.yml so it is callable from release.yml and
dispatchable for a pre-release, matching release-android.yml. The tag is
verified by cloning and building it before the push.

tag-release.sh reads the required paths out of Package.swift so the file set
cannot drift from what SwiftPM looks for.

Also fixes the manifest guard: 9b708fb replaced 'let useLocalBinary = false'
with the ProcessInfo form, but release.yml still grepped for the literal, so
the follow-up commit step could never pass.
Also compiles the ios-swift CI gate from a clean clone of the tag rather than
in place, so a generated file the tag does not carry fails the PR instead of
the next consumer.
@valentinfernandez1
valentinfernandez1 marked this pull request as ready for review September 2, 2026 15:46
@valentinfernandez1
valentinfernandez1 requested a review from a team September 2, 2026 15:46
Comment thread ios/truapi-host/scripts/tag-release.sh Outdated
Comment thread ios/truapi-host/scripts/tag-release.sh Outdated

@TarikGul TarikGul left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just two small nits

valentinfernandez1 and others added 2 commits September 7, 2026 11:34
…ommit

The rerun guard only diffed Package.swift, so cutting the tag, regenerating
bindings and rerunning the release at the same version exited zero and left
consumers pinned to the old generated sources. It now diffs everything the tag
carries. The generated sources are git-ignored, so they are staged into a
scratch GIT_INDEX_FILE seeded from the tag: git can diff them there, the
caller's own index is untouched on the no-op path, and any difference is
reported by path and fails. The manifest and existence guards move above the
check, which needs the declared paths.

The release commit is made with -c commit.gpgsign=false. CI has no signing
configured, but the manual fallback is the case that matters and it died on
machines with signing on globally.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three conflicts, resolved as follows.

`truapi-codegen`'s `generate()` gained a `schema_hash` parameter on main and
the `mod.rs` emission here; the two are orthogonal, so it now takes the
parameter and writes all three files. `mod.rs` and `TRUAPI_WIRE_SCHEMA_HASH`
both appear in the regenerated output.

`ios/truapi-host/Sources/TrUAPIHost/truapi_server.swift` and
`rust/crates/truapi-server/src/generated/wire_table.rs` were regenerated on
main and are untracked here. They stay deleted: both are matched by .gitignore
and their content derives from source the merge already carries.
@valentinfernandez1
valentinfernandez1 added this pull request to the merge queue Sep 7, 2026
Merged via the queue into main with commit daca158 Sep 7, 2026
19 checks passed
@valentinfernandez1
valentinfernandez1 deleted the untrack-generated-files branch September 7, 2026 15:26
filvecchiato added a commit that referenced this pull request Sep 7, 2026
Resolutions:

- The four generated outputs main untracked in #551 (`generated/dispatcher.rs`,
  `generated/wire_table.rs`, `wasm/generated_bridge.rs` and the iOS
  `truapi_platform.swift`) conflicted modify/delete, this branch having edited
  them. Took main's deletion: all four are gitignored now and `make codegen`
  reproduces them, contacts entries included.

- `worker-provider.test.ts` asserts the worker init message, which both sides
  extended — this branch with the `contacts` capability, main with
  `debuggerUrl`. Kept both; the merged producer in
  `create-worker-host-runtime.ts` emits both fields.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants