Skip to content

Add native gem precompilation support#4080

Open
flavorjones wants to merge 3 commits intoruby:mainfrom
flavorjones:native-gem-precompilation
Open

Add native gem precompilation support#4080
flavorjones wants to merge 3 commits intoruby:mainfrom
flavorjones:native-gem-precompilation

Conversation

@flavorjones
Copy link
Copy Markdown
Contributor

@flavorjones flavorjones commented Apr 16, 2026

WIP: Looking for CI feedback on this approach.

Closes #4026

Package precompiled native gems for Ruby 3.3, 3.4, and 4.0 across 10 platforms: aarch64-linux-gnu, aarch64-linux-musl, aarch64-mingw-ucrt, arm-linux-gnu, arm-linux-musl, arm64-darwin, x64-mingw-ucrt, x86_64-darwin, x86_64-linux-gnu, and x86_64-linux-musl.

  • Add Ruby-version-aware extension loader in lib/prism.rb with GLIBC error message for musl/glibc mismatches
  • Add cross-compilation support to Rakefile using rake-compiler-dock
  • Add a reusable build-gems.yml workflow called by both the CI workflow (native-gem-precompilation.yml) and the publish workflow, building source and native gems across all platforms and Ruby versions
  • Add scripts for building, verifying, and testing gems

Update publish-gem.yml to handle multi-gem publishing: rubygems/release-gem only supports a single gem, so this uses rubygems/configure-rubygems-credentials and loops gem push over each built .gem. Add a workflow_dispatch trigger alongside the existing tag-push trigger, and create/update a GitHub release for the tag with all gems plus a CHECKSUMS.txt manifest.

Move testing of the "ruby" platform gem (compile-from-source) out of main.yml into test-gems.yml so we're testing the gem built by the workflow tooling.

Pin all GitHub Actions to commit SHAs and apply zizmor-style hardening across the new and modified workflows.


This approach uses rake-compiler-dock to build a broad set of platform gems, including GNU and Musl linux variants.

I also have an experimental branch that uses https://github.com/shopify/cibuildgem, however I think for a gem like prism it may be prudent to choose the more mature rake-compiler-dock toolchain, until cibuildgem is at parity.

@kddnewton
Copy link
Copy Markdown
Collaborator

Only minor comment (that can be left until after merge) is that I'd love these bin scripts to be in rake. Otherwise the approach looks reasonable.

It seems like right now the GitHub action triggers on a bunch of stuff, but presumably after you get CI passing we'll switch it to just tags right?

@flavorjones
Copy link
Copy Markdown
Contributor Author

Only minor comment (that can be left until after merge) is that I'd love these bin scripts to be in rake. Otherwise the approach looks reasonable.

I don't think it's obvious that they should be rake tasks, but I do want to try to follow the project conventions. If that's what you want, we can try to make it work.

The shell scripts are intentionally operating outside of bundler, and test-gem-file-contents is a self-contained minitest suite. build-gems will be replaced with a publishing workflow.

It seems like right now the GitHub action triggers on a bunch of stuff, but presumably after you get CI passing we'll switch it to just tags right?

I mean, it's obviously up to you when you want to run these tests, but my implementation decision to run these tests on every pull request is intentional. The new workflow isn't publishing, it's testing:

  • ensure that the gem can be precompiled on all target platforms
  • run the test suite against the installed precompiled gem to make sure it's functionally correct and does not generate runtime linker errors

and I think these are valuable tests for any PR that modifies the source code or the build process.

@flavorjones flavorjones force-pushed the native-gem-precompilation branch from 3108b0e to aa1bf2d Compare April 16, 2026 17:51
@flavorjones

This comment was marked as outdated.

@flavorjones flavorjones force-pushed the native-gem-precompilation branch 2 times, most recently from 6d2af66 to 74cde70 Compare April 20, 2026 19:47
@flavorjones flavorjones force-pushed the native-gem-precompilation branch from 74cde70 to bf9cd77 Compare May 7, 2026 18:01
@flavorjones
Copy link
Copy Markdown
Contributor Author

Force-pushed some changes:

  • extracted a reusable workflow that's used for both testing native gems and publishing native gems
  • did some security hardening (pinning actions, extracting matrix variables into env vars to prevent injection, etc.)

@flavorjones flavorjones force-pushed the native-gem-precompilation branch from 9c5f12c to 4785ba2 Compare May 7, 2026 20:50
because in git worktrees, it's a file, not a directory.
sparklemotion/nokogiri#3530 is merged fixing the loader config on
windows-11-arm, but has not yet landed in a release.

Pin to `main` on the affected platform only, to unblock CI until a
fixed nokogiri release ships.
Package precompiled native gems for Ruby 3.3, 3.4, and 4.0 across 10
platforms: aarch64-linux-gnu, aarch64-linux-musl, aarch64-mingw-ucrt,
arm-linux-gnu, arm-linux-musl, arm64-darwin, x64-mingw-ucrt,
x86_64-darwin, x86_64-linux-gnu, and x86_64-linux-musl.

- Add Ruby-version-aware extension loader in lib/prism.rb with GLIBC
  error message for musl/glibc mismatches
- Add cross-compilation support to Rakefile using rake-compiler-dock
- Add a reusable `build-gems.yml` workflow called by both the CI
  workflow (`native-gem-precompilation.yml`) and the publish workflow,
  building source and native gems across all platforms and Ruby versions
- Add scripts for building, verifying, and testing gems

Update `publish-gem.yml` to handle multi-gem publishing:
`rubygems/release-gem` only supports a single gem, so this uses
`rubygems/configure-rubygems-credentials` and loops `gem push` over
each built `.gem`. Add a `workflow_dispatch` trigger alongside the
existing tag-push trigger, and create/update a GitHub release for the
tag with all gems plus a `CHECKSUMS.txt` manifest.

Move testing of the "ruby" platform gem (compile-from-source) out of
`main.yml` into `test-gems.yml` so we're testing the gem built by the
workflow tooling.

Pin all GitHub Actions to commit SHAs and apply zizmor-style hardening
across the new and modified workflows.
@flavorjones flavorjones force-pushed the native-gem-precompilation branch 2 times, most recently from 02d6e4b to 15d8e4d Compare May 7, 2026 21:24
@flavorjones flavorjones marked this pull request as ready for review May 7, 2026 21:35
@flavorjones
Copy link
Copy Markdown
Contributor Author

@kddnewton This is ready for your review. LMK if you have any questions, happy to jump on slack or whatever! ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support pre-compiled gems in our rubygems publishing workflow

2 participants