Install arbitrary bundler versions (heroku#1695)

* Inject bundler version into BundlerWrapper

* Install injected Bundler Version

Previously the BundlerWrapper reads the Gemfile.lock to extract BUNDLED WITH and then resolves this to a specific **known** good version of bundler.

Now, the version is still parsed from the Gemfile.lock, but this is passed in. Instead of resolving to a specific bundler version, we use the version specified in the customer's Gemfile.lock.

Keep in mind that the version of bundler used by the platform will use the largest version present on the system, which could also be the default bundler version that ships with Ruby.

* Add warning

* Remove "escape valve" as this is no longer needed

This feature prevented bundler from re-exec-ing with the version of bundler in the `Gemfile.lock`. Since we're now installing an exact match, this is no longer needed.

* Remove unused code

* Prefer inline private

* Remove unused errors

These errors are not constructed anymore

* Changelog

* Improve lockfile parser error

Previously when this script failed it was difficult to understand

```
     StandardError:
       Command: 'ruby -e require\ \"json\"'
       'require\ \"bundler\"'
       ''
       'path\ \=\ ARGV\[0\]\ or\ raise\ \"First\ argument\ must\ be\ the\ path\ to\ the\ Gemfile.lock\"'
       'specs\ \=\ Bundler::LockfileParser.new\(File.read\(path\)\)'
       '\ \ .specs'
       '\ \ .each_with_object\(\{\}\)\ \{\|spec,\ hash\|\ hash\[spec.name.to_s\]\ \=\ spec.version.to_s\ \}'
       'puts\ specs.to_json'
       ' ''' failed unexpectedly:
     # ./lib/language_pack/shell_helpers.rb:80:in `block in run!'
```

This is from the Gemfile.lock file being `nil`.

Part of the problem is that the script is shell encoded and present in the output. The other problem is that redirecting stderr to `/dev/null` hides the error message "No such file or directory" from the output.

This change pushes the IO failures and input failures outside of the script and uses Open3 and a tempfile to separate out stdout from stderr without losing stderr.

* fixup! Inject bundler version into BundlerWrapper

Author: schneems

CHANGELOG.md

@@ -2,6 +2,13 @@
 
 ## [Unreleased]
 
+- Bundler version installed now directly matches the value in `BUNDLED WITH` from the `Gemfile.lock`
+  Previously, this value was converted to a "known good version." For example:
+
+  `BUNDLED WITH` 2.7.x installs `bundler 2.7.2`
+
+  Now, the exact version from the `Gemfile.lock` is installed instead. Applications without
+  a `BUNDLED WITH` value will receive a default bundler version. (https://github.com/heroku/heroku-buildpack-ruby/pull/1695)
 
 ## [v340] - 2026-01-06
 

changelogs/unreleased/exact-bundler-version.md

@@ -0,0 +1,22 @@
+## Ruby applications now receive exact bundler version
+
+The exact bundler version from the `Gemfile.lock` is now installed with no conversion.
+
+If your application fails after this change, you can manually adjust your `BUNDLED WITH` value in the `Gemfile.lock` to match the previously used exact version:
+
+- `BUNDLED WITH 1.x.` installs `bundler 1.17.3`
+- `BUNDLED WITH 2.0.x to 2.3.x` installs `bundler 2.3.25`
+- `BUNDLED WITH 2.4.x` installs `bundler 2.4.22`
+- `BUNDLED WITH 2.5.x` installs `bundler 2.5.23`
+- `BUNDLED WITH 2.6.x` installs `bundler 2.6.9`
+- `BUNDLED WITH 2.7.x` installs `bundler 2.7.2`
+- `BUNDLED WITH 4.0.x` installs `bundler 4.0.0`
+
+For example, if your application started failing, using bundler 2.6.1, you could adjust it to instead use the 2.6.x series above which would be:
+
+```
+BUNDLED WITH
+   2.6.9
+```
+
+Applications without a `BUNDLED WITH` value will receive a [default bundler version ](https://devcenter.heroku.com/articles/ruby-support-reference#default-bundler-version).

lib/language_pack.rb

@@ -27,7 +27,10 @@ def self.call(app_path:, cache_path:, gemfile_lock:, bundle_default_without:, en
     warn_io = LanguagePack::ShellHelpers::WarnIO.new
     user_env_hash = LanguagePack::ShellHelpers.user_env_hash
     bundler_cache = LanguagePack::BundlerCache.new(cache, stack)
-    bundler_version = LanguagePack::Helpers::BundlerWrapper.detect_bundler_version(contents: gemfile_lock.contents)
+    bundler_version = LanguagePack::Helpers::BundlerWrapper.resolve_bundler_version(
+      warn_io: warn_io,
+      gemfile_lock: gemfile_lock,
+    )
 
     metadata = LanguagePack::Metadata.new(cache_path: cache_path)
     new_app = metadata.empty?
@@ -50,7 +53,7 @@ def self.call(app_path:, cache_path:, gemfile_lock:, bundle_default_without:, en
       io: warn_io
     )
 
-    bundler = Helpers::BundlerWrapper.new(bundler_path: ruby_version.bundler_directory).install
+    bundler = Helpers::BundlerWrapper.new(bundler_path: ruby_version.bundler_directory, bundler_version: bundler_version).install
     default_config_vars = Ruby.default_config_vars(metadata: metadata, ruby_version: ruby_version, bundler: bundler, environment_name: environment_name)
     Ruby.setup_language_pack_environment(
       app_path: app_path.expand_path,

lib/language_pack/helpers/bundler_wrapper.rb

@@ -9,10 +9,10 @@
 #
 # Example:
 #
-#   bundler = LanguagePack::Helpers::BundlerWrapper.new(bundler_path: "vendor/bundle/ruby/3.2.0")
+#   bundler = LanguagePack::Helpers::BundlerWrapper.new(bundler_path: "vendor/bundle/ruby/3.2.0", bundler_version: "2.5.7")
 #   bundler.install
-#   bundler.version                 => "1.15.2"
-#   bundler.dir_name                => "bundler-1.15.2"
+#   bundler.version                 => "2.5.23"
+#   bundler.dir_name                => "bundler-2.5.23"
 #   bundler.has_gem?("railties")    => true
 #   bundler.gem_version("railties") => "5.2.2"
 #   bundler.clean
@@ -22,116 +22,33 @@
 # of an isolated dyno, you must call `BundlerWrapper#clean`. To reset the environment
 # variable:
 #
-#   bundler = LanguagePack::Helpers::BundlerWrapper.new(bundler_path: "vendor/bundle/ruby/3.2.0")
+#   bundler = LanguagePack::Helpers::BundlerWrapper.new(bundler_path: "vendor/bundle/ruby/3.2.0", bundler_version: "2.5.7")
 #   bundler.install
 #   bundler.clean # <========== IMPORTANT =============
 #
 class LanguagePack::Helpers::BundlerWrapper
   include LanguagePack::ShellHelpers
 
-  BLESSED_BUNDLER_VERSIONS = {}
   # Heroku-22's oldest Ruby version is 3.1
-  BLESSED_BUNDLER_VERSIONS["2.3"] = "2.3.25"
-  BLESSED_BUNDLER_VERSIONS["2.4"] = "2.4.22"
-  BLESSED_BUNDLER_VERSIONS["2.5"] = "2.5.23"
-  BLESSED_BUNDLER_VERSIONS["2.6"] = "2.6.9"
-  BLESSED_BUNDLER_VERSIONS["2.7"] = "2.7.2"
-  BLESSED_BUNDLER_VERSIONS["4.0"] = "4.0.0"
-
-  SORTED_KEYS = BLESSED_BUNDLER_VERSIONS.keys.map { |k| Gem::Version.new(k) }.sort
-  BUNDLER_2_SORTED_KEYS = SORTED_KEYS.select { |k| k.segments.first == 2 }
-  BUNDLER_2_SMALLEST = BUNDLER_2_SORTED_KEYS.first.to_s
-  BUNDLER_2_LARGEST = BUNDLER_2_SORTED_KEYS.last.to_s
-
-  BUNDLER_4_SORTED_KEYS = SORTED_KEYS.select { |k| k.segments.first == 4 }
-  BUNDLER_4_SMALLEST = BUNDLER_4_SORTED_KEYS.first.to_s
-  BUNDLER_4_LARGEST = BUNDLER_4_SORTED_KEYS.last.to_s
-
-  DEFAULT_VERSION = BLESSED_BUNDLER_VERSIONS["2.3"]
-
-  # Convert arbitrary `<Major>.<Minor>.x` versions
-  BLESSED_BUNDLER_VERSIONS.default_proc = Proc.new do |hash, key|
-    case Gem::Version.new(key).segments.first
-    when 4
-      hash[BUNDLER_4_LARGEST]
-    when 2
-      if Gem::Version.new(key) > Gem::Version.new(BUNDLER_2_LARGEST)
-        hash[BUNDLER_2_LARGEST]
-      elsif Gem::Version.new(key) < Gem::Version.new(BUNDLER_2_SMALLEST)
-        hash[BUNDLER_2_SMALLEST]
-      else
-        raise UnsupportedBundlerVersion.new(hash, key)
-      end
-    else
-      raise UnsupportedBundlerVersion.new(hash, key)
-    end
-  end
-
-  def self.detect_bundler_version(contents: , bundled_with: contents.match(BUNDLED_WITH_REGEX))
-    if bundled_with
-      major = bundled_with[:major]
-      minor = bundled_with[:minor]
-      version = BLESSED_BUNDLER_VERSIONS["#{major}.#{minor}"]
-      version
-    else
-      DEFAULT_VERSION
-    end
-  end
-
-  BUNDLED_WITH_REGEX = /^BUNDLED WITH$(\r?\n) {2,3}(?<version>(?<major>\d+)\.(?<minor>\d+)\.\d+)/m
-
-  class GemfileParseError < BuildpackError
-    def initialize(error)
-      msg = String.new("There was an error parsing your Gemfile, we cannot continue\n")
-      msg << error
-      super msg
-    end
-  end
-
-  class UnsupportedBundlerVersion < BuildpackError
-    def initialize(version_hash, major_minor)
-      msg = String.new("Your Gemfile.lock indicates you need bundler `#{major_minor}.x`\n")
-      msg << "which is not currently supported. You can deploy with bundler version:\n"
-      version_hash.keys.each do |v|
-        msg << "  - `#{v}.x`\n"
-      end
-      msg << "\nTo use another version of bundler, update your `Gemfile.lock` to point\n"
-      msg << "to a supported version. For example:\n"
-      msg << "\n"
-      msg << "```\n"
-      msg << "BUNDLED WITH\n"
-      msg << "   #{DEFAULT_VERSION}\n"
-      msg << "```\n"
-      super msg
-    end
-  end
+  DEFAULT_VERSION = "2.3.25"
 
   attr_reader :bundler_path
 
   def initialize(
       bundler_path:,
+      bundler_version:,
       gemfile_path: Pathname.new("./Gemfile"),
       report: HerokuBuildReport::GLOBAL
     )
     @report               = report
     @gemfile_path         = gemfile_path
     @gemfile_lock_path    = Pathname.new("#{@gemfile_path}.lock")
 
-    contents = @gemfile_lock_path.read(mode: "rt")
-    bundled_with = contents.match(BUNDLED_WITH_REGEX)
     dot_ruby_version_file = @gemfile_lock_path.join("..").join(".ruby-version")
     @report.capture(
-      "bundler.bundled_with" => bundled_with&.[]("version") || "empty",
-      # We use this bundler class to detect the Requested ruby version from the Gemfile.lock
-      # Rails 8 stopped generating `RUBY VERSION` in the Gemfile.lock and started generating
-      # a `.ruby-version` file. This will observe the formats to help guide implementation
-      # decisions
       "ruby.dot_ruby_version" => dot_ruby_version_file.exist? ? dot_ruby_version_file.read&.strip : nil
     )
-    @version = self.class.detect_bundler_version(
-      contents: contents,
-      bundled_with: bundled_with
-    )
+    @version = bundler_version
     parts = @version.split(".")
     @report.capture(
       "bundler.version_installed" => @version,
@@ -175,28 +92,10 @@ def dir_name
     @dir_name
   end
 
-  def self.platform_to_version(bundle_platform_output)
-    if bundle_platform_output.match(/No ruby version specified/)
-      ""
-    else
-      bundle_platform_output.strip.sub('(', '').sub(')', '').sub(/(p-?\d+)/, ' \1').split.join('-')
-    end
-  end
-
-  def bundler_version_escape_valve!
-    topic("Removing BUNDLED WITH version in the Gemfile.lock")
-    contents = File.read(@gemfile_lock_path, mode: "rt")
-    File.open(@gemfile_lock_path, "w") do |f|
-      f.write contents.sub(/^BUNDLED WITH$(\r?\n) {2,3}(?<major>\d+)\.\d+\.\d+/m, '')
-    end
-  end
-
-  private
-  def fetch_bundler
+  private def fetch_bundler
     return true if Dir.exist?(bundler_path.join("gems", dir_name))
 
     topic("Installing bundler #{@version}")
-    bundler_version_escape_valve!
 
     # Install directory structure (as of Bundler 2.1.4):
     # - cache
@@ -211,7 +110,38 @@ def fetch_bundler
   end
 
   # Runs a Ruby subprocess to parse the Gemfile.lock and return specs as a hash.
-  def specs_from_lockfile
+  private def specs_from_lockfile
     LanguagePack::Helpers::LockfileShellParser.call(lockfile_path: @gemfile_lock_path)
   end
+
+  def self.resolve_bundler_version(gemfile_lock:, warn_io: )
+    version = gemfile_lock.bundler.version
+    if version
+      version
+    else
+      warn_io.warn(<<~WARNING)
+        Using default bundler version `#{DEFAULT_VERSION}`
+
+        The Ruby buildpack uses the `BUNDLED WITH` value in your `Gemfile.lock` to determine the version
+        of bundler to install. Your `Gemfile.lock` does not contain this section, so a default version
+        of bundler will be installed instead.
+
+        Heroku recommends that you have both a `RUBY VERSION` and `BUNDLED WITH` version listed in your `Gemfile.lock`.
+        You can add it to your project by running:
+
+        ```
+        $ bundle update --bundler
+        ```
+
+        Commit the results to git before redeploying:
+
+        ```
+        $ git add Gemfile.lock
+        $ git commit -m "Add BUNDLED WITH version"
+        ```
+      WARNING
+
+      DEFAULT_VERSION
+    end
+  end
 end

lib/language_pack/helpers/lockfile_shell_parser.rb

@@ -1,6 +1,8 @@
 # frozen_string_literal: true
 
 require "json"
+require "tempfile"
+require "open3"
 
 module LanguagePack
   module Helpers
@@ -22,14 +24,11 @@ module Helpers
     #   specs["rake"] # => #<Gem::Version "13.2.1">
     #
     module LockfileShellParser
-      extend LanguagePack::ShellHelpers
-
       RUBY_PARSER_CODE = <<~RUBY
         require "json"
         require "bundler"
 
-        path = ARGV[0] or raise "First argument must be the path to the Gemfile.lock"
-        specs = Bundler::LockfileParser.new(File.read(path))
+        specs = Bundler::LockfileParser.new(STDIN.read)
           .specs
           .each_with_object({}) {|spec, hash| hash[spec.name.to_s] = spec.version.to_s }
         puts specs.to_json
@@ -47,8 +46,32 @@ module LockfileShellParser
       #   specs["nokogiri"] # => #<Gem::Version "1.15.0">
       #
       def self.call(lockfile_path:)
-        output = run!("ruby -e #{RUBY_PARSER_CODE.shellescape} #{lockfile_path.to_s.shellescape}", out: "2>/dev/null")
-        JSON.parse(output).transform_values { |version| Gem::Version.new(version) }
+        lockfile_path = Pathname(lockfile_path)
+        Tempfile.create(['lockfile_parser', '.rb']) do |tempfile|
+          tempfile.write(RUBY_PARSER_CODE)
+          tempfile.flush
+
+          stdout, stderr, status = Open3.capture3("ruby", tempfile.path, stdin_data: lockfile_path.read(mode: "rt"))
+          if status.success?
+            JSON.parse(stdout).transform_values { |version| Gem::Version.new(version) }
+          else
+            raise <<~ERROR
+              Cannot parse `Gemfile.lock` file at path `#{lockfile_path}`
+
+              The Ruby buildpack runs a Ruby script that uses Bundler::LockfileParser to parse the lockfile of your application.
+              This information is needed to set environment variables based on requested gems such as `RAILS_ENV`
+              before gems are installed via `bundle install`.
+
+              This script failed to parse `#{lockfile_path}` and the buildpack cannot continue.
+
+              Debugging information:
+
+              status: #{status}
+              stdout: #{stdout}
+              stderr: #{stderr}
+            ERROR
+          end
+        end
       end
     end
   end

spec/hatchet/rails6_spec.rb

@@ -3,7 +3,10 @@
 describe "Rails 6" do
   it "should detect successfully" do
     Hatchet::App.new('rails61').in_directory_fork do
-      bundler = LanguagePack::Helpers::BundlerWrapper.new(bundler_path: Dir.mktmpdir)
+      bundler = LanguagePack::Helpers::BundlerWrapper.new(
+        bundler_path: Dir.mktmpdir,
+        bundler_version: LanguagePack::Helpers::BundlerWrapper::DEFAULT_VERSION
+      )
       expect(LanguagePack::Rails5.use?(bundler: bundler)).to eq(false)
       expect(LanguagePack::Rails6.use?(bundler: bundler)).to eq(true)
     end

spec/hatchet/rails7_spec.rb

@@ -3,7 +3,10 @@
 describe "Rails 7" do
   it "should detect successfully" do
     Hatchet::App.new('rails-jsbundling').in_directory_fork do
-      bundler = LanguagePack::Helpers::BundlerWrapper.new(bundler_path: Dir.mktmpdir)
+      bundler = LanguagePack::Helpers::BundlerWrapper.new(
+        bundler_path: Dir.mktmpdir,
+        bundler_version: LanguagePack::Helpers::BundlerWrapper::DEFAULT_VERSION
+      )
       expect(LanguagePack::Rails6.use?(bundler: bundler)).to eq(false)
       expect(LanguagePack::Rails7.use?(bundler: bundler)).to eq(true)
     end