Skip to content

Commit bec38d1

Browse files
authored
Introduce bundle list based parser (heroku#1610)
* Introduce `bundle list` based parser This allows us to determine what versions of gems an application is using after `bundle install` has run. Currently, the buildpack works by loading bundler internals into memory and using those to parse the `Gemfile.lock`. That approach introduces strong compatibility requirements for coupling between Ruby versions and bundler versions. Additionally, recent versions of `bundle install` no longer output the list of gems installed: ``` $ bundle _2.3.27_ install Using rake 13.2.1 Using base64 0.2.0 ... Using toml-rb 4.0.0 Bundle complete! 11 Gemfile dependencies, 31 gems now installed. Use `bundle info [gemname]` to see where a bundled gem is installed. ``` ``` $ bundle _2.4.22_ install Bundle complete! 11 Gemfile dependencies, 31 gems now installed. Use `bundle info [gemname]` to see where a bundled gem is installed. ``` It's helpful to have a list of installed gems on build output for debugging purposes. We can use this `bundle list` output as a proxy for the old `bundle install` output. * Run `bundle list` to list installed gems Stream `bundle list` when `bundle install` emits no gem information. This output is retained and parsed to build a list of Ruby dependencies. This will eventually replace the bundler internals that are currently in use.
1 parent be1af49 commit bec38d1

5 files changed

Lines changed: 170 additions & 0 deletions

File tree

CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@
33
## [Unreleased]
44

55

6+
- Stream `bundle list` when `bundle install` emits no gem information. This condition happens when bundler 2.4+ runs with no gem additions or deletions (https://github.com/heroku/heroku-buildpack-ruby/pull/1610)
7+
68
## [v310] - 2025-05-27
79

810
- Introduce internal metrics for deriving Ruby version directly from the `Gemfile.lock` to avoid needing to call `bundle platform --ruby` in the future. No change in behavior is expected. (https://github.com/heroku/heroku-buildpack-ruby/pull/1603)

lib/language_pack.rb

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ def self.detect(app_path:, cache_path:, gemfile_lock: )
4545

4646
require "language_pack/helpers/plugin_installer"
4747
require "language_pack/helpers/stale_file_cleaner"
48+
require "language_pack/helpers/bundle_list"
4849
require "language_pack/helpers/rake_runner"
4950
require "language_pack/helpers/rails_runner"
5051
require "language_pack/helpers/bundler_wrapper"
Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
require "language_pack/shell_helpers"
2+
3+
module LanguagePack::Helpers
4+
class BundleList
5+
class CmdError < BuildpackError
6+
def initialize(command:, output:)
7+
super(<<~EOF)
8+
Error detecting dependencies
9+
10+
The Ruby buildpack requires information about your application’s dependencies to
11+
complete the build. Without this information, the Ruby buildpack cannot continue.
12+
13+
Command failed: `#{command}`
14+
15+
#{output}
16+
EOF
17+
end
18+
end
19+
20+
# Runs `bundle list` and builds a BundleList object
21+
# from the output
22+
#
23+
# Announces the run, optionally streams results to the user.
24+
# This is useful for the case where bundler doesn't print
25+
# any version info.
26+
#
27+
# Example:
28+
#
29+
# BundleList::Command.new(
30+
# stream_to_user: stream_to_user
31+
# ).call
32+
class HumanCommand
33+
include LanguagePack::ShellHelpers
34+
private attr_reader :stream_to_user, :io
35+
36+
def initialize(stream_to_user:, io: self)
37+
@io = io
38+
@stream_to_user = stream_to_user
39+
end
40+
41+
def call
42+
command = "bundle list"
43+
io.puts "Running: #{command}"
44+
45+
output = if stream_to_user
46+
io.pipe(command, user_env: true)
47+
else
48+
io.run(command, user_env: true)
49+
end
50+
51+
if $?.success?
52+
BundleList.new(
53+
output: output
54+
)
55+
else
56+
raise CmdError.new(output: output, command: command)
57+
end
58+
end
59+
end
60+
61+
def initialize(output: )
62+
@raw = output
63+
@gems = {}
64+
@raw.scan(/\* (?<name>\S+) \((?<version>[a-zA-Z0-9\.]+)(?<git_sha> [a-zA-Z0-9]+)?\)/) do
65+
captures = Regexp.last_match.named_captures
66+
@gems[captures["name"]] = captures["version"]
67+
end
68+
end
69+
70+
def has_gem?(name)
71+
@gems[name]
72+
end
73+
74+
def length
75+
@gems.length
76+
end
77+
78+
def gem_version(name)
79+
if version = @gems[name]
80+
Gem::Version.new(version)
81+
end
82+
end
83+
end
84+
end

lib/language_pack/ruby.rb

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,34 @@ def cleanup
116116
def config_detect
117117
end
118118

119+
# Runs `bundle list` and optionally streams the result to the user
120+
#
121+
# Streaming helps with build log visibility i.e. "what version of X" am I using at a glance.
122+
#
123+
# Checks if the information from `bundle list` matches information collected from bundler internals
124+
# if not, emits the difference. The goal is to eventually replace requiring bundler internals with
125+
# information retrieved from `bundle list`.
126+
private def bundle_list(stream_to_user: )
127+
bundle_list = LanguagePack::Helpers::BundleList::HumanCommand.new(
128+
stream_to_user: stream_to_user
129+
).call
130+
differences = bundler.specs.filter_map do |(name, spec)|
131+
expected = Gem::Version.new(spec.version)
132+
actual = bundle_list.gem_version(name)
133+
if expected != actual
134+
"#{name}: (`#{expected}` `#{actual}`)"
135+
end
136+
end
137+
138+
if !differences.empty?
139+
@report.capture(
140+
"bundle_list.differences" => differences.join(", "),
141+
)
142+
end
143+
144+
bundle_list
145+
end
146+
119147
private
120148

121149
# A bad shebang line looks like this:
@@ -703,6 +731,10 @@ def build_bundler
703731

704732
# Keep gem cache out of the slug
705733
FileUtils.rm_rf("#{slug_vendor_base}/cache")
734+
735+
bundle_list(
736+
stream_to_user: !bundler_output.match?(/Installing|Fetching|Using/)
737+
)
706738
else
707739
error_message = "Failed to install gems via Bundler."
708740
puts "Bundler Output: #{bundler_output}"

spec/helpers/bundle_list_spec.rb

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
2+
require 'spec_helper'
3+
4+
describe "Bundle list" do
5+
it "parses bundle list output" do
6+
output = <<~EOF
7+
Gems included by the bundle:
8+
* actioncable (6.1.4.1)
9+
* actionmailbox (6.1.4.1)
10+
* actionmailer (6.1.4.1)
11+
* actionpack (6.1.4.1)
12+
* actiontext (6.1.4.1)
13+
* actionview (6.1.4.1)
14+
* activejob (6.1.4.1)
15+
* activemodel (6.1.4.1)
16+
* activerecord (6.1.4.1)
17+
* activestorage (6.1.4.1)
18+
* activesupport (6.1.4.1)
19+
* addressable (2.8.0)
20+
* ast (2.4.2)
21+
* railties (6.1.4.1)
22+
Use `bundle info` to print more detailed information about a gem
23+
EOF
24+
25+
bundle_list = LanguagePack::Helpers::BundleList.new(
26+
output: output
27+
)
28+
expect(bundle_list.has_gem?("railties")).to be_truthy
29+
expect(bundle_list.gem_version("railties")).to eq(Gem::Version.new("6.1.4.1"))
30+
expect(bundle_list.has_gem?("nope")).to be_falsey
31+
32+
expect(bundle_list.length).to eq(14)
33+
end
34+
35+
it "handles git SHA gems" do
36+
output = <<~EOF
37+
Gems included by the bundle:
38+
* railties (6.1.4.1 asdf1)
39+
Use `bundle info` to print more detailed information about a gem
40+
EOF
41+
42+
bundle_list = LanguagePack::Helpers::BundleList.new(
43+
output: output
44+
)
45+
expect(bundle_list.has_gem?("railties")).to be_truthy
46+
expect(bundle_list.gem_version("railties")).to eq(Gem::Version.new("6.1.4.1"))
47+
expect(bundle_list.has_gem?("nope")).to be_falsey
48+
49+
expect(bundle_list.length).to eq(1)
50+
end
51+
end

0 commit comments

Comments
 (0)