| Age | Commit message (Collapse) | Author |
|---|
|
https://github.com/ruby/timeout/commit/cd51eac3ca
|
|
- ### TL;DR
Bundler is heavily limited by the connection pool which manages a
single connection. By increasing the number of connection, we can
drastiscally speed up the installation process when many gems need
to be downloaded and installed.
### Benchmark
There are various factors that are hard to control such as
compilation time and network speed but after dozens of tests I
can consistently get aroud 70% speed increase when downloading and
installing 472 gems, most having no native extensions (on purpose).
```
# Before
bundle install 28.60s user 12.70s system 179% cpu 23.014 total
# After
bundle install 30.09s user 15.90s system 281% cpu 16.317 total
```
You can find on this gist how this was benchmarked and the Gemfile
used https://gist.github.com/Edouard-chin/c8e39148c0cdf324dae827716fbe24a0
### Context
A while ago in #869, Aaron introduced a connection pool which
greatly improved Bundler speed. It was noted in the PR description
that managing one connection was already good enough and it wasn't
clear whether we needed more connections. Aaron also had the
intuition that we may need to increase the pool for downloading
gems and he was right.
> We need to study how RubyGems uses connections and make a decision
> based on request usage (e.g. only use one connection for many small
> requests like bundler API, and maybe many connections for
> downloading gems)
When bundler downloads and installs gem in parallel https://github.com/ruby/rubygems/blob/4f85e02fdd89ee28852722dfed42a13c9f5c9193/bundler/lib/bundler/installer/parallel_installer.rb#L128
most threads have to wait for the only connection in the pool to be
available which is not efficient.
### Solution
This commit modifies the pool size for the fetcher that Bundler
uses. RubyGems fetcher will continue to use a single connection.
The bundler fetcher is used in 2 places.
1. When downloading gems https://github.com/ruby/rubygems/blob/4f85e02fdd89ee28852722dfed42a13c9f5c9193/bundler/lib/bundler/source/rubygems.rb#L481-L484
2. When grabing the index (not the compact index) using the
`bundle install --full-index` flag.
https://github.com/ruby/rubygems/blob/4f85e02fdd89ee28852722dfed42a13c9f5c9193/bundler/lib/bundler/fetcher/index.rb#L9
Having more connections in 2) is not any useful but tweaking the
size based on where the fetcher is used is a bit tricky so I opted
to modify it at the class level.
I fiddle with the pool size and found that 5 seems to be the sweet
spot at least for my environment.
https://github.com/ruby/rubygems/commit/6063fd9963
|
|
https://github.com/ruby/rubygems/commit/9d744beb56
|
|
https://github.com/ruby/rubygems/commit/a55c485226
|
|
Otherwise, it uses the latest prism version
https://github.com/ruby/prism/commit/86406f63aa
|
|
over lockfile method in Gemfile
It would be simpler to do `options[:lockfile] ||= ENV["BUNDLE_LOCKFILE"]`,
but that doesn't work as `options` is frozen.
Fixes https://github.com/ruby/rubygems/pull/9117
https://github.com/ruby/rubygems/commit/6e3603a0e9
|
|
https://github.com/ruby/optparse/commit/6e2709a5fd
|
|
Co-authored-by: Benoit Daloze <eregontp@gmail.com>
https://github.com/ruby/rubygems/commit/463488b439
Co-authored-by: Patrik Ragnarsson <patrik@starkast.net>
|
|
This is more concise and ruby does a better job performance-wise.
This used to be `bsearch_index` already but https://github.com/ruby/prism/commit/6d8358c08395438d5924777c1fc3001a5ebf0aa3 changed it.
https://github.com/ruby/prism/pull/1733#discussion_r1373702087 said:
> Yeah the edge case was that the value matched an element exactly
But surely there would be a test to show this behaviour?
Gets called as part of pretty-printing nodes.
Further reduces the time for `SnapshotsTest` by ~16% for me.
https://github.com/ruby/prism/commit/f448e2b995
|
|
https://github.com/ruby/tempfile/commit/7fa7436baa
|
|
https://github.com/ruby/tempfile/commit/96361e9e42
|
|
https://github.com/ruby/rubygems/commit/5fd95f38d3
|
|
|
|
https://github.com/ruby/rubygems/commit/9e44b5ebc4
|
|
As mentioned in https://github.com/ruby/rubygems/issues/9124,
the intent for changing the default command was to be more welcoming.
I think we can acheive that by attempting to install, but to print
that same help message if there is no Gemfile.
That should address both concerns.
https://github.com/ruby/rubygems/commit/f3f505c02a
|
|
Fix: https://github.com/ruby/rubygems/issues/9124
This behavior is a deeply entrenched convention and changing it
will annoy lots of developers with unclear gains.
https://github.com/ruby/rubygems/commit/628e0ede46
|
|
Performance/StringInclude cop
https://github.com/ruby/rubygems/commit/fdd3419144
|
|
https://github.com/ruby/rubygems/commit/bb4d791cb4
|
|
|
|
- Depending on the native extension, it can greatly reduce compilation
time when executing recipes simultaneously.
For example on Prism:
```
# Before
time gem install prism
Building native extensions. This could take a while...
Successfully installed prism-1.6.0
1 gem installed
gem install prism 3.61s user 0.80s system 95% cpu 4.595 total
```
```
# After
time MAKEFLAGS="-j" gem install prism
Building native extensions. This could take a while...
Successfully installed prism-1.6.0
1 gem installed
MAKEFLAGS="-j" gem install prism 4.47s user 1.27s system 246% cpu 2.330 total
```
I don't think adding `-j` as a default is harmful, but I'm admitedly
not very knowledgable when it comes to compiler.
https://github.com/ruby/rubygems/commit/61340081c6
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
|
|
Removed public visibility from the candidate method.
https://github.com/ruby/optparse/commit/9a784a89a2
|
|
https://github.com/ruby/optparse/commit/5478354d4f
|
|
https://github.com/ruby/rubygems/commit/b8529f48bf
|
|
This allows for specifying the lockfile to read and write. It mirrors
the --gemfile option, and has higher priority than the lockfile
method in the Gemfile. It also mirrors the bundle lock --lockfile
option.
When the --lockfile option is used, it is applied twice. First, before
the Gemfile is read, to specify the lockfile to operate on, and again
after the Gemfile is read, so that if the Gemfile has a lockfile
method that overrides the defintion's lockfile, the --lockfile option
still has higher precedence.
https://github.com/ruby/rubygems/commit/17acdd4a89
|
|
A `password` is cleared when change a user now.
https://github.com/ruby/uri/commit/af6714473c
|
|
ruby2_keywords on Ruby 2.7+
On Ruby 3.4+, generic argument forwarding is significantly faster
as it does not allocate.
https://github.com/ruby/forwardable/commit/b606c3bf0a
|
|
This pattern is extremely common across the ecosystem, I don't think
it's reasonable to deprecate it.
I understand the performance argument, but perhaps the dependency
resolution algorithm can use another method that is private API
and only works with two `Version` instance.
https://github.com/ruby/rubygems/commit/024b4b547a
|
|
|
|
(https://github.com/ruby/rubygems/pull/9095)
* Rescue when deleting a non-existent cached gem file
When a gem was in the cache, but another process deletes it first, this
delete command fails.
To work around this, I'm rescuing from Errno::ENOENT and swalling the
error. The file is gone, and we can move on.
* Apply suggestion from @kou
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
---------
https://github.com/ruby/rubygems/commit/b30bcbc648
Co-authored-by: Hiroshi SHIBATA <hsbt@ruby-lang.org>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
|
|
If we use "system" variable in BUNDLE_VERSION on Bundler configuration,
we can use bundler version provided by system installation.
But the current logic returns the first activated version of bundler
like 2.7.2. It makes to confuse users.
https://github.com/ruby/rubygems/commit/4eb66d9549
|
|
Comparing version objects is a huge bottleneck in dependency solvers
(like inside Bundler). I would like to make comparing version objects
cheaper. Right now we support comparing version objects with strings by
trying to coerce the string to a version. So for example:
```ruby
Gem::Version.new("1") <=> "12"
```
I would like to deprecate and remove support for this feature so that we
can reduce the overhead of `def <=>`.
I'm not sure what version of RubyGems we could remove this from though.
https://github.com/ruby/rubygems/commit/81b7602183
|
|
https://github.com/ruby/rubygems/commit/c1e3d4d63b
|
|
```
Offenses:
Rakefile:18:1: C: [Correctable] Layout/EmptyLines: Extra blank line detected.
Diff:
@@ -11,4 +11,5 @@
ext.lib_dir = "lib/test_gem"
end
+
task default: :compile
https://github.com/ruby/rubygems/commit/8c414729df
|
|
https://github.com/ruby/rubygems/commit/64f92d2da0
|
|
https://github.com/ruby/rubygems/commit/f360af8e3b
|
|
This specifies the lockfile location. This allows for easy support
of different lockfiles per Ruby version or platform.
https://github.com/ruby/rubygems/commit/b54d65bc0a
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Co-authored-by: Colby Swandale <996377+colby-swandale@users.noreply.github.com>
|
|
This allows for the same behavior as including `lockfile false`
in the Gemfile. This allows you to get the behavior without
modifying the Gemfile, which is useful if you do not control the
Gemfile.
This is similar to the --no-lock option already supported by
`gem install -g Gemfile`.
https://github.com/ruby/rubygems/commit/6c94623881
Co-authored-by: Colby Swandale <996377+colby-swandale@users.noreply.github.com>
|
|
This allows you to specify the lockfile to use. This is useful if
you want to use different lockfiles for different ruby versions or
platforms. You can also skip writing the lockfile by using a false
value.
https://github.com/ruby/rubygems/commit/2896aa3fc2
Co-authored-by: Colby Swandale <996377+colby-swandale@users.noreply.github.com>
|
|
- The Logger is not thread safe when calling `with_level`.
This now becomes problematic because we are using multiple
threads during the resolution phase in order to fetch git gems.
https://github.com/ruby/rubygems/commit/380653ae74
|
|
- ### Problem
When you have a Gemfile that contains git gems, each repository will
be fetched one by one. This is extremelly slow.
A simple Gemfile with 5 git gems (small repositories) can take up
to 10 seconds just to fetch the repos.
We can speed this up by running multiple git process and fetching
repositories silmutaneously.
### Solution
The repositories are fetched in Bundler when `Source::Git#specs` is
called.
The problem is that `source.specs` is called in various places
depending on Gemfile.
I think the issue is that calling `source.specs` feels like that
as a "side effect" we are going to clone repositories. I believe
that fetching repositories should be an explicit call.
For instance:
```ruby
source "https://rubygems.org"
gem "foo", github: "foo/foo"
# The repository foo will be fetched as a side effect to the call to `source.spec_names`
# https://github.com/ruby/rubygems/blob/6cc7d71dac3d0275c9727cf200c7acfbf6c78d37/bundler/lib/bundler/source_map.rb#L21
```
```ruby
source "https://rubygems.org"
gem "bar", source: "https://example.org"
gem "foo", github: "foo/foo"
# The repository foo will be fetched on a different codepath
# https://github.com/ruby/rubygems/blob/6cc7d71dac3d0275c9727cf200c7acfbf6c78d37/bundler/lib/bundler/source/rubygems_aggregate.rb#L35
# That is because the gem "bar" has a source that doesn't have the `/dependencies` API
# endpoint and therefore Bundler enters a different branch condition.
```
I opted to add a self explanatory call to fetch the git source
repositories just before we start the resolution, and *just*
before any other calls to `source.specs` is performed.
https://github.com/ruby/rubygems/commit/f0ef526f23
|
|
For subclasses from Set, require `set/subclass_compatible`, and
extend the subclass and include a module in it that makes it more
backwards compatible with the pure Ruby Set implementation used
before Ruby 4.
The module included in the subclass contains a near-copy of the
previous Set implementation, with the following changes:
* Accesses to `@hash` are generally replaced with `super` calls. In
some cases, they are replaced with a call to another instance method.
* Some methods that only accessed `@hash` and nothing else are not
defined, so they inherit behavior from core Set.
* The previous `Set#divide` implementation is not used, to avoid
depending on tsort.
This fixes the following two issues:
* [Bug #21375] Set[] does not call #initialize
* [Bug #21396] Set#initialize should call Set#add on items passed in
It should also fix the vast majority of backwards compatibility issues
in other cases where code subclassed Set and depended on implementation
details (such as which methods call which other methods).
This does not affect Set internals, so Set itself remains fast. For
users who want to subclass Set but do not need to worry about
backwards compatibility, they can subclass from Set::CoreSet, a Set
subclass that does not have the backward compatibility layer included.
|
|
https://github.com/ruby/rubygems/commit/9be811c01a
|
|
Previously calling a singleton method with invalid arguments would give:
RuntimeError: Incompatible locations
This is because `join` wants the operator to come before the location
https://github.com/ruby/error_highlight/commit/44920551dd
|
|
https://github.com/ruby/rubygems/commit/6cc7d71dac
|
|
- In https://github.com/ruby/rubygems/commit/31d67ecc056fb5a9193bc66a6e69e21576a87702
we enforced the new behaviour where running `bundle` no longer
installs gems but displays the help.
Users now have a way to configure their preferred default command using
the `BUNDLE_DEFAULT_CLI_COMMAND` flag.
With the preview of Ruby 4.0 now being released, some people will
start to see this new change.
The problem is that the previous behaviour had existed for like an
eternity and we didn't warn users about this change in advance.
I'd like to provide a deprecation/warning cycle because this is
confusing users already and this breaks various CI setup that now
needs to be changed immediately.
https://github.com/ruby/rubygems/commit/e415480ac5
|
|
This is useful, in case you're using Docker, and an upstream
Dockerfile sets BUNDLER_VERSION to something you don't want.
It's impossible to unset it... only override to be the empty
string.
https://github.com/ruby/rubygems/commit/ffa3eb9ac6
|
|
(https://github.com/ruby/rubygems/pull/8183)
* Add new gem templates
* Add `--ext=go` in `bundle gem`
* Add setup-go to .github/workflows/main.yml
* Embed go version in go.mod
* Use go in bundler CI
* Add example method to template
* Install Go in .circleci/config.yml
* Install Go in .gitlab-ci.yml
* Allow hard tabs in go template
* Run `rake update_manifest`
* Fix test
* Move go_gem to gemspec
Respect to 9b0ec80
* nits: :golf:
* includes valid module name in go.mod
* generate header file
* Run `go mod tidy` to create `go.sum`
* Check if `go.sum` is generated only when Go is installed
To avoid test failure in environments where Go is not installed
* Run CI
* Workaround for hung up
c.f. https://github.com/rubygems/rubygems/actions/runs/11639408044/job/32415545422
* Write man for --ext=go
* Re-generate man with `./bin/rake man:build`
* pinning :pushpin:
* Update with `./bin/rake man:build`
* nits: Extract to method
* nits: Use `sys_exec` instead of `system`
* Clean go module cache after test
Workaround following error
```
1) bundle gem gem naming with underscore --ext parameter set with go includes go_gem extension in extconf.rb
Failure/Error: FileUtils.rm_r(dir)
Errno::EACCES:
Permission denied @ apply2files - /home/runner/work/rubygems/rubygems/bundler/tmp/2.2/home/go/pkg/mod/gopkg.in/yaml.v3@v3.0.1/decode_test.go
# ./spec/support/helpers.rb:37:in `block in reset!'
# ./spec/support/helpers.rb:21:in `each'
# ./spec/support/helpers.rb:21:in `reset!'
# ./spec/spec_helper.rb:130:in `block (2 levels) in <top (required)>'
# /home/runner/work/rubygems/rubygems/lib/rubygems.rb:303:in `load'
# /home/runner/work/rubygems/rubygems/lib/rubygems.rb:303:in `activate_and_load_bin_path'
```
Files installed with `go get` have permissions set to 444
ref. https://github.com/golang/go/issues/35615
```
$ ls -l /home/runner/work/rubygems/rubygems/bundler/tmp/2.2/home/go/pkg/mod/gopkg.in/yaml.v3@v3.0.1/decode_test.go
-r--r--r-- 1 runner runner 42320 Nov 15 06:38 /home/runner/work/rubygems/rubygems/bundler/tmp/2.2/home/go/pkg/mod/gopkg.in/yaml.v3@v3.0.1/decode_test.go
```
So they cannot be deleted by `FileUtils.rm_r`.
Therefore, this is necessary to execute `go clean -modcache` separately from `FileUtils.rm_r` to circumvent it.
* Remove needless changes
ref. https://github.com/ruby/rubygems/pull/8183#discussion_r2532902051
* ci: setup-go is needless
* Don't run go command in `bundle gem`
ref. https://github.com/ruby/rubygems/pull/8183#discussion_r2532765470
* Revert unrelated date changes
---------
https://github.com/ruby/rubygems/commit/260d7d60b3
Co-authored-by: Hiroshi SHIBATA <hsbt@ruby-lang.org>
|
|
`Gem::Installer.instance_methods(false).include?(:generate_plugins)` is
63x slower than `Gem::Installer.method_defined?(:generate_plugins)` in a
microbenchmark. The latter is a direct lookup, whereas the former will
create an array, which will be slower.
```ruby
require "benchmark/ips"
Benchmark.ips do |x|
x.report "instance_methods" do
Gem::Installer.instance_methods(false).include?(:generate_plugins)
end
x.report "method_defined" do
Gem::Installer.method_defined?(:generate_plugins)
end
x.compare!
end
```
```
$ ruby -I lib/ benchmark_methods.rb
ruby 3.4.4 (2025-05-14 revision https://github.com/ruby/rubygems/commit/a38531fd3f) +PRISM [arm64-darwin23]
Warming up --------------------------------------
instance_methods 58.449k i/100ms
method_defined 3.375M i/100ms
Calculating -------------------------------------
instance_methods 541.874k (± 5.7%) i/s (1.85 μs/i) - 2.747M in 5.087825s
method_defined 34.263M (± 1.1%) i/s (29.19 ns/i) - 172.135M in 5.024524s
Comparison:
method_defined: 34263189.1 i/s
instance_methods: 541874.3 i/s - 63.23x slower
```
This does not make much difference in an overall benchmark or profile,
but this is more idiomatic Ruby than the prior code.
https://github.com/ruby/rubygems/commit/49dec52cb2
|
|
https://github.com/ruby/rubygems/commit/fee8dd2f08
|
|
https://github.com/ruby/rubygems/commit/40ace48651
|
