| Age | Commit message (Collapse) | Author |
|
https://github.com/ruby/rubygems/commit/aa7632161e
|
|
|
|
|
|
|
|
(cherry picked from commit https://github.com/ruby/rubygems/commit/26c1db5a65a8)
https://github.com/ruby/rubygems/commit/bbb5b767d0
|
|
- ### TL;DR
Bundler is heavily limited by the connection pool which manages a
single connection. By increasing the number of connection, we can
drastiscally speed up the installation process when many gems need
to be downloaded and installed.
### Benchmark
There are various factors that are hard to control such as
compilation time and network speed but after dozens of tests I
can consistently get aroud 70% speed increase when downloading and
installing 472 gems, most having no native extensions (on purpose).
```
# Before
bundle install 28.60s user 12.70s system 179% cpu 23.014 total
# After
bundle install 30.09s user 15.90s system 281% cpu 16.317 total
```
You can find on this gist how this was benchmarked and the Gemfile
used https://gist.github.com/Edouard-chin/c8e39148c0cdf324dae827716fbe24a0
### Context
A while ago in #869, Aaron introduced a connection pool which
greatly improved Bundler speed. It was noted in the PR description
that managing one connection was already good enough and it wasn't
clear whether we needed more connections. Aaron also had the
intuition that we may need to increase the pool for downloading
gems and he was right.
> We need to study how RubyGems uses connections and make a decision
> based on request usage (e.g. only use one connection for many small
> requests like bundler API, and maybe many connections for
> downloading gems)
When bundler downloads and installs gem in parallel https://github.com/ruby/rubygems/blob/4f85e02fdd89ee28852722dfed42a13c9f5c9193/bundler/lib/bundler/installer/parallel_installer.rb#L128
most threads have to wait for the only connection in the pool to be
available which is not efficient.
### Solution
This commit modifies the pool size for the fetcher that Bundler
uses. RubyGems fetcher will continue to use a single connection.
The bundler fetcher is used in 2 places.
1. When downloading gems https://github.com/ruby/rubygems/blob/4f85e02fdd89ee28852722dfed42a13c9f5c9193/bundler/lib/bundler/source/rubygems.rb#L481-L484
2. When grabing the index (not the compact index) using the
`bundle install --full-index` flag.
https://github.com/ruby/rubygems/blob/4f85e02fdd89ee28852722dfed42a13c9f5c9193/bundler/lib/bundler/fetcher/index.rb#L9
Having more connections in 2) is not any useful but tweaking the
size based on where the fetcher is used is a bit tricky so I opted
to modify it at the class level.
I fiddle with the pool size and found that 5 seems to be the sweet
spot at least for my environment.
https://github.com/ruby/rubygems/commit/6063fd9963
|
|
https://github.com/ruby/rubygems/commit/a55c485226
|
|
https://github.com/ruby/rubygems/commit/b8529f48bf
|
|
https://github.com/ruby/rubygems/commit/9be811c01a
|
|
https://github.com/ruby/rubygems/commit/74c9eaf1f0
|
|
https://github.com/ruby/rubygems/commit/a65a4b775e
|
|
https://github.com/rubygems/rubygems/commit/a51334ba99
|
|
|
|
|
|
Next version for both will be 4.0.0, however, extra work is necessary to
get CI passing against the new major. So for now, I'm bumping just the
minor version.
|
|
Also removed the helper to install real gems during specs to avoid the
temptation of introducing network stuff again.
https://github.com/rubygems/rubygems/commit/a1ab5e319a
|
|
https://github.com/rubygems/rubygems/commit/672722cd4d
|
|
They cause flakies when different tests start them in parallel, and also
make the specs more complicated.
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/13484
|
|
CGI::Cookie
https://github.com/rubygems/rubygems/commit/cecc280f61
|
|
https://github.com/rubygems/rubygems/commit/5d5e37bf23
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/13275
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/13275
|
|
|
|
https://github.com/rubygems/rubygems/commit/a7c6696ffd
Notes:
Merged: https://github.com/ruby/ruby/pull/12804
|
|
In the past, bundler updated platform without normalized variable like arm64-darwin-23.
We ignored that update. But the current bundler uses arm64-darwin for that.
Notes:
Merged: https://github.com/ruby/ruby/pull/12570
|
|
|
|
Pick from https://github.com/rubygems/rubygems/commit/7a7b234721c375874b7e22b1c5b14925b943f04e
|
|
https://github.com/rubygems/rubygems/commit/125f9fece9
|
|
from https://github.com/rubygems/rubygems/commit/72fd3dd2096af16d797ad0cd8e0d2a8869e240b3
Notes:
Merged: https://github.com/ruby/ruby/pull/7025
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/7020
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6987
|
|
Pick from https://github.com/rubygems/rubygems/commit/084f7d1f21f6fc3e2bb685b7bda3653fb2891c6e
Notes:
Merged: https://github.com/ruby/ruby/pull/6936
|
|
from https://github.com/rubygems/rubygems/commit/bfb0ae69776069155d2092702bfbb5a12617d85a
Notes:
Merged: https://github.com/ruby/ruby/pull/6906
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6124
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/5350
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/5342
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/5334
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/5325
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/5317
|
|
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/4383
|
|
https://github.com/rubygems/rubygems/commit/9f15b595ab
Notes:
Merged: https://github.com/ruby/ruby/pull/3184
|
|
https://github.com/rubygems/rubygems/commit/08374d0e4a
Notes:
Merged: https://github.com/ruby/ruby/pull/3184
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/3086
|