| Age | Commit message (Collapse) | Author |
|
|
|
|
|
|
|
|
|
|
|
(cherry picked from commit https://github.com/ruby/rubygems/commit/26c1db5a65a8)
https://github.com/ruby/rubygems/commit/bbb5b767d0
|
|
- ### TL;DR
Bundler is heavily limited by the connection pool which manages a
single connection. By increasing the number of connection, we can
drastiscally speed up the installation process when many gems need
to be downloaded and installed.
### Benchmark
There are various factors that are hard to control such as
compilation time and network speed but after dozens of tests I
can consistently get aroud 70% speed increase when downloading and
installing 472 gems, most having no native extensions (on purpose).
```
# Before
bundle install 28.60s user 12.70s system 179% cpu 23.014 total
# After
bundle install 30.09s user 15.90s system 281% cpu 16.317 total
```
You can find on this gist how this was benchmarked and the Gemfile
used https://gist.github.com/Edouard-chin/c8e39148c0cdf324dae827716fbe24a0
### Context
A while ago in #869, Aaron introduced a connection pool which
greatly improved Bundler speed. It was noted in the PR description
that managing one connection was already good enough and it wasn't
clear whether we needed more connections. Aaron also had the
intuition that we may need to increase the pool for downloading
gems and he was right.
> We need to study how RubyGems uses connections and make a decision
> based on request usage (e.g. only use one connection for many small
> requests like bundler API, and maybe many connections for
> downloading gems)
When bundler downloads and installs gem in parallel https://github.com/ruby/rubygems/blob/4f85e02fdd89ee28852722dfed42a13c9f5c9193/bundler/lib/bundler/installer/parallel_installer.rb#L128
most threads have to wait for the only connection in the pool to be
available which is not efficient.
### Solution
This commit modifies the pool size for the fetcher that Bundler
uses. RubyGems fetcher will continue to use a single connection.
The bundler fetcher is used in 2 places.
1. When downloading gems https://github.com/ruby/rubygems/blob/4f85e02fdd89ee28852722dfed42a13c9f5c9193/bundler/lib/bundler/source/rubygems.rb#L481-L484
2. When grabing the index (not the compact index) using the
`bundle install --full-index` flag.
https://github.com/ruby/rubygems/blob/4f85e02fdd89ee28852722dfed42a13c9f5c9193/bundler/lib/bundler/fetcher/index.rb#L9
Having more connections in 2) is not any useful but tweaking the
size based on where the fetcher is used is a bit tricky so I opted
to modify it at the class level.
I fiddle with the pool size and found that 5 seems to be the sweet
spot at least for my environment.
https://github.com/ruby/rubygems/commit/6063fd9963
|
|
https://github.com/ruby/rubygems/commit/a55c485226
|
|
https://github.com/ruby/rubygems/commit/b8529f48bf
|
|
https://github.com/ruby/rubygems/commit/9be811c01a
|
|
|
|
|
|
|
|
https://github.com/ruby/rubygems/commit/74c9eaf1f0
|
|
https://github.com/ruby/rubygems/commit/a65a4b775e
|
|
https://github.com/ruby/rubygems/commit/be579a9c8c
|
|
https://github.com/ruby/rubygems/commit/8df0e4e63e
|
|
https://github.com/ruby/rubygems/commit/bc77ec0bf2
|
|
https://github.com/rubygems/rubygems/commit/82d46d3b28
|
|
https://github.com/rubygems/rubygems/commit/a51334ba99
|
|
|
|
|
|
https://github.com/rubygems/rubygems/commit/8078a747b3
|
|
Next version for both will be 4.0.0, however, extra work is necessary to
get CI passing against the new major. So for now, I'm bumping just the
minor version.
|
|
https://github.com/rubygems/rubygems/commit/afbbc02763
|
|
Also removed the helper to install real gems during specs to avoid the
temptation of introducing network stuff again.
https://github.com/rubygems/rubygems/commit/a1ab5e319a
|
|
https://github.com/rubygems/rubygems/commit/672722cd4d
|
|
They cause flakies when different tests start them in parallel, and also
make the specs more complicated.
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/13484
|
|
CGI::Cookie
https://github.com/rubygems/rubygems/commit/cecc280f61
|
|
https://github.com/rubygems/rubygems/commit/5d5e37bf23
|
|
net-http-persistent
https://github.com/rubygems/rubygems/commit/b9a4722d5e
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/13275
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/13275
|
|
|
|
My application spends more than 30% of time during `bundle update`
comparing versions due to versions being sorted inside
next_package_to_try. This has been addressed in pub_grub by defining a
strategy interface (a `#next_package_and_version` method) which allows
consumers to have finer control over the heuristic to select the next
package to try.
This commit implements the new strategy interface to remove extraneous
version sorting (previously in next_package_to_try) since only the final
count of versions is used.
Combined with a previous change to pub_grub (already applied to
Bundler), this commit results in `bundle update` taking only half the
time it did on 2.6.5.
https://github.com/rubygems/rubygems/commit/62f69e27f0
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/12983
|
|
https://github.com/rubygems/rubygems/commit/3aaa75e7b9
Notes:
Merged: https://github.com/ruby/ruby/pull/12968
|
|
|
|
https://github.com/ruby/prism/commit/2afe89f8ce
|
|
This gets our daily Bundler CI back to green.
https://github.com/rubygems/rubygems/commit/1bb70f75d2
Notes:
Merged: https://github.com/ruby/ruby/pull/12890
|
|
- new version restores previously removed method
https://github.com/rubygems/rubygems/commit/5348e88d07
Notes:
Merged: https://github.com/ruby/ruby/pull/12840
|
|
https://github.com/rubygems/rubygems/commit/176dc7421c
Notes:
Merged: https://github.com/ruby/ruby/pull/12840
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/12840
|
|
https://github.com/rubygems/rubygems/commit/a7c6696ffd
Notes:
Merged: https://github.com/ruby/ruby/pull/12804
|
|
In the past, bundler updated platform without normalized variable like arm64-darwin-23.
We ignored that update. But the current bundler uses arm64-darwin for that.
Notes:
Merged: https://github.com/ruby/ruby/pull/12570
|
|
Since we now vendor uri, it does not buy us anything to include it in
the gemfile explicitly.
Notes:
Merged: https://github.com/ruby/ruby/pull/12568
|
|
I think logger is only used to figure out which methods need to be made
noops in order to silence webrick during tests.
However, it seems possible to do the same using webrick's builtin logger
and the current method does not seem even correct since it's not
guaranteed that the logger gem and webrick's logger will use the same
methods.
Notes:
Merged: https://github.com/ruby/ruby/pull/12568
|
|
Tests don't use it directly, so no need to specify it in our Gemfile.
I think it may have only been added because sinatra was not declaring
its dependency at the time.
Notes:
Merged: https://github.com/ruby/ruby/pull/12568
|
|
We switched to nronn because ronn-ng felt abandoned, but it seems
it has activity again, so switch back.
Notes:
Merged: https://github.com/ruby/ruby/pull/12568
|