summaryrefslogtreecommitdiff
path: root/lib/rubygems/request
AgeCommit message (Collapse)Author
2025-12-04[ruby/rubygems] Increase connection pool to allow for up to 70% speed increase:Edouard CHIN
- ### TL;DR Bundler is heavily limited by the connection pool which manages a single connection. By increasing the number of connection, we can drastiscally speed up the installation process when many gems need to be downloaded and installed. ### Benchmark There are various factors that are hard to control such as compilation time and network speed but after dozens of tests I can consistently get aroud 70% speed increase when downloading and installing 472 gems, most having no native extensions (on purpose). ``` # Before bundle install 28.60s user 12.70s system 179% cpu 23.014 total # After bundle install 30.09s user 15.90s system 281% cpu 16.317 total ``` You can find on this gist how this was benchmarked and the Gemfile used https://gist.github.com/Edouard-chin/c8e39148c0cdf324dae827716fbe24a0 ### Context A while ago in #869, Aaron introduced a connection pool which greatly improved Bundler speed. It was noted in the PR description that managing one connection was already good enough and it wasn't clear whether we needed more connections. Aaron also had the intuition that we may need to increase the pool for downloading gems and he was right. > We need to study how RubyGems uses connections and make a decision > based on request usage (e.g. only use one connection for many small > requests like bundler API, and maybe many connections for > downloading gems) When bundler downloads and installs gem in parallel https://github.com/ruby/rubygems/blob/4f85e02fdd89ee28852722dfed42a13c9f5c9193/bundler/lib/bundler/installer/parallel_installer.rb#L128 most threads have to wait for the only connection in the pool to be available which is not efficient. ### Solution This commit modifies the pool size for the fetcher that Bundler uses. RubyGems fetcher will continue to use a single connection. The bundler fetcher is used in 2 places. 1. When downloading gems https://github.com/ruby/rubygems/blob/4f85e02fdd89ee28852722dfed42a13c9f5c9193/bundler/lib/bundler/source/rubygems.rb#L481-L484 2. When grabing the index (not the compact index) using the `bundle install --full-index` flag. https://github.com/ruby/rubygems/blob/4f85e02fdd89ee28852722dfed42a13c9f5c9193/bundler/lib/bundler/fetcher/index.rb#L9 Having more connections in 2) is not any useful but tweaking the size based on where the fetcher is used is a bit tricky so I opted to modify it at the class level. I fiddle with the pool size and found that 5 seems to be the sweet spot at least for my environment. https://github.com/ruby/rubygems/commit/6063fd9963
2023-12-13[rubygems/rubygems] Vendor net-http and net-protocol in RubyGemsDavid Rodríguez
https://github.com/rubygems/rubygems/commit/99d91c9ed2
2023-03-23util/rubocop -A --only Layout/EmptyLineAfterMagicCommentHiroshi SHIBATA
Notes: Merged: https://github.com/ruby/ruby/pull/7582
2023-03-17util/rubocop -A --only Style/SymbolProcHiroshi SHIBATA
2023-03-17util/rubocop -A --only Performance/CasecmpHiroshi SHIBATA
2022-08-09Merge rubygems/bundler HEAD.Hiroshi SHIBATA
Pick from https://github.com/rubygems/rubygems/commit/dfbb5a38114640e0d8d616861607f3de73ee0199 Notes: Merged: https://github.com/ruby/ruby/pull/6224
2022-07-22RubyGems: Enable Style/StringLiterals copTakuya Noguchi
Signed-off-by: Takuya Noguchi <takninnovationresearch@gmail.com>
2021-07-07Sync RubyGems and Bundler with upstreamHiroshi SHIBATA
Notes: Merged: https://github.com/ruby/ruby/pull/4634
2020-09-23Revert "Manually merged from https://github.com/rubygems/rubygems/pull/2636"Hiroshi SHIBATA
31a6eaabc165d8a222e176f2c809d90622d88ec2 is obsoleted with https://github.com/rubygems/rubygems/pull/3820
2020-09-23Manually merged from https://github.com/rubygems/rubygems/pull/2636Hiroshi SHIBATA
Enable Style/EmptyLinesAroundClassBody rubocop cop.
2020-07-31Enforce no empty lines around class body in rubygemsDavid Rodríguez
To normalize the code style with `bundler`. Notes: Merged: https://github.com/ruby/ruby/pull/3379
2020-06-15Use space inside block braces everywhereDavid Rodríguez
To make rubygems code style consistent with bundler. Notes: Merged: https://github.com/ruby/ruby/pull/3229
2019-06-01Merge rubygems master from upstream.Hiroshi SHIBATA
I picked the commit from 3c469e0da538428a0ddd94f99aa73c32da22e8ba
2019-02-14Merge RubyGems master@9be7858f7f17eae3058204f3c03e4b798ba18b9chsbt
This version contains the some style changes by RuboCop. * https://github.com/rubygems/rubygems/commit/9d810be0ede925fb2e3af535848582c3f8e0e72f * https://github.com/rubygems/rubygems/commit/61ea98a727fb1b76b6fac52d74107ee4b02aaef2 * https://github.com/rubygems/rubygems/commit/795893dce3c5f8540804fc08144cc6a90f086b13 * https://github.com/rubygems/rubygems/commit/9be7858f7f17eae3058204f3c03e4b798ba18b9c git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67074 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-21Merge master branch from rubygems/rubygems upstream.hsbt
* Enable Style/MethodDefParentheses in Rubocop https://github.com/rubygems/rubygems/pull/2478 * Enable Style/MultilineIfThen in Rubocop https://github.com/rubygems/rubygems/pull/2479 * Fix required_ruby_version with prereleases and improve error message https://github.com/rubygems/rubygems/pull/2344 * Fix bundler rubygems binstub not properly looking for bundler https://github.com/rubygems/rubygems/pull/2426 git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65904 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-27Merge master branch from rubygems upstream.hsbt
* It's preparation to release RubyGems 3.0.0.beta2 and Ruby 2.6.0 preview 3. * https://github.com/rubygems/rubygems/compare/v3.0.0.beta1...fad2eb15a282b19dfcb4b48bc95b8b39ebb4511f git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64555 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-30Merge RubyGems 3.0.0.beta1.hsbt
* It drop to support < Ruby 2.2 * Cleanup deprecated methods and classes. * Mark obsoleted methods to deprecate. * and other enhancements. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63528 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-02-01* lib/rubygems.rb, lib/rubygems/*, test/rubygems/*: Update rubygems-2.5.2.hsbt
It supports to enable frozen string literal and add `--norc` option for disable to `.gemrc` configuration. See 2.5.2 release notes for other fixes and enhancements. https://github.com/rubygems/rubygems/blob/a8aa3bac723f045c52471c7b9328310a048561e0/History.txt#L3 git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53707 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-12-16Add frozen_string_literal: false for all filesnaruse
When you change this to true, you may need to add more tests. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53141 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-07-01* lib/rubygems: Update to RubyGems HEAD(c202db2).hsbt
this version contains many enhancements see http://git.io/vtNwF * test/rubygems: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51092 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-17* lib/rubygems/*, test/rubygems/*: Update to RubyGems 2.4.4hsbt
master (2f6e42e). git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48468 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-09-14* lib/rubygems: Update to RubyGems 2.4.1 master(713ab65)hsbt
Complete history at: https://github.com/rubygems/rubygems/blob/master/History.txt#L3-L216 * test/rubygems: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47582 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.25.1) at 2026-01-16 06:38:52 +0000