- ### TL;DR

  Bundler is heavily limited by the connection pool which manages a
  single connection. By increasing the number of connection, we can
  drastiscally speed up the installation process when many gems need
  to be downloaded and installed.

  ### Benchmark

  There are various factors that are hard to control such as
  compilation time and network speed but after dozens of tests I
  can consistently get aroud 70% speed increase when downloading and
  installing 472 gems, most having no native extensions (on purpose).

  ```
  # Before
  bundle install  28.60s user 12.70s system 179% cpu 23.014 total

  # After
  bundle install  30.09s user 15.90s system 281% cpu 16.317 total
  ```

  You can find on this gist how this was benchmarked and the Gemfile
  used https://gist.github.com/Edouard-chin/c8e39148c0cdf324dae827716fbe24a0

  ### Context

  A while ago in #869, Aaron introduced a connection pool which
  greatly improved Bundler speed. It was noted in the PR description
  that managing one connection was already good enough and it wasn't
  clear whether we needed more connections. Aaron also had the
  intuition that we may need to increase the pool for downloading
  gems and he was right.

  > We need to study how RubyGems uses connections and make a decision
  > based on request usage (e.g. only use one connection for many small
  > requests like bundler API, and maybe many connections for
  > downloading gems)

  When bundler downloads and installs gem in parallel https://github.com/ruby/rubygems/blob/4f85e02fdd89ee28852722dfed42a13c9f5c9193/bundler/lib/bundler/installer/parallel_installer.rb#L128
  most threads have to wait for the only connection in the pool to be
  available which is not efficient.

  ### Solution

  This commit modifies the pool size for the fetcher that Bundler
  uses. RubyGems fetcher will continue to use a single connection.

  The bundler fetcher is used in 2 places.

  1. When downloading gems https://github.com/ruby/rubygems/blob/4f85e02fdd89ee28852722dfed42a13c9f5c9193/bundler/lib/bundler/source/rubygems.rb#L481-L484
  2. When grabing the index (not the compact index) using the
    `bundle install --full-index` flag.
    https://github.com/ruby/rubygems/blob/4f85e02fdd89ee28852722dfed42a13c9f5c9193/bundler/lib/bundler/fetcher/index.rb#L9

  Having more connections in 2) is not any useful but tweaking the
  size based on where the fetcher is used is a bit tricky so I opted
  to modify it at the class level.
  I fiddle with the pool size and found that 5 seems to be the sweet
  spot at least for my environment.

https://github.com/ruby/rubygems/commit/6063fd9963

The remote fetcher only works with certain schemes (`http`, `https`,
`s3`, and `file`).  It's possible for other schemes to show up in this
code and it can cause bugs.

Before this patch, doing `gem install path:///hello` would result in an
infinite loop because this function would do `send "fetch_path"`,
calling itself forever.  Now we see an exception.

I think we should validate gem names earlier, but it's really best
practice to restrict the possible strings passed to `send`.

https://github.com/rubygems/rubygems/commit/54e2781b73

https://github.com/rubygems/rubygems/commit/b58829a868

https://github.com/rubygems/rubygems/commit/35fc7f9547

We sometimes send HEAD requests. The s3_uri_signer.rb code allways assumed GETs.
This lead to consistently getting 403 responses back from S3. Recently, S3
attempted to change the behaviour of how 403s are handled when TCP connections
are reused,  which escalated this bug from "just noise" to "breaks gem installs".
They've reverted that behaviour, so the severity of this problem is back to
"just noise". Either way, it's a bug in rubygems and warrants a fix it.

https://github.com/rubygems/rubygems/commit/c38f502b73

Fixed https://github.com/rubygems/rubygems/pull/7996

https://github.com/rubygems/rubygems/commit/16bfcac883

https://github.com/rubygems/rubygems/commit/cfc908c8c1

https://github.com/rubygems/rubygems/commit/e2e7440ede