| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- Added a new `-j` option to `gem install` and `gem update`.
This option allows to specify the number of jobs we pass to `make`
when compiling gem with native extensions.
By default its the number of processors, but users may want a way
to control this.
You can use it like so: `gem install json -j8`
https://github.com/ruby/rubygems/commit/67aad88ca6
|
|
- ### Problem
Since https://github.com/ruby/rubygems/pull/9131, we are now
compiling make rules simultaneously. The number of jobs
is equal to the number of processors.
This may be problematic for some users as they want to control
this value.
### Solution
The number of jobs passed to `make` will now be equal to the
`BUNDLE_JOBS` value.
### Side note
It's also worth to note that since Bundler installs gems in
parallel, we may end up running multiple `make -j<JOB>` in parallel
which would cause exhaust the number of processors we have.
This problem can be fixed by implementing a GNU jobserver, which I
plan to do. But I felt that this would be too much change in one PR.
https://github.com/ruby/rubygems/commit/d51995deb9
|
|
This change updates `write_binary` to use a new class,
`AtomicFileWriter.open` to write the gem's files. This implementation
is borrowed from Active Support's [`atomic_write`](https://github.com/rails/rails/blob/main/activesupport/lib/active_support/core_ext/file/atomic.rb).
Atomic write will write the files to a temporary file and then once
created, sets permissions and renames the file. If the file is corrupted
- ie on failed download, an error occurs, or for some other reason, the
real file will not be created. The changes made here make `verify_gz`
obsolete, we don't need to verify it if we have successfully created the
file atomically. If it exists, it is not corrupt. If it is corrupt, the
file won't exist on disk.
While writing tests for this functionality I replaced the
`RemoteFetcher` stub with `FakeFetcher` except for where we really do
need to overwrite the `RemoteFetcher`. The new test implementation is much
clearer on what it's trying to accomplish versus the prior test
implementation.
https://github.com/ruby/rubygems/commit/0cd4b54291
|
|
installation.
https://github.com/ruby/rubygems/commit/a70e573973
|
|
- ### TL;DR
Bundler is heavily limited by the connection pool which manages a
single connection. By increasing the number of connection, we can
drastiscally speed up the installation process when many gems need
to be downloaded and installed.
### Benchmark
There are various factors that are hard to control such as
compilation time and network speed but after dozens of tests I
can consistently get aroud 70% speed increase when downloading and
installing 472 gems, most having no native extensions (on purpose).
```
# Before
bundle install 28.60s user 12.70s system 179% cpu 23.014 total
# After
bundle install 30.09s user 15.90s system 281% cpu 16.317 total
```
You can find on this gist how this was benchmarked and the Gemfile
used https://gist.github.com/Edouard-chin/c8e39148c0cdf324dae827716fbe24a0
### Context
A while ago in #869, Aaron introduced a connection pool which
greatly improved Bundler speed. It was noted in the PR description
that managing one connection was already good enough and it wasn't
clear whether we needed more connections. Aaron also had the
intuition that we may need to increase the pool for downloading
gems and he was right.
> We need to study how RubyGems uses connections and make a decision
> based on request usage (e.g. only use one connection for many small
> requests like bundler API, and maybe many connections for
> downloading gems)
When bundler downloads and installs gem in parallel https://github.com/ruby/rubygems/blob/4f85e02fdd89ee28852722dfed42a13c9f5c9193/bundler/lib/bundler/installer/parallel_installer.rb#L128
most threads have to wait for the only connection in the pool to be
available which is not efficient.
### Solution
This commit modifies the pool size for the fetcher that Bundler
uses. RubyGems fetcher will continue to use a single connection.
The bundler fetcher is used in 2 places.
1. When downloading gems https://github.com/ruby/rubygems/blob/4f85e02fdd89ee28852722dfed42a13c9f5c9193/bundler/lib/bundler/source/rubygems.rb#L481-L484
2. When grabing the index (not the compact index) using the
`bundle install --full-index` flag.
https://github.com/ruby/rubygems/blob/4f85e02fdd89ee28852722dfed42a13c9f5c9193/bundler/lib/bundler/fetcher/index.rb#L9
Having more connections in 2) is not any useful but tweaking the
size based on where the fetcher is used is a bit tricky so I opted
to modify it at the class level.
I fiddle with the pool size and found that 5 seems to be the sweet
spot at least for my environment.
https://github.com/ruby/rubygems/commit/6063fd9963
|
|
https://github.com/ruby/rubygems/commit/5fd95f38d3
|
|
|
|
Performance/StringInclude cop
https://github.com/ruby/rubygems/commit/fdd3419144
|
|
|
|
- Depending on the native extension, it can greatly reduce compilation
time when executing recipes simultaneously.
For example on Prism:
```
# Before
time gem install prism
Building native extensions. This could take a while...
Successfully installed prism-1.6.0
1 gem installed
gem install prism 3.61s user 0.80s system 95% cpu 4.595 total
```
```
# After
time MAKEFLAGS="-j" gem install prism
Building native extensions. This could take a while...
Successfully installed prism-1.6.0
1 gem installed
MAKEFLAGS="-j" gem install prism 4.47s user 1.27s system 246% cpu 2.330 total
```
I don't think adding `-j` as a default is harmful, but I'm admitedly
not very knowledgable when it comes to compiler.
https://github.com/ruby/rubygems/commit/61340081c6
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
|
|
This pattern is extremely common across the ecosystem, I don't think
it's reasonable to deprecate it.
I understand the performance argument, but perhaps the dependency
resolution algorithm can use another method that is private API
and only works with two `Version` instance.
https://github.com/ruby/rubygems/commit/024b4b547a
|
|
|
|
If we use "system" variable in BUNDLE_VERSION on Bundler configuration,
we can use bundler version provided by system installation.
But the current logic returns the first activated version of bundler
like 2.7.2. It makes to confuse users.
https://github.com/ruby/rubygems/commit/4eb66d9549
|
|
Comparing version objects is a huge bottleneck in dependency solvers
(like inside Bundler). I would like to make comparing version objects
cheaper. Right now we support comparing version objects with strings by
trying to coerce the string to a version. So for example:
```ruby
Gem::Version.new("1") <=> "12"
```
I would like to deprecate and remove support for this feature so that we
can reduce the overhead of `def <=>`.
I'm not sure what version of RubyGems we could remove this from though.
https://github.com/ruby/rubygems/commit/81b7602183
|
|
This specifies the lockfile location. This allows for easy support
of different lockfiles per Ruby version or platform.
https://github.com/ruby/rubygems/commit/b54d65bc0a
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Co-authored-by: Colby Swandale <996377+colby-swandale@users.noreply.github.com>
|
|
This is useful, in case you're using Docker, and an upstream
Dockerfile sets BUNDLER_VERSION to something you don't want.
It's impossible to unset it... only override to be the empty
string.
https://github.com/ruby/rubygems/commit/ffa3eb9ac6
|
|
|
|
|
|
https://github.com/ruby/rubygems/commit/778426fb73
|
|
https://github.com/ruby/rubygems/commit/4e02243f66
|
|
https://github.com/ruby/rubygems/commit/b6deff99c9
|
|
https://github.com/ruby/rubygems/commit/07f2daf51e
|
|
It seems like we were trying to deprecate passing `nil` to
Gem::Version.new. This breaks existing code, and I don't think there is
a good reason to deprecate this usage.
I believe what we want to prevent is the following code:
```ruby
Gem::Specification.new do |spec|
spec.version = nil
# suddenly the spec version is 0!
p spec.version
end
```
This commit allows people to manually construct `Gem::Version.new(nil)`,
but when someone assigns `nil` as the Gem specification version, it sets
the spec version to `nil` (making the specification invalid). People
who manually construct `Gem::Version` objects and use nil should be
allowed to do it, and `Gem::Version.new(nil) == Gem::Version.new("0")`,
but people who assign `nil` in a gemspec will get an invalid gemspec.
I think deprecation started
[here](https://github.com/ruby/rubygems/pull/2203) but there doesn't
seem to be a reason to do it.
Fixes https://github.com/ruby/rubygems/pull/9052
https://github.com/ruby/rubygems/commit/ded5e909c2
|
|
https://github.com/ruby/rubygems/commit/3471646d43
|
|
https://github.com/ruby/rubygems/commit/a49d315ecd
|
|
and has_rdoc?
https://github.com/ruby/rubygems/commit/b043538576
|
|
https://github.com/ruby/rubygems/commit/9b19e1f555
|
|
https://github.com/ruby/rubygems/commit/de269cfbb6
|
|
Gem::DependencyInstaller#find_gems_with_sources
https://github.com/ruby/rubygems/commit/1b3f3bf194
|
|
https://github.com/ruby/rubygems/commit/94d4e633d1
|
|
https://github.com/ruby/rubygems/commit/728269cc4a
|
|
validate_dependencies and validate_permissions
https://github.com/ruby/rubygems/commit/fbf38fc190
|
|
https://github.com/ruby/rubygems/commit/84ceaff1b7
|
|
https://github.com/ruby/rubygems/commit/f4b4f12f91
|
|
https://github.com/ruby/rubygems/commit/96cef34041
|
|
Gem::BasicSpecification.default_specifications_dir
https://github.com/ruby/rubygems/commit/60f0b87d47
|
|
https://github.com/ruby/rubygems/commit/be3b09c786
|
|
https://github.com/ruby/rubygems/commit/18f64c6b29
|
|
https://github.com/ruby/rubygems/commit/b59917447c
|
|
This reverts commit 136157e772ab2b2ea08555d0ad821da7dc2bde96.
|
|
I would like to start making some of the methods in Gem::Package
private so that we can refactor them better. Right now we have many
methods that are public, and since they are public we can't refactor
them. Historically, I think "private" methods have just been tagged
with :nodoc:, but I would like to be more strict about our APIs
https://github.com/ruby/rubygems/commit/fb352e9176
|
