summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)Author
2022-08-31[ruby/net-protocol] Improve BufferedIO performanceJean Boussier
`BufferedIO` is a bit inefficient for reading large responses because it use the classic `buffer.slice!` technique which cause a lot of unnecessary string copying. This is particularly visible on line based protocol when reading line by line. Instead of repeatedly shifting the string, we can keep track of which offset we're at, to know how many bytes are left in the buffer. This change also open the door to further optimization by increasing the buffer size, as previously `slice!` would get slower the larger the buffer is. Benchmark results: ``` === 1k === Warming up -------------------------------------- 1k 1.234k i/100ms 1k opt 1.283k i/100ms Calculating ------------------------------------- 1k 12.615k (± 0.9%) i/s - 64.168k in 5.086995s 1k opt 12.856k (± 0.9%) i/s - 65.433k in 5.090051s Comparison: 1k: 12615.2 i/s 1k opt: 12856.0 i/s - 1.02x (± 0.00) faster === 10k === Warming up -------------------------------------- 10k 1.165k i/100ms 10k opt 1.269k i/100ms Calculating ------------------------------------- 10k 11.550k (± 2.4%) i/s - 58.250k in 5.046378s 10k opt 12.736k (± 1.0%) i/s - 64.719k in 5.081969s Comparison: 10k: 11550.3 i/s 10k opt: 12736.3 i/s - 1.10x (± 0.00) faster === 100k === Warming up -------------------------------------- 100k 809.000 i/100ms 100k opt 926.000 i/100ms Calculating ------------------------------------- 100k 8.054k (± 3.0%) i/s - 40.450k in 5.028299s 100k opt 9.286k (± 2.2%) i/s - 47.226k in 5.088841s Comparison: 100k: 8053.6 i/s 100k opt: 9285.5 i/s - 1.15x (± 0.00) faster === 1M === Warming up -------------------------------------- 1M 249.000 i/100ms 1M opt 315.000 i/100ms Calculating ------------------------------------- 1M 2.448k (± 2.5%) i/s - 12.450k in 5.089744s 1M opt 3.119k (± 2.6%) i/s - 15.750k in 5.053772s Comparison: 1M: 2447.8 i/s 1M opt: 3118.8 i/s - 1.27x (± 0.00) faster ``` Profiling before (1MB responses): ``` ================================== Mode: wall(1000) Samples: 5276 (0.00% miss rate) GC: 394 (7.47%) ================================== TOTAL (pct) SAMPLES (pct) FRAME 1622 (30.7%) 1622 (30.7%) IO#wait_readable 777 (14.7%) 777 (14.7%) IO#read_nonblock 365 (6.9%) 365 (6.9%) (sweeping) 2705 (51.3%) 364 (6.9%) Net::BufferedIO#rbuf_fill 264 (5.0%) 264 (5.0%) String#index 223 (4.2%) 223 (4.2%) String#sub 221 (4.2%) 221 (4.2%) String#slice! 185 (3.5%) 185 (3.5%) String#split 108 (2.0%) 108 (2.0%) IO#write_nonblock 101 (1.9%) 101 (1.9%) String#downcase 66 (1.3%) 66 (1.3%) Net::BufferedIO#LOG 57 (1.1%) 57 (1.1%) String#count 51 (1.0%) 51 (1.0%) String#to_s 391 (7.4%) 50 (0.9%) Net::HTTPGenericRequest#write_header 50 (0.9%) 50 (0.9%) String#capitalize 49 (0.9%) 49 (0.9%) Array#join 47 (0.9%) 47 (0.9%) String#b 106 (2.0%) 36 (0.7%) Net::HTTPHeader#set_field 34 (0.6%) 34 (0.6%) Module#=== 33 (0.6%) 33 (0.6%) String#[] 140 (2.7%) 29 (0.5%) Net::BufferedIO#write0 29 (0.5%) 29 (0.5%) (marking) 281 (5.3%) 27 (0.5%) Net::BufferedIO#rbuf_consume 1195 (22.6%) 25 (0.5%) Net::HTTPResponse#read_body 1024 (19.4%) 25 (0.5%) Net::HTTPResponse.each_response_header 86 (1.6%) 24 (0.5%) Net::HTTPHeader#set_field 23 (0.4%) 23 (0.4%) Net::HTTP#proxy_uri 51 (1.0%) 23 (0.4%) Net::HTTPHeader#initialize_http_header 2225 (42.2%) 22 (0.4%) Net::BufferedIO#readuntil 20 (0.4%) 20 (0.4%) Regexp#=== ``` Profiling after (1MB responses): ``` ================================== Mode: wall(1000) Samples: 15180 (0.00% miss rate) GC: 1688 (11.12%) ================================== TOTAL (pct) SAMPLES (pct) FRAME 4534 (29.9%) 4534 (29.9%) IO#read_nonblock 10650 (70.2%) 3944 (26.0%) Net::HTTPOpt::BufferedIOOpt#rbuf_fill 2101 (13.8%) 2101 (13.8%) IO#wait_readable 1442 (9.5%) 1442 (9.5%) (sweeping) 360 (2.4%) 360 (2.4%) String#sub 312 (2.1%) 312 (2.1%) String#split 265 (1.7%) 265 (1.7%) String#bytesize 246 (1.6%) 246 (1.6%) (marking) 151 (1.0%) 151 (1.0%) IO#write_nonblock 125 (0.8%) 125 (0.8%) String#downcase 116 (0.8%) 116 (0.8%) String#index 113 (0.7%) 113 (0.7%) Module#=== 162 (1.1%) 89 (0.6%) Net::HTTPOpt::BufferedIOOpt#rbuf_consume_all_shareable! 158 (1.0%) 65 (0.4%) Net::HTTPHeader#set_field 63 (0.4%) 63 (0.4%) String#capitalize 63 (0.4%) 63 (0.4%) BasicObject#equal? 58 (0.4%) 58 (0.4%) Regexp#match 58 (0.4%) 58 (0.4%) String#[] 449 (3.0%) 56 (0.4%) Net::HTTPGenericRequest#write_header 53 (0.3%) 53 (0.3%) String#to_s 52 (0.3%) 52 (0.3%) Net::HTTPOpt::BufferedIOOpt#LOG 52 (0.3%) 52 (0.3%) String#count 44 (0.3%) 44 (0.3%) String#byteslice 44 (0.3%) 44 (0.3%) Array#join 1096 (7.2%) 42 (0.3%) Net::HTTPResponse.each_response_header 2617 (17.2%) 40 (0.3%) Net::HTTPOpt::BufferedIOOpt#readuntil 132 (0.9%) 30 (0.2%) Net::HTTPOpt::BufferedIOOpt#rbuf_consume 28 (0.2%) 28 (0.2%) Regexp#=== 27 (0.2%) 27 (0.2%) Net::HTTP#proxy_uri 8862 (58.4%) 27 (0.2%) Net::HTTPResponse#read_body ```` Benchmark code: ```ruby require "fileutils" DIR = "/tmp/www" FileUtils.mkdir_p(DIR) HOST = "127.0.0.1" PORT = 8080 CONF = <<~EOS daemon off; worker_processes 2; events { worker_connections 128; } http { server_tokens off; charset utf-8; server { server_name localhost; listen #{HOST}:#{PORT}; keepalive_requests 10000000; keepalive_timeout 3600s; error_page 500 502 503 504 /50x.html; location / { root #{DIR}; } } } EOS File.write(File.join(DIR, "1k.txt"), 'a' * 1024) File.write(File.join(DIR, "10k.txt"), 'a' * 1024 * 10) File.write(File.join(DIR, "100k.txt"), 'a' * 1024 * 100) File.write(File.join(DIR, "1M.txt"), 'a' * 1024 * 1024) File.write(File.join(DIR, "nginx.conf"), CONF) require "benchmark/ips" require "net/http" nginx_pid = Process.spawn('nginx', '-c', File.join(DIR, "nginx.conf")) module Net class HTTPOpt < HTTP class BufferedIOOpt < ::Net::BufferedIO #:nodoc: internal use only def initialize(io, read_timeout: 60, write_timeout: 60, continue_timeout: nil, debug_output: nil) @io = io @read_timeout = read_timeout @write_timeout = write_timeout @continue_timeout = continue_timeout @debug_output = debug_output @rbuf = ''.b @rbuf_offset = 0 end attr_reader :io attr_accessor :read_timeout attr_accessor :write_timeout attr_accessor :continue_timeout attr_accessor :debug_output def inspect "#<#{self.class} io=#{@io}>" end def eof? @io.eof? end def closed? @io.closed? end def close @io.close end # # Read # public def read(len, dest = ''.b, ignore_eof = false) LOG "reading #{len} bytes..." read_bytes = 0 begin while read_bytes + rbuf_size < len if s = rbuf_consume_all_shareable! read_bytes += s.bytesize dest << s end rbuf_fill end s = rbuf_consume(len - read_bytes) read_bytes += s.bytesize dest << s rescue EOFError raise unless ignore_eof end LOG "read #{read_bytes} bytes" dest end def read_all(dest = ''.b) LOG 'reading all...' read_bytes = 0 begin while true if s = rbuf_consume_all_shareable! read_bytes += s.bytesize dest << s end rbuf_fill end rescue EOFError ; end LOG "read #{read_bytes} bytes" dest end def readuntil(terminator, ignore_eof = false) offset = @rbuf_offset begin until idx = @rbuf.index(terminator, offset) offset = @rbuf.bytesize rbuf_fill end return rbuf_consume(idx + terminator.bytesize - @rbuf_offset) rescue EOFError raise unless ignore_eof return rbuf_consume end end def readline readuntil("\n").chop end private BUFSIZE = 1024 * 16 def rbuf_fill tmp = @rbuf_empty ? @rbuf : nil case rv = @io.read_nonblock(BUFSIZE, tmp, exception: false) when String @rbuf_empty = false if rv.equal?(tmp) @rbuf_offset = 0 else @rbuf << rv rv.clear end return when :wait_readable (io = @io.to_io).wait_readable(@read_timeout) or raise Net::ReadTimeout.new(io) # continue looping when :wait_writable # OpenSSL::Buffering#read_nonblock may fail with IO::WaitWritable. # http://www.openssl.org/support/faq.html#PROG10 (io = @io.to_io).wait_writable(@read_timeout) or raise Net::ReadTimeout.new(io) # continue looping when nil raise EOFError, 'end of file reached' end while true end def rbuf_flush if @rbuf_empty @rbuf.clear @rbuf_offset = 0 end nil end def rbuf_size @rbuf.bytesize - @rbuf_offset end # Warning: this method may share the buffer to avoid # copying. The caller must no longer use the returned # string once rbuf_fill has been called again def rbuf_consume_all_shareable! @rbuf_empty = true buf = if @rbuf_offset == 0 @rbuf else @rbuf.byteslice(@rbuf_offset..-1) end @rbuf_offset = @rbuf.bytesize buf end def rbuf_consume(len = nil) if @rbuf_offset == 0 && (len.nil? || len == @rbuf.bytesize) s = @rbuf @rbuf = ''.b @rbuf_offset = 0 @rbuf_empty = true elsif len.nil? s = @rbuf.byteslice(@rbuf_offset..-1) @rbuf = ''.b @rbuf_offset = 0 @rbuf_empty = true else s = @rbuf.byteslice(@rbuf_offset, len) @rbuf_offset += len @rbuf_empty = @rbuf_offset == @rbuf.bytesize rbuf_flush end @debug_output << %Q[-> #{s.dump}\n] if @debug_output s end # # Write # public def write(*strs) writing { write0(*strs) } end alias << write def writeline(str) writing { write0 str + "\r\n" } end private def writing @written_bytes = 0 @debug_output << '<- ' if @debug_output yield @debug_output << "\n" if @debug_output bytes = @written_bytes @written_bytes = nil bytes end def write0(*strs) @debug_output << strs.map(&:dump).join if @debug_output orig_written_bytes = @written_bytes strs.each_with_index do |str, i| need_retry = true case len = @io.write_nonblock(str, exception: false) when Integer @written_bytes += len len -= str.bytesize if len == 0 if strs.size == i+1 return @written_bytes - orig_written_bytes else need_retry = false # next string end elsif len < 0 str = str.byteslice(len, -len) else # len > 0 need_retry = false # next string end # continue looping when :wait_writable (io = @io.to_io).wait_writable(@write_timeout) or raise Net::WriteTimeout.new(io) # continue looping end while need_retry end end # # Logging # private def LOG_off @save_debug_out = @debug_output @debug_output = nil end def LOG_on @debug_output = @save_debug_out end def LOG(msg) return unless @debug_output @debug_output << msg + "\n" end end BufferedIO = BufferedIOOpt # Unchanged from ruby 3.1.1, only allow to lookup the mofidied BufferedIO def connect if use_ssl? # reference early to load OpenSSL before connecting, # as OpenSSL may take time to load. @ssl_context = OpenSSL::SSL::SSLContext.new end if proxy? then conn_addr = proxy_address conn_port = proxy_port else conn_addr = conn_address conn_port = port end D "opening connection to #{conn_addr}:#{conn_port}..." begin s = Socket.tcp conn_addr, conn_port, @local_host, @local_port, connect_timeout: @open_timeout rescue => e e = Net::OpenTimeout.new(e) if e.is_a?(Errno::ETIMEDOUT) #for compatibility with previous versions raise e, "Failed to open TCP connection to " + "#{conn_addr}:#{conn_port} (#{e.message})" end s.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1) D "opened" if use_ssl? if proxy? plain_sock = BufferedIO.new(s, read_timeout: @read_timeout, write_timeout: @write_timeout, continue_timeout: @continue_timeout, debug_output: @debug_output) buf = "CONNECT #{conn_address}:#{@port} HTTP/#{HTTPVersion}\r\n" buf << "Host: #{@address}:#{@port}\r\n" if proxy_user credential = ["#{proxy_user}:#{proxy_pass}"].pack('m0') buf << "Proxy-Authorization: Basic #{credential}\r\n" end buf << "\r\n" plain_sock.write(buf) HTTPResponse.read_new(plain_sock).value # assuming nothing left in buffers after successful CONNECT response end ssl_parameters = Hash.new iv_list = instance_variables SSL_IVNAMES.each_with_index do |ivname, i| if iv_list.include?(ivname) value = instance_variable_get(ivname) unless value.nil? ssl_parameters[SSL_ATTRIBUTES[i]] = value end end end @ssl_context.set_params(ssl_parameters) @ssl_context.session_cache_mode = OpenSSL::SSL::SSLContext::SESSION_CACHE_CLIENT | OpenSSL::SSL::SSLContext::SESSION_CACHE_NO_INTERNAL_STORE @ssl_context.session_new_cb = proc {|sock, sess| @ssl_session = sess } D "starting SSL for #{conn_addr}:#{conn_port}..." s = OpenSSL::SSL::SSLSocket.new(s, @ssl_context) s.sync_close = true # Server Name Indication (SNI) RFC 3546 s.hostname = @address if s.respond_to? :hostname= if @ssl_session and Process.clock_gettime(Process::CLOCK_REALTIME) < @ssl_session.time.to_f + @ssl_session.timeout s.session = @ssl_session end ssl_socket_connect(s, @open_timeout) if (@ssl_context.verify_mode != OpenSSL::SSL::VERIFY_NONE) && @ssl_context.verify_hostname s.post_connection_check(@address) end D "SSL established, protocol: #{s.ssl_version}, cipher: #{s.cipher[0]}" end @socket = BufferedIO.new(s, read_timeout: @read_timeout, write_timeout: @write_timeout, continue_timeout: @continue_timeout, debug_output: @debug_output) @last_communicated = nil on_connect rescue => exception if s D "Conn close because of connect error #{exception}" s.close end raise end private :connect end end begin sleep 0.2 connection = Net::HTTP.start(HOST, PORT) connection.keep_alive_timeout = 3600 connection_opt = Net::HTTPOpt.start(HOST, PORT) connection_opt.keep_alive_timeout = 3600 unless connection.request_get("/100k.txt").body == connection_opt.request_get("/100k.txt").body abort("bug?") end if ARGV.first == "profile" require 'stackprof' require 'json' StackProf.run(mode: :wall, out: "/tmp/stackprof-net-http.dump", raw: true) do 40_000.times do connection.request_get("/1M.txt").body end end File.write("/tmp/stackprof-net-http.json", JSON.dump(Marshal.load(File.binread("/tmp/stackprof-net-http.dump")))) system("stackprof", "/tmp/stackprof-net-http.rb") StackProf.run(mode: :wall, out: "/tmp/stackprof-net-http-opt.dump", raw: true) do 40_000.times do connection_opt.request_get("/1M.txt").body end end File.write("/tmp/stackprof-net-http-opt.json", JSON.dump(Marshal.load(File.binread("/tmp/stackprof-net-http-opt.dump")))) system("stackprof", "/tmp/stackprof-net-http-opt.dump") else %w(1k 10k 100k 1M).each do |size| puts "=== #{size} ===" Benchmark.ips do |x| path = "/#{size}.txt" x.report("#{size}") { connection.request_get(path).body } x.report("#{size} opt") { connection_opt.request_get(path).body } x.compare!(order: :baseline) end puts end end ensure Process.kill('TERM', nginx_pid) Process.wait(nginx_pid) end ``` https://github.com/ruby/net-protocol/commit/781e400389
2022-08-29[rubygems/rubygems] Add platform :windows as a shortcut for all Windows ↵shields
platforms https://github.com/rubygems/rubygems/commit/f3c49ad3f7
2022-08-28[rubygems/rubygems] Bundler: make to_lock consistent between ↵Aleksandr Varnin
Gem::Dependency and Bundler::Dependency https://github.com/rubygems/rubygems/commit/971d57cf5a
2022-08-26[ruby/irb] Require RDoc in `input-method.rb` again in a limited scope.Jun Aruga
RDoc is implemented as soft dependency in IRB. See how the rdoc is required in the files. I reverted the commit below. ``` $ grep -ril rdoc lib/ lib/irb/cmd/help.rb lib/irb/completion.rb lib/irb/easter-egg.rb lib/irb/input-method.rb ``` --- Revert "Remove `require` in signal handler to avoid ThreadError" This reverts commit https://github.com/ruby/irb/commit/5f749c613c89. https://github.com/ruby/irb/commit/b24852058f
2022-08-25[rubygems/rubygems] Fix another regression for sorbetDavid Rodríguez
Recently a changed was introduced to update the resolver platforms after it has been created, in order to remove the "ruby" platform from it if it's to be removed from the lockfile. However, it did not update the `@resolving_only_for_ruby` instance variable in that case, so the resolver was not properly doing the right thing anymore. To fix this, I tweaked the code to restore not changing resolver platforms after the resolver has been instantiated. https://github.com/rubygems/rubygems/commit/8fbc30a1d0
2022-08-24[rubygems/rubygems] Backport non-gnu libc on linux platform matching to BundlerDavid Rodríguez
https://github.com/rubygems/rubygems/commit/703373b41f Co-authored-by: Loic Nageleisen <loic.nageleisen@gmail.com>
2022-08-23[ruby/irb] Drop hard dependency on RDoc.Vít Ondruch
This has been introduced in https://github.com/ruby/irb/commit/026700499dfd, but it seems that this is just be mistake, otherwise the later handling of `LoadError` would not be needed. https://github.com/ruby/irb/commit/54c8df06ff
2022-08-23[ruby/fileutils] FileUtils.rm* methods swallows only Errno::ENOENT when ↵Yusuke Endoh
force is true ... instead of any StandardError. To behave like the standard `rm` command, it should only ignore exceptions about not existing files, not every exception. This should make debugging some errors easier, because the expectation is that `rm -rf` will succeed if and only if, all given files (previously existent or not) are removed. However, due to this exception swallowing, this is not always the case. From the `rm` man page > COMPATIBILITY > > The rm utility differs from historical implementations in that the -f > option only masks attempts to remove non-existent files instead of > masking a large variety of errors. https://github.com/ruby/fileutils/commit/fa65d676ec Co-Authored-By: David Rodríguez <deivid.rodriguez@riseup.net>
2022-08-23[ruby/fileutils] Narrow the scope of ensureYusuke Endoh
The ensure in postorder_traverse was added for [Bug #6756]. The intention was to try to delete the parent directory if it failed to get the children. (It may be possible to delete the directory if it is empty.) However, the ensure region rescue'ed not only "failure to get children" but also "failure to delete each child". Thus, the following raised Errno::ENOTEMPTY, but we expect it to raise Errno::EACCES. ``` $ mkdir foo $ touch foo/bar $ chmod 555 foo $ ruby -rfileutils -e 'FileUtils.rm_rf("foo")' ``` This changeset narrows the ensure region so that it rescues only "failure to get children". https://github.com/ruby/fileutils/commit/ec5d3b84ea
2022-08-23[rubygems/rubygems] Bundler: update the link suggested on error with the new oneTakuya Noguchi
Also typo is fixed. Signed-off-by: Takuya Noguchi <takninnovationresearch@gmail.com> https://github.com/rubygems/rubygems/commit/9c1ea52ddf
2022-08-23Merge rubygems/bundler HEADHiroshi SHIBATA
Pick from https://github.com/rubygems/rubygems/commit/6b3a5a9ab0453463381a8164efb6298ea9eb776f Notes: Merged: https://github.com/ruby/ruby/pull/6268
2022-08-23[rubygems/rubygems] Fix conservative update downgrading top level gemsDavid Rodríguez
When `--conservative` is passed, explicit unlocks are set for top level gems via `@unlock[:gems]`, so that only those particular gems are allowed to be updated. When we compute the "base resolve" from the lockfile (the set of gems whose versions should be kept pinned by the resolver), we always exclude gems explicitly unlocked through `@unlock[:gems]` from it. This is done by the `converge_specs` method. However, the `converge_specs` method is also used for figuring out additional lower bound requirements from the lockfile. But in this case, even if gems are explicitly unlock in `@unlock[:gems]`, we still want to add the additional requirement, so that gems are not downgraded by the resolver. So the solution is to move the line filtering out gems in `@unlock[:gems]` from the `converged_specs` method out of that method, so that it only applies for computing the "base resolve", but not the addtional lower bound requirements. https://github.com/rubygems/rubygems/commit/405119bd7b Notes: Merged: https://github.com/ruby/ruby/pull/6268
2022-08-23[rubygems/rubygems] Refactor building metadata dependenciesDavid Rodríguez
https://github.com/rubygems/rubygems/commit/fa60f1fe43 Notes: Merged: https://github.com/ruby/ruby/pull/6268
2022-08-23[rubygems/rubygems] Remove unnecessary mixin inclusionDavid Rodríguez
It's already included by the parent. https://github.com/rubygems/rubygems/commit/3ffe389c44 Notes: Merged: https://github.com/ruby/ruby/pull/6268
2022-08-23[rubygems/rubygems] Centralize loading `Bundler::MatchPlatform` mixinDavid Rodríguez
It's explicitly loaded when monkeypatching RubyGems, which we do very early. So neither autoloading it, nor explicitly loading it anywhere else is necessary. https://github.com/rubygems/rubygems/commit/fbc7a57161 Notes: Merged: https://github.com/ruby/ruby/pull/6268
2022-08-23[rubygems/rubygems] Implement extra rules for libc versioningDavid Rodríguez
https://github.com/rubygems/rubygems/commit/7e976d790a
2022-08-23[rubygems/rubygems] Fix `gem install` still choosing musl incorrectlyDavid Rodríguez
https://github.com/rubygems/rubygems/commit/1b9f7f50a5
2022-08-23[rubygems/rubygems] Handle non-gnu libc on linux platforms in RubyGemsDavid Rodríguez
Attempting to install a gem published as both *-linux and *-linux-musl results in the incorrect gem being picked up, causing build failures due to binary incompatibility. This is caused by the `nil` wildcard swallowing the libc information upon version comparison. Handle the linux case by performing only non-wildcard equality on the version and asserting 'gnu' and nil equivalence, while preserving the current behaviour for other OSes. https://github.com/rubygems/rubygems/commit/9eead86abc Co-authored-by: Loic Nageleisen <loic.nageleisen@gmail.com>
2022-08-23[rubygems/rubygems] Test platform's version-ness consistentlyLoic Nageleisen
The symmetry with the "for command line" case is made more apparent. https://github.com/rubygems/rubygems/commit/ab85d3558f
2022-08-22[ruby/fileutils] Fix mkdir_p hanging on Windows when trying to create a file ↵David Rodríguez
on a offline drive https://github.com/ruby/fileutils/commit/9cc6a082d7
2022-08-21[rubygems/rubygems] Fix edge case where `bundler/inline` unintentionally ↵David Rodríguez
skips install If the application has the `no_install` setting set for `bundle package`, then `bundler/inline` would silently skip installing any gems. https://github.com/rubygems/rubygems/commit/7864f49b27
2022-08-21[rubygems/rubygems] Fix `bundle platform` crash when there's a lockfile with ↵David Rodríguez
no Ruby locked https://github.com/rubygems/rubygems/commit/49fc54e87d
2022-08-19[ruby/error_highlight] Apply ErrorHighlight::CoreExt to TypeError and ↵Yusuke Endoh
ArgumentError https://github.com/ruby/error_highlight/commit/defcaf1beb
2022-08-19Sync SyntaxSuggestschneems
``` $ tool/sync_default_gems.rb syntax_suggest ``` Notes: Merged: https://github.com/ruby/ruby/pull/5859
2022-08-16[ruby/cgi] Implement `CGI.url_encode` and `CGI.url_decode`Jean Boussier
[Feature #18822] Ruby is somewhat missing an RFC 3986 compliant escape method. https://github.com/ruby/cgi/commit/c2729c7f33
2022-08-15[rubygems/rubygems] Fix Ruby platform incorrectly removed on `bundle update`David Rodríguez
https://github.com/rubygems/rubygems/commit/0d321c9e3a
2022-08-12[ruby/error_highlight] Add a note about the current limitation of ↵Yusuke Endoh
ErrorHighlight.spot https://github.com/ruby/error_highlight/commit/489ce80a62
2022-08-10[ruby/error_highlight] Bump versionYusuke Endoh
https://github.com/ruby/error_highlight/commit/6edf0a0a5d
2022-08-10[ruby/error_highlight] Make backtrace_location keyword workYusuke Endoh
We had to keep backtrace_location before opts is overwritten. https://github.com/ruby/error_highlight/commit/2735e4681a
2022-08-10[ruby/error_highlight] Make ErrorHighlight.spot accept Exception ↵Yusuke Endoh
(https://github.com/ruby/error_highlight/pull/25) ... and move things from core_ext.rb to base.rb. This will confine CRuby-dependent things to ErrorHighlight.spot. https://github.com/ruby/error_highlight/commit/22d1dd7824
2022-08-09Merge rubygems/bundler HEAD.Hiroshi SHIBATA
Pick from https://github.com/rubygems/rubygems/commit/dfbb5a38114640e0d8d616861607f3de73ee0199 Notes: Merged: https://github.com/ruby/ruby/pull/6224
2022-08-08[ruby/rdoc] Allow multiple footnotes without in-between blank linesNobuyoshi Nakada
https://github.com/ruby/ruby/commit/e4e054e3ce40 used four footnotes without blank lines. And the ChangeLog generated from that commit resulted in ``undefined method `parts' for nil`` error. For now, let a footnote terminated by the next footnote mark. Also refined the error message when undefined footnote is used. https://github.com/ruby/rdoc/commit/a7f290130b
2022-08-07[ruby/rdoc] [DOC] Removes remaining old Markup Reference ↵Burdette Lamar
(https://github.com/ruby/rdoc/pull/910) https://github.com/ruby/rdoc/commit/4e44c9c6cf
2022-08-06[rubygems/rubygems] Fix yanked gems being unintentionally update when other ↵David Rodríguez
gems are unlocked This is a regression from a change intended to raise errors when user puts a gem under an incorrect source in the Gemfile by mistake. To fix the issue, we revert the change that caused it and implement it in a different way that restores the resolver independency from real specifications. Now it deals only with names and versions and does not try to materialize anything into real specifications before resolving. https://github.com/rubygems/rubygems/commit/d2bf1b86eb
2022-08-06[rubygems/rubygems] Move comment where the actual replacement happensDavid Rodríguez
https://github.com/rubygems/rubygems/commit/d60acdf80d
2022-08-06[rubygems/rubygems] Remove unclear commentDavid Rodríguez
https://github.com/rubygems/rubygems/commit/3a843c1ac7
2022-08-06[rubygems/rubygems] Extract `SourceList#get_with_fallback`David Rodríguez
https://github.com/rubygems/rubygems/commit/9dbc4757a8
2022-08-05[rubygems/rubygems] Include backtrace with crashes by defaultDavid Rodríguez
https://github.com/rubygems/rubygems/commit/3cc3bfd371
2022-08-05[rubygems/rubygems] Automatically remove "ruby" from lockfile if incompleteDavid Rodríguez
https://github.com/rubygems/rubygems/commit/69d0b4e10b
2022-08-04Use $(bindir) for path to executable in mkmfAlan Wu
For the macOS -bundle_loader linker option, we need a path to the Ruby exectuable. $(RUBY) is not necessarily a path since it could be a command line invocation. That happens during build with runruby.rb and can happen post installation if the user passes the --ruby option to a extconf.rb. Use $(bindir) to locate the executable instead. Before installation, $(bindir) doesn't exist, so we need to be able to override $(BUILTRUBY) in such situations so test-spec and bundled extensions could build. Use a new mkmf global, $builtruby, to do this; set it in fake.rb and in extmk.rb. Our build system is quite complex... Notes: Merged: https://github.com/ruby/ruby/pull/6193
2022-08-04Link ext bundles with bundle loader option for newer ld64Yuta Saito
ld64 shipped with Xcode 14 emits a warning when using `-undefined dynamic_lookup`. ``` ld: warning: -undefined dynamic_lookup may not work with chained fixups ``` Actually, `-undefined dynamic_lookup` doesn't work when: 1. Link a *shared library* with the option 2. Link it with a program that uses the chained-fixup introduced from macOS 12 and iOS 15 because `-undefined dynamic_lookup` uses lazy-bindings and they won't be bound while dyld fixes-up by traversing chained-fixup info. However, we build exts as *bundles* and they are loaded only through `dlopen`, so it's safe to use `-undefined dynamic_lookup` in theory. So the warning produced by ld64 is false-positive, and it results failure of option checking in configuration. Therefore, it would be an option to ignore the warning during our configuration. On the other hand, `-undefined dynamic_lookup` is already deprecated on all darwin platforms except for macOS, so it's good time to get rid of the option. ld64 also provides `-bundle_loader <executable>` option, which allows to resolve symbols defined in the executable symtab while linking. It behaves almost the same with `-undefined dynamic_lookup`, but it makes the following changes: 1. Require that unresolved symbols among input objects must be defined in the executable. 2. Lazy symbol binding will lookup only the symtab of the bundle loader executable. (`-undefined dynamic_lookup` lookups all symtab as flat namespace) This patch adds `-bundle_loader $(RUBY)` when non-EXTSTATIC configuration by assuming ruby executable can be linked before building exts. See "New Features" subsection under "Linking" section for chained fixup https://developer.apple.com/documentation/xcode-release-notes/xcode-13-release-notes Notes: Merged: https://github.com/ruby/ruby/pull/6193
2022-08-04[rubygems/rubygems] Warn dangling symlinksDavid Rodríguez
https://github.com/rubygems/rubygems/commit/425b78637f
2022-08-04[rubygems/rubygems] Extract entry.full_name to a variableDavid Rodríguez
https://github.com/rubygems/rubygems/commit/3973773005
2022-08-04[ruby/irb] shortcut colorize_code to speedup pretty_printtompng
https://github.com/ruby/irb/commit/8a074a6904
2022-08-03[rubygems/rubygems] Fix `bundle outdated --strict`David Rodríguez
It should be an alias of `--filter-strict`. `--update-strict` is essentially a dummy option with no special behavior associated and should be deprecated. https://github.com/rubygems/rubygems/commit/ec1e5d83c8
2022-08-03[rubygems/rubygems] Fix conservative updates regardless of `--strict`David Rodríguez
https://github.com/rubygems/rubygems/commit/c9a1d69a8d
2022-08-03Merge rubygems/bundler HEADHiroshi SHIBATA
Pick from https://github.com/rubygems/rubygems/commit/8331e63263081a6aa690d8025d2957f30c4e814a Notes: Merged: https://github.com/ruby/ruby/pull/6209
2022-08-03[rubygems/rubygems] fix platform matching for index specsIlya Dyakonov
https://github.com/rubygems/rubygems/commit/f087f1b590
2022-08-03[rubygems/rubygems] Warn (rather than crash) when setting `nil` ↵David Rodríguez
specification versions https://github.com/rubygems/rubygems/commit/a4ba1a4d97
2022-08-02[rubygems/rubygems] Array is already uniq, no need to deduplicate itDavid Rodríguez
https://github.com/rubygems/rubygems/commit/3212ae14b7