summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)Author
2025-03-18[ruby/prism] Further refine string handling in the parser translatorEarlopain
Mostly around newlines and line continuation. * percent arrays need special backslash handling in the ast * Fix offset issue for heredocs with many line continuations (used wrong variable as index access) * More refined rules on when to simplify string tokens * Handle line continuations in squiggly heredocs * Correctly dedent squiggly heredocs with interpolation * Consider `':foo:` and `%s[foo]` to not be interpolation https://github.com/ruby/prism/commit/4edfe9d981
2025-03-18[ruby/prism] Add a custom builder class for the parser translatorEarlopain
I want to add new node types to the parser translator, for example `itblock`. The bulk of the work is already done by prism itself. In the `parser` builder, this would be a 5-line change at most but we don't control that here. Instead, we can add our own builder and either overwrite the few methods we need, or just inline the complete builder. I'm not sure yet which would be better. `rubocop-ast` uses its own builder for `parser`. For this to correctly work, it must explicitly choose to extend the prism builder and use it, same as it currently chooses to use a different parser when prism is used. I'd like to enforce that the builder for prism extends its custom one since it will lead to some pretty weird issues otherwise. But first, I'd like to change `rubocop-ast` to make use of this. https://github.com/ruby/prism/commit/b080e608a8
2025-03-18[ruby/prism] Fix parser translator when unescaping invalid utf8Earlopain
1. The string starts out as binary 2. `ち` is appended, forcing it back into utf-8 3. Some invalid byte sequences are tried to append > incompatible character encodings: UTF-8 and BINARY (ASCII-8BIT) This makes use of my wish to use `append_as_bytes`. Unfortunatly that method is rather new so it needs a fallback https://github.com/ruby/prism/commit/e31e94a775
2025-03-18[rubygems/rubygems] Speed up Version#<=> ~20-50% when lengths differHartley McGuire
Previously, the comparison code would loop through segments up to the longest of the two versions being compared. However, this is inefficient because once one version has more segments than the other we can do a lot less work. This commit optimizes the differing segment length case by specializing the logic once the iteration has passed the shorter of the two segment lengths. At this point we only need to continue looking at the longer version's segment, and we know that any String encountered means the version is less than (pre), and any non-zero Integer means the version is greater than. Benchmark: ``` { first: [Gem::Version.new("1.2.3"), Gem::Version.new("2.2.3")], second: [Gem::Version.new("1.2.3"), Gem::Version.new("1.3.3")], third: [Gem::Version.new("1.2.3"), Gem::Version.new("1.2.4")], length: [Gem::Version.new("1.2.3"), Gem::Version.new("1.2.3.4")], left_s_second: [Gem::Version.new("1.a.3"), Gem::Version.new("1.2.3")], left_s_third: [Gem::Version.new("1.2.a"), Gem::Version.new("1.2.3")], right_s_second: [Gem::Version.new("1.2.3"), Gem::Version.new("1.a.3")], right_s_third: [Gem::Version.new("1.2.3"), Gem::Version.new("1.2.a")], left_s_length: [Gem::Version.new("8.0.1.pre"), Gem::Version.new("8.0.1")], right_s_length: [Gem::Version.new("8.0.1"), Gem::Version.new("8.0.1.pre")], both_s: [Gem::Version.new("8.0.2.pre1"), Gem::Version.new("8.0.2.pre2")], }.each do |name, v| puts "== #{name} ==" raise name unless v[0].fast_comp(v[1]) == (v[0] <=> v[1]) Benchmark.ips do |x| x.report("fast") { v[0].fast_comp(v[1]) } x.report("original") { v[0] <=> v[1] } x.compare!(order: :baseline) end end == first == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 208.555k i/100ms original 199.789k i/100ms Calculating ------------------------------------- fast 2.075M (± 6.0%) i/s (481.93 ns/i) - 10.428M in 5.055818s original 2.045M (± 3.9%) i/s (488.94 ns/i) - 10.389M in 5.090034s Comparison: fast: 2075002.8 i/s original: 2045227.4 i/s - same-ish: difference falls within error == second == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 192.395k i/100ms original 183.000k i/100ms Calculating ------------------------------------- fast 1.892M (± 3.8%) i/s (528.62 ns/i) - 9.620M in 5.094104s original 1.824M (± 3.5%) i/s (548.11 ns/i) - 9.150M in 5.023163s Comparison: fast: 1891722.2 i/s original: 1824435.3 i/s - same-ish: difference falls within error == third == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 172.788k i/100ms original 162.934k i/100ms Calculating ------------------------------------- fast 1.719M (± 9.0%) i/s (581.72 ns/i) - 8.467M in 5.025861s original 1.638M (± 3.6%) i/s (610.36 ns/i) - 8.310M in 5.080344s Comparison: fast: 1719042.9 i/s original: 1638389.6 i/s - same-ish: difference falls within error == length == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 191.741k i/100ms original 155.952k i/100ms Calculating ------------------------------------- fast 1.920M (± 3.9%) i/s (520.74 ns/i) - 9.587M in 5.002328s original 1.576M (± 6.2%) i/s (634.42 ns/i) - 7.954M in 5.072507s Comparison: fast: 1920362.1 i/s original: 1576240.9 i/s - 1.22x slower == left_s_second == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 176.441k i/100ms original 164.879k i/100ms Calculating ------------------------------------- fast 1.609M (± 7.3%) i/s (621.51 ns/i) - 8.116M in 5.083414s original 1.620M (± 8.3%) i/s (617.43 ns/i) - 8.079M in 5.028525s Comparison: fast: 1608994.8 i/s original: 1619606.5 i/s - same-ish: difference falls within error == left_s_third == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 160.562k i/100ms original 152.799k i/100ms Calculating ------------------------------------- fast 1.591M (± 3.6%) i/s (628.40 ns/i) - 8.028M in 5.052029s original 1.528M (± 3.6%) i/s (654.31 ns/i) - 7.640M in 5.007526s Comparison: fast: 1591334.1 i/s original: 1528320.6 i/s - same-ish: difference falls within error == right_s_second == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 135.938k i/100ms original 132.907k i/100ms Calculating ------------------------------------- fast 1.367M (± 1.2%) i/s (731.77 ns/i) - 6.933M in 5.074030s original 1.320M (± 2.4%) i/s (757.35 ns/i) - 6.645M in 5.036155s Comparison: fast: 1366548.7 i/s original: 1320386.4 i/s - same-ish: difference falls within error == right_s_third == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 129.971k i/100ms original 123.802k i/100ms Calculating ------------------------------------- fast 1.273M (± 4.1%) i/s (785.25 ns/i) - 6.369M in 5.011805s original 1.215M (± 1.8%) i/s (823.04 ns/i) - 6.190M in 5.096330s Comparison: fast: 1273487.0 i/s original: 1215002.9 i/s - same-ish: difference falls within error == left_s_length == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 211.093k i/100ms original 155.784k i/100ms Calculating ------------------------------------- fast 2.120M (± 1.9%) i/s (471.63 ns/i) - 10.766M in 5.079336s original 1.565M (± 6.7%) i/s (638.87 ns/i) - 7.789M in 5.007522s Comparison: fast: 2120296.1 i/s original: 1565258.0 i/s - 1.35x slower == right_s_length == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 213.977k i/100ms original 142.990k i/100ms Calculating ------------------------------------- fast 2.154M (± 1.3%) i/s (464.15 ns/i) - 10.913M in 5.066124s original 1.446M (± 1.8%) i/s (691.75 ns/i) - 7.292M in 5.046172s Comparison: fast: 2154455.3 i/s original: 1445607.9 i/s - 1.49x slower == both_s == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 154.903k i/100ms original 131.011k i/100ms Calculating ------------------------------------- fast 1.515M (± 4.0%) i/s (659.97 ns/i) - 7.590M in 5.019890s original 1.280M (± 5.3%) i/s (781.28 ns/i) - 6.420M in 5.035387s Comparison: fast: 1515223.3 i/s original: 1279957.8 i/s - 1.18x slower ``` https://github.com/rubygems/rubygems/commit/7195e77152
2025-03-17[ruby/optparse] Fix completion of key-value pairs arrayNobuyoshi Nakada
Enum array may be the list of pairs of key and value. Check if only key is completable, not pair. Fix https://github.com/ruby/optparse/pull/93 Fix https://github.com/ruby/optparse/pull/94 https://github.com/ruby/optparse/commit/a8d0ba8dac
2025-03-13[ruby/prism] Use Set.new over to_setKevin Newton
https://github.com/ruby/prism/commit/422d5c4c64
2025-03-13[ruby/prism] Use `reverse_each` in the parser translatorEarlopain
Avoids an array allocation which matters more and more the larger the file is. I have it at 14% of runtime. https://github.com/ruby/prism/commit/f65b90f27d
2025-03-13[ruby/prism] Optimize array inclusion checks in the parser translatorEarlopain
I see `Array.include?` as 2.4% runtime. Probably because of `LPAREN_CONVERSION_TOKEN_TYPES` but the others will be faster as well. Also remove some inline array checks. They are specifically optimized in Ruby since 3.4, but for now prism is for >= 2.7 https://github.com/ruby/prism/commit/ca9500a3fc
2025-03-13[ruby/prism] Ensure backwards compatibility with the custom parser builderEarlopain
Temoprary backwards-compat code so that current users don't break. Eventually the Translation::Parser initializer should asser that the correct class is passed in. https://github.com/ruby/prism/commit/66b0162b35
2025-03-13[rubygems/rubygems] Remove array allocation from Candidate#<=>Hartley McGuire
In a large application I profiled allocations while running `bundle update` and found that this method was ~60% of allocations while resolving (and Candidate#<=> is almost half of the total runtime). This commit removes the array allocation in Candidate#<=> (and similar methods since the implementations are so simple). The array is always the same two elements so they can just be compared directly. https://github.com/rubygems/rubygems/commit/6a7c411ba7
2025-03-12[ruby/prism] Revert "Mark extension as Ractor-safe"Kevin Newton
https://github.com/ruby/prism/commit/56eaf53732
2025-03-12[ruby/prism] Mark extension as Ractor-safeKevin Newton
https://github.com/ruby/prism/commit/10e5431b38
2025-03-12[rubygems/rubygems] Update docs for with/without consistencyJacob Atzen
The with and without flags accepts both comma and space separated values. https://github.com/rubygems/rubygems/commit/b6149f61e3
2025-03-12[rubygems/rubygems] Fix `ENAMETOOLONG` error when creating compact index cacheDavid Rodríguez
If a custom rubygems source URI is long enough, Bundler may end up raising an `ENAMETOOLONG` error and crash. This commit fixes the problem by trimming the cache slug size to fit usual OS requirements. https://github.com/rubygems/rubygems/commit/df40ff1e14 Co-authored-by: mbclu <mbclu@users.noreply.github.com> Co-authored-by: martinemde <martinemde@users.noreply.github.com>
2025-03-10[ruby/prism] Support `itblock` for `Prism::Translation::Parser`Koichi ITO
## Summary `itblock` node is added to support the `it` block parameter syntax introduced in Ruby 3.4. ```console $ ruby -Ilib -rprism -rprism/translation/parser34 -e 'buffer = Parser::Source::Buffer.new("path"); buffer.source = "proc { it }"; \ p Prism::Translation::Parser34.new.tokenize(buffer)[0]' s(:itblock, s(:send, nil, :proc), :it, s(:lvar, :it)) ``` This node design is similar to the `numblock` node, which was introduced for the numbered parameter syntax in Ruby 2.7. ``` $ ruby -Ilib -rprism -rprism/translation/parser34 -e 'buffer = Parser::Source::Buffer.new("path"); buffer.source = "proc { _1 }"; \ p Prism::Translation::Parser34.new.tokenize(buffer)[0]' s(:numblock, s(:send, nil, :proc), 1, s(:lvar, :_1)) ``` The difference is that while numbered parameters can have multiple parameters, the `it` block parameter syntax allows only a single parameter. In Ruby 3.3, the conventional node prior to the `it` block parameter syntax is returned. ```console $ ruby -Ilib -rprism -rprism/translation/parser33 -e 'buffer = Parser::Source::Buffer.new("path"); buffer.source = "proc { it }"; \ p Prism::Translation::Parser33.new.tokenize(buffer)[0]' s(:block, s(:send, nil, :proc), s(:args), s(:send, nil, :it)) ``` ## Development Note The Parser gem does not yet support the `it` block parameter syntax. This is the first case where Prism's node design precedes that of the Parser gem. When implementing https://github.com/whitequark/parser/issues/962, this node design will need to be taken into consideration. https://github.com/ruby/prism/commit/c141e1420a
2025-03-10[ruby/optparse] bump up to 0.7.0.dev.1 [ci skip]Nobuyoshi Nakada
https://github.com/ruby/optparse/commit/f4d64b0b17
2025-03-10[ruby/optparse] [DOC] Extract description from READMENobuyoshi Nakada
https://github.com/ruby/optparse/commit/83e8c23d68
2025-03-10[ruby/optparse] Fix LESS environment variable setup in OptionParser#help_exitKouhei Yanagita
If the original value of LESS ends with an option starting with "--", simply appending "Fe" would result in an invalid option string. https://github.com/ruby/optparse/commit/30571f91d3
2025-03-10[ruby/optparse] Make the result of `tty?` obtainable with flexible stdoutKoichi ITO
In mock testing for stdout, `StringIO.new` is sometimes used to redirect the output. In such cases, the assignment is done with `$stdout = StringIO.new`, not the constant `STDOUT`. e.g., https://github.com/rubocop/rubocop/blob/v1.71.1/lib/rubocop/rspec/shared_contexts.rb#L154-L164 After assigning `StringIO.new`, `$stdout.tty?` returns `false`, allowing the standard output destination to be switched during test execution. ```ruby STDOUT.tty? # => true StringIO.new.tty? # => false ``` However, since `STDOUT.tty?` returns `true`, a failure occurred in environments where the environment variables `RUBY_PAGER` or `PAGER` are set. e.g., https://github.com/rubocop/rubocop/pull/13784 To address this, `STDOUT` has been updated to `$stdout` so that the result of `tty?` can be flexibly overridden. A potential concern is that `$stdout`, unlike `STDOUT`, does not always represent the standard output at the time the Ruby process started. However, no concrete examples of issues related to this have been identified. `STDOUT.tty?` is the logic of optparse introduced in https://github.com/ruby/optparse/pull/70. This PR replaces `STDOUT` with `$stdout` throughout, based on the assumption that `$stdout` is sufficient for use with optparse. https://github.com/ruby/optparse/commit/262cf6f9ac
2025-03-10[ruby/optparse] Add post-check of valueNobuyoshi Nakada
Fix https://github.com/ruby/optparse/pull/80 https://github.com/ruby/optparse/commit/050a87d029
2025-03-10[ruby/optparse] Update argument check with save navigation operatorNobuyoshi Nakada
https://github.com/ruby/optparse/commit/71e2b31824
2025-03-10[ruby/optparse] Remove extra blank lines [ci skip]Nobuyoshi Nakada
https://github.com/ruby/optparse/commit/d7dec6808f
2025-03-10[ruby/optparse] [DOC] Update documents to use single quotes instead of ↵Nobuyoshi Nakada
backqoutes https://github.com/ruby/optparse/commit/5e71a70cb5
2025-03-10[rubygems/rubygems] Fix `gem rdoc` not working with newer versions of rdocDavid Rodríguez
https://github.com/rubygems/rubygems/commit/369f9b9311 Notes: Merged: https://github.com/ruby/ruby/pull/12890
2025-03-10[rubygems/rubygems] Don't treat a git-sourced gem install as complete if ↵Tara Bass
only the '.git' directory is present. This recovers cases where a git-sourced install can be left in a partially installed state. https://github.com/rubygems/rubygems/commit/d132b7008d Notes: Merged: https://github.com/ruby/ruby/pull/12890
2025-03-10[rubygems/rubygems] Switch inject to use shorthand hash syntaxSean Collins
https://github.com/rubygems/rubygems/commit/ba5a62fd04 Notes: Merged: https://github.com/ruby/ruby/pull/12890
2025-03-10[rubygems/rubygems] Use shorthand hash syntax for bundle addSean Collins
https://github.com/rubygems/rubygems/commit/9691097036 Notes: Merged: https://github.com/ruby/ruby/pull/12890
2025-03-09[ruby/optparse] Allow non-string enum list #79Nobuyoshi Nakada
Command line arguments are strings, convert enum list elements to strings to match. https://github.com/ruby/optparse/commit/c5ec052efc
2025-03-09[ruby/optparse] Use `\A` instead of `^` as the beginning of stringNobuyoshi Nakada
https://github.com/ruby/optparse/commit/a3f1029815
2025-03-09[ruby/optparse] [DOC] Manage rdoc options only in .rdoc_options fileNobuyoshi Nakada
Make `rdoc .` and `rake rdoc` consistent. https://github.com/ruby/optparse/commit/61b4ea0704
2025-03-03[rubygems/rubygems] docs(bundle-exec): recommend non-deprecated methodsMichael Chui
https://github.com/rubygems/rubygems/commit/3b4934fb69 Notes: Merged: https://github.com/ruby/ruby/pull/12840
2025-03-03[rubygems/rubygems] docs(bundle-config): hint default group when using only ↵Mateo
option https://github.com/rubygems/rubygems/commit/c258e45b44 Notes: Merged: https://github.com/ruby/ruby/pull/12840
2025-03-03[rubygems/rubygems] Bring man pages up to dateJosef Šimánek
https://github.com/rubygems/rubygems/commit/591d2c0503 Notes: Merged: https://github.com/ruby/ruby/pull/12840
2025-03-03[rubygems/rubygems] Update vendored uri to 1.0.3Hiroshi SHIBATA
https://github.com/rubygems/rubygems/commit/176dc7421c Notes: Merged: https://github.com/ruby/ruby/pull/12840
2025-02-28[rubygems/rubygems] Retry gracefully on blank partial response in compact indexMartin Emde
https://github.com/rubygems/rubygems/commit/fafb9ae090
2025-02-27[ruby/cgi] Bump up v0.4.2Hiroshi SHIBATA
https://github.com/ruby/cgi/commit/ab84b7fe66
2025-02-27[ruby/cgi] Bump up 0.4.2.beta2Hiroshi SHIBATA
https://github.com/ruby/cgi/commit/8e6fb1041b
2025-02-27[rubygems/rubygems] Remove MD5 digesting of compact index responsesMartin Emde
It has been over a year since the release, so let's stop MD5ing everything https://github.com/rubygems/rubygems/commit/29ef4ca30b
2025-02-27[rubygems/rubygems] Improve error message when on read-only filesystemsDavid Rodríguez
If we fail to write the lockfile, give a better error. https://github.com/rubygems/rubygems/commit/81a08d6eda
2025-02-27[ruby/uri] Use a fully qualified name in warning messagesyuuji.yaginuma
Currently, some warning messages don't contain a `URI` like the following. ```ruby warning: URI::ABS_URI is obsolete. Use RFC2396_PARSER.regexp[:ABS_URI] explicitly. ``` But, without `URI` prefix, the suggested value doesn't work. So I think we should use a fully qualified name to avoid confusion. https://github.com/ruby/uri/commit/428eb10e44
2025-02-27[ruby/uri] Fix the mention to removed `URI.escape/URI::Escape`Yuji Yaginuma
This was removed by #9. https://github.com/ruby/uri/commit/fec924238f
2025-02-26[ruby/cgi] Escape/unescape unclosed tags as wellHiroshi SHIBATA
https://github.com/ruby/cgi/commit/cd1eb08076 Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
2025-02-26[ruby/cgi] Use String#concat instead of String#+ for reducing cpu usageHiroshi SHIBATA
https://github.com/ruby/cgi/commit/9907b76dad Co-authored-by: "Yusuke Endoh" <mame@ruby-lang.org>
2025-02-26[ruby/uri] Bump up v1.0.3Hiroshi SHIBATA
https://github.com/ruby/uri/commit/3213f4a0f8
2025-02-26[ruby/uri] Fix merger of URI with authority componentHiroshi SHIBATA
https://hackerone.com/reports/2957667 https://github.com/ruby/uri/commit/2789182478 Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
2025-02-26[ruby/uri] Truncate userinfo with URI#join, URI#merge and URI#+Hiroshi SHIBATA
https://github.com/ruby/uri/commit/3675494839
2025-02-26[ruby/cgi] Bump up 0.4.2.beta1Hiroshi SHIBATA
https://github.com/ruby/cgi/commit/3f5b4ed9e9
2025-02-25[ruby/prism] Restore a comment for `Prism::Translation::Parser#initialize`Koichi ITO
This restores the missing method comments in https://github.com/ruby/prism/pull/3479. https://github.com/ruby/prism/commit/78b8f67dee
2025-02-25[ruby/prism] Fix merge mishapEarlopain
Caused by https://github.com/ruby/prism/pull/3478 and https://github.com/ruby/prism/pull/3443 I also made the builder reference more explicit to clearly distinquish between `::Parser` and `Prism::Translation::Parser` https://github.com/ruby/prism/commit/d52aaa75b6
2025-02-25[ruby/prism] Add a custom builder class for the parser translatorEarlopain
I want to add new node types to the parser translator, for example `itblock`. The bulk of the work is already done by prism itself. In the `parser` builder, this would be a 5-line change at most but we don't control that here. Instead, we can add our own builder and either overwrite the few methods we need, or just inline the complete builder. I'm not sure yet which would be better. `rubocop-ast` uses its own builder for `parser`. For this to correctly work, it must explicitly choose to extend the prism builder and use it, same as it currently chooses to use a different parser when prism is used. I'd like to enforce that the builder for prism extends its custom one since it will lead to some pretty weird issues otherwise. But first, I'd like to change `rubocop-ast` to make use of this. https://github.com/ruby/prism/commit/b080e608a8
generated by cgit v1.2.3 (git 2.25.1) at 2026-01-19 18:34:25 +0000