summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-04-06YJIT: Stack temp register allocation for arm64 (#7659)Takashi Kokubun
* YJIT: Stack temp register allocation for arm64 * Update a comment Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com> * Update comments about assertion * Update a comment Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com> --------- Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com> Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2023-04-06Update VPATH for socket, & dependenciesMatt Valentine-House
The socket extensions rubysocket.h pulls in the "private" include/gc.h, which now depends on vm_core.h. vm_core.h pulls in id.h when tool/update-deps generates the dependencies for the makefiles, it generates the line for id.h to be based on VPATH, which is configured in the extconf.rb for each of the extensions. By default VPATH does not include the actual source directory of the current Ruby so the dependency fails to resolve and linking fails. We need to append the topdir and top_srcdir to VPATH to have the dependancy picked up correctly (and I believe we need both of these to cope with in-tree and out-of-tree builds). I copied this from the approach taken in https://github.com/ruby/ruby/blob/master/ext/objspace/extconf.rb#L3 Notes: Merged: https://github.com/ruby/ruby/pull/7393
2023-04-06[Feature #19474] Refactor NEWOBJ macrosMatt Valentine-House
NEWOBJ_OF is now our canonical newobj macro. It takes an optional ec Notes: Merged: https://github.com/ruby/ruby/pull/7393
2023-04-06Remove dependancy of vm_core.h on shape.hMatt Valentine-House
so that now shape can happily include gc.h Notes: Merged: https://github.com/ruby/ruby/pull/7393
2023-04-06Pull the shape tree out of the vm objectMatt Valentine-House
Notes: Merged: https://github.com/ruby/ruby/pull/7393
2023-04-06Remove newobj_of_crMatt Valentine-House
We can just make newobj_of take a ractor Notes: Merged: https://github.com/ruby/ruby/pull/7393
2023-04-06Revert "[ruby/syntax_suggest] Introduce binstubs to set RUBYOPT for development"Hiroshi SHIBATA
This reverts commit d2eef4b7867578df302cfa9d7c97eebbef8d9ee4.
2023-04-06Revert accidentally commit of wrong fileHiroshi SHIBATA
Revert "[ruby/syntax_suggest] Remove unnecessary `--color` option" This reverts commit 588dd44d418d56dce3f2a388c4021d11f9aa4324.
2023-04-06Update default gems list at 6ca1f3eec4222924c7e23642a68b51 [ci skip]git
2023-04-06Load only SyntaxSuggest::VERSION for version checkHiroshi SHIBATA
2023-04-06v1.0.4schneems
2023-04-06v1.0.3schneems
Fix a CI error and add a test to ensure we're testing the current version: ``` Run bundle exec rake test bundler: failed to load command: rake (/home/runner/work/syntax_suggest/syntax_suggest/vendor/bundle/ruby/3.2.0/bin/rake) /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/runtime.rb:309:in `check_for_activated_spec!': You have already activated syntax_suggest 1.0.2, but your Gemfile requires syntax_suggest 1.0.3. Since syntax_suggest is a default gem, you can either remove your dependency on it or try updating to a newer version of bundler that supports syntax_suggest as a default gem. (Gem::LoadError) from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/runtime.rb:25:in `block in setup' from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/spec_set.rb:138:in `each' from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/spec_set.rb:138:in `each' from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/runtime.rb:24:in `map' from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/runtime.rb:24:in `setup' from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler.rb:151:in `setup' from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/setup.rb:20:in `block in <top (required)>' from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/ui/shell.rb:136:in `with_level' from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/ui/shell.rb:88:in `silence' from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/setup.rb:20:in `<top (required)>' from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/cli/exec.rb:56:in `require_relative' from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/cli/exec.rb:56:in `kernel_load' from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/cli/exec.rb:23:in `run' from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/cli.rb:483:in `exec' from /opt/hostedtoolcache/Ruby/3.2.1/x64/lib/ruby/gems/3.2.0/gems/bundler-2.3.14/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run' ```
2023-04-06[ruby/syntax_suggest] Introduce binstubs to set RUBYOPT for developmentschneems
Because `syntax_suggest` is a default gem you can get conflicts when working on this project with Ruby 3.2+. To fix conflicts you can disable loading `syntax_suggest` as a default gem by using then environment variable `RUBYOPT` with the value `--disable=syntax_suggest`. The `RUBYOPT` environment variable works the same as if we had entered those flags directly in the ruby cli (i.e. `ruby --disable=syntax_suggest` is the same as `RUBYOPT="--disable=syntax_suggest" ruby`). It's needed because we don't always directly execute Ruby and RUBYOPT will be picked up when other commands load ruby (`rspec`, `rake`, or `bundle` etc.). There are some binstubs that already have this done for you. Instead of running `bundle exec rake` you can run `bin/rake`. Binstubs provided: - `bin/rake` - `bin/rspec` https://github.com/ruby/syntax_suggest/commit/342093706d
2023-04-06[ruby/syntax_suggest] The annotation must end with a new lineYusuke Endoh
syntax_suggest did not work great when there is no new line at the end of the input file. Input: ``` def foo end end # No newline at end of file ``` Previous output: ``` $ ruby test.rb test.rb: --> test.rb Unmatched `end', missing keyword (`do', `def`, `if`, etc.) ? > 1 def foo > 2 end > 3 end # No newline at end of filetest.rb:3: syntax error, unexpected `end' (SyntaxError) end # No newline at end of file ^~~ ``` Note that "test.rb:3: ..." is appended to the last line of the annotation. This change makes sure that the annotation ends with a new line. New output: ``` $ ruby test.rb test.rb: --> test.rb Unmatched `end', missing keyword (`do', `def`, `if`, etc.) ? > 1 def foo > 2 end > 3 end # No newline at end of file test.rb:3: syntax error, unexpected `end' (SyntaxError) end # No newline at end of file ^~~ ``` https://github.com/ruby/syntax_suggest/commit/db4cf9147d
2023-04-06[ruby/syntax_suggest] Rollback comment indentation behaviorschneems
Originally I fixed https://github.com/ruby/syntax_suggest/pull/177 by making the process of comment removal indentation aware. The next commit is the more general fix and means we don't need to carry that additional logic/overhead. Also: Update syntax via linter
2023-04-06[ruby/syntax_suggest] Add comments and refactor AroundBlockScan methodsschneems
https://github.com/ruby/syntax_suggest/commit/cecd12292c
2023-04-06[ruby/syntax_suggest] Fix sibling bug to #177schneems
While #177 is reported as being caused by a comment, the underlying behavior is a problem due to the newline that we generated (from a comment). The prior commit fixed that problem by preserving whitespace before the comment. That guarantees that a block will form there from the frontier before it will be expanded there via a "neighbors" method. Since empty lines are valid ruby code, it will be hidden and be safe. ## Problem setup This failure mode is not fixed by the prior commit, because the indentation is 0. To provide good results, we must make the algorithm less greedy. One heuristic/signal to follow is developer added newlines. If a developer puts a newline between code, it's more likely they're unrelated. For example: ``` port = rand(1000...9999) stub_request(:any, "localhost:#{port}") query = Cutlass::FunctionQuery.new( port: port ).call expect(WebMock).to have_requested(:post, "localhost:#{port}"). with(body: "{}") ``` This code is split into three chunks by the developer. Each are likely (but not guaranteed) to be intended to stand on their own (in terms of syntax). This behavior is good for scanning neighbors (same indent or higher) within a method, but bad for parsing neighbors across methods. ## Problem Code is expanded to capture all neighbors, and then it decreases indent level which allows it to capture surrounding scope (think moving from within the method to also capturing the `def/end` definition. Once the indentation level has been increased, we go back to scanning neighbors, but now neighbors also contain keywords. For example: ``` 1 def bark 2 3 end 4 5 def sit 6 end ``` In this case if lines 4, 5, and 6 are in a block when it tries to expand neighbors it will expand up. If it stops after line 2 or 3 it may cause problems since there's a valid kw/end pair, but the block will be checked without it. TLDR; It's good to stop scanning code after hitting a newline when you're in a method...it causes a problem scanning code between methods when everything inside of one of the methods is an empty line. In this case it grabs the end on line 3 and since the problem was an extra end, the program now compiles correctly. It incorrectly assumes that the block it captured was causing the problem. ## Extra bit of context One other technical detail is that after we've decided to stop scanning code for a new neighbor block expansion, we look around the block and grab any empty newlines. Basically adding empty newlines before of after a code block do not affect the parsing of that block. ## The fix Since we know that this problem only happens when there's a newline inside of a method and we know this particular failure mode is due to having an invalid block (capturing an extra end, but not it's keyword) we have all the metadata we need to detect this scenario and correct it. We know that the next line above our block must be code or empty (since we grabbed extra newlines). Same for code below it. We can count all the keywords and ends in the block. If they are balanced, it's likely (but not guaranteed) we formed the block correctly. If they're imbalanced, look above or below (depending on the nature of the imbalance), check to see if adding that line would balance the count. This concept of balance and "leaning" comes from work in https://github.com/ruby/syntax_suggest/pull/152 and has proven useful, but not been formally introduced into the main branch. ## Outcome Adding this extra check introduced no regressions and fixed the test case. It might be possible there's a mirror or similar problem that we're not handling. That will come out in time. It might also be possible that this causes a worse case in some code not under test. That too would come out in time. One other possible concern to adding logic in this area (which is a hot codepath), is performance. This extra count check will be performed for every block. In general the two most helpful performance strategies I've found are reducing total number of blocks (therefore reducing overall N internal iterations) and making better matches (the parser to determine if a close block is valid or not is a major bottleneck. If we can split valid code into valid blocks, then it's only evaluated by the parser once, where as invalid code must be continuously re-checked by the parser until it becomes valid, or is determined to be the cause of the core problem. This extra logic should very rarely result in a change, but when it does it should tend to produce slightly larger blocks (by one line) and more accurate blocks. Informally it seems to have no impact on performance: `` This branch: DEBUG_DISPLAY=1 bundle exec rspec spec/ --format=failures 3.01s user 1.62s system 113% cpu 4.076 total ``` ``` On main: DEBUG_DISPLAY=1 bundle exec rspec spec/ --format=failures 3.02s user 1.64s system 113% cpu 4.098 total ``` https://github.com/ruby/syntax_suggest/commit/13739c6946
2023-04-06[ruby/syntax_suggest] Preserve whitespace in front of commentsschneems
When removing comments I previously replaced them with a newline. This loses some context and may affect the order of the indent search which in turn affects the final result. By preserving whitespace in front of the comment, we preserve the "natural" indentation order of the line while also allowing the parser/lexer to see and join naturally consecutive (method chain) lines. close https://github.com/ruby/syntax_suggest/pull/177
2023-04-06[ruby/syntax_suggest] Handle alias already exists when debuggingschneems
When `tmp/alias` already exists, I'm now getting phantom folders in the directory pointing at older aliases which is distracting/confusing. By checking and removing that alias before symlinking we can prevent this strange behavior (possibly caused by newer Mac OS?).
2023-04-06[ruby/syntax_suggest] Remove unnecessary `--color` optionNobuyoshi Nakada
It is enabled automatically if possible. Forcing this option makes https://github.com/rspec/rspec-core/pull/3017 useless. https://github.com/ruby/syntax_suggest/commit/8e7141b472
2023-04-06Add missing test for Data.initializeMarc-Andre Lafortune
Notes: Merged: https://github.com/ruby/ruby/pull/7666
2023-04-06[rubygems/rubygems] Improve comment explaining the neccessity of ↵Vít Ondruch
`write_default_spec` method. The intention is not obvious from the commit log and it might avoid temptation to remove the method without further consideration. https://github.com/rubygems/rubygems/commit/8e17c50f64
2023-04-06[rubygems/rubygems] Fix resolver hangs when dealing with an incomplete lockfileDavid Rodríguez
While working on locking multiple platforms by default, I got an infinite resolution loop in one of our resolver specs. The culprit ended up being that when dealing with lockfile specs with incomplete dependencies (spec appears in lockfile, but its dependencies don't), those specs were not being properly expired and that tripped up resolution. The issue for some reason only manifests when dealing with multiple lockfile platforms, that's why it only manifested when working on locking multiple platforms by default. https://github.com/rubygems/rubygems/commit/4ca72913bb
2023-04-06Revert "Refactor incomplete specs handling"David Rodríguez
This reverts commit 69580f8b72f41c58cae57d1ada4db909922b3891.
2023-04-06[rubygems/rubygems] Revert "Reuse `SpecSet#materialize` logic"David Rodríguez
This reverts commit https://github.com/rubygems/rubygems/commit/a20585b4512d. https://github.com/rubygems/rubygems/commit/aa9102659e
2023-04-06[rubygems/rubygems] Use TestCase#util_spec instead of custom code.Vít Ondruch
https://github.com/rubygems/rubygems/commit/74d485ff1e
2023-04-06Bump github/codeql-action from 2.2.9 to 2.2.10dependabot[bot]
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.2.9 to 2.2.10. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](https://github.com/github/codeql-action/compare/04df1262e6247151b5ac09cd2c303ac36ad3f62b...8c8d71dde4abced210732d8486586914b97752e8) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Notes: Merged: https://github.com/ruby/ruby/pull/7668
2023-04-05Add BIN as an entry in the glossary (#7667)Adam Hess
I initially thought this macro is referencing binary, not basic instruction name Notes: Merged-By: k0kubun <takashikkbn@gmail.com>
2023-04-05[ruby/irb] Drop Ruby 2.6 supportStan Lo
(https://github.com/ruby/irb/pull/555) * Remove all Ruby 2.6 support * Drop Ruby 2.6 specific testing conditions * Only run Ruby 2.7+ on CI * Bump Ruby requirement to 2.7+ https://github.com/ruby/irb/commit/3f714b616c
2023-04-05YJIT: Add codegen for Integer methods (#7665)Takashi Kokubun
* YJIT: Add codegen for Integer methods * YJIT: Update dependencies * YJIT: Fix Integer#[] for argc=2 Notes: Merged-By: k0kubun <takashikkbn@gmail.com>
2023-04-05Ensure ruby_xfree won't segfault if called after vm_destructMike Dalessio
[Bug #19580] The real-world scenario motivating this change is libxml2's pthread code which uses `pthread_key_create` to set up a destructor that is called at thread exit to free thread-local storage. There is a small window of time -- after ruby_vm_destruct but before the process exits -- in which a pthread may exit and the destructor is called, leading to a segfault. Please note that this window of time may be relatively large if `atexit` is being used. Notes: Merged: https://github.com/ruby/ruby/pull/7663
2023-04-06core_assertions.rb: Prefer CPU time clocksNobuyoshi Nakada
To prevent influence from other processes. Notes: Merged: https://github.com/ruby/ruby/pull/7661
2023-04-06core_assertions.rb: Extract common code blockNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/7661
2023-04-06core_assertions.rb: Tweak timeout limitNobuyoshi Nakada
Increase the timeout limit when variance at rehearsal is small. Notes: Merged: https://github.com/ruby/ruby/pull/7661
2023-04-05[DOC] Clarify behavior of abort() with no argumentDaniel DeLorme
Since Ruby 2.7 it prints the message and backtrace of $! Fixes [Bug #19578] Notes: Merged: https://github.com/ruby/ruby/pull/7664
2023-04-05YJIT: Count the number of actually written bytes (#7658)Takashi Kokubun
Notes: Merged-By: maximecb <maximecb@ruby-lang.org>
2023-04-05YJIT: Smoke test on Rust 1.58.0Alan Wu
Since warnings might show up on older version but not newer ones. Notes: Merged: https://github.com/ruby/ruby/pull/7657
2023-04-05YJIT: Eanble `unsafe_op_in_unsafe_fn` on crate::coreAlan Wu
Encourages commenting about soundness of `unsafe` usages. Notes: Merged: https://github.com/ruby/ruby/pull/7657
2023-04-05Revert "YJIT: Suppress unnecessary `unsafe` block (GH-7634)"Alan Wu
This reverts commit 9e678cdbd054f78576a8f21b3f97cccc395ade22. Without the `unsafe` annotations, the SAFETY comments make less sense. I want to keep the SAFETY comments. Notes: Merged: https://github.com/ruby/ruby/pull/7657
2023-04-05[rubygems/rubygems] util/rubocop -A --only Style/RegexpLiteralHiroshi SHIBATA
https://github.com/rubygems/rubygems/commit/9264d83421
2023-04-05RJIT: Skip a class guard if known to be T_STRINGTakashi Kokubun
2023-04-05RJIT: Handle include_all argument of respond_to?Takashi Kokubun
2023-04-04RJIT: Remove unused variablesTakashi Kokubun
2023-04-04RJIT: Always use guard_two_fixnumsTakashi Kokubun
2023-04-04RJIT: Eliminate known-result guards for blockargTakashi Kokubun
2023-04-04RJIT: Eliminate known-result branchesTakashi Kokubun
2023-04-05[rubygems/rubygems] Removed unused Gem::Deprecate from Gem::TestCaseHiroshi SHIBATA
https://github.com/rubygems/rubygems/commit/d8a75a5f84
2023-04-05[rubygems/rubygems] Extract class method used by self.methodHiroshi SHIBATA
https://github.com/rubygems/rubygems/commit/d900453ca2
2023-04-05[rubygems/rubygems] We don't use process_based_port and self.process_based_portHiroshi SHIBATA
https://github.com/rubygems/rubygems/commit/f1a20c4c69
2023-04-05[rubygems/rubygems] We don't use self.make_commandHiroshi SHIBATA
https://github.com/rubygems/rubygems/commit/241f376c32