| Age | Commit message (Collapse) | Author |
|---|
|
|
|
[Misc #18830]
Notes:
Merged-By: k0kubun <takashikkbn@gmail.com>
|
|
|
|
And re-embed any strings that can now fit inside the slot they've been
moved to
Notes:
Merged: https://github.com/ruby/ruby/pull/5986
|
|
http://rubyci.s3.amazonaws.com/ubuntu1804/ruby-master/log/20220613T003003Z.log.html.gz
```
/home/chkbuild/chkbuild/tmp/build/20220613T003003Z/ruby/test/ruby/test_yjit_exit_locations.rb:96: warning: assigned but unused variable - out
/home/chkbuild/chkbuild/tmp/build/20220613T003003Z/ruby/test/ruby/test_yjit_exit_locations.rb:96: warning: assigned but unused variable - err
/home/chkbuild/chkbuild/tmp/build/20220613T003003Z/ruby/test/ruby/test_yjit_exit_locations.rb:96: warning: assigned but unused variable - status
```
|
|
Previously, because opt_aref and opt_aset don't push a frame, when they
would call rb_hash to determine the hash value of the key, the initial
level of recursion would incorrectly use the method id at the top of the
stack instead of "hash".
This commit replaces rb_exec_recursive_outer with
rb_exec_recursive_outer_mid, which takes an explicit method id, so that
we can make the hash calculation behave consistently.
rb_exec_recursive_outer was documented as being internal, so I believe
this should be okay to change.
Notes:
Merged: https://github.com/ruby/ruby/pull/6004
|
|
df317151a5b4e0c5a30fcc321a9dc6abad63f7ed removed the code to free
rb_hook_list_t, so repeated targeting of the same bmethod started
to leak the hook list. You can observe how the maximum memory use
scales with input size in the following script with `/usr/bin/time -v`.
```ruby
o = Object.new
o.define_singleton_method(:foo) {}
trace = TracePoint.new(:return) {}
bmethod = o.method(:foo)
ARGV.first.to_i.times { trace.enable(target:bmethod){} }
4.times {GC.start}
```
After this change the maximum doesn't grow as quickly.
To plug the leak, check whether the hook list is already allocated
when enabling the targeting TracePoint for the bmethod. This fix
also allows multiple TracePoints to target the same bmethod, similar
to other valid TracePoint targets.
Finally, free the rb_hook_list_t struct when freeing the method
definition it lives on. Freeing in the GC is a good way to avoid
lifetime problems similar to the one fixed in df31715.
[Bug #18031]
Notes:
Merged: https://github.com/ruby/ruby/pull/4651
|
|
I thought about using it in 2931957d6ff16b5c095f6e8095384c98130133ad
once and then ended up not using it.
|
|
I originally added the check for
RubyVM::YJIT.trace_exit_locations_enabled? to fix errors when these
tests run without the stats feature enabled. However I forgot that this
will never be true when this test is booting, so nothing was running
when the stats feature is turned on.
Instead I've decided to make a new hash in the dump file and check if
exit locations are enabled there. If they aren't enabled we return early
to avoid checking for keys that won't exit in the dumped exit locations.
I chose to add this additional enabled check because empty exit
locations might not indicate that stats isn't enabled, it could mean the
feature is entirely broken. I do want these tests to fail if stats are
on and nothing was collected.
Followup to #5970
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
When running with `--yjit-stats` turned on, yjit can inform the user
what the most common exits are. While this is useful information it
doesn't tell you the source location of the code that exited or what the
code that exited looks like. This change intends to fix that.
To use the feature, run yjit with the `--yjit-trace-exits` option,
which will record the backtrace for every exit that occurs. This functionality
requires the stats feature to be turned on. Calling `--yjit-trace-exits`
will automatically set the `--yjit-stats` option.
Users must call `RubyVM::YJIT.dump_exit_locations(filename)` which will
Marshal dump the contents of `RubyVM::YJIT.exit_locations` into a file
based on the passed filename.
*Example usage:*
Given the following script, we write to a file called
`concat_array.dump` the results of `RubyVM::YJIT.exit_locations`.
```ruby
def concat_array
["t", "r", *x = "u", "e"].join
end
1000.times do
concat_array
end
RubyVM::YJIT.dump_exit_locations("concat_array.dump")
```
When we run the file with this branch and the appropriate flags the
stacktrace will be recorded. Note Stackprof needs to be installed or you
need to point to the library directly.
```
./ruby --yjit --yjit-call-threshold=1 --yjit-trace-exits -I/Users/eileencodes/open_source/stackprof/lib test.rb
```
We can then read the dump file with Stackprof:
```
./ruby -I/Users/eileencodes/open_source/stackprof/lib/ /Users/eileencodes/open_source/stackprof/bin/stackprof --text concat_array.dump
```
Results will look similar to the following:
```
==================================
Mode: ()
Samples: 1817 (0.00% miss rate)
GC: 0 (0.00%)
==================================
TOTAL (pct) SAMPLES (pct) FRAME
1001 (55.1%) 1001 (55.1%) concatarray
335 (18.4%) 335 (18.4%) invokeblock
178 (9.8%) 178 (9.8%) send
140 (7.7%) 140 (7.7%) opt_getinlinecache
...etc...
```
Simply inspecting the `concatarray` method will give `SOURCE
UNAVAILABLE` because the source is insns.def.
```
./ruby -I/Users/eileencodes/open_source/stackprof/lib/ /Users/eileencodes/open_source/stackprof/bin/stackprof --text concat_array.dump --method concatarray
```
Result:
```
concatarray (nonexistent.def:1)
samples: 1001 self (55.1%) / 1001 total (55.1%)
callers:
1000 ( 99.9%) Object#concat_array
1 ( 0.1%) Gem.suffixes
callees (0 total):
code:
SOURCE UNAVAILABLE
```
However if we go deeper to the callee we can see the exact
source of the `concatarray` exit.
```
./ruby -I/Users/eileencodes/open_source/stackprof/lib/ /Users/eileencodes/open_source/stackprof/bin/stackprof --text concat_array.dump --method Object#concat_array
```
```
Object#concat_array (/Users/eileencodes/open_source/rust_ruby/test.rb:1)
samples: 0 self (0.0%) / 1000 total (55.0%)
callers:
1000 ( 100.0%) block in <main>
callees (1000 total):
1000 ( 100.0%) concatarray
code:
| 1 | def concat_array
1000 (55.0%) | 2 | ["t", "r", *x = "u", "e"].join
| 3 | end
```
The `--walk` option is recommended for this feature as it make it
easier to traverse the tree of exits.
*Goals of this feature:*
This feature is meant to give more information when working on YJIT.
The idea is that if we know what code is exiting we can decide what
areas to prioritize when fixing exits. In some cases this means adding
prioritizing avoiding certain exits in yjit. In more complex cases it
might mean changing the Ruby code to be more performant when run with
yjit. Ultimately the more information we have about what code is exiting
AND why, the better we can make yjit.
*Known limitations:*
* Due to tracing exits, running this on large codebases like Rails
can be quite slow.
* On complex methods it can still be difficult to pinpoint the exact cause of
an exit.
* Stackprof is a requirement to to view the backtrace information from
the dump file.
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
Fixes [Bug #18771]
Notes:
Merged: https://github.com/ruby/ruby/pull/5953
|
|
|
|
We could fix it, but removing files in the directory recursively is
tedious in C and --mjit-debug is not a concern for users. We have
TestMJITDebug for detecting linker problems that are ignored by -O. It's
not really for maintaining --mjit-debug itself.
|
|
force_major_gc_count is the number of times the size pool forced major
GC to run.
|
|
This reverts commit 9d927204e7b86eb00bfd07a060a6383139edf741.
Notes:
Merged: https://github.com/ruby/ruby/pull/5981
|
|
... only when the message string has a newline.
`p StandardError.new("foo\nbar")` now prints `#<StandardError: "foo\nbar">'
instead of:
#<StandardError:
bar>
[Bug #18170]
Notes:
Merged: https://github.com/ruby/ruby/pull/4857
|
|
Invalid escapes are handled at multiple levels. The first level
is in parse.y, so skip invalid unicode escape checks for regexps
in parse.y.
Make rb_reg_preprocess and unescape_nonascii accept the regexp
options. In unescape_nonascii, if the regexp is an extended
regexp, when "#" is encountered, ignore all characters until the
end of line or end of regexp.
Unfortunately, in extended regexps, you can use "#" as a non-comment
character inside a character class, so also parse "[" and "]"
specially for extended regexps, and only skip comments if "#" is
not inside a character class. Handle nested character classes as well.
This issue doesn't just affect extended regexps, it also affects
"(#?" comments inside all regexps. So for those comments, scan
until trailing ")" and ignore content inside.
I'm not sure if there are other corner cases not handled. A
better fix would be to redesign the regexp parser so that it
unescaped during parsing instead of before parsing, so you already
know the current parsing state.
Fixes [Bug #18294]
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
Notes:
Merged: https://github.com/ruby/ruby/pull/5721
Merged-By: jeremyevans <code@jeremyevans.net>
|
|
If an autoload exists for a constant, but the path for the autoload
was required, const_source_location would return [false, 0] instead
of the actual file and line. This fixes it by setting the appropriate
file and line in rb_const_set, and saving the file and line in
const_tbl_update before they get reset by current_autoload_data.
Fixes [Bug #18624]
Notes:
Merged: https://github.com/ruby/ruby/pull/5646
|
|
Previously `(2..).cover?("2"..)` was false, but
`(..2).cover?(.."2")` was true. This changes it so both are false,
treating beginless ranges the same as endless ranges in regards to
type checks.
This also adds documentation to #cover? to describe behavior with
beginless and endless ranges, testing each documentation example,
which is how this bug was found.
Fixes [Bug #18155]
Notes:
Merged: https://github.com/ruby/ruby/pull/5831
|
|
Implements [Feature #12655]
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
Notes:
Merged: https://github.com/ruby/ruby/pull/5733
Merged-By: jeremyevans <code@jeremyevans.net>
|
|
Define `GC.verify_compaction_references` as a built-in ruby method,
according to GC compaction support via `GC::OPTS`.
Notes:
Merged: https://github.com/ruby/ruby/pull/5972
|
|
|
|
|
|
`vm_trace_hook()` runs global hooks before running local hooks.
Previously, we read the local hook list before running the global hooks
which led to use-after-free when a global hook frees the local hook
list. A global hook can do this by disabling a local TracePoint, for
example.
Delay local hook list loading until after running the global hooks.
Issue discovered by Jeremy Evans in GH-5862.
[Bug #18730]
Notes:
Merged: https://github.com/ruby/ruby/pull/5865
|
|
|
|
|
|
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/5941
|
|
Fixes [Bug #18779]
Define the following methods as `rb_f_notimplement` on unsupported
platforms:
- GC.compact
- GC.auto_compact
- GC.auto_compact=
- GC.latest_compact_info
- GC.verify_compaction_references
This change allows users to call `GC.respond_to?(:compact)` to
properly test for compaction support. Previously, it was necessary to
invoke `GC.compact` or `GC.verify_compaction_references` and check if
those methods raised `NotImplementedError` to determine if compaction
was supported.
This follows the precedent set for other platform-specific
methods. For example, in `process.c` for methods such as
`Process.fork`, `Process.setpgid`, and `Process.getpriority`.
Notes:
Merged: https://github.com/ruby/ruby/pull/5934
|
|
forgot to commit this in ead96e7b44b98bef4896d836239345012821f1d2
|
|
to avoid confusion with YJIT
|
|
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
during GC. (#5911)
Notes:
Merged-By: ioquatix <samuel@codeotaku.com>
|
|
* Update naming of critical section assertions macros.
* Improved locking for autoload.
Notes:
Merged-By: ioquatix <samuel@codeotaku.com>
|
|
This implements the getblockparam instruction.
There are two cases we need to handle depending on whether or not
VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM is set in the environment flag.
When the modified flag is unset, we need to call rb_vm_bh_to_procval to
get a proc from our passed block, save the proc in the environment, and
set the modified flag.
In the case that the modified flag is set we are able to just use the
existing proc in the environment.
One quirk of this is that we need to call jit_prepare_routine_call early
and ensure we update PC and SP regardless of the branch taken, so that
we have a consistent SP offset at the start of the next instruction.
We considered using a chain guard to generate these two paths
separately, but decided against it because it's very common to see both
and the modified case is basically a subset of the instructions in the
unmodified case.
This includes tests for both getblockparam and getblockparamproxy which
was previously missing a test.
Notes:
Merged: https://github.com/ruby/ruby/pull/5881
|
|
For string concat, see if compile-time encoding of strings matches.
If so, use simple buffer string concat at runtime. Otherwise, use
encoding-checking string concat.
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
|
|
Notes:
Merged-By: ioquatix <samuel@codeotaku.com>
|
|
Notes:
Merged-By: ioquatix <samuel@codeotaku.com>
|
|
Notes:
Merged-By: maximecb <maximecb@ruby-lang.org>
|
|
Previously it made object references without using write barriers,
creating GC inconsistencies.
See: http://ci.rvm.jp/results/trunk-gc-asserts@phosphorus-docker/3925529
Notes:
Merged: https://github.com/ruby/ruby/pull/5851
|
|
In December 2021, we opened an [issue] to solicit feedback regarding the
porting of the YJIT codebase from C99 to Rust. There were some
reservations, but this project was given the go ahead by Ruby core
developers and Matz. Since then, we have successfully completed the port
of YJIT to Rust.
The new Rust version of YJIT has reached parity with the C version, in
that it passes all the CRuby tests, is able to run all of the YJIT
benchmarks, and performs similarly to the C version (because it works
the same way and largely generates the same machine code). We've even
incorporated some design improvements, such as a more fine-grained
constant invalidation mechanism which we expect will make a big
difference in Ruby on Rails applications.
Because we want to be careful, YJIT is guarded behind a configure
option:
```shell
./configure --enable-yjit # Build YJIT in release mode
./configure --enable-yjit=dev # Build YJIT in dev/debug mode
```
By default, YJIT does not get compiled and cargo/rustc is not required.
If YJIT is built in dev mode, then `cargo` is used to fetch development
dependencies, but when building in release, `cargo` is not required,
only `rustc`. At the moment YJIT requires Rust 1.60.0 or newer.
The YJIT command-line options remain mostly unchanged, and more details
about the build process are documented in `doc/yjit/yjit.md`.
The CI tests have been updated and do not take any more resources than
before.
The development history of the Rust port is available at the following
commit for interested parties:
https://github.com/Shopify/ruby/commit/1fd9573d8b4b65219f1c2407f30a0a60e537f8be
Our hope is that Rust YJIT will be compiled and included as a part of
system packages and compiled binaries of the Ruby 3.2 release. We do not
anticipate any major problems as Rust is well supported on every
platform which YJIT supports, but to make sure that this process works
smoothly, we would like to reach out to those who take care of building
systems packages before the 3.2 release is shipped and resolve any
issues that may come up.
[issue]: https://bugs.ruby-lang.org/issues/18481
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Co-authored-by: Noah Gibbs <the.codefolio.guy@gmail.com>
Co-authored-by: Kevin Newton <kddnewton@gmail.com>
Notes:
Merged: https://github.com/ruby/ruby/pull/5826
|
|
Should fix issues with parallel testing sometimes not running all
tests.
This should be viewed skipping whitespace changes.
Fixes [Bug #18731]
Notes:
Merged: https://github.com/ruby/ruby/pull/5839
|
|
Not sure if this is the correct fix. It does raise LocalJumpError in
the yielding thread as you would expect, but the value yielded to the calling
thread is still yielded without an exception.
Fixes [Bug #18649]
Notes:
Merged: https://github.com/ruby/ruby/pull/5692
|
|
|
|
Previously in some when classes were duped (specifically those with a
prepended module), they would not correctly have their "superclasses"
array or depth filled in.
This could cause ancestry checks (like is_a? and Module comparisons) to
return incorrect results.
This happened because rb_mod_init_copy builds origin classes in an order
that doesn't have the super linked list fully connected until it's
finished. This commit fixes the previous issue by calling
rb_class_update_superclasses before returning the cloned class. This is
similar to what's already done in make_metaclass.
Notes:
Merged: https://github.com/ruby/ruby/pull/5808
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/5803
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/5803
|
|
Allocating a string of length MAXPATHLEN and then shrinking the string
is inefficient when the resulting path is short. Preallocating a large
string is also a problem for Variable Width Allocation since we can't
easily downsize the capacity.
I ran the following benchmark:
```ruby
Benchmark.ips do |x|
{
"empty" => "",
"short" => "a/" * 10,
"medium" => "a/" * 100,
"long" => "a/" * 500
}.each do |name, path|
x.report(name) do |times|
i = 0
while i < times
File.expand_path(path)
i += 1
end
end
end
end
```
On this commit:
```
empty 97.486k (± 0.7%) i/s - 492.915k in 5.056507s
short 96.026k (± 2.4%) i/s - 486.489k in 5.068966s
medium 86.304k (± 1.3%) i/s - 435.336k in 5.045112s
long 59.395k (± 1.7%) i/s - 302.175k in 5.089026s
```
On master:
```
empty 94.138k (± 1.4%) i/s - 472.158k in 5.016590s
short 92.043k (± 1.4%) i/s - 468.180k in 5.087496s
medium 84.910k (± 2.3%) i/s - 425.750k in 5.017007s
long 61.503k (± 2.7%) i/s - 309.723k in 5.039429s
```
Notes:
Merged: https://github.com/ruby/ruby/pull/5789
|
|
https://hackerone.com/reports/1248108
Notes:
Merged: https://github.com/ruby/ruby/pull/5794
|
