| Age | Commit message (Collapse) | Author |
|---|
|
Instead of silently ignoring the key, we should raise to clearly tell
the user that this key is read-only.
Notes:
Merged: https://github.com/ruby/ruby/pull/11872
|
|
And a default and readonly key to the GC.config hash that names the
current GC implementation.
This is provided by each implementation by the API function
rb_gc_impl_active_gc_name
Notes:
Merged: https://github.com/ruby/ruby/pull/11872
|
|
I caught a reproduction of this test failing under rr, and was able to
replay it to isolate the failure. The call to
`before_stat_heap = GC.stat_heap` is itself allocating a hash, which in
unlucky circumstances can result in a new page being allocated and thus
`before_stats[:heap_allocated_pages]` no longer equals
`after_stats[:heap_allocated_pages]`.
The solution is to use the form of GC.stat/stat_heap which takes a hash
as an argument, and thus needs to perform no Ruby allocations itself.
Notes:
Merged: https://github.com/ruby/ruby/pull/11997
|
|
GC.config is always defined.
Notes:
Merged: https://github.com/ruby/ruby/pull/11867
|
|
|
|
Now that we've inlined the eden_heap into the size_pool, we should
rename the size_pool to heap. So that Ruby contains multiple heaps, with
different sized objects.
The term heap as a collection of memory pages is more in memory
management nomenclature, whereas size_pool was a name chosen out of
necessity during the development of the Variable Width Allocation
features of Ruby.
The concept of size pools was introduced in order to facilitate
different sized objects (other than the default 40 bytes). They wrapped
the eden heap and the tomb heap, and some related state, and provided a
reasonably simple way of duplicating all related concerns, to provide
multiple pools that all shared the same structure but held different
objects.
Since then various changes have happend in Ruby's memory layout:
* The concept of tomb heaps has been replaced by a global free pages list,
with each page having it's slot size reconfigured at the point when it
is resurrected
* the eden heap has been inlined into the size pool itself, so that now
the size pool directly controls the free_pages list, the sweeping
page, the compaction cursor and the other state that was previously
being managed by the eden heap.
Now that there is no need for a heap wrapper, we should refer to the
collection of pages containing Ruby objects as a heap again rather than
a size pool
Notes:
Merged: https://github.com/ruby/ruby/pull/11771
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/11635
|
|
This adds keys heap_empty_pages and heap_allocatable_slots to GC.stat.
|
|
|
|
It's possible for a GC to run between the calls of GC.latest_gc_info,
which would cause the test to fail. We can disable GC so that GC only
triggers manually.
|
|
When assertions are enabled, the following code triggers an assertion
error:
GC.disable
GC.start(immediate_mark: false, immediate_sweep: false)
10_000_000.times { Object.new }
This is because the GC.start ignores that the GC is disabled and will
start incremental marking and lazy sweeping. But the assertions in
gc_marks_continue and gc_sweep_continue assert that GC is not disabled.
This commit changes it for the assertion to pass if the GC was triggered
from a method.
Notes:
Merged: https://github.com/ruby/ruby/pull/11391
|
|
`RBOOL` is a macro to convert C boolean to Ruby boolean.
Notes:
Merged: https://github.com/ruby/ruby/pull/11351
|
|
|
|
This feature provides a new method `GC.config` that configures internal
GC configuration variables provided by an individual GC implementation.
Implemented in this PR is the option `full_mark`: a boolean value that
will determine whether the Ruby GC is allowed to run a major collection
while the process is running.
It has the following semantics
This feature configures Ruby's GC to only run minor GC's. It's designed
to give users relying on Out of Band GC complete control over when a
major GC is run. Configuring `full_mark: false` does two main things:
* Never runs a Major GC. When the heap runs out of space during a minor
and when a major would traditionally be run, instead we allocate more
heap pages, and mark objspace as needing a major GC.
* Don't increment object ages. We don't promote objects during GC, this
will cause every object to be scanned on every minor. This is an
intentional trade-off between minor GC's doing more work every time,
and potentially promoting objects that will then never be GC'd.
The intention behind not aging objects is that users of this feature
should use a preforking web server, or some other method of pre-warming
the oldgen (like Nakayoshi fork)before disabling Majors. That way most
objects that are going to be old will have already been promoted.
This will interleave major and minor GC collections in exactly the same
what that the Ruby GC runs in versions previously to this. This is the
default behaviour.
* This new method has the following extra semantics:
- `GC.config` with no arguments returns a hash of the keys of the
currently configured GC
- `GC.config` with a key pair (eg. `GC.config(full_mark: true)` sets
the matching config key to the corresponding value and returns the
entire known config hash, including the new values. If the key does
not exist, `nil` is returned
* When a minor GC is run, Ruby sets an internal status flag to determine
whether the next GC will be a major or a minor. When `full_mark:
false` this flag is ignored and every GC will be a minor.
This status flag can be accessed at
`GC.latest_gc_info(:needs_major_by)`. Any value other than `nil` means
that the next collection would have been a major.
Thus it's possible to use this feature to check at a predetermined
time, whether a major GC is necessary and run one if it is. eg. After
a request has finished processing.
```ruby
if GC.latest_gc_info(:needs_major_by)
GC.start(full_mark: true)
end
```
[Feature #20443]
|
|
We only collect GC.stat_heap(nil, stat_heap_all)
once, outside of the loop, but assert_equal could
allocate objects which can cause a GC to run and
cause stat_heap_all to be out-of-sync.
|
|
This commit splits gc.c into two files:
- gc.c now only contains code not specific to Ruby GC. This includes
code to mark objects (which the GC implementation may choose not to
use) and wrappers for internal APIs that the implementation may need
to use (e.g. locking the VM).
- gc_impl.c now contains the implementation of Ruby's GC. This includes
marking, sweeping, compaction, and statistics. Most importantly,
gc_impl.c only uses public APIs in Ruby and a limited set of functions
exposed in gc.c. This allows us to build gc_impl.c independently of
Ruby and plug Ruby's GC into itself.
|
|
|
|
This reverts commit 3680981c7b71df8c3a426164787ccefe5296bb25.
|
|
(maybe) from 9cf754b the test fails on some environments:
https://rubyci.s3.amazonaws.com/icc-x64/ruby-master/log/20240325T200004Z.fail.html.gz
```
1) Failure:
TestGc#test_gc_stress_at_startup [/home/chkbuild/chkbuild/tmp/build/20240325T200004Z/ruby/test/ruby/test_gc.rb:771]:
[Bug #15784]
pid 1087168 killed by SIGSEGV (signal 11) (core dumped).
1. [3/3] Assertion for "success?"
| Expected #<Process::Status: pid 1087168 SIGSEGV (signal 11) (core dumped)> to be success?.
```
https://rubyci.s3.amazonaws.com/freebsd14/ruby-master/log/20240325T203002Z.fail.html.gz
https://rubyci.s3.amazonaws.com/osx1200arm-no-yjit/ruby-master/log/20240325T195003Z.fail.html.gz
https://rubyci.s3.amazonaws.com/s390x/ruby-master/log/20240325T210003Z.fail.html.gz
...
so just skipt it until it works.
|
|
|
|
When error on finalizer, the exception will be ignored.
To restart the code, we need to restore the stack pointer.
fix [Bug #20042]
|
|
The test sometimes fails with:
1) Failure:
TestGc#test_stat_heap [/tmp/ruby/src/trunk-repeat50/test/ruby/test_gc.rb:169]:
Expected 33434403 to be <= 33434354.
|
|
It randomly fails like this:
https://github.com/ruby/ruby/actions/runs/7191443542/job/19586164973
|
|
|
|
|
|
This test creates a lot of Objects held in an array, and a set of weak
references to them using WeakMap. It then clears the array and frees it
and asserts that all the weak references to it are also gone.
This test is failing because one of the dummy objects in our weakmap is
ending up on the stack, and so is being marked, even though we thought
that we'd removed the only reference to it.
This behaviour has changed since this commit:
https://github.com/ruby/ruby/commit/5b5ae3d9e064e17e2a7d8d21d739fcc62ae1075c
which rewrites `Integer#times` from C into Ruby. This change is somehow
causing the last object we append to our array to consistently end up on
the stack during GC.
This commit fixes the specific weakmap test by using an enumerator and
each, instead of `Integer#times`, and thus avoids having our last object
created end up on the stack.
Notes:
Merged: https://github.com/ruby/ruby/pull/8402
|
|
The old algorithm could calculate an undercount for the initial pages
due to two issues:
1. It did not take into account that some heap pages will have one less
slot due to alignment. It assumed that every heap page would be able
to be fully filled with slots. Pages that are unaligned with the slot
size will lose one slot. The new algorithm assumes that every page
will be unaligned.
2. It performed integer division, which truncates down. This means that
the number of pages might not actually satisfy the number of slots.
This can cause the heap to grow in `gc_sweep_finish_size_pool` after
allocating all of the allocatable pages because the total number of
slots would be less than the initial configured number of slots.
Notes:
Merged: https://github.com/ruby/ruby/pull/8333
|
|
This commit changes RUBY_GC_HEAP_INIT_SIZE_{40,80,160,320,640}_SLOTS to
RUBY_GC_HEAP_{0,1,2,3,4}_INIT_SLOTS. This is easier to use because the
user does not need to determine the slot sizes (which can vary between
32 and 64 bit systems). They now just use the heap names
(`GC.stat_heap.keys`).
Notes:
Merged: https://github.com/ruby/ruby/pull/8335
|
|
If initial slots is set, then during a minor GC, if we have allocatable
pages but the heap is mostly full, then we will set `grow_heap` to true
since `total_slots` does not count allocatable pages so it will be less
than `init_slots`. This can cause `allocatable_pages` to grow to much
higher than desired since it will appear that the heap is mostly full.
Notes:
Merged: https://github.com/ruby/ruby/pull/8310
|
|
assert_in_out_err adds --disable=gems so we don't need to add
--disable-gems in the args list.
Notes:
Merged: https://github.com/ruby/ruby/pull/8303
|
|
This environment variable is replaced by
`RUBY_GC_HEAP_INIT_SIZE_%d_SLOTS`, so it doesn't make sense to keep it.
Notes:
Merged: https://github.com/ruby/ruby/pull/8147
|
|
[Feature #19783]
This commit adds stats about weak references to `GC.latest_gc_info`.
It adds the following two keys:
- `weak_references_count`: number of weak references registered during
the last GC.
- `retained_weak_references_count`: number of weak references that
survived the last GC.
Notes:
Merged: https://github.com/ruby/ruby/pull/8113
|
|
This commit moves the `total_freed_objects` statistic to the size pool
which allows for `total_freed_objects` key in `GC.stat_heap`.
Notes:
Merged: https://github.com/ruby/ruby/pull/8231
|
|
This commit moves the `total_allocated_objects` statistic to the size
pool which allows for `total_allocated_objects` key in `GC.stat_heap`.
Notes:
Merged: https://github.com/ruby/ruby/pull/8231
|
|
This commit adds key force_incremental_marking_finish_count to
GC.stat_heap. This statistic returns the number of times the size pool
has forced incremental marking to finish due to running out of slots.
|
|
assert_separately adds --disable=gems so we don't need to add
--disable-gems when calling assert_separately.
Notes:
Merged: https://github.com/ruby/ruby/pull/8162
|
|
Not setting a value to global_init_slots causes get_envparam_size to
output a broken default value.
|
|
If the stack is not cleared (e.g. compiling with -O0), then `ary` could
remain on the stack, which would be marked. Clear the array first to
make sure all the objects can be GC'd.
|
|
This commit stores the initial slots per size pool, configured with
the environment variables `RUBY_GC_HEAP_INIT_SIZE_%d_SLOTS`. This
ensures that the configured initial slots remains a low bound for the
number of slots in the heap, which can prevent heaps from thrashing in
size.
Notes:
Merged: https://github.com/ruby/ruby/pull/8116
|
|
Add comment for 7299c8c0f165247853fac2fe337e7c2678e653c9.
|
|
assert_not_nil could allocate objects which may trigger the major GC, so
don't run the assertions until the major GC has been ran.
Notes:
Merged: https://github.com/ruby/ruby/pull/8055
|
|
|
|
Fix:
```
1) Failure:
TestGc#test_latest_gc_info_need_major_by [/home/runner/work/ruby/ruby/src/test/ruby/test_gc.rb:266]:
<nil> expected to not be nil.
```
`GC.stat(:major_gc_count)` can be bumped while `GC.latest_gc_info(:need_major_by)` is still nil.
Notes:
Merged: https://github.com/ruby/ruby/pull/7895
|
|
[Feature #19678]
References from an old object to a write barrier protected young object
will not immediately promote the young object. Instead, the young object
will age just like any other object, meaning that it has to survive
three collections before being promoted to the old generation.
References from an old object to a write barrier unprotected object will
place the parent object in the remember set for marking during minor
collections. This allows the child object to be reclaimed in minor
collections at the cost of increased time for minor collections.
On one of [Shopify's highest traffic Ruby apps, Storefront
Renderer](https://shopify.engineering/how-shopify-reduced-storefront-response-times-rewrite),
we saw significant improvements after deploying this feature in
production. We compare the GC time and response time of web workers that
have the original behaviour (non-experimental group) and this new
behaviour (experimental group). We see that with this feature we spend
significantly less time in the GC, 0.81x on average, 0.88x on p99, and
0.45x on p99.9.
This translates to improvements in average response time (0.96x) and p99
response time (0.92x).
Notes:
Merged: https://github.com/ruby/ruby/pull/7821
|
|
[Feature #19571]
This commit adds the environment variable
`RUBY_GC_HEAP_REMEMBERED_WB_UNPROTECTED_OBJECTS_LIMIT_RATIO` which is
used to calculate the `remembered_wb_unprotected_objects_limit` using a
ratio of `old_objects`. This should improve performance by reducing
major GC because, in a major GC, we mark all of the old objects, so we
should have more uncollectible WB unprotected objects before starting a
major GC. The default has been set to 0.01 (1% of old objects).
On one of [Shopify's highest traffic Ruby apps, Storefront Renderer](https://shopify.engineering/how-shopify-reduced-storefront-response-times-rewrite),
we saw significant improvements after deploying this patch in
production. In the graphs below, we have the `tuned` group which uses
`RUBY_GC_HEAP_REMEMBERED_WB_UNPROTECTED_OBJECTS_LIMIT_RATIO=0.01` (the
default value), and an `untuned` group, which turns this feature off
with `RUBY_GC_HEAP_REMEMBERED_WB_UNPROTECTED_OBJECTS_LIMIT_RATIO=0`. We
see that the tuned group spends significantly less time in GC, on
average 0.67x of the time compared to the untuned group and 0.49x for
p99. We see this improvement in GC time translate to improvements in
response times. The average response time is now 0.96x of the time
compared to the untuned group and 0.86x for p99.
https://user-images.githubusercontent.com/15860699/229559078-e23e8ce4-5f1f-4a2f-b5ef-5769f92b8c70.png
Notes:
Merged: https://github.com/ruby/ruby/pull/7577
|
|
There is a `time` key in GC.stat that gives us the total time spent in
GC. However, we don't know what proportion of the time is spent between
marking and sweeping. This makes it difficult to tune the GC as we're
not sure where to focus our efforts on.
This PR adds keys `marking_time` and `sweeping_time` to GC.stat for the
time spent marking and sweeping, in milliseconds.
[Feature #19437]
Notes:
Merged: https://github.com/ruby/ruby/pull/7304
|
|
The old RUBY_GC_HEAP_INIT_SLOTS isn't really usable anymore as
it initalize all the pools by the same factor, but it's unlikely
that pools will need similar sizes.
In production our 40B pool is 5 to 6 times bigger than our 80B pool.
Notes:
Merged: https://github.com/ruby/ruby/pull/7235
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/7263
|
|
The test added in 90a80eb0 fails if Ruby is verbose, it outputs the
following line to stderr:
RUBY_GC_HEAP_INIT_SLOTS=100 (default value: 10000)
|
|
There is an integer underflow when the environment variable
RUBY_GC_HEAP_INIT_SLOTS is less than the number of slots currently
in the Ruby heap.
[Bug #19284]
Notes:
Merged: https://github.com/ruby/ruby/pull/7044
|
