summaryrefslogtreecommitdiff
path: root/gc.c
AgeCommit message (Collapse)Author
2022-01-14Separately allocate class_serial on 32-bit systemsPeter Zhu
On 32-bit systems, VWA causes class_serial to not be aligned (it only guarantees 4 byte alignment but class_serial is 8 bytes and requires 8 byte alignment). This commit uses a hack to allocate class_serial through malloc. Once VWA allocates with 8 byte alignment in the future, we will revert this commit. Notes: Merged: https://github.com/ruby/ruby/pull/5442
2022-01-07Improve string info in rb_raw_obj_infoPeter Zhu
Improve rb_raw_obj_info to output additional into about strings including the length, capacity, and whether or not it is embedded. Notes: Merged: https://github.com/ruby/ruby/pull/5414
2022-01-05Remove assertion causing read barrier to triggerPeter Zhu
GET_HEAP_PAGE reads the page. If during compaction there is a read barrier on the page, it causes the read barrier to trigger. Notes: Merged: https://github.com/ruby/ruby/pull/5394
2022-01-04Switch `is_pointer_to_heap` to use library bsearchMatt Valentine-House
This commit switches from a custom implemented bsearch algorithm to use the one provided by the C standard library. Because `is_pointer_to_heap` will only return true if the pointer being searched for is a valid slot starting address within the heap page body, we've extracted the bsearch call site into a more general function so we can use it elsewhere. The new function `heap_page_for_ptr` returns the heap page for any heap page pointer, regardless of whether that is at the start of a slot or in the middle of one. We then use this function as the basis of `is_pointer_to_heap`. Notes: Merged: https://github.com/ruby/ruby/pull/5187
2022-01-04[Feature #18364] Add GC.stat_heap to get stats for memory heapsPeter Zhu
GC.stat_heap will return stats for memory heaps. This is used for the Variable Width Allocation feature. Notes: Merged: https://github.com/ruby/ruby/pull/5177
2022-01-01Negative RBOOL usageNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/5385
2021-12-29On 64bit macOS, enlarge heap pages to reduce mmap calls [Bug #18447]Nobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/5371
2021-12-26Remove deprecate rb_cData [Bug #18433]Nobuyoshi Nakada
Also enable the warning for T_DATA allocator. Notes: Merged: https://github.com/ruby/ruby/pull/5348
2021-12-23`finalize_deferred` doesn't need VM lockKoichi Sasada
`finalize_list()` acquires VM lock to manipulate objspace state. Notes: Merged: https://github.com/ruby/ruby/pull/5326
2021-12-23undef `rb_vm_lookup_overloaded_cme()`Koichi Sasada
Some callable method entries (cme) can be a key of `overloaded_cme_table` and the keys should be pinned because the table is numtable (VALUE is a key). Before the patch GC checks the cme is in `overloaded_cme_table` by looking up the table, but it needs VM locking. It works well in normal GC marking because it is protected by the VM lock, but it doesn't work on `rb_objspace_reachable_objects_from` because it doesn't use VM lock. Now, the number of target cmes are small enough, I decide to pin down all possible cmes instead of using looking up the table. Notes: Merged: https://github.com/ruby/ruby/pull/5327
2021-12-21make `overloaded_cme_table` truly weak key mapKoichi Sasada
`overloaded_cme_table` keeps cme -> monly_cme pairs to manage corresponding `monly_cme` for `cme`. The lifetime of the `monly_cme` should be longer than `monly_cme`, but the previous patch losts the reference to the living `monly_cme`. Now `overloaded_cme_table` values are always root (keys are only weak reference), it means `monly_cme` does not freed until corresponding `cme` is invalidated. To make managing easy, move `overloaded_cme_table` to `rb_vm_t`. Notes: Merged: https://github.com/ruby/ruby/pull/5316
2021-12-21`mandatory_only_cme` should not be in `def`Koichi Sasada
`def` (`rb_method_definition_t`) is shared by multiple callable method entries (cme, `rb_callable_method_entry_t`). There are two issues: * old -> young reference: `cme1->def->mandatory_only_cme = monly_cme` if `cme1` is young and `monly_cme` is young, there is no problem. Howevr, another old `cme2` can refer `def`, in this case, old `cme2` points young `monly_cme` and it violates gengc assumption. * cme can have different `defined_class` but `monly_cme` only has one `defined_class`. It does not make sense and `monly_cme` should be created for a cme (not `def`). To solve these issues, this patch allocates `monly_cme` per `cme`. `cme` does not have another room to store a pointer to the `monly_cme`, so this patch introduces `overloaded_cme_table`, which is weak key map `[cme] -> [monly_cme]`. `def::body::iseqptr::monly_cme` is deleted. The first issue is reported by Alan Wu. Notes: Merged: https://github.com/ruby/ruby/pull/5311
2021-12-20Show whether object is garbage in rb_raw_obj_info()Alan Wu
When using `rp(obj)` for debugging during development, it may be useful to know that an object is soon to be swept. Add a new letter to the object dump for whether the object is garbage. It's easy to forget about lazy sweep. Notes: Merged: https://github.com/ruby/ruby/pull/5310 Merged-By: XrXr
2021-12-14Remove compaction support detection using sysconfPeter Zhu
Except on Windows and MinGW, we can only use compaction on systems that use mmap (only systems that use mmap can use the read barrier that compaction requires). We don't need to separately detect whether we can support compaction or not. Notes: Merged: https://github.com/ruby/ruby/pull/5260
2021-12-07Fixed the check order in wmap_live_p [Bug #18392]Nobuyoshi Nakada
Check if the object is a pointer to heap before check the flag in that object. Notes: Merged: https://github.com/ruby/ruby/pull/5224
2021-12-07ObjectSpace::WeakMap#inspect: check if living object [Bug #18392]Nobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/5224
2021-12-03Refactor GC functions to have consistent namingPeter Zhu
Refactor function names for consistency. Function with name xyz_page should have a corresponding function named xyz_plane. Notes: Merged: https://github.com/ruby/ruby/pull/5204
2021-12-02Lazily create singletons on instance_{exec,eval} (#5146)John Hawthorn
* Lazily create singletons on instance_{exec,eval} Previously when instance_exec or instance_eval was called on an object, that object would be given a singleton class so that method definitions inside the block would be added to the object rather than its class. This commit aims to improve performance by delaying the creation of the singleton class unless/until one is needed for method definition. Most of the time instance_eval is used without any method definition. This was implemented by adding a flag to the cref indicating that it represents a singleton of the object rather than a class itself. In this case CREF_CLASS returns the object's existing class, but in cases that we are defining a method (either via definemethod or VM_SPECIAL_OBJECT_CBASE which is used for undef and alias). This also happens to fix what I believe is a bug. Previously instance_eval behaved differently with regards to constant access for true/false/nil than for all other objects. I don't think this was intentional. String::Foo = "foo" "".instance_eval("Foo") # => "foo" Integer::Foo = "foo" 123.instance_eval("Foo") # => "foo" TrueClass::Foo = "foo" true.instance_eval("Foo") # NameError: uninitialized constant Foo This also slightly changes the error message when trying to define a method through instance_eval on an object which can't have a singleton class. Before: $ ruby -e '123.instance_eval { def foo; end }' -e:1:in `block in <main>': no class/module to add method (TypeError) After: $ ./ruby -e '123.instance_eval { def foo; end }' -e:1:in `block in <main>': can't define singleton (TypeError) IMO this error is a small improvement on the original and better matches the (both old and new) message when definging a method using `def self.` $ ruby -e '123.instance_eval{ def self.foo; end }' -e:1:in `block in <main>': can't define singleton (TypeError) Co-authored-by: Matthew Draper <matthew@trebex.net> * Remove "under" argument from yield_under * Move CREF_SINGLETON_SET into vm_cref_new * Simplify vm_get_const_base * Fix leaf VM_SPECIAL_OBJECT_CONST_BASE Co-authored-by: Matthew Draper <matthew@trebex.net> Notes: Merged-By: jhawthorn <john@hawthorn.email>
2021-12-02Don't clear the constant cache when finishing compactionMatt Valentine-House
References are being updated correctly, so this is no longer necessary Notes: Merged: https://github.com/ruby/ruby/pull/5198
2021-12-02Cast tv_usec to int32_t to fit in tv_nsecYuta Saito
suseconds_t, which is the type of tv_usec, may be defined with a longer size type than tv_nsec's type (long). So usec to nsec conversion needs an explicit casting. Notes: Merged: https://github.com/ruby/ruby/pull/5200
2021-11-26Fix a function name in an error messageKazuhiro NISHIYAMA
Notes: Merged: https://github.com/ruby/ruby/pull/5181
2021-11-26Remove unused function `size_pool_for_size`Kazuhiro NISHIYAMA
``` compiling ../gc.c ../gc.c:2444:1: warning: unused function 'size_pool_for_size' [-Wunused-function] size_pool_for_size(rb_objspace_t *objspace, size_t size) ^ 1 warning generated. ``` Notes: Merged: https://github.com/ruby/ruby/pull/5181
2021-11-26initialize allocated memory by VWA for assertionsKoichi Sasada
When `RGENGC_CHECK_MODE` is enable, `obj_memsize_of` is called in `newobj_init` and it expect the memory is zero-cleared.
2021-11-25Revert "Add GC.stat_size_pool to get stats for a size pool"Peter Zhu
This reverts commit 6157619bb68e4307cdf065cb73d5bfcec30d042d. We'll wait for comments in the open ticket: https://bugs.ruby-lang.org/issues/18364 Notes: Merged: https://github.com/ruby/ruby/pull/5176
2021-11-25Add GC.stat_size_pool to get stats for a size poolPeter Zhu
GC.stat_size_pool will return stats for a particular size pool. This is used for the Variable Width Allocation feature. Notes: Merged: https://github.com/ruby/ruby/pull/5169
2021-11-23Speed up Ractors for Variable Width AllocationPeter Zhu
This commit adds a Ractor cache for every size pool. Previously, all VWA allocated objects used the slowpath and locked the VM. On a micro-benchmark that benchmarks String allocation: VWA turned off: 29.196591 0.889709 30.086300 ( 9.434059) VWA before this commit: 29.279486 41.477869 70.757355 ( 12.527379) VWA after this commit: 16.782903 0.557117 17.340020 ( 4.255603) Notes: Merged: https://github.com/ruby/ruby/pull/5151
2021-11-22Removes unused HEAP_PAGE_BITMAP_PLANES constant from gc.cJemma Issroff
Notes: Merged: https://github.com/ruby/ruby/pull/4154
2021-11-22Make RCLASS_EXT(c)->subclasses a doubly linked listMatt Valentine-House
Updating RCLASS_PARENT_SUBCLASSES and RCLASS_MODULE_SUBCLASSES while compacting can trigger the read barrier. This commit makes RCLASS_SUBCLASSES a doubly linked list with a dedicated head object so that we can add and remove entries from the list without having to touch an object in the Ruby heap Notes: Merged: https://github.com/ruby/ruby/pull/5125
2021-11-19gc.c: Fix a compile error on some crossbuildsYusuke Endoh
http://rubyci.s3.amazonaws.com/crossruby/crossruby-master-wasm32_emscripten/log/20211118T233311Z.log.html.gz#make ``` compiling gc.c gc.c:10629:47: error: implicit conversion loses integer precision: 'unsigned long long' to 'size_t' (aka 'unsigned long') [-Werror,-Wshorten-64-to-32] SET(time, objspace->profile.total_time_ns / (1000 * 1000) /* ns -> ms */); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ gc.c:10624:9: note: expanded from macro 'SET' return attr; \ ~~~~~~ ^~~~ gc.c:10629:47: error: implicit conversion loses integer precision: 'unsigned long long' to 'unsigned long' [-Werror,-Wshorten-64-to-32] SET(time, objspace->profile.total_time_ns / (1000 * 1000) /* ns -> ms */); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ gc.c:10626:68: note: expanded from macro 'SET' rb_hash_aset(hash, gc_stat_symbols[gc_stat_sym_##name], SIZET2NUM(attr)); ~~~~~~~~~ ^~~~ 2 errors generated. ```
2021-11-19GC measurement featureKoichi Sasada
* `GC.measure_total_time = true` enables total time measurement (default: true) * `GC.measure_total_time` returns current flag. * `GC.total_time` returns measured total time in nano seconds. * `GC.stat(:time)` (and Hash) returns measured total time in milli seconds. Notes: Merged: https://github.com/ruby/ruby/pull/4757
2021-11-19support `GC.stat(:time)` take 2Koichi Sasada
Notes: Merged: https://github.com/ruby/ruby/pull/4757
2021-11-15`Primitive.mandatory_only?` for fast pathKoichi Sasada
Compare with the C methods, A built-in methods written in Ruby is slower if only mandatory parameters are given because it needs to check the argumens and fill default values for optional and keyword parameters (C methods can check the number of parameters with `argc`, so there are no overhead). Passing mandatory arguments are common (optional arguments are exceptional, in many cases) so it is important to provide the fast path for such common cases. `Primitive.mandatory_only?` is a special builtin function used with `if` expression like that: ```ruby def self.at(time, subsec = false, unit = :microsecond, in: nil) if Primitive.mandatory_only? Primitive.time_s_at1(time) else Primitive.time_s_at(time, subsec, unit, Primitive.arg!(:in)) end end ``` and it makes two ISeq, ``` def self.at(time, subsec = false, unit = :microsecond, in: nil) Primitive.time_s_at(time, subsec, unit, Primitive.arg!(:in)) end def self.at(time) Primitive.time_s_at1(time) end ``` and (2) is pointed by (1). Note that `Primitive.mandatory_only?` should be used only in a condition of an `if` statement and the `if` statement should be equal to the methdo body (you can not put any expression before and after the `if` statement). A method entry with `mandatory_only?` (`Time.at` on the above case) is marked as `iseq_overload`. When the method will be dispatch only with mandatory arguments (`Time.at(0)` for example), make another method entry with ISeq (2) as mandatory only method entry and it will be cached in an inline method cache. The idea is similar discussed in https://bugs.ruby-lang.org/issues/16254 but it only checks mandatory parameters or more, because many cases only mandatory parameters are given. If we find other cases (optional or keyword parameters are used frequently and it hurts performance), we can extend the feature. Notes: Merged: https://github.com/ruby/ruby/pull/5112
2021-11-11Remove RCLASS(obj)->ptr when RVARGC is enabledMatt Valentine-House
With RVARGC we always store the rb_classext_t in the same slot as the RClass struct that refers to it. So we don't need to store the pointer or access through the pointer anymore and can switch the RCLASS_EXT macro to use an offset Notes: Merged: https://github.com/ruby/ruby/pull/5101
2021-11-11fix a memory leak introduced in 8bbd319Matt Valentine-House
This commit fixes a memory leak introduced in an early part of the variable width allocation project that would prevent the rb_classext_t struct from being free'd when the class is swept. Notes: Merged: https://github.com/ruby/ruby/pull/5103
2021-11-08[Feature #18290] Deprecate rb_gc_force_recycle and remove ↵Peter Zhu
invalidate_mark_stack_chunk This commit deprecates rb_gc_force_recycle and coverts it to a no-op function. Also removes invalidate_mark_stack_chunk since only rb_gc_force_recycle uses it. Notes: Merged: https://github.com/ruby/ruby/pull/4363
2021-10-29make obj_free return true when it frees an objectMatt Valentine-House
Previously obj_free returned true when it could not free a slot because of a finalizer, and false when it successfully frees a slot. Notes: Merged: https://github.com/ruby/ruby/pull/5055
2021-10-29Prefer size pool heap macros over direct accessMatt Valentine-House
Notes: Merged: https://github.com/ruby/ruby/pull/5054
2021-10-28Fix a warningKazuhiro NISHIYAMA
``` ../gc.c:2342:45: warning: comparison of integers of different signs: 'short' and 'size_t' (aka 'unsigned long') [-Wsign-compare] GC_ASSERT(size_pools[pool_id].slot_size == slot_size); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~ ``` Add cast to short, because `GC_ASSERT`s in `size_pool_for_size` already use cast to short.
2021-10-28Fix a warningKazuhiro NISHIYAMA
``` ../gc.c:2342:25: warning: array subscript is of type 'char' [-Wchar-subscripts] GC_ASSERT(size_pools[pool_id].slot_size == slot_size); ^~~~~~~~ ```
2021-10-27Align `RFloat` at VALUE boundaryNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/5027
2021-10-25[Feature #18239] Implement VWA for stringsPeter Zhu
This commit adds support for embedded strings with variable capacity and uses Variable Width Allocation to allocate strings. Notes: Merged: https://github.com/ruby/ruby/pull/4933
2021-10-25[Feature #18239] Refactor RVARGC alloc functionsPeter Zhu
The allocation functions no longer assume that one RVALUE needs to be allocated. Notes: Merged: https://github.com/ruby/ruby/pull/4933
2021-10-24Suppress false warning for freed pointerNobuyoshi Nakada
2021-10-21Push compaction page alignment check downAaron Patterson
It seems like `gc_verify_compaction_references` is not protected in case alignment is wrong. This commit pushes the alignment check down to `gc_start_internal` so that anyone trying to compact will check page alignment I think this method may be getting called on PowerPC and the alignment might be wrong. http://rubyci.s3.amazonaws.com/ppc64le/ruby-master/log/20211021T190006Z.fail.html.gz Notes: Merged: https://github.com/ruby/ruby/pull/5001
2021-10-20Partial revert of ceebc7fc98dAaron Patterson
I'm looking through the places where YJIT needs notifications. It looks like these changes to gc.c and vm_callinfo.h have become unnecessary since 84ab77ba592. This commit just makes the diff against upstream smaller, but otherwise shouldn't change any behavior.
2021-10-20MicroJIT: generate less code for CFUNCsAlan Wu
Added UJIT_CHECK_MODE. Set to 1 to double check method dispatch in generated code. It's surprising to me that we need to watch both cc and cme. There might be opportunities to simplify there.
2021-10-20Print errno when mprotect failsAaron Patterson
Trying to figure out the problem on s390x. Notes: Merged: https://github.com/ruby/ruby/pull/4996
2021-10-04Move rb_ractor_p definitionS.H
Notes: Merged: https://github.com/ruby/ruby/pull/4422 Merged-By: nobu <nobu@ruby-lang.org>
2021-10-03Using NIL_P macro instead of `== Qnil`S.H
Notes: Merged: https://github.com/ruby/ruby/pull/4925 Merged-By: nobu <nobu@ruby-lang.org>
2021-10-03Cast to void pointer to suppress -Wformat-pedantic in RUBY_DEBUG_LOGNobuyoshi Nakada