summaryrefslogtreecommitdiff
path: root/gc.c
AgeCommit message (Collapse)Author
2022-04-13Parenthize macro argumentsNobuyoshi Nakada
2022-04-12Fix a typo [ci skip]Kazuhiro NISHIYAMA
2022-04-09[DOC]Some link prefix replaceS-H-GAMELINKS
Notes: Merged: https://github.com/ruby/ruby/pull/5783
2022-04-07Update `heap_pages_deferred_final` atomicallyNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/5775
2022-04-04ruby_gc_set_params: update malloc_limit when env is setEric Wong
During VM startup, rb_objspace_alloc sets malloc_limit (objspace->malloc_params.limit) before ruby_gc_set_params is called, thus nullifying the effect of RUBY_GC_MALLOC_LIMIT before the initial GC run. The call sequence is as follows: main.c::main() ruby_init ruby_setup Init_BareVM rb_objspace_alloc // malloc_limit = gc_params.malloc_limit_min; ruby_options ruby_process_options process_options ruby_gc_set_params // RUBY_GC_MALLOC_LIMIT => gc_params.malloc_limit_min With ruby_gc_set_params setting malloc_limit, RUBY_GC_MALLOC_LIMIT affects the process sooner. [ruby-core:107170]
2022-04-04Disable mmap on WASMPeter Zhu
WASM does not have proper support for mmap. Notes: Merged: https://github.com/ruby/ruby/pull/5749
2022-04-04Make heap page sizes 64KiB by defaultPeter Zhu
Commit dde164e968e382d50b07ad4559468885cbff33ef decoupled incremental marking from page sizes. This commit changes Ruby heap page sizes to 64KiB. Doing so will have several benefits: 1. We can use compaction on systems with 64KiB system page sizes (e.g. PowerPC). 2. Larger page sizes will allow Variable Width Allocation to increase slot sizes and embed larger objects. 3. Since commit 002fa2859962f22de8afdbeece04966ea57b7da9, macOS has 64 KiB pages. Making page sizes 64 KiB will bring these systems to parity. I have attached some bechmark results below. Discourse: On Discourse, we saw much better p99 performance (e.g. for "categories" it went from 214ms on master to 134ms on branch, for "home" it went from 265ms to 251ms). We don’t see much change in p60, p75, and p90 performance. We also see a slight decrease in memory usage by 1.04x. Branch RSS: 354.9MB Master RSS: 368.2MB railsbench: On rails bench, we don’t see a big change in RPS or p99 performance. We don’t see a big difference in memory usage. Branch RPS: 826.27 Master RPS: 824.85 Branch p99: 1.67 Master p99: 1.72 Branch RSS: 88.72MB Master RSS: 88.48MB liquid: We don’t see a significant change in liquid performance. Branch parse & render: 28.653 I/s Master parse & render: 28.563 i/s Notes: Merged: https://github.com/ruby/ruby/pull/5749
2022-04-01extract magic number from gc_sweep_stepMatt Valentine-House
Notes: Merged: https://github.com/ruby/ruby/pull/5746
2022-04-01Use mmap for heap page allocation onlyPeter Zhu
Currently, rb_aligned_malloc uses mmap if Ruby heap pages can be allocated through mmap (when system heap page size <= Ruby heap page size). If Ruby heap page sizes is increased to 64KiB, then mmap will be used on systems with 64KiB system page sizes. However, the transient heap also uses rb_aligned_malloc and requires 32KiB alignment. This would break in the current implementation since it would allocate sizes through mmap that is not a multiple of the system page size. This commit adds heap_page_body_allocate which will use mmap when possible and changes rb_aligned_malloc to not use mmap (and only use posix_memalign).
2022-04-01[Feature #18619] remove FL_FROM_FREELISTMatt Valentine-House
Notes: Merged: https://github.com/ruby/ruby/pull/5637
2022-04-01[Feature #18619] Remove redundant compaction pathMatt Valentine-House
Notes: Merged: https://github.com/ruby/ruby/pull/5637
2022-04-01[Feature #18619] Reverse the order of compaction movementMatt Valentine-House
This commit changes the way compaction moves objects and sweeps pages in order to better facilitate object movement between size pools. Previously we would move the scan cursor first until we found an empty slot and then we'd decrement the compact cursor until we found something to move into that slot. We would sweep the page that contained the scan cursor before trying to fill it In this algorithm we first move the compact cursor down until we find an object to move - We then take a free page from the desired destination heap (always the same heap in this current iteration of the code). If there is no free page we sweep the page at the sweeping_page cursor, add it to the free pages, and advance the cursor to the next page, and try again. We sweep one page from each size pool in this way, and then repeat that process until all the size pools are compacted (all the cursors have met), and then we update references and sweep the rest of the heap. Notes: Merged: https://github.com/ruby/ruby/pull/5637
2022-03-31Remove hard-coded swept slots thresholdMatt Valentine-House
Notes: Merged: https://github.com/ruby/ruby/pull/5720
2022-03-30Decouple incremental marking step from page sizesPeter Zhu
Currently, the number of incremental marking steps is calculated based on the number of pooled pages available. This means that if we make Ruby heap pages larger, it would run fewer incremental marking steps (which would mean each incremental marking step takes longer). This commit changes incremental marking to run after every INCREMENTAL_MARK_STEP_ALLOCATIONS number of allocations. This means that the behaviour of incremental marking remains the same regardless of the Ruby heap page size. I've benchmarked against discourse benchmarks and did not get a significant change in response times beyond the margin of error. This is expected as this new incremental marking algorithm behaves very similarly to the previous one. Notes: Merged: https://github.com/ruby/ruby/pull/5732
2022-03-30Prefix ccan headers (#4568)Nobuyoshi Nakada
* Prefixed ccan headers * Remove unprefixed names in ccan/build_assert * Remove unprefixed names in ccan/check_type * Remove unprefixed names in ccan/container_of * Remove unprefixed names in ccan/list Co-authored-by: Samuel Williams <samuel.williams@oriontransfer.co.nz> Notes: Merged-By: ioquatix <samuel@codeotaku.com>
2022-03-28Remove unneeded function declarations in gc.cPeter Zhu
2022-03-24Add ISEQ_BODY macroPeter Zhu
Use ISEQ_BODY macro to get the rb_iseq_constant_body of the ISeq. Using this macro will make it easier for us to change the allocation strategy of rb_iseq_constant_body when using Variable Width Allocation. Notes: Merged: https://github.com/ruby/ruby/pull/5698
2022-03-03Dedup superclass array in leaf sibling classesJohn Hawthorn
Previously, we would build a new `superclasses` array for each class, even though for all immediate subclasses of a class, the array is identical. This avoids duplicating the arrays on leaf classes (those without subclasses) by calculating and storing a "superclasses including self" array on a class when it's first inherited and sharing that among all superclasses. An additional trick used is that the "superclass array including self" is valid as "self"'s superclass array. It just has it's own class at the end. We can use this to avoid an extra pointer of storage and can use one bit of a flag to track that we've "upgraded" the array. Notes: Merged: https://github.com/ruby/ruby/pull/5604
2022-02-23Constant time class to class ancestor lookupJohn Hawthorn
Previously when checking ancestors, we would walk all the way up the ancestry chain checking each parent for a matching class or module. I believe this was especially unfriendly to CPU cache since for each step we need to check two cache lines (the class and class ext). This check is used quite often in: * case statements * rescue statements * Calling protected methods * Class#is_a? * Module#=== * Module#<=> I believe it's most common to check a class against a parent class, to this commit aims to improve that (unfortunately does not help checking for an included Module). This is done by storing on each class the number and an array of all parent classes, in order (BasicObject is at index 0). Using this we can check whether a class is a subclass of another in constant time since we know the location to expect it in the hierarchy. Notes: Merged: https://github.com/ruby/ruby/pull/5568
2022-02-16Change darray size to size_t and add functions that use GC mallocPeter Zhu
Changes size and capacity of darray to size_t to support more elements. Adds functions to darray that use GC allocation functions. Notes: Merged: https://github.com/ruby/ruby/pull/5546
2022-02-16`wmap#each` should check liveness of keysKoichi Sasada
`ObjectSpace::WeakMap#each*` should check key's liveness. fix [Bug #18586] Notes: Merged: https://github.com/ruby/ruby/pull/5556
2022-02-14fix GC event synchronizationKoichi Sasada
(1) gc_verify_internal_consistency() use barrier locking for consistency while `during_gc == true` at the end of the sweep on `RGENGC_CHECK_MODE >= 2`. (2) `rb_objspace_reachable_objects_from()` is called without VM synchronization and it checks `during_gc != true`. So (1) and (2) causes BUG because of `during_gc == true`. To prevent this error, wait for VM barrier on `during_gc == false` and introduce VM locking on `rb_objspace_reachable_objects_from()`. http://ci.rvm.jp/results/trunk-asserts@phosphorus-docker/3830088 Notes: Merged: https://github.com/ruby/ruby/pull/5552
2022-02-10Free cached mark stack chunks when freeing objspacePeter Zhu
Cached mark stack chunks should also be freed when freeing objspace. Notes: Merged: https://github.com/ruby/ruby/pull/5536
2022-02-03Move total_freed_pages to size poolPeter Zhu
Notes: Merged: https://github.com/ruby/ruby/pull/5523
2022-02-03Move total_allocated_pages to size poolPeter Zhu
Notes: Merged: https://github.com/ruby/ruby/pull/5523
2022-02-03Fix case when gc_marks_continue does not yield slotsPeter Zhu
gc_marks_continue will start sweeping when it finishes marking. However, if the heap we are trying to allocate into is full, then the sweeping may not yield any free slots. If we don't call gc_sweep_continue immediate after this, then another GC will be started halfway during lazy sweeping. gc_sweep_continue will either grow the heap or finish sweeping. Notes: Merged: https://github.com/ruby/ruby/pull/5521
2022-02-02Decouple GC slot sizes from RVALUEPeter Zhu
Add a new macro BASE_SLOT_SIZE that determines the slot size. For Variable Width Allocation (compiled with USE_RVARGC=1), all slot sizes are powers-of-2 multiples of BASE_SLOT_SIZE. For USE_RVARGC=0, BASE_SLOT_SIZE is set to sizeof(RVALUE). Notes: Merged: https://github.com/ruby/ruby/pull/5517
2022-01-31Fix heap page iteration in gc_verify_heap_pagePeter Zhu
The for loops are not correctly iterating heap pages in gc_verify_heap_page. Notes: Merged: https://github.com/ruby/ruby/pull/5503
2022-01-29[Bug#18556] Fallback `MAP_ ANONYMOUS`Nobuyoshi Nakada
Define `MAP_ANONYMOUS` to `MAP_ANON` if undefined on old systems. Notes: Merged: https://github.com/ruby/ruby/pull/5506 Merged-By: nobu <nobu@ruby-lang.org>
2022-01-26Fix typo in assertion in gc.cPeter Zhu
2022-01-26Unpoison the cached object in the exact sizeNobuyoshi Nakada
2022-01-25Call rb_id_table_foreach_values insteadPeter Zhu
These places never replace the value, so call rb_id_table_foreach_values instead of rb_id_table_foreach_values_with_replace. Notes: Merged: https://github.com/ruby/ruby/pull/5486
2022-01-25Rename rb_id_table_foreach_with_replacePeter Zhu
Renames rb_id_table_foreach_with_replace to rb_id_table_foreach_values_with_replace and passes only the value to the callback. We can use this in GC compaction when we cannot access the global symbol array. Notes: Merged: https://github.com/ruby/ruby/pull/5486
2022-01-25Remove redundant if statement in try_movePeter Zhu
The if statement is redundant since if `index == 0` then `BITS_BITLENGTH * index == 0`. Notes: Merged: https://github.com/ruby/ruby/pull/5479
2022-01-24Keep right operand within width when right shiftingPeter Zhu
NUM_IN_PAGE could return a value much larger than 64. According to the C11 spec 6.5.7 paragraph 3 this is undefined behavior: > If the value of the right operand is negative or is greater than or > equal to the width of the promoted left operand, the behavior is > undefined. On most platforms, this is usually not a problem as the architecture will mask off all out-of-range bits. Notes: Merged: https://github.com/ruby/ruby/pull/5478
2022-01-24[wasm] Disallow compactionPeter Zhu
WebAssembly doesn't support signals so we can't use read barriers so we can't use compaction. Notes: Merged: https://github.com/ruby/ruby/pull/5475
2022-01-19Fix format size qualifier on IL32P64Nobuyoshi Nakada
2022-01-19[wasm] gc.c: scan wasm locals and c stack to mark living objectsYuta Saito
WebAssembly has function local infinite registers and stack values, but there is no way to scan the values in a call stack for now. This implementation uses Asyncify to spilling out wasm locals into linear memory. Notes: Merged: https://github.com/ruby/ruby/pull/5407
2022-01-19[wasm] gc.c: disable read signal barrier for wasiYuta Saito
WASI currently does not yet support signal Notes: Merged: https://github.com/ruby/ruby/pull/5407
2022-01-19[wasm] eval_inter.h gc.c vm_core.h: include wasm/setjmp.h instead of sysroot ↵Yuta Saito
header Notes: Merged: https://github.com/ruby/ruby/pull/5407
2022-01-14Separately allocate class_serial on 32-bit systemsPeter Zhu
On 32-bit systems, VWA causes class_serial to not be aligned (it only guarantees 4 byte alignment but class_serial is 8 bytes and requires 8 byte alignment). This commit uses a hack to allocate class_serial through malloc. Once VWA allocates with 8 byte alignment in the future, we will revert this commit. Notes: Merged: https://github.com/ruby/ruby/pull/5442
2022-01-07Improve string info in rb_raw_obj_infoPeter Zhu
Improve rb_raw_obj_info to output additional into about strings including the length, capacity, and whether or not it is embedded. Notes: Merged: https://github.com/ruby/ruby/pull/5414
2022-01-05Remove assertion causing read barrier to triggerPeter Zhu
GET_HEAP_PAGE reads the page. If during compaction there is a read barrier on the page, it causes the read barrier to trigger. Notes: Merged: https://github.com/ruby/ruby/pull/5394
2022-01-04Switch `is_pointer_to_heap` to use library bsearchMatt Valentine-House
This commit switches from a custom implemented bsearch algorithm to use the one provided by the C standard library. Because `is_pointer_to_heap` will only return true if the pointer being searched for is a valid slot starting address within the heap page body, we've extracted the bsearch call site into a more general function so we can use it elsewhere. The new function `heap_page_for_ptr` returns the heap page for any heap page pointer, regardless of whether that is at the start of a slot or in the middle of one. We then use this function as the basis of `is_pointer_to_heap`. Notes: Merged: https://github.com/ruby/ruby/pull/5187
2022-01-04[Feature #18364] Add GC.stat_heap to get stats for memory heapsPeter Zhu
GC.stat_heap will return stats for memory heaps. This is used for the Variable Width Allocation feature. Notes: Merged: https://github.com/ruby/ruby/pull/5177
2022-01-01Negative RBOOL usageNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/5385
2021-12-29On 64bit macOS, enlarge heap pages to reduce mmap calls [Bug #18447]Nobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/5371
2021-12-26Remove deprecate rb_cData [Bug #18433]Nobuyoshi Nakada
Also enable the warning for T_DATA allocator. Notes: Merged: https://github.com/ruby/ruby/pull/5348
2021-12-23`finalize_deferred` doesn't need VM lockKoichi Sasada
`finalize_list()` acquires VM lock to manipulate objspace state. Notes: Merged: https://github.com/ruby/ruby/pull/5326
2021-12-23undef `rb_vm_lookup_overloaded_cme()`Koichi Sasada
Some callable method entries (cme) can be a key of `overloaded_cme_table` and the keys should be pinned because the table is numtable (VALUE is a key). Before the patch GC checks the cme is in `overloaded_cme_table` by looking up the table, but it needs VM locking. It works well in normal GC marking because it is protected by the VM lock, but it doesn't work on `rb_objspace_reachable_objects_from` because it doesn't use VM lock. Now, the number of target cmes are small enough, I decide to pin down all possible cmes instead of using looking up the table. Notes: Merged: https://github.com/ruby/ruby/pull/5327
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-02 12:17:59 +0000