<feed xmlns='http://www.w3.org/2005/Atom'>
<title>ruby.git/test/ruby/test_gc_compact.rb, branch v3_4_9</title>
<subtitle>The Ruby Programming Language</subtitle>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/'/>
<entry>
<title>Rename supports_auto_compact? to supports_compact?</title>
<updated>2024-10-11T19:29:10+00:00</updated>
<author>
<name>Peter Zhu</name>
<email>peter@peterzhu.ca</email>
</author>
<published>2024-10-11T18:08:59+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=8aeb60aec88dd68fdfbaa75ca06e65188233ccbf'/>
<id>8aeb60aec88dd68fdfbaa75ca06e65188233ccbf</id>
<content type='text'>
It's testing whether GC compaction is supported in general.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
It's testing whether GC compaction is supported in general.
</pre>
</div>
</content>
</entry>
<entry>
<title>Used respond_to? check for compaction in test_gc_compact.rb</title>
<updated>2024-10-11T19:29:10+00:00</updated>
<author>
<name>Peter Zhu</name>
<email>peter@peterzhu.ca</email>
</author>
<published>2024-10-11T18:01:08+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=6a88e9d43088f51d3d9dde9c89f24836ab715810'/>
<id>6a88e9d43088f51d3d9dde9c89f24836ab715810</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Rename size_pool -&gt; heap</title>
<updated>2024-10-03T20:20:09+00:00</updated>
<author>
<name>Matt Valentine-House</name>
<email>matt@eightbitraptor.com</email>
</author>
<published>2024-10-03T12:53:49+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=8e7df4b7c674cf408fa570b9593811167bbff04a'/>
<id>8e7df4b7c674cf408fa570b9593811167bbff04a</id>
<content type='text'>
Now that we've inlined the eden_heap into the size_pool, we should
rename the size_pool to heap. So that Ruby contains multiple heaps, with
different sized objects.

The term heap as a collection of memory pages is more in memory
management nomenclature, whereas size_pool was a name chosen out of
necessity during the development of the Variable Width Allocation
features of Ruby.

The concept of size pools was introduced in order to facilitate
different sized objects (other than the default 40 bytes). They wrapped
the eden heap and the tomb heap, and some related state, and provided a
reasonably simple way of duplicating all related concerns, to provide
multiple pools that all shared the same structure but held different
objects.

Since then various changes have happend in Ruby's memory layout:

* The concept of tomb heaps has been replaced by a global free pages list,
  with each page having it's slot size reconfigured at the point when it
  is resurrected
* the eden heap has been inlined into the size pool itself, so that now
  the size pool directly controls the free_pages list, the sweeping
  page, the compaction cursor and the other state that was previously
  being managed by the eden heap.

Now that there is no need for a heap wrapper, we should refer to the
collection of pages containing Ruby objects as a heap again rather than
a size pool
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Now that we've inlined the eden_heap into the size_pool, we should
rename the size_pool to heap. So that Ruby contains multiple heaps, with
different sized objects.

The term heap as a collection of memory pages is more in memory
management nomenclature, whereas size_pool was a name chosen out of
necessity during the development of the Variable Width Allocation
features of Ruby.

The concept of size pools was introduced in order to facilitate
different sized objects (other than the default 40 bytes). They wrapped
the eden heap and the tomb heap, and some related state, and provided a
reasonably simple way of duplicating all related concerns, to provide
multiple pools that all shared the same structure but held different
objects.

Since then various changes have happend in Ruby's memory layout:

* The concept of tomb heaps has been replaced by a global free pages list,
  with each page having it's slot size reconfigured at the point when it
  is resurrected
* the eden heap has been inlined into the size pool itself, so that now
  the size pool directly controls the free_pages list, the sweeping
  page, the compaction cursor and the other state that was previously
  being managed by the eden heap.

Now that there is no need for a heap wrapper, we should refer to the
collection of pages containing Ruby objects as a heap again rather than
a size pool
</pre>
</div>
</content>
</entry>
<entry>
<title>Don't use SEGV signal when timeout in test_gc_compact</title>
<updated>2024-06-07T15:55:38+00:00</updated>
<author>
<name>Peter Zhu</name>
<email>peter@peterzhu.ca</email>
</author>
<published>2024-06-07T13:50:17+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=caac56584ccafd9a103f79a1fc9d389fe438181f'/>
<id>caac56584ccafd9a103f79a1fc9d389fe438181f</id>
<content type='text'>
Using a SEGV signal for timeout makes it difficult to tell if it's a real
SEGV or if it timed out, so we should just use the default signals.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Using a SEGV signal for timeout makes it difficult to tell if it's a real
SEGV or if it timed out, so we should just use the default signals.
</pre>
</div>
</content>
</entry>
<entry>
<title>Extend timeout of TestGCCompect#test_moving_objects_between_size_pools</title>
<updated>2024-06-07T13:09:44+00:00</updated>
<author>
<name>Yusuke Endoh</name>
<email>mame@ruby-lang.org</email>
</author>
<published>2024-06-07T12:26:50+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=0e5640b56f85104293cabe34ae08b8fc94241c7d'/>
<id>0e5640b56f85104293cabe34ae08b8fc94241c7d</id>
<content type='text'>
It is too flaky on macOS GitHub Actions
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
It is too flaky on macOS GitHub Actions
</pre>
</div>
</content>
</entry>
<entry>
<title>Ensure test suite is compatible with --frozen-string-literal</title>
<updated>2024-03-14T16:56:15+00:00</updated>
<author>
<name>Jean Boussier</name>
<email>byroot@ruby-lang.org</email>
</author>
<published>2024-03-13T11:50:11+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=09d8c99cdcb04fb6c6c8e61c9dea28927a3a0b46'/>
<id>09d8c99cdcb04fb6c6c8e61c9dea28927a3a0b46</id>
<content type='text'>
As preparation for https://bugs.ruby-lang.org/issues/20205
making sure the test suite is compatible with frozen string
literals is making things easier.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
As preparation for https://bugs.ruby-lang.org/issues/20205
making sure the test suite is compatible with frozen string
literals is making things easier.
</pre>
</div>
</content>
</entry>
<entry>
<title>Remove never used method</title>
<updated>2024-02-20T11:14:37+00:00</updated>
<author>
<name>Nobuyoshi Nakada</name>
<email>nobu@ruby-lang.org</email>
</author>
<published>2024-02-20T11:08:52+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=386b956ad8fb6883512d17646d6eddd02e6e0d6a'/>
<id>386b956ad8fb6883512d17646d6eddd02e6e0d6a</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Remove no longer used methods</title>
<updated>2024-02-20T11:14:37+00:00</updated>
<author>
<name>Nobuyoshi Nakada</name>
<email>nobu@ruby-lang.org</email>
</author>
<published>2024-02-20T11:07:07+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=e3917fc7da2c4dfbf0144ad988ec1047fd290642'/>
<id>e3917fc7da2c4dfbf0144ad988ec1047fd290642</id>
<content type='text'>
`find_object_in_recycled_slot` and `memory_location` have not been
used since commit:b99833baec2e567e38758f4fd017c90c7ce57d75.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
`find_object_in_recycled_slot` and `memory_location` have not been
used since commit:b99833baec2e567e38758f4fd017c90c7ce57d75.
</pre>
</div>
</content>
</entry>
<entry>
<title>Add a fudge factor to the GC compaction move up/down tests</title>
<updated>2023-12-09T20:49:51+00:00</updated>
<author>
<name>KJ Tsanaktsidis</name>
<email>kj@kjtsanaktsidis.id.au</email>
</author>
<published>2023-12-08T23:55:12+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=c0b6ea7c8b5dc6e48ecf6e14e1dbd135d079f0fc'/>
<id>c0b6ea7c8b5dc6e48ecf6e14e1dbd135d079f0fc</id>
<content type='text'>
There seems to be another manifestation of bug #20021, where some of the
compaction tests are failing on i686 for unrelated PR's because of fake
"live" references to moved objects on the machine stack.

We _could_ solve this by counting how many objects are pinned during
compaction, but doing that involves pushing down the mark &amp; pin bitset
merge into gc_compact_plane and out of gc_compact_page, which I thought
was pretty ugly.

Now that we've solved bug #20022 though, we're able to compact
arbitrarily many objects with GC.verify_compaction_references, so the
number of objects we're moving is now 50,000 instead of 500. Since
that's now much larger than the number of objects likely to be pinned, I
think it's safe enough to just add a fudge-factor to the tests.

Any _other_ change in GC.verify_compaction_references that breaks
compaction is now highly likely to break the assertion by more than 10
objects, since it's operating on so many more in the first place.

[Bug #20021]
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
There seems to be another manifestation of bug #20021, where some of the
compaction tests are failing on i686 for unrelated PR's because of fake
"live" references to moved objects on the machine stack.

We _could_ solve this by counting how many objects are pinned during
compaction, but doing that involves pushing down the mark &amp; pin bitset
merge into gc_compact_plane and out of gc_compact_page, which I thought
was pretty ugly.

Now that we've solved bug #20022 though, we're able to compact
arbitrarily many objects with GC.verify_compaction_references, so the
number of objects we're moving is now 50,000 instead of 500. Since
that's now much larger than the number of objects likely to be pinned, I
think it's safe enough to just add a fudge-factor to the tests.

Any _other_ change in GC.verify_compaction_references that breaks
compaction is now highly likely to break the assertion by more than 10
objects, since it's operating on so many more in the first place.

[Bug #20021]
</pre>
</div>
</content>
</entry>
<entry>
<title>Fix GC.verify_compaction_references not moving every object</title>
<updated>2023-12-07T15:19:35+00:00</updated>
<author>
<name>KJ Tsanaktsidis</name>
<email>kj@kjtsanaktsidis.id.au</email>
</author>
<published>2023-11-27T05:50:05+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=cbc0e0bef08f9389f5bbe76de016795f01c3bc76'/>
<id>cbc0e0bef08f9389f5bbe76de016795f01c3bc76</id>
<content type='text'>
The intention of GC.verify_compaction_references is, I believe, to force
every single movable object to be moved, so that it's possible to debug
native extensions which not correctly updating their references to
objects they mark as movable.

To do this, it doubles the number of allocated pages for each size pool,
and sorts the heap pages so that the free ones are swept first; thus,
every object in an old page should be moved into a free slot in one of
the new pages.

This worked fine until movement of objects _between_ size pools during
compaction was implemented. That causes some problems for
verify_compaction_references:

* We were doubling the number of pages in each size pool, but actually
  if some objects need to move into a _different_ pool, there's no
  guarantee that they'll be enough room in that one.
* It's possible for the sweep &amp; compact cursors to meet in one size pool
  before all the objects that want to move into that size pool from
  another are processed by the compaction.

You can see these problems by changing some of the movement tests in
test_gc_compact.rb to try and move e.g. 50,000 objects instead of
500; the test is not able to actually move all of the objects in a
single compaction run.

To fix this, we do two things in verify_compaction_references:

* Firstly, we add enough pages to every size pool to make them the same
  size. This ensures that their compact cursors will all have space to
  move during compaction (even if that means empty pages are
  pointlessly compacted)
* Then, we examine every object and determine where it _wants_ to be
  compacted into. We use this information to add additional pages to
  each size pool to handle all objects which should live there.

With these two changes, we can move arbitrary amounts of objects into
the correct size pool in a single call to verify_compaction_references.

My _motivation_ for performing this work was to try and fix some test
stability issues in test_gc_compact.rb. I now no longer think that we
actually see this particular bug in rubyci.org, but I also think
verify_compaction_references should do what it says on the tin, so it's
worth keeping.

[Bug #20022]
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The intention of GC.verify_compaction_references is, I believe, to force
every single movable object to be moved, so that it's possible to debug
native extensions which not correctly updating their references to
objects they mark as movable.

To do this, it doubles the number of allocated pages for each size pool,
and sorts the heap pages so that the free ones are swept first; thus,
every object in an old page should be moved into a free slot in one of
the new pages.

This worked fine until movement of objects _between_ size pools during
compaction was implemented. That causes some problems for
verify_compaction_references:

* We were doubling the number of pages in each size pool, but actually
  if some objects need to move into a _different_ pool, there's no
  guarantee that they'll be enough room in that one.
* It's possible for the sweep &amp; compact cursors to meet in one size pool
  before all the objects that want to move into that size pool from
  another are processed by the compaction.

You can see these problems by changing some of the movement tests in
test_gc_compact.rb to try and move e.g. 50,000 objects instead of
500; the test is not able to actually move all of the objects in a
single compaction run.

To fix this, we do two things in verify_compaction_references:

* Firstly, we add enough pages to every size pool to make them the same
  size. This ensures that their compact cursors will all have space to
  move during compaction (even if that means empty pages are
  pointlessly compacted)
* Then, we examine every object and determine where it _wants_ to be
  compacted into. We use this information to add additional pages to
  each size pool to handle all objects which should live there.

With these two changes, we can move arbitrary amounts of objects into
the correct size pool in a single call to verify_compaction_references.

My _motivation_ for performing this work was to try and fix some test
stability issues in test_gc_compact.rb. I now no longer think that we
actually see this particular bug in rubyci.org, but I also think
verify_compaction_references should do what it says on the tin, so it's
worth keeping.

[Bug #20022]
</pre>
</div>
</content>
</entry>
</feed>
