| Age | Commit message (Collapse) | Author |
|---|
|
The argument to `is_data_encoding` is assumed to be `T_DATA`.
|
|
|
|
When debugging the fstring table, I found "UTF-8" to be the most common
interned strings in many benchmarks.
We have a fixed, limited number of these strings, so we might as well permanently
cache their fstrings.
|
|
Encodings are RTypedData, not the deprecated RData. Although the
structures are compatible we should use the correct API.
|
|
to adopt strict shareable rule.
* (basically) shareable objects only refer shareable objects
* (exception) shareable objects can refere unshareable objects
but should not leak reference to unshareable objects to Ruby world
|
|
Previously we were looping over the enc_table, but when I added an
assertion the only thing that loop was doing is the equivalent of
ENC_TO_ENCINDEX(base). However we don't even need the index of base.
Instead we should be able to just use the badirectly.
|
|
Previously we would sometimes call enc_register_at several times in
order to update the encoding values after the base encoding may have
been loaded.
This updates enc_register_at to only be used via enc_register, when an
actual new encoding at a new index is being registered.
Other callers (which in all cases found the index by the name matching)
now call enc_load_from_base which is only responsibly for loading the
encoding from the base values.
|
|
|
|
Previously when we copied base_encoding on top of the encoding, other
threads could briefly see the name and ruby_encoding_index of the base
encoding.
|
|
|
|
Not all ractor-related encoding issues were fixed by 1afc07e815051e2f73493f055f2130cb642ba12a.
I found more by running my test-all branch with 3 ractors for each test.
|
|
Calling rb_ary_replace(copy, orig) can modify orig, which is not safe
across ractors because orig is shared (it's the global encoding list).
Hoping to address CI failures such as https://ci.rvm.jp/results/trunk-gc-asserts@ruby-sp2-noble-docker/5890058
|
|
Make sure VM lock is not held when calling `load_transcoder_entry`, as
that causes deadlock inside ractors. `String#encode` now works inside
ractors, among others.
Atomic load the rb_encoding_list
Without this, wbcheck would sometimes hit a missing write barrier.
Co-authored-by: John Hawthorn <john.hawthorn@shopify.com>
Hold VM lock when iterating over global_enc_table.names
This st_table can be inserted into at runtime when autoloading
encodings.
minor optimization when calling Encoding.list
|
|
|
|
|
|
Otherwise we may be checking the index while the encoding
is being autoloaded by another ractor.
|
|
Using `encoding->max_enc_len` as a way to check if the encoding
has been loaded isn't atomic, because it's not atomically set
last.
Intead we can use a dedicated atomic value inside the encoding table.
|
|
None of the datastructures involved in the require process are
safe to call on a secondary ractor, however when autoloading
encodings, we do so from the current ractor.
So all sorts of corruption can happen when using an autoloaded
encoding for the first time from a secondary ractor.
|
|
This reverts commit cf4d37fbc079116453e69cf08ea8007d0e1c73e6.
|
|
This reverts commit dda5a04f2b4835582dba09ba33797258a61efafe.
|
|
Also, make sure autoloading of encodings is safe across ractors.
|
|
This fixes segfaults and errors of the type "Encoding not found" when
using encoding-related methods and internal encoding c functions across
ractors.
Example of a possible segfault in release mode or assertion error in debug mode:
```ruby
rs = []
100.times do
rs << Ractor.new do
"abc".force_encoding(Encoding.list.shuffle.first)
end
end
while rs.any?
r, obj = Ractor.select(*rs)
rs.delete(r)
end
```
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/13439
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/13154
|
|
While profiling msgpack-ruby I noticed a very substantial amout of time
spent in `rb_enc_associate_index`, called by `rb_utf8_str_new`.
On that benchmark, `rb_utf8_str_new` is 33% of the total runtime,
in big part because it cause GC to trigger often, but even then
`5.3%` of the total runtime is spent in `rb_enc_associate_index`
called by `rb_utf8_str_new`.
After closer inspection, it appears that it's performing a lot of
safety check we can assert we don't need, and other extra useless
operations, because strings are first created and filled as ASCII-8BIT
and then later reassociated to the desired encoding.
By directly allocating the string with the right encoding, it allow
to skip a lot of duplicated and useless operations.
After this change, the time spent in `rb_utf8_str_new` is down
to `28.4%` of total runtime, and most of that is GC.
Notes:
Merged: https://github.com/ruby/ruby/pull/12076
|
|
|
|
[Bug #20598]
Just like [Bug #20595], Encoding#name_list and Encoding#aliases can have
their strings corrupted when Encoding.default_internal is set to nil.
Co-authored-by: Matthew Valentine-House <matt@eightbitraptor.com>
|
|
[Bug #20595]
enc_set_default_encoding will free the C string if the encoding is nil,
but the C string can be used by the encoding name string. This will cause
the encoding name string to be corrupted.
Consider the following code:
Encoding.default_internal = Encoding::ASCII_8BIT
names = Encoding.default_internal.names
p names
Encoding.default_internal = nil
p names
It outputs:
["ASCII-8BIT", "BINARY", "internal"]
["ASCII-8BIT", "BINARY", "\x00\x00\x00\x00\x00\x00\x00\x00"]
Co-authored-by: Matthew Valentine-House <matt@eightbitraptor.com>
|
|
[Feature #18576]
Since outright renaming `ASCII-8BIT` is deemed to backward incompatible,
the next best thing would be to only change its `#inspect`, particularly
in exception messages.
|
|
This `st_table` is used to both mark and pin classes
defined from the C API. But `vm->mark_object_ary` already
does both much more efficiently.
Currently a Ruby process starts with 252 rooted classes,
which uses `7224B` in an `st_table` or `2016B` in an `RArray`.
So a baseline of 5kB saved, but since `mark_object_ary` is
preallocated with `1024` slots but only use `405` of them,
it's a net `7kB` save.
`vm->mark_object_ary` is also being refactored.
Prior to this changes, `mark_object_ary` was a regular `RArray`, but
since this allows for references to be moved, it was marked a second
time from `rb_vm_mark()` to pin these objects.
This has the detrimental effect of marking these references on every
minors even though it's a mostly append only list.
But using a custom TypedData we can save from having to mark
all the references on minor GC runs.
Addtionally, immediate values are now ignored and not appended
to `vm->mark_object_ary` as it's just wasted space.
|
|
There is a memory leak in Encoding.default_external= and
Encoding.default_internal= because the duplicated name is not freed
when overwriting.
10.times do
1_000_000.times do
Encoding.default_internal = nil
end
puts `ps -o rss= -p #{$$}`
end
Before:
25664
41504
57360
73232
89168
105056
120944
136816
152720
168576
After:
9648
9648
9648
9680
9680
9680
9680
9680
9680
9680
|
|
when the RUBY_FREE_ON_SHUTDOWN environment variable is set, manually free memory at shutdown.
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
Co-authored-by: Peter Zhu <peter@peterzhu.ca>
|
|
It doesn't even have a mark function.
It's only about a hundred objects, but not reason
to scan them every time.
Notes:
Merged: https://github.com/ruby/ruby/pull/7254
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/7079
|
|
now `enc_table->list` is not a pointer.
|
|
This reduces global lock acquiring for reading.
https://bugs.ruby-lang.org/issues/18949
Notes:
Merged: https://github.com/ruby/ruby/pull/6935
|
|
UTF-16/UTF-32
* And simplify callers of get_actual_encoding().
* See [Feature #18949].
* See https://github.com/ruby/ruby/pull/6322#issuecomment-1242758474
|
|
* See [Feature #18949].
|
|
[Misc #18891]
Notes:
Merged: https://github.com/ruby/ruby/pull/6094
|
|
Otherwise it's way too easy to confuse it with US_ASCII.
Notes:
Merged: https://github.com/ruby/ruby/pull/6127
|
|
Main doc for encodings moved from encoding.c to doc/encodings.rdoc
Notes:
Merged-By: BurdetteLamar <BurdetteLamar@Yahoo.com>
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/5015
|
|
Less macros == huge win.
Notes:
Merged: https://github.com/ruby/ruby/pull/4909
|
|
Instead of relying on setting an unsetting ruby_verbose, which is
not thread-safe, restructure require_internal and load_lock to
accept a warn argument for whether to warn, and add
rb_require_internal_silent to require without warnings. Use
rb_require_internal_silent when loading encoding.
Note this does not modify ruby_debug and errinfo handling, those
remain thread-unsafe.
Also silent requires when loading transcoders.
Notes:
Merged: https://github.com/ruby/ruby/pull/4887
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/4759
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/4695
Merged-By: nobu <nobu@ruby-lang.org>
|
|
If called with an autoloaded encoding that was not yet
initialized, `rb_enc_interned_str` would crash with
a NULL pointer exception.
See: https://github.com/ruby/ruby/pull/4119#issuecomment-800189841
Notes:
Merged: https://github.com/ruby/ruby/pull/4290
|
|
rb_enc_from_index(index) doesn't need locking if index specify
ASCII/US_ASCCII/UTF-8.
rb_enc_from_index() is called frequently so it has impact.
user system total real
r_parallel/miniruby 174 0.000209 0.000000 5.559872 ( 1.811501)
r_parallel/master_mini 175 0.000238 0.000000 12.664707 ( 3.523641)
(repeat x1000 `s.split(/,/)` where s = '0,,' * 1000)
Notes:
Merged: https://github.com/ruby/ruby/pull/3916
|
|
* Use UTF-8 as default for Encoding.default_external on Windows
* Document UTF-8 change on Windows to Encoding.default_external
fix https://bugs.ruby-lang.org/issues/16604
Notes:
Merged-By: nurse <naruse@airemix.jp>
|
|
To make some kind of Ractor related extensions, some functions
should be exposed.
* include/ruby/thread_native.h
* rb_native_mutex_*
* rb_native_cond_*
* include/ruby/ractor.h
* RB_OBJ_SHAREABLE_P(obj)
* rb_ractor_shareable_p(obj)
* rb_ractor_std*()
* rb_cRactor
and rm ractor_pub.h
and rename srcdir/ractor.h to srcdir/ractor_core.h
(to avoid conflict with include/ruby/ractor.h)
Notes:
Merged: https://github.com/ruby/ruby/pull/3775
|
