summaryrefslogtreecommitdiff
path: root/test/ruby/test_encoding.rb
AgeCommit message (Collapse)Author
2026-01-02Prefer dedicated assertionsNobuyoshi Nakada
2025-08-22Fixes to encoding/transcoding for ractors.Luke Gruber
Not all ractor-related encoding issues were fixed by 1afc07e815051e2f73493f055f2130cb642ba12a. I found more by running my test-all branch with 3 ractors for each test.
2025-07-07Autoload encodings on the main ractorJean Boussier
None of the datastructures involved in the require process are safe to call on a secondary ractor, however when autoloading encodings, we do so from the current ractor. So all sorts of corruption can happen when using an autoloaded encoding for the first time from a secondary ractor.
2025-07-03Revert "Add locks around accesses/modifications to global encodings table"John Hawthorn
This reverts commit cf4d37fbc079116453e69cf08ea8007d0e1c73e6.
2025-07-03Revert "Make get/set default internal/external encoding lock-free"John Hawthorn
This reverts commit dda5a04f2b4835582dba09ba33797258a61efafe.
2025-07-03Make get/set default internal/external encoding lock-freeLuke Gruber
Also, make sure autoloading of encodings is safe across ractors.
2025-07-03Add locks around accesses/modifications to global encodings tableLuke Gruber
This fixes segfaults and errors of the type "Encoding not found" when using encoding-related methods and internal encoding c functions across ractors. Example of a possible segfault in release mode or assertion error in debug mode: ```ruby rs = [] 100.times do rs << Ractor.new do "abc".force_encoding(Encoding.list.shuffle.first) end end while rs.any? r, obj = Ractor.select(*rs) rs.delete(r) end ```
2025-06-17Refactor generic fields to use `T_IMEMO/fields` objects.Jean Boussier
Followup: https://github.com/ruby/ruby/pull/13589 This simplify a lot of things, as we no longer need to manually manage the memory, we can use the Read-Copy-Update pattern and avoid numerous race conditions. Co-Authored-By: Étienne Barrié <etienne.barrie@gmail.com> Notes: Merged: https://github.com/ruby/ruby/pull/13626
2025-05-31`Ractor::Port`Koichi Sasada
* Added `Ractor::Port` * `Ractor::Port#receive` (support multi-threads) * `Rcator::Port#close` * `Ractor::Port#closed?` * Added some methods * `Ractor#join` * `Ractor#value` * `Ractor#monitor` * `Ractor#unmonitor` * Removed some methods * `Ractor#take` * `Ractor.yield` * Change the spec * `Racotr.select` You can wait for multiple sequences of messages with `Ractor::Port`. ```ruby ports = 3.times.map{ Ractor::Port.new } ports.map.with_index do |port, ri| Ractor.new port,ri do |port, ri| 3.times{|i| port << "r#{ri}-#{i}"} end end p ports.each{|port| pp 3.times.map{port.receive}} ``` In this example, we use 3 ports, and 3 Ractors send messages to them respectively. We can receive a series of messages from each port. You can use `Ractor#value` to get the last value of a Ractor's block: ```ruby result = Ractor.new do heavy_task() end.value ``` You can wait for the termination of a Ractor with `Ractor#join` like this: ```ruby Ractor.new do some_task() end.join ``` `#value` and `#join` are similar to `Thread#value` and `Thread#join`. To implement `#join`, `Ractor#monitor` (and `Ractor#unmonitor`) is introduced. This commit changes `Ractor.select()` method. It now only accepts ports or Ractors, and returns when a port receives a message or a Ractor terminates. We removes `Ractor.yield` and `Ractor#take` because: * `Ractor::Port` supports most of similar use cases in a simpler manner. * Removing them significantly simplifies the code. We also change the internal thread scheduler code (thread_pthread.c): * During barrier synchronization, we keep the `ractor_sched` lock to avoid deadlocks. This lock is released by `rb_ractor_sched_barrier_end()` which is called at the end of operations that require the barrier. * fix potential deadlock issues by checking interrupts just before setting UBF. https://bugs.ruby-lang.org/issues/21262 Notes: Merged: https://github.com/ruby/ruby/pull/13445
2024-12-24Fix calls to require_internal in multi-ractor modelukeg
After a ractor is started (multi-ractor mode), any calls to require_internal will hang the process due to deadlock. For example, loading a new encoding will deadlock after a ractor starts. Fixes [Bug #19562] Notes: Merged: https://github.com/ruby/ruby/pull/7656
2023-08-03Remove --disable-gems for assert_separatelyPeter Zhu
assert_separately adds --disable=gems so we don't need to add --disable-gems when calling assert_separately. Notes: Merged: https://github.com/ruby/ruby/pull/8162
2023-01-11Remove Encoding#replicateBenoit Daloze
Notes: Merged: https://github.com/ruby/ruby/pull/7079
2022-12-16fixed encoding tableKoichi Sasada
This reduces global lock acquiring for reading. https://bugs.ruby-lang.org/issues/18949 Notes: Merged: https://github.com/ruby/ruby/pull/6935
2022-09-10Deprecate Encoding#replicateBenoit Daloze
* See [Feature #18949].
2021-02-16Fix a failureKazuhiro NISHIYAMA
http://ci.rvm.jp/results/trunk-test@ruby-sky1/3359644 ``` : 1) : TestEncoding#test_nonascii_library_path [/tmp/ruby/v3/src/trunk-test/test/ruby/test_encoding.rb:157]: : Exception(LoadError) with message matches to /\[Bug #16382\]/. ```
2020-10-15Check encoding name to replicateNobuyoshi Nakada
https://hackerone.com/reports/954433
2020-02-13Get rid of warnings/exceptions at cleanupNobuyoshi Nakada
After the encoding index instance variable is removed when all instance variables are removed in `obj_free`, then `rb_str_free` causes uninitialized instance variable warning and nil-to-integer conversion exception. Both cases result in object allocation during GC, and crashes.
2020-02-12Copy non-inlined encoding indexNobuyoshi Nakada
2020-02-12Make temporary lock string encoding freeNobuyoshi Nakada
As a temporary lock string is hidden, it can not have instance variables, including non-inlined encoding index.
2020-02-12Workaround of instance variable on hidden objectNobuyoshi Nakada
Since 9d9aea7fe50f6340829faa105d9ffe08ebaee658, generic instance variables need `iv_index_tbl` in the object's class. As hidden objects, however, have no class, access to the variables causes a segfault. Get rid of that segfault by raising an exception, for the time being.
2020-02-12Isolate the test for Encoding#replicateNobuyoshi Nakada
It has global side effect which cannot be reverted.
2020-01-28support multi-run for test/ruby/test_encoding.rbKoichi Sasada
Unique encoding name is required.
2019-12-03Fixed stack overflow [Bug #16382]Nobuyoshi Nakada
Get rid of infinite recursion in expanding a load path to the real path while loading a transcoder. Notes: Merged: https://github.com/ruby/ruby/pull/2714
2019-11-18Deprecate taint/trust and related methods, and make the methods no-opsJeremy Evans
This removes the related tests, and puts the related specs behind version guards. This affects all code in lib, including some libraries that may want to support older versions of Ruby. Notes: Merged: https://github.com/ruby/ruby/pull/2476
2019-03-12Assertion for r67226nobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67227 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-08-29test/ruby: tweaked heredocsnobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59682 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-12-16Add frozen_string_literal: false for all filesnaruse
When you change this to true, you may need to add more tests. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53141 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-12-02encoding.c: defer finding encodingnobu
* encoding.c (enc_m_loader): defer finding encoding object not to be infected by marshal source. [ruby-core:71793] [Bug #11760] * marshal.c (r_object0): enable compatible loader on USERDEF class. the loader function is called with the class itself, instead of an allocated object, and the loaded data. * marshal.c (compat_allocator_table): intialize compat_allocator_tbl on demand. * object.c (rb_undefined_alloc): extract from rb_obj_alloc. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52856 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-13* test/lib/envutil.rb: Moved from test/ruby/.akr
* test/lib/find_executable.rb: Ditto. * test/lib/memory_status.rb: Ditto. * test/lib/test/unit.rb: require envutil. * test/: Don't require envutil in test files. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48409 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-03-26encoding.c: fix encoding of dynsymnobu
* encoding.c (rb_enc_get_index): the encoding of dynamic symbol comes from fstr. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45438 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-03-26encoding.c: enc_capable symbolnobu
* encoding.c (enc_capable): Symbol is now encoding capable. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45436 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-12-13test/ruby: better assertionsnobu
* test/ruby: use better assertions instead of mere assert. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44173 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-10-21encoding.c: preserve outer errinfonobu
* encoding.c (load_encoding): should preserve outer errinfo, so that expected exception may not be lost. [ruby-core:57949] [Bug #9038] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@43376 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-06-12* safe.c (rb_set_safe_level, safe_setter): raise an ArgumentErrorshugo
when $SAFE is set to 4. $SAFE=4 is now obsolete. [ruby-core:55222] [Feature #8468] * object.c (rb_obj_untrusted, rb_obj_untrust, rb_obj_trust): Kernel#untrusted?, untrust, and trust are now deprecated. Their behavior is same as tainted?, taint, and untaint, respectively. * include/ruby/ruby.h (OBJ_UNTRUSTED, OBJ_UNTRUST): OBJ_UNTRUSTED() and OBJ_UNTRUST() are aliases of OBJ_TAINTED() and OBJ_TAINT(), respectively. * array.c, class.c, debug.c, dir.c, encoding.c, error.c, eval.c, ext/curses/curses.c, ext/dbm/dbm.c, ext/dl/cfunc.c, ext/dl/cptr.c, ext/dl/dl.c, ext/etc/etc.c, ext/fiddle/fiddle.c, ext/fiddle/pointer.c, ext/gdbm/gdbm.c, ext/readline/readline.c, ext/sdbm/init.c, ext/socket/ancdata.c, ext/socket/basicsocket.c, ext/socket/socket.c, ext/socket/udpsocket.c, ext/stringio/stringio.c, ext/syslog/syslog.c, ext/tk/tcltklib.c, ext/win32ole/win32ole.c, file.c, gc.c, hash.c, io.c, iseq.c, load.c, marshal.c, object.c, proc.c, process.c, random.c, re.c, safe.c, string.c, thread.c, transcode.c, variable.c, vm_insnhelper.c, vm_method.c, vm_trace.c: remove code for $SAFE=4. * test/dl/test_dl2.rb, test/erb/test_erb.rb, test/readline/test_readline.rb, test/readline/test_readline_history.rb, test/ruby/test_alias.rb, test/ruby/test_array.rb, test/ruby/test_dir.rb, test/ruby/test_encoding.rb, test/ruby/test_env.rb, test/ruby/test_eval.rb, test/ruby/test_exception.rb, test/ruby/test_file_exhaustive.rb, test/ruby/test_hash.rb, test/ruby/test_io.rb, test/ruby/test_method.rb, test/ruby/test_module.rb, test/ruby/test_object.rb, test/ruby/test_pack.rb, test/ruby/test_rand.rb, test/ruby/test_regexp.rb, test/ruby/test_settracefunc.rb, test/ruby/test_struct.rb, test/ruby/test_thread.rb, test/ruby/test_time.rb: remove tests for $SAFE=4. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@41259 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-02-05* encoding.c (rb_enc_compatible): return ASCII-8BIT even if 2nd stringnaruse
is ascii only string. [ruby-core:42354] [Bug #5968] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@34433 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-01-21* encoding.c (rb_enc_compatible): fix segv on symbols.naruse
[ruby-core:42204] [Bug #5921] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@34350 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-09-25* encoding.c (require_enc): reject only loading from untrustednobu
load paths. [ruby-dev:44541] [Bug #5279] * transcode.c (load_transcoder_entry): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@33328 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-09-06* encoding.c (load_encoding): predefined encoding names are safe.nobu
[ruby-dev:44469] [Bug #5279] * transcode.c (load_transcoder_entry): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@33201 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-09-03* test/ruby/test_encoding.rb (test_find): mingw is not Unix, of course.nobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@33169 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-08-02* encoding.c (enc_find): mistakenly remained !. [Bug #5150]nobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@32823 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-04-26* test/ruby/test_encoding.rb (test_encoding): tests for r31362.nobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@31364 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-04-29* lib/test/unit/assertions.rb (Test::Unit::Assertions#assert):nobu
use more descriptive assertions. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@27545 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-04-11* test/ruby/test_encoding.rb (TestEncoding#test_replicate): morenobu
test. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@27302 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-04-11* encoding.c (enc_replicate): new encoding name must be validnobu
c-string. [ruby-dev:40954] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@27301 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-12-12Add rdoc and test of Encoding#replicate.naruse
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26068 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-11-18* encoding.c (enc_set_default_encoding): reset filesytemnaruse
encoding because on resetting default_external because Unix's filesystem encoding depends on default_external. * encoding.c (enc_set_filesystem_encoding): added. * ruby.c (process_options): don't call rb_filesystem_encoding because filesystem encoding is reset when default_external is reset. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25837 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-10-28* encoding.c (get_filesystem_encoding): removed.naruse
* encoding.c (rb_locale_encindex): added. * encoding.c (rb_filesystem_encindex): added. * encoding.c (rb_filesystem_encindex): add an alias 'filesystem'. [ruby-dev:39574] * encoding.c (enc_find): add rdoc about special aliases. * gem_prelude.rb (Gem.set_home): use Encoding.find('filesystem'). * gem_prelude.rb (Gem.set_paths): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25529 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-08-08New API Encoding#ascii_compatible?.naruse
* encoding.c (enc_ascii_compatible_p): added. [ruby-core:24793] (Init_Encoding): New API Encoding#ascii_compatible?. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24480 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-06-13* marshal.c (r_ivar): should not set internal encoding ivar as annobu
ordinary ivar. [ruby-dev:38596] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23677 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-03-06* {ext,lib,test}/**/*.rb: removed trailing spaces.nobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@22784 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.25.1) at 2026-01-16 12:05:32 +0000