summaryrefslogtreecommitdiff
path: root/encoding.c
AgeCommit message (Collapse)Author
2021-10-05include/ruby/encoding.h: convert macros into inline functions卜部昌平
Less macros == huge win. Notes: Merged: https://github.com/ruby/ruby/pull/4909
2021-10-02Make encoding loading not issue warningJeremy Evans
Instead of relying on setting an unsetting ruby_verbose, which is not thread-safe, restructure require_internal and load_lock to accept a warn argument for whether to warn, and add rb_require_internal_silent to require without warnings. Use rb_require_internal_silent when loading encoding. Note this does not modify ruby_debug and errinfo handling, those remain thread-unsafe. Also silent requires when loading transcoders. Notes: Merged: https://github.com/ruby/ruby/pull/4887
2021-08-22Add rb_encoding_check functionS-H-GAMELINKS
Notes: Merged: https://github.com/ruby/ruby/pull/4759
2021-08-02Using RBOOL macroS.H
Notes: Merged: https://github.com/ruby/ruby/pull/4695 Merged-By: nobu <nobu@ruby-lang.org>
2021-03-22rb_enc_interned_str: handle autoloaded encodingsJean Boussier
If called with an autoloaded encoding that was not yet initialized, `rb_enc_interned_str` would crash with a NULL pointer exception. See: https://github.com/ruby/ruby/pull/4119#issuecomment-800189841 Notes: Merged: https://github.com/ruby/ruby/pull/4290
2020-12-17No sync on ASCII/US_ASCCII/UTF-8Koichi Sasada
rb_enc_from_index(index) doesn't need locking if index specify ASCII/US_ASCCII/UTF-8. rb_enc_from_index() is called frequently so it has impact. user system total real r_parallel/miniruby 174 0.000209 0.000000 5.559872 ( 1.811501) r_parallel/master_mini 175 0.000238 0.000000 12.664707 ( 3.523641) (repeat x1000 `s.split(/,/)` where s = '0,,' * 1000) Notes: Merged: https://github.com/ruby/ruby/pull/3916
2020-12-08Set default for Encoding.default_external to UTF-8 on Windows (#2877)Lars Kanis
* Use UTF-8 as default for Encoding.default_external on Windows * Document UTF-8 change on Windows to Encoding.default_external fix https://bugs.ruby-lang.org/issues/16604 Notes: Merged-By: nurse <naruse@airemix.jp>
2020-11-18fix public interfaceKoichi Sasada
To make some kind of Ractor related extensions, some functions should be exposed. * include/ruby/thread_native.h * rb_native_mutex_* * rb_native_cond_* * include/ruby/ractor.h * RB_OBJ_SHAREABLE_P(obj) * rb_ractor_shareable_p(obj) * rb_ractor_std*() * rb_cRactor and rm ractor_pub.h and rename srcdir/ractor.h to srcdir/ractor_core.h (to avoid conflict with include/ruby/ractor.h) Notes: Merged: https://github.com/ruby/ruby/pull/3775
2020-10-21Don't redefine #rb_intern over and over againStefan Stüben
Notes: Merged: https://github.com/ruby/ruby/pull/3589
2020-10-20Revert "reduce lock for encoding"Koichi Sasada
This reverts commit de17e2dea137bc5ba9f00e3acec32792d0dbb2eb. This patch can introduce race condition because of conflicting read/write access for enc_table::default_list. Maybe we need to freeze default_list at the end of Init_encdb() in enc/encdb.c.
2020-10-19reduce lock for encodingKoichi Sasada
To reduce the number of locking for encoding manipulation, enc_table::list is splited to ::default_list and ::additional_list. ::default_list is pre-allocated and no need locking to access to the ::default_list. If additional encoding space is needed, use ::additional_list and this list need to use locking. However, most of case, ::default_list is enough. Notes: Merged: https://github.com/ruby/ruby/pull/3664
2020-10-15Check encoding name to replicateNobuyoshi Nakada
https://hackerone.com/reports/954433
2020-10-14freeze Encoding objectsKoichi Sasada
Encoding objects can be accessed in multi-ractors and there is no state to mutate. So we can mark it as frozen and shareable. [Bug #17188] Notes: Merged: https://github.com/ruby/ruby/pull/3654
2020-10-14sync enc_table and rb_encoding_listKoichi Sasada
enc_table which manages Encoding information. rb_encoding_list also manages Encoding objects. Both are accessed/modified by ractors simultaneously so that they should be synchronized. For enc_table, this patch introduced GLOBAL_ENC_TABLE_ENTER/LEAVE/EVAL to access this table with VM lock. To make shortcut, three new global variables global_enc_ascii, global_enc_utf_8, global_enc_us_ascii are also introduced. For rb_encoding_list, we split it to rb_default_encoding_list (256 entries) and rb_additional_encoding_list. rb_default_encoding_list is fixed sized Array so we don't need to synchronized (and most of apps only needs it). To manage 257 or more encoding objects, they are stored into rb_additional_encoding_list. To access rb_additional_encoding_list., VM lock is needed. Notes: Merged: https://github.com/ruby/ruby/pull/3654
2020-05-16Fallback to built-in UTF-8 for minirubyNobuyoshi Nakada
Source code encoding is defaulted to UTF-8 now too.
2020-05-11sed -i 's|ruby/impl|ruby/internal|'卜部昌平
To fix build failures. Notes: Merged: https://github.com/ruby/ruby/pull/3079
2020-05-11sed -i s|ruby/3|ruby/impl|g卜部昌平
This shall fix compile errors. Notes: Merged: https://github.com/ruby/ruby/pull/3079
2020-05-11Added more NORETURN declarationsNobuyoshi Nakada
2020-04-08Merge pull request #2991 from shyouhei/ruby.h卜部昌平
Split ruby.h Notes: Merged-By: shyouhei <shyouhei@ruby-lang.org>
2020-02-13Get rid of warnings/exceptions at cleanupNobuyoshi Nakada
After the encoding index instance variable is removed when all instance variables are removed in `obj_free`, then `rb_str_free` causes uninitialized instance variable warning and nil-to-integer conversion exception. Both cases result in object allocation during GC, and crashes.
2020-02-10rb_enc_str_asciionly_p expects T_STRING卜部昌平
This `str2` variable can be non-string (regexp etc.) but the previous code passed it directly to rb_enc_str_asciionly_p(), which expects its argument be a string. Let's enforce that constraint.
2020-02-07more on NULL versus functions.卜部昌平
Function pointers are not void*. See also ce4ea956d24eab5089a143bba38126f2b11b55b6 8427fca49bd85205f5a8766292dd893f003c0e48
2020-02-03Fix description of Encoding.default_(in|ex)ternalLars Kanis
Data written to files is not transcoded per default, but only when default_internal is set. The default for default_internal is nil and doesn't depend on the source file encoding. Notes: Merged: https://github.com/ruby/ruby/pull/2878
2019-12-26decouple internal.h headers卜部昌平
Saves comitters' daily life by avoid #include-ing everything from internal.h to make each file do so instead. This would significantly speed up incremental builds. We take the following inclusion order in this changeset: 1. "ruby/config.h", where _GNU_SOURCE is defined (must be the very first thing among everything). 2. RUBY_EXTCONF_H if any. 3. Standard C headers, sorted alphabetically. 4. Other system headers, maybe guarded by #ifdef 5. Everything else, sorted alphabetically. Exceptions are those win32-related headers, which tend not be self- containing (headers have inclusion order dependencies). Notes: Merged: https://github.com/ruby/ruby/pull/2711
2019-12-22[Doc] Fix sample in Encoding#namesQuestionDriven
Notes: Merged: https://github.com/ruby/ruby/pull/2760
2019-12-22[Doc] Fix wrong example in Encoding.aliasesQuestionDriven
Notes: Merged: https://github.com/ruby/ruby/pull/2760
2019-11-18Deprecate taint/trust and related methods, and make the methods no-opsJeremy Evans
This removes the related tests, and puts the related specs behind version guards. This affects all code in lib, including some libraries that may want to support older versions of Ruby. Notes: Merged: https://github.com/ruby/ruby/pull/2476
2019-11-18Warn on access/modify of $SAFE, and remove effects of modifying $SAFEJeremy Evans
This removes the security features added by $SAFE = 1, and warns for access or modification of $SAFE from Ruby-level, as well as warning when calling all public C functions related to $SAFE. This modifies some internal functions that took a safe level argument to no longer take the argument. rb_require_safe now warns, rb_require_string has been added as a version that takes a VALUE and does not warn. One public C function that still takes a safe level argument and that this doesn't warn for is rb_eval_cmd. We may want to consider adding an alternative method that does not take a safe level argument, and warn for rb_eval_cmd. Notes: Merged: https://github.com/ruby/ruby/pull/2476
2019-11-05Moved Init_encoding from wrong place [Bug #16292]Nobuyoshi Nakada
2019-10-09avoid overflow in integer multiplication卜部昌平
This changeset basically replaces `ruby_xmalloc(x * y)` into `ruby_xmalloc2(x, y)`. Some convenient functions are also provided for instance `rb_xmalloc_mul_add(x, y, z)` which allocates x * y + z byes. Notes: Merged: https://github.com/ruby/ruby/pull/2540
2019-08-04Better wording for __ENCODING__Lars Kanis
"locale encoding" is misleading since it doesn't mean Encoding.find("locale") but the encoding used to interpret the script file. It's therefore better to call it "script encoding" as in the paragraphs above. Closes: https://github.com/ruby/ruby/pull/1611
2019-07-23Let the index boundary check in rb_enc_from_index be flagged as unlikelyLourens Naudé
[Misc #15806] Closes: https://github.com/ruby/ruby/pull/2128
2019-07-23Explicitly initialise encodings on init to remove branches on encoding lookupLourens Naudé
[Misc #15806] Closes: https://github.com/ruby/ruby/pull/2128
2019-07-15respect RUBY_DEBUG tooKoichi Sasada
2019-07-15encoding.c (enc_table_expand): prefer xrealloc to reallocYusuke Endoh
And raise an exception when failed to register an encoding
2019-03-22[DOC] fix markups [ci skip]nobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67337 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-09-15encoding.c: check external encodingnobu
* encoding.c (rb_enc_get_index): external encoding may not be Data object. [ruby-core:89016] [Bug #15122] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64753 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-03encoding.c (rb_enc_get_index): return -1 for non-encoding capable objectseregon
* Clarify logic and add spec. * Now passes test-all with the JSON fix. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64178 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-03Revert "encoding.c (rb_enc_get_index): return -1 for non-encoding capable ↵eregon
objects" * This reverts commit fb253d2032e51f333e6577aa4fcf0c03f9c6fc02. * The CI is failing, this seems a bug in the JSON C extension. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64174 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-03encoding.c (rb_enc_get_index): return -1 for non-encoding capable objectseregon
* Clarify logic and add spec. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64173 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-08-03encoding.c (enc_set_index): raise instead of rb_bug() for non-encoding ↵eregon
capable objects * Add spec. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64172 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-06-28check enc_capable.ko1
* encoding.c (rb_enc_capable): make it extern to check enc_capable. enc_index can be set to limited types such as T_STRING, T_REGEX and so on. This function check an object is this kind of types. * include/ruby/encoding.h: ditto. * encoding.c (enc_set_index): check a given object is enc_capable. * include/ruby/encoding.h (PUREFUNC): * marshal.c (encoding_name): check `rb_enc_capable` first. * marshal.c (r_ivar): ditto. If it is not enc_capable, it should be malformed data. * spec/ruby/optional/capi/encoding_spec.rb: remove tests depending on the wrong feature: all objects can set enc_index. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63777 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-06-05encoding.c: remove rb_gc_mark_encodingsnormal
rb_gc_mark_encodings has been empty for a decade (since r17875 / 28b216ac45262819f8bc7be0b1417a707974f94b). Just remove it and its only caller in gc.c git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63582 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-24load.c: use fstring instead of OBJ_FREEZEnormal
These strings already exist (or will exist soon) in the fstring table, so avoid the duplicate, sooner. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62030 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09internal.h: remove dependecy on ruby/encoding.hnobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61713 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-22encoding.c: rb_enc_find_index2nobu
* string.c (str_undump): use rb_enc_find_index2 to find encoding by unterminated string. check the format before encoding name. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61396 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-12-14encoding.c: handle needmore error from rb_enc_precise_mbclen()rhe
rb_enc_ascget() erroneously reports success even if the given byte sequence is incomplete, for non-ASCII compatible encoding strings. rb_enc_precise_mbclen() may return a negative value on error, and thus rb_enc_ascget() must not store the return value in 'unsigned int'; otherwise the subsequent MBCLEN_CHARFOUND_P() check won't catch the error. [ruby-core:78646] [Bug #13034] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57078 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-07-05* string.c (rb_str_change_terminator_length): New function to changengoto
termlen and resize heap for the terminator. This is split from rb_str_fill_terminator (str_fill_term) because filling terminator and changing terminator length are different things. [Bug #12536] * internal.h: declaration for rb_str_change_terminator_length. * string.c (str_fill_term): Simplify only to zero-fill the terminator. For non-shared strings, it assumes that (capa + termlen) bytes of heap is allocated. This partially reverts r55557. * encoding.c (rb_enc_associate_index): rb_str_change_terminator_length is used, and it should be called whenever the termlen is changed. * string.c (str_capacity): New static function to return capacity of a string with the given termlen, because the termlen may sometimes be different from TERM_LEN(str) especially during changing termlen or filling terminator with specific termlen. * string.c (rb_str_capacity): Use str_capacity. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55575 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-04-25* encoding.c: Fix return value of `Encoding::ISO8859_1.name`hsbt
[Bug #12313][ruby-core:75147][ci skip] * ext/bigdecimal/bigdecimal.c: Fix code sample of `BigDecimal.new` git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54755 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-03-29* include/ruby/ruby.h (rb_isupper, rb_islower, rb_isalpha, rb_isdigit,naruse
rb_isalnum, rb_isxdigit, rb_isblank, rb_isspace, rb_isblank, rb_iscntrl, rb_isprint, rb_ispunct, rb_isgraph, rb_tolower, rb_toupper): use inline function to avoid function call. * include/ruby/ruby.h (rb_isascii): use inline function to clarify the logic. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54391 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.25.1) at 2024-04-27 11:46:15 +0000