summaryrefslogtreecommitdiff
path: root/enc
AgeCommit message (Collapse)Author
2022-09-21reserved_word: just use gperf 3.1 declaration卜部昌平
The reason why this was commented out was because of gperf 3.0 vs 3.1 differences (see [Feature #13883]). Five years passed, I am pretty confident that we can drop support of old versions here. Ditto for uniname2ctype_p(), onig_jis_property(), and zonetab(). Notes: Merged: https://github.com/ruby/ruby/pull/6358
2022-09-17Move case-folding.rb to tooldir with enc-prefixNobuyoshi Nakada
2022-09-12Remove get_actual_encoding() and the dynamic endian detection for dummy ↵Benoit Daloze
UTF-16/UTF-32 * And simplify callers of get_actual_encoding(). * See [Feature #18949]. * See https://github.com/ruby/ruby/pull/6322#issuecomment-1242758474
2022-08-19Support Encoding::Converter newline: :lf and :lf_newline optionsJeremy Evans
Previously, newline: :lf was accepted but ignored. Where it should have been used was commented out code that didn't work, but unlike all other invalid values, using newline: :lf did not raise an error. This adds support for newline: :lf and :lf_newline, for consistency with newline: :cr and :cr_newline. This is basically the same as universal_newline, except that it only affects writing and not reading due to RUBY_ECONV_NEWLINE_DECORATOR_WRITE_MASK. Add tests for the File.open :newline option while here. Fixes [Bug #12436] Notes: Merged: https://github.com/ruby/ruby/pull/4590
2022-08-04Resolve abi symbol references from miniruby to avoid circular depsYuta Saito
Adding `ruby` to `PREP` causes the following circular dependencies because `PREP` is used as a prerequisite by some targets required to build `ruby` target itself. ``` make: Circular .rbconfig.time <- ruby dependency dropped. make: Circular builtin_binary.inc <- ruby dependency dropped. make: Circular ext/extinit.c <- ruby dependency dropped. make: Circular ruby <- ruby dependency dropped. ``` Adding a new Make variable like `EXTPREP` only for exts may be also reasonable, but it would introduce another complexity into our build system. `-bundle_loader` doesn't care that link-time and run-time loader executables are different as long as bound symbols are provided, so it's ok to resolve from miniruby to simplify our build. Notes: Merged: https://github.com/ruby/ruby/pull/6193
2022-08-04Link ext bundles with bundle loader option for newer ld64Yuta Saito
ld64 shipped with Xcode 14 emits a warning when using `-undefined dynamic_lookup`. ``` ld: warning: -undefined dynamic_lookup may not work with chained fixups ``` Actually, `-undefined dynamic_lookup` doesn't work when: 1. Link a *shared library* with the option 2. Link it with a program that uses the chained-fixup introduced from macOS 12 and iOS 15 because `-undefined dynamic_lookup` uses lazy-bindings and they won't be bound while dyld fixes-up by traversing chained-fixup info. However, we build exts as *bundles* and they are loaded only through `dlopen`, so it's safe to use `-undefined dynamic_lookup` in theory. So the warning produced by ld64 is false-positive, and it results failure of option checking in configuration. Therefore, it would be an option to ignore the warning during our configuration. On the other hand, `-undefined dynamic_lookup` is already deprecated on all darwin platforms except for macOS, so it's good time to get rid of the option. ld64 also provides `-bundle_loader <executable>` option, which allows to resolve symbols defined in the executable symtab while linking. It behaves almost the same with `-undefined dynamic_lookup`, but it makes the following changes: 1. Require that unresolved symbols among input objects must be defined in the executable. 2. Lazy symbol binding will lookup only the symtab of the bundle loader executable. (`-undefined dynamic_lookup` lookups all symtab as flat namespace) This patch adds `-bundle_loader $(RUBY)` when non-EXTSTATIC configuration by assuming ruby executable can be linked before building exts. See "New Features" subsection under "Linking" section for chained fixup https://developer.apple.com/documentation/xcode-release-notes/xcode-13-release-notes Notes: Merged: https://github.com/ruby/ruby/pull/6193
2022-07-19Rename ENCINDEX_ASCII to ENCINDEX_ASCII_8BITJean Boussier
Otherwise it's way too easy to confuse it with US_ASCII. Notes: Merged: https://github.com/ruby/ruby/pull/6127
2022-03-16Remove Unicode 13.0.0 related filesMartin Dürst
2022-03-13update Unicode Version to 14.0.0 and Emoji version to 14.0Martin Dürst
2022-03-02Revert "enc/depend: fix out-of-src build with --with-static-linked-ext" (#5616)Nobuyoshi Nakada
This reverts commit 32ad8df9d1e07e1b2435a8890d070802fcd2989f, which broke out-of-src build with the pre-generated transcoder sources. Notes: Merged-By: nobu <nobu@ruby-lang.org>
2022-03-02enc/depend: fix out-of-src build with --with-static-linked-extYuta Saito
When out-of-src build, at the beginning of a build, `make -f enc.mk srcs` generates trans C sources under build dir. On the other hand, enc/trans/*.o were built from trans C sources generated under srcdir due to the following auto-generated rules from enc/depend. ``` encsrcdir = ../src/enc ... enc/trans/big5.$(OBJEXT): $(encsrcdir)/trans/big5.c ``` Therefore, trans C sources are generated twice under srcdir and build dir during a build. Ideally, trans C sources have always been built before compilation of enc/trans/*.o because the source generation is prereq, so making enc/trans/*.o doesn't trigger trans C source generation and shouldn't require MINIRUBY as a make arg for enc.mk. However, the second trans C source gen is unintentionally triggered by enc/trans/*.o, so `make -f enc.mk libencs` requires MINIRUBY for now. When no `--with-static-linked-ext`, `make -f enc.mk libencs` is triggered from common.mk with MINIRUBY, so there is no problem. But when `--with-static-linked-ext`, libencs should be statically-linked to ruby, so `make -f enc.mk libencs` is triggered from exts.mk, and exts.mk invokes it without MINIRUBY. Therefore, when out-of-src build and with `--with-static-linked-ext`, the second trans C source gen fails due to missing MINIRUBY. This issue is deterministically reproducible without -j because common.mk's `main` rule also has libencs prerequisite. This patch supresses the second trans C source gen. Notes: Merged: https://github.com/ruby/ruby/pull/5611
2022-02-22[Feature #18249] Update dependenciesPeter Zhu
Notes: Merged: https://github.com/ruby/ruby/pull/5474
2022-02-22[Feature #18249] Include ruby.h in extensions to have ABI versionPeter Zhu
All shared libraries must have `include/ruby/internal/abi.h` to include the ABI version. Including `ruby.h` will guarantee that. Notes: Merged: https://github.com/ruby/ruby/pull/5474
2021-11-21Update dependenciesNobuyoshi Nakada
2021-10-05ruby tool/update-deps --fix卜部昌平
Notes: Merged: https://github.com/ruby/ruby/pull/4909
2021-07-09Remove no longer needed include files (Unicode Version 12.1.0)Martin Dürst
2021-07-08Add directory and include files for Unicode version 13.0.0Martin Dürst
- Add directory enc/unicode/13.0.0 - Add include files casefold.h and name2ctype.h for Unicode version 13.0.0
2021-07-03Use $ignore_error defined in mkmf.rbNobuyoshi Nakada
2021-04-13dependency updates卜部昌平
Notes: Merged: https://github.com/ruby/ruby/pull/4371
2020-12-29transcode-tblgen.rb: make silent a little when just -vNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/4011
2020-11-22Add string encoding IBM720 alias CP720 (#3803)Lars Kanis
The mapping table is generated from the ICU project: https://github.com/unicode-org/icu/blob/master/icu4c/source/data/mappings/ibm-720_P100-1997.ucm Fixes bug 16233 : https://bugs.ruby-lang.org/issues/16233 Notes: Merged-By: nurse <naruse@airemix.jp>
2020-08-27sed -i '/rmodule.h/d'卜部昌平
Notes: Merged: https://github.com/ruby/ruby/pull/3347
2020-08-27sed -i '/r_cast.h/d'卜部昌平
Notes: Merged: https://github.com/ruby/ruby/pull/3346
2020-08-27sed -i '\,2/extern.h,d'卜部昌平
Notes: Merged: https://github.com/ruby/ruby/pull/3338
2020-07-28Use https instead of httpKazuhiro NISHIYAMA
2020-07-10Encode ' as &apos; when using encode(xml: :attr)Jeremy Evans
Fixes [Bug #16922] Notes: Merged: https://github.com/ruby/ruby/pull/3177
2020-05-11sed -i 's|ruby/impl|ruby/internal|'卜部昌平
To fix build failures. Notes: Merged: https://github.com/ruby/ruby/pull/3079
2020-05-11sed -i s|ruby/3|ruby/impl|g卜部昌平
This shall fix compile errors. Notes: Merged: https://github.com/ruby/ruby/pull/3079
2020-05-04Suppress warnings by gcc 10.1.0-RC-20200430Nobuyoshi Nakada
* Folding results should not be empty. If `OnigCodePointCount(to->n)` were 0, `for` loop using `fn` wouldn't execute and `ncs` elements are not initialized. ``` enc/unicode.c:557:21: warning: 'ncs[0]' may be used uninitialized in this function [-Wmaybe-uninitialized] 557 | for (i = 0; i < ncs[0]; i++) { | ~~~^~~ ``` * Cast to `enum yytokentype` Additional enums for scanner events by ripper are not included in `yytokentype`. ``` ripper.y:7274:28: warning: implicit conversion from 'enum <anonymous>' to 'enum yytokentype' [-Wenum-conversion] ```
2020-04-08Merge pull request #2991 from shyouhei/ruby.h卜部昌平
Split ruby.h Notes: Merged-By: shyouhei <shyouhei@ruby-lang.org>
2020-04-05Suppress warnings: reserved for numbered parameterKazuki Tsujimoto
2020-04-05Added tooldir variableNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/3003
2020-02-07more on NULL versus functions.卜部昌平
Function pointers are not void*. See also ce4ea956d24eab5089a143bba38126f2b11b55b6 8427fca49bd85205f5a8766292dd893f003c0e48
2019-12-26update dependencies卜部昌平
Notes: Merged: https://github.com/ruby/ruby/pull/2711
2019-12-26decouple internal.h headers卜部昌平
Saves comitters' daily life by avoid #include-ing everything from internal.h to make each file do so instead. This would significantly speed up incremental builds. We take the following inclusion order in this changeset: 1. "ruby/config.h", where _GNU_SOURCE is defined (must be the very first thing among everything). 2. RUBY_EXTCONF_H if any. 3. Standard C headers, sorted alphabetically. 4. Other system headers, maybe guarded by #ifdef 5. Everything else, sorted alphabetically. Exceptions are those win32-related headers, which tend not be self- containing (headers have inclusion order dependencies). Notes: Merged: https://github.com/ruby/ruby/pull/2711
2019-12-24enc/x_emoji.h: fixed dead-links [ci skip]Nobuyoshi Nakada
English version pages seem no longer provided.
2019-12-22Fixed misspellingsNobuyoshi Nakada
Fixed misspellings reported at [Bug #16437], missed and a new typo.
2019-11-18Update dependenciesNobuyoshi Nakada
2019-08-10Init function is need to link staticallyNobuyoshi Nakada
2019-08-10Removed unnecessary headersNobuyoshi Nakada
2019-08-10Use ENC_REPLICATE to copy an encodingNobuyoshi Nakada
2019-08-10Revert "Removed unused includes"Yusuke Endoh
This reverts commit c9eb8f82e9febeb634a23bec6aeea915eb25fe26. The change caused "implicit declaration" warning and actual segfault. ``` /tmp/ruby/v2/src/trunk-gc-asserts/enc/gb2312.c: In function ‘Init_gb2312’: /tmp/ruby/v2/src/trunk-gc-asserts/enc/gb2312.c:6:31: warning: implicit declaration of function ‘rb_enc_find’ [-Wimplicit-function-declaration] rb_enc_register("GB2312", rb_enc_find("EUC-KR")); ^~~~~~~~~~~ /tmp/ruby/v2/src/trunk-gc-asserts/enc/gb2312.c:6:31: warning: passing argument 2 of ‘rb_enc_register’ makes pointer from integer without a cast [-Wint-conversion] <command-line>:0:19: note: expected ‘OnigEncoding {aka const struct OnigEncodingTypeST *}’ but argument is of type ‘int’ /tmp/ruby/v2/src/trunk-gc-asserts/regenc.h:231:12: note: in expansion of macro ‘ONIG_ENC_REGISTER’ extern int ONIG_ENC_REGISTER(const char *, OnigEncoding); ^~~~~~~~~~~~~~~~~ ```
2019-08-09Removed unused includesNobuyoshi Nakada
2019-07-14Include ruby/assert.h in ruby/ruby.h so that assertions can be thereNobuyoshi Nakada
2019-07-14Update dependencies for 369ff79394765ce198ac7cee872a8c739d895aaaTakashi Kokubun
Just copy-pasting diff from https://travis-ci.org/ruby/ruby/jobs/558407687
2019-07-14add encoding conversion from/to CESU-8Martin Dürst
Add encoding conversion (transcoding) from UTF-8 to CESU-8 and back. CESU-8 is an encoding similar to UTF-8, but encodes codepoints above U+FFFF as two surrogates, these surrogates again being encoded as if they were UTF-8 codepoints. This preserves the same binary sorting order as in UTF-16. It is also somewhat similar (although not exactly identical) to an encoding used internally by Java. This completes issue #15995. enc/trans/cesu_8.trans: Add encoding conversion from/to CESU-8 test/ruby/test_transcode.rb: Add tests for above
2019-07-09Update dependenciesNobuyoshi Nakada
2019-06-24remove UNREACHABLENARUSE, Yui
2019-06-24Add new encoding CESU-8 [Feature #15931]NARUSE, Yui
2019-04-05remove Unicode 12.0.0 related directory and generated filesduerst
This completes issue #15195. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67453 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-30 06:58:10 +0000