ruby.git/test/test_unicode_normalize.rb, branch v4.0.3

Add test for Unicode normalization of Gurung Khema.

2025-11-02T00:27:54+00:00

Add regression test for bug 21559.

2025-11-02T00:27:54+00:00

Improve performance of UnicodeNormalize.canonical_ordering_one

2025-10-25T12:19:29+00:00

Use array_of_integer.sort! instead of buble-sort-like algorithm

Prefer to use File.readlines instead of IO.readlines

2023-02-27T09:49:18+00:00

Removed unused variables

2019-06-28T14:21:37+00:00

update to Unicode Version 12.1.0 (beta)

2019-04-05T00:58:51+00:00

Unicode Version 12.1.0 adds one single character, U+32FF SQUARE ERA NAME REIWA,
for the new Japanese era starting on May 1st. 12.1.0 will be finalized only on
May 7th, so we go with the beta version because further changes in the data we
need are highly unlikely, and we want to make sure Ruby is ready for the new era.

* common.mk: change UNICODE_VERSION to 12.1.0, UNICODE_BETA to YES

* enc/unicode/12.1.0, enc/unicode/12.1.0/casefold.h, enc/unicode/12.1.0/name2ctype.h:
  add directory and generated data files for new version

* lib/unicode_normalize/tables.rb: update for new character

* test/ruby/test_regexp.rb: add test for character property age=12.1

* test/test_unicode_normalize.rb: add test for NFKC decomposition of new character

This (mostly) completes issue #15195.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67441 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

fix range check for Hangul jamo trailers in Unicode normalization

2018-07-28T09:44:33+00:00

* lib/unicode_normalize/normalize.rb: Fix the range check for trailing
  Hangul jamo characters in Unicode normalization. Different from
  leading or vowel jamos, where LBASE and VBASE are actual characters,
  a value equal to TBASE expresses the absence of a trailing jamo.
  This fix is technically correct, but there was no bug because
  the regular expressions in lib/unicode_normalize/tables.rb
  eliminate jamos equal to TBASE from normalization processing.

* test/test_unicode_normalize.rb: Add preventive test
  test_no_trailing_jamo based on
  https://github.com/python/cpython/commit/d134809cd3764c6a634eab7bb8995e3e2eff14d5
  just for the case we ever get a regression.

This closes issue #14934, thanks to MaLin (Lin Ma) for reporting.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64087 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

always run unicode normalization that do not depend on data file

2018-07-28T09:11:13+00:00

* test/test_unicode_normalize.rb: extract tests that do not depend
  on NormalizationTest.txt data file from conditionally constructed
  part of TestUnicodeNormalize class, to always run them even if
  the data file isn't found.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64086 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

update unicode data files directory for normalization tests

2018-07-28T08:56:59+00:00

* test/test_unicode_normalize.rb: search ucd directory
  first if it exists. This change follows r61415.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64085 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* test/ruby/enc/test_case_comprehensive.rb, test_regex_casefold.rb,

2016-07-03T09:51:46+00:00

  test/test_unicode_normalize.rb: Replace UNICODE_VERSION from
  UnicodeNormalize with RbConfig::CONFIG['UNICODE_VERSION'] from
  feature 12460.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55567 b2dd03c8-39d4-4d8f-98ff-823fe69b080e