summaryrefslogtreecommitdiff
path: root/test/ruby/test_regexp.rb
AgeCommit message (Collapse)Author
2025-11-06[Backport #13671] Fix that "ss" in look-behind causes syntax errorK.Takata
Fixes k-takata/Onigmo#92. This fix was ported from oniguruma: https://github.com/kkos/oniguruma/commit/257082dac8c6019198b56324012f0bd1830ff4ba https://github.com/k-takata/Onigmo/commit/b1a5445fbeba97b3e94a733c2ce11c033453af73
2025-08-27Make word prop match join_control to conform to UTS 18Janosch Müller
See <https://bugs.ruby-lang.org/issues/19417#note-3>. https://unicode.org/reports/tr18/#word states word should match join_control chars. It did not previously: ```ruby [*0x0..0xD799, *0xE000..0x10FFFF].map { |n| n.chr 'utf-8' } => all_chars all_chars.grep(/\p{join_control}/) => jc jc.count # => 2 jc.grep(/\p{word}/).count # => 0 ``` [Backport #19417] --- Backporting note: I regenerated `enc/unicode/15.0.0/name2ctype.h` using `make update-unicode`.
2024-11-11Fix regex timeout double-free after stack_doubleJohn Hawthorn
As of 10574857ce167869524b97ee862b610928f6272f, it's possible to crash on a double free due to `stk_alloc` AKA `msa->stack_p` being freed twice, once at the end of match_at and a second time in `FREE_MATCH_ARG` in the parent caller. Fixes [Bug #20886] Notes: Merged: https://github.com/ruby/ruby/pull/12030
2024-07-25Fix memory leak in Regexp capture group when timeoutPeter Zhu
[Bug #20650] The capture group allocates memory that is leaked when it times out. For example: re = Regexp.new("^#{"(a*)" * 10_000}x$", timeout: 0.000001) str = "a" * 1000000 + "x" 10.times do 100.times do re =~ str rescue Regexp::TimeoutError end puts `ps -o rss= -p #{$$}` end Before: 34688 56416 78288 100368 120784 140704 161904 183568 204320 224800 After: 16288 16288 16880 16896 16912 16928 16944 17184 17184 17200 Notes: Merged: https://github.com/ruby/ruby/pull/11238
2024-07-16Add MatchData#bytebegin and MatchData#byteendShugo Maeda
These methods return the byte-based offset of the beginning or end of the specified match. [Feature #20576]
2024-06-07TestRegexp#test_match_cache_positive_look_behind: Extend the timeout limitYusuke Endoh
2024-06-07TestRegexp#test_timeout_shorter_than_global: Extend the timeout limitYusuke Endoh
2024-06-07TestRegexp#test_s_timeout: accept timeout errors more tolerantlyYusuke Endoh
This test seems flaky on macOS GitHub Actions
2024-04-25Don't use assert_separately in Bug 20453 testDaniel Colson
https://github.com/ruby/ruby/pull/10630#discussion_r1579565056 The PR was merged before I had a chance to address this feedback. `assert_separately` is not necessary for this test if I don't use a global timeout.
2024-04-25[Bug #20453] segfault in Regexp timeoutDaniel Colson
https://bugs.ruby-lang.org/issues/20228 started freeing `stk_base` to avoid a memory leak. But `stk_base` is sometimes stack allocated (using `xalloca`), so the free only works if the regex stack has grown enough to hit `stack_double` (which uses `xmalloc` and `xrealloc`). To reproduce the problem on master and 3.3.1: ```ruby Regexp.timeout = 0.001 /^(a*)x$/ =~ "a" * 1000000 + "x"' ``` Some details about this potential fix: `stk_base == stk_alloc` on [init](https://github.com/ruby/ruby/blob/dde99215f2bc60c22a00fc941ff7f714f011e920/regexec.c#L1153), so if `stk_base != stk_alloc` we can be sure we called [`stack_double`](https://github.com/ruby/ruby/blob/dde99215f2bc60c22a00fc941ff7f714f011e920/regexec.c#L1210) and it's safe to free. It's also safe to free if we've [saved](https://github.com/ruby/ruby/blob/dde99215f2bc60c22a00fc941ff7f714f011e920/regexec.c#L1187-L1189) the stack to `msa->stack_p`, since we do the `stk_base != stk_alloc` check before saving. This matches the check we do inside [`stack_double`](https://github.com/ruby/ruby/blob/dde99215f2bc60c22a00fc941ff7f714f011e920/regexec.c#L1221)
2024-02-22Skip under_gc_compact_stress on s390x (#10073)Takashi Kokubun
2024-02-13Fix [Bug #20246]: Don't set next_head_exact if a capture is called (#9897)Hiroya Fujinami
2024-02-02Add memory leak test for Regexp timeoutPeter Zhu
[Bug #20228]
2024-01-29Fix RegExp warning causing flaky Ripper failureAlan Wu
Sometimes this file get picked up and break Ripper tests: TestRipper::Generic#test_parse_files:test/ruby assert_separately failed with error message pid 63392 exit 0 | test_regexp.rb:2025: warning: character class has duplicated range https://github.com/ruby/ruby/actions/runs/7699956651/job/20982702553#step:12:103
2024-01-29Correctly handle consecutive lookarounds (#9738)Hiroya Fujinami
Fix [Bug #20207] Fix [Bug #20212] Handling consecutive lookarounds in init_cache_opcodes is buggy, so it causes invalid memory access reported in [Bug #20207] and [Bug #20212]. This fixes it by using recursive functions to detected lookarounds nesting correctly.
2024-01-11Prevent syntax warnings in test/ruby/test_regexp.rbYusuke Endoh
2024-01-10Fix test case for `test_match_cache_with_peek_optimization` (#9466)Hiroya Fujinami
2024-01-10Fix to work match cache with peek next optimization (#9459)Hiroya Fujinami
2024-01-01Don't create T_MATCH object if /regexp/.match(string) doesn't matchLuke Gruber
Fixes [Bug #20104]
2023-12-29Fix [Bug #20098]: set counter value for {n,m} repetition correctly (#9391)Hiroya Fujinami
2023-12-28Fix [Bug #20083]: correct a cache point size for atomic groups (#9367)Hiroya Fujinami
2023-12-24Fix Regexp#inspect for GC compactionPeter Zhu
rb_reg_desc was not safe for GC compaction because it took in the C string and length but not the backing String object so it get moved during compaction. This commit changes rb_reg_desc to use the string from the Regexp object. The test fails when RGENGC_CHECK_MODE is turned on: TestRegexp#test_inspect_under_gc_compact_stress [test/ruby/test_regexp.rb:474]: <"(?-mix:\\/)|"> expected but was <"/\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00/">.
2023-12-24Fix Regexp#match for GC compactionPeter Zhu
The test fails when RGENGC_CHECK_MODE is turned on: TestRegexp#test_match_under_gc_compact_stress: NoMethodError: undefined method `match' for nil test_regexp.rb:878:in `block in test_match_under_gc_compact_stress'
2023-12-23Fix Regexp#to_s for GC compactionPeter Zhu
The test fails when RGENGC_CHECK_MODE is turned on: TestRegexp#test_to_s_under_gc_compact_stress = 13.46 s 1) Failure: TestRegexp#test_to_s_under_gc_compact_stress [test/ruby/test_regexp.rb:81]: <"(?-mix:abcd\u3042)"> expected but was <"(?-mix:\u5C78\u3030\u5C78\u3030\u5C78\u3030\u5C78\u3030\u5C78\u3030)">.
2023-12-06Copy encoding flags when copying a regex [Bug #20039]Dustin Brown
* :bug: Fixes [Bug #20039](https://bugs.ruby-lang.org/issues/20039) When a Regexp is initialized with another Regexp, we simply copy the properties from the original. However, the flags on the original were not being copied correctly. This caused an issue when the original had multibyte characters and was being compared with an ASCII string. Without the forced encoding flag (`KCODE_FIXED`) transferred on to the new Regexp, the comparison would fail. See the included test for an example. Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
2023-11-08Improve error and memory handlingAdam Hess
Apply Nobu's suggestions which improve style, memory handling and error correction. Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
2023-10-30Optimize regexp matching for look-around and atomic groups (#7931)Hiroya Fujinami
2023-10-18Skip some timeout tests on s390xYusuke Endoh
They are too unstable on the machine. ``` 1) Failure: TestRegexp#test_timeout_shorter_than_global [/home/chkbuild/chkbuild/tmp/build/20231018T230003Z/ruby/test/ruby/test_regexp.rb:1788]: Expected |0.2 - 0.962938869| (0.7629388690000001) to be <= 0.15000000000000002. ``` https://rubyci.s3.amazonaws.com/s390x/ruby-master/log/20231018T230003Z.fail.html.gz ``` 1) Failure: TestRegexp#test_timeout_longer_than_global [/home/chkbuild/chkbuild/tmp/build/20231017T140006Z/ruby/test/ruby/test_regexp.rb:1788]: Expected |0.5 - 1.040696078| (0.5406960780000001) to be <= 0.375. ``` https://rubyci.s3.amazonaws.com/s390x/ruby-master/log/20231017T140006Z.fail.html.gz
2023-10-01Move repeating `matches` and `unmatches` to keyword argumentsNobuyoshi Nakada
And default to the corresponding instance variables.
2023-10-01Add tests for Unicode age property 15.0Nobuyoshi Nakada
2023-05-22Allow the match cache optimization for atomic groups (#7804)TSUYUSATO Kitsune
Notes: Merged-By: makenowjust <make.just.on@gmail.com>
2023-04-23Use UTF-8 encoding for literal extended regexps with UTF-8 characters in ↵Jeremy Evans
comments Fixes [Bug #19455] Notes: Merged: https://github.com/ruby/ruby/pull/7592
2023-04-19* remove trailing spaces. [ci skip]git
2023-04-19Refactor `Regexp#match` cache implementation (#7724)TSUYUSATO Kitsune
* Refactor Regexp#match cache implementation Improved variable and function names Fixed [Bug 19537] (Maybe fixed in https://github.com/ruby/ruby/pull/7694) * Add a comment of the glossary for "match cache" * Skip to reset match cache when no cache point on null check Notes: Merged-By: makenowjust <make.just.on@gmail.com>
2023-04-19MatchData#named_captures: add optional symbolize_names keyword (#6952)Vladimir Dementyev
Notes: Merged-By: ioquatix <samuel@codeotaku.com>
2023-04-12[Bug #19587] Fix `reset_match_cache` argumentsNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/7694
2023-03-18core_assertions.rb: Relax `assert_linear_performance`Nobuyoshi Nakada
* Use an `Enumerable` as factors, instead of three arguments. * Include `assert_operator` time in rehearsal time. * Round up max expected time. Notes: Merged: https://github.com/ruby/ruby/pull/7554
2023-03-16Revert "core_assertions.rb: Refine `assert_linear_performance`"Takashi Kokubun
This reverts commit cae4342dd559e34c1ce6219593f77f0ad80286da. This is failing a lot of CIs and nobody is actively looking into fixing it. Let me revert this until we have a solution to it.
2023-03-16core_assertions.rb: Refine `assert_linear_performance`Nobuyoshi Nakada
* Use an `Enumerable` as factors, instead of three arguments.
2023-03-13[Bug #19476]: correct cache index computation for repetition (#7457)TSUYUSATO Kitsune
Notes: Merged-By: makenowjust <make.just.on@gmail.com>
2023-03-13* remove trailing spaces. [ci skip]git
2023-03-13[Bug #19467] correct cache points and counting failure on ↵TSUYUSATO Kitsune
`OP_ANYCHAR_STAR_PEEK_NEXT` (#7454) Notes: Merged-By: makenowjust <make.just.on@gmail.com>
2023-03-12Add test for linear performanceNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/7506
2023-03-03[Bug #19471] `Regexp.compile` should handle keyword argumentsNobuyoshi Nakada
As well as `Regexp.new`, it should pass keyword arguments to the `Regexp#initialize` method. Notes: Merged: https://github.com/ruby/ruby/pull/7431
2023-03-01Remove support for the Regexp.new 3rd argumentJeremy Evans
This was deprecated in Ruby 3.2. Fixes [Bug #18797] Notes: Merged: https://github.com/ruby/ruby/pull/7039
2023-01-30Fix parsing of regexps that toggle extended mode on/off inside regexpJeremy Evans
This was broken in ec3542229b29ec93062e9d90e877ea29d3c19472. That commit didn't handle cases where extended mode was turned on/off inside the regexp. There are two ways to turn extended mode on/off: ``` /(?-x:#y)#z /x =~ '#y' /(?-x)#y(?x)#z /x =~ '#y' ``` These can be nested inside the same regexp: ``` /(?-x:(?x)#x (?-x)#y)#z /x =~ '#y' ``` As you can probably imagine, this makes handling these regexps somewhat complex. Due to the nesting inside portions of regexps, the unassign_nonascii function needs to be recursive. In recursive mode, it needs to track both opening and closing parentheses, similar to how it already tracked opening and closing brackets for character classes. When scanning the regexp and coming to `(?` not followed by `#`, scan for options, and use `x` and `i` to determine whether to turn on or off extended mode. For `:`, indicting only the current regexp section should have the extended mode switched, recurse with the extended mode set or unset. For `)`, indicating the remainder of the regexp (or current regexp portion if already recursing) should turn extended mode on or off, just change the extended mode flag and keep scanning. While testing this, I noticed that `a`, `d`, and `u` are accepted as options, in addition to `i`, `m`, and `x`, but I can't see where those options are documented. I'm not sure whether or not handling `a`, `d`, and `u` as options is a bug. Fixes [Bug #19379] Notes: Merged: https://github.com/ruby/ruby/pull/7192
2022-12-28Fix [Bug 19273], set correct value to `outer_repeat` on `OP_REPEAT` (#7035)TSUYUSATO Kitsune
Notes: Merged-By: makenowjust <make.just.on@gmail.com>
2022-12-22Always issue deprecation warning when calling Regexp.new with 3rd positional ↵Jeremy Evans
argument Previously, only certain values of the 3rd argument triggered a deprecation warning. First step for fix for bug #18797. Support for the 3rd argument will be removed after the release of Ruby 3.2. Fix minor fallout discovered by the tests. Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org> Notes: Merged: https://github.com/ruby/ruby/pull/6976
2022-12-22Share argument parsing in `Regexp#initialize` and `Regexp.linear_time?`Nobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/6988
2022-12-14Add `Regexp.linear_time?` (#6901)TSUYUSATO Kitsune
Notes: Merged-By: makenowjust <make.just.on@gmail.com>
generated by cgit v1.2.3 (git 2.25.1) at 2026-01-17 07:07:51 +0000