summaryrefslogtreecommitdiff
path: root/test/ruby/test_regexp.rb
AgeCommit message (Collapse)Author
2025-11-02Fix initialization of the table for quick searchK.Takata
This fixes k-takata/Onigmo#120. The commit k-takata/Onigmo@9c13de8d0684ebde97e3709d7693997c81ca374b was insufficient. https://github.com/k-takata/Onigmo/commit/1de602ddff140d91419e3f86dd35c81d7bd2d8e7
2025-11-02[Bug #13671] Fix that "ss" in look-behind causes syntax errorK.Takata
Fixes k-takata/Onigmo#92. This fix was ported from oniguruma: https://github.com/kkos/oniguruma/commit/257082dac8c6019198b56324012f0bd1830ff4ba https://github.com/k-takata/Onigmo/commit/b1a5445fbeba97b3e94a733c2ce11c033453af73
2024-11-12[Bug #20886] Avoid double-free in regex timeout after stack_double (#12063)John Hawthorn
Fix regex timeout double-free after stack_double As of 10574857ce167869524b97ee862b610928f6272f, it's possible to crash on a double free due to `stk_alloc` AKA `msa->stack_p` being freed twice, once at the end of match_at and a second time in `FREE_MATCH_ARG` in the parent caller. Fixes [Bug #20886]
2024-07-25[Bug #20650] Fix memory leak in Regexp capture group when timeout (#11244)Peter Zhu
Fix memory leak in Regexp capture group when timeout [Bug #20650] The capture group allocates memory that is leaked when it times out. For example: re = Regexp.new("^#{"(a*)" * 10_000}x$", timeout: 0.000001) str = "a" * 1000000 + "x" 10.times do 100.times do re =~ str rescue Regexp::TimeoutError end puts `ps -o rss= -p #{$$}` end Before: 34688 56416 78288 100368 120784 140704 161904 183568 204320 224800 After: 16288 16288 16880 16896 16912 16928 16944 17184 17184 17200
2024-05-29merge revision(s) d292a9b98ce03c76dbe13138d20b9fbf613cc02d: [Backport #20453]Takashi Kokubun
[Bug #20453] segfault in Regexp timeout https://bugs.ruby-lang.org/issues/20228 started freeing `stk_base` to avoid a memory leak. But `stk_base` is sometimes stack allocated (using `xalloca`), so the free only works if the regex stack has grown enough to hit `stack_double` (which uses `xmalloc` and `xrealloc`). To reproduce the problem on master and 3.3.1: ```ruby Regexp.timeout = 0.001 /^(a*)x$/ =~ "a" * 1000000 + "x"' ``` Some details about this potential fix: `stk_base == stk_alloc` on [init](https://github.com/ruby/ruby/blob/dde99215f2bc60c22a00fc941ff7f714f011e920/regexec.c#L1153), so if `stk_base != stk_alloc` we can be sure we called [`stack_double`](https://github.com/ruby/ruby/blob/dde99215f2bc60c22a00fc941ff7f714f011e920/regexec.c#L1210) and it's safe to free. It's also safe to free if we've [saved](https://github.com/ruby/ruby/blob/dde99215f2bc60c22a00fc941ff7f714f011e920/regexec.c#L1187-L1189) the stack to `msa->stack_p`, since we do the `stk_base != stk_alloc` check before saving. This matches the check we do inside [`stack_double`](https://github.com/ruby/ruby/blob/dde99215f2bc60c22a00fc941ff7f714f011e920/regexec.c#L1221)
2024-05-29Skip under_gc_compact_stress on s390x (#10073)Takashi Kokubun
2024-05-28Fix RegExp warning causing flaky Ripper failureAlan Wu
Sometimes this file get picked up and break Ripper tests: TestRipper::Generic#test_parse_files:test/ruby assert_separately failed with error message pid 63392 exit 0 | test_regexp.rb:2025: warning: character class has duplicated range https://github.com/ruby/ruby/actions/runs/7699956651/job/20982702553#step:12:103
2024-03-21merge revision(s) ↵NARUSE, Yui
18ee7c9a108bf3424814565377c8796e5e455cf7,4a6384ed9358e8fb8464f6e37efb5477182f01db: [Backport #20246] (#10309) Clear all refined CCs on reopening refinement mod In cfd7729ce7a31c8b6ec5dd0e99c67b2932de4732 we started using inline caches for refinements. However, we weren't clearing inline caches when defined on a reopened refinement module. Fixes [Bug #20246] Fix [Bug #20246]: Don't set next_head_exact if a capture is called (#9897)
2024-03-20merge revision(s) 3e6e3ca2627b1aa71b17de902cc1b8188246a828: [Backport ↵NARUSE, Yui
#20207] (#10299) Correctly handle consecutive lookarounds (#9738) Fix [Bug #20207] Fix [Bug #20212] Handling consecutive lookarounds in init_cache_opcodes is buggy, so it causes invalid memory access reported in [Bug #20207] and [Bug #20212]. This fixes it by using recursive functions to detected lookarounds nesting correctly.
2024-03-20merge revision(s) ↵NARUSE, Yui
01bfd1a2bf013a9ed92a9722ac5228187e05e6a8,1c120efe02d079b0a1dea573cf0fd7978d9cc857,31378dc0969f4466b2122d730b7298dd7004acdf: [Backport #20228] (#10301) Fix memory leak in OnigRegion when match raises [Bug #20228] rb_reg_onig_match can raise a Regexp::TimeoutError, which would cause the OnigRegion to leak. Fix memory leak in stk_base when Regexp timeout [Bug #20228] If rb_reg_check_timeout raises a Regexp::TimeoutError, then the stk_base will leak. Add memory leak test for Regexp timeout [Bug #20228]
2024-03-20merge revision(s) bb59696614083660241ef272f222628cbfa95844: [Backport ↵NARUSE, Yui
#20098] (#10298) Fix [Bug #20098]: set counter value for {n,m} repetition correctly (#9391)
2024-02-01merge revision(s) 597955a,8b65d15: [Backport #20173] (#9794)NARUSE, Yui
Fix to work match cache with peek next optimization (#9459) --- regexec.c | 3 ++- test/ruby/test_regexp.rb | 9 +++++++++ 2 files changed, 11 insertions(+), 1 deletion(-) Fix test case for `test_match_cache_with_peek_optimization` (#9466) --- test/ruby/test_regexp.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
2024-01-30merge revision(s) e12d4c654e3cb7a4473014610bc3bae41aaf811e: [Backport #20104]NARUSE, Yui
Don't create T_MATCH object if /regexp/.match(string) doesn't match Fixes [Bug #20104] --- re.c | 9 ++++++--- test/ruby/test_regexp.rb | 12 ++++++++++++ tool/lib/envutil.rb | 8 ++++++++ 3 files changed, 26 insertions(+), 3 deletions(-)
2024-01-30merge revision(s) d8702ddbfbe8cc7fc601a9a4d19842ef9c2b76c1: [Backport #20083]NARUSE, Yui
Fix [Bug #20083]: correct a cache point size for atomic groups (#9367) --- regexec.c | 2 +- test/ruby/test_regexp.rb | 8 ++++++++ 2 files changed, 9 insertions(+), 1 deletion(-)
2023-12-24Fix Regexp#inspect for GC compactionPeter Zhu
rb_reg_desc was not safe for GC compaction because it took in the C string and length but not the backing String object so it get moved during compaction. This commit changes rb_reg_desc to use the string from the Regexp object. The test fails when RGENGC_CHECK_MODE is turned on: TestRegexp#test_inspect_under_gc_compact_stress [test/ruby/test_regexp.rb:474]: <"(?-mix:\\/)|"> expected but was <"/\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00/">.
2023-12-24Fix Regexp#match for GC compactionPeter Zhu
The test fails when RGENGC_CHECK_MODE is turned on: TestRegexp#test_match_under_gc_compact_stress: NoMethodError: undefined method `match' for nil test_regexp.rb:878:in `block in test_match_under_gc_compact_stress'
2023-12-23Fix Regexp#to_s for GC compactionPeter Zhu
The test fails when RGENGC_CHECK_MODE is turned on: TestRegexp#test_to_s_under_gc_compact_stress = 13.46 s 1) Failure: TestRegexp#test_to_s_under_gc_compact_stress [test/ruby/test_regexp.rb:81]: <"(?-mix:abcd\u3042)"> expected but was <"(?-mix:\u5C78\u3030\u5C78\u3030\u5C78\u3030\u5C78\u3030\u5C78\u3030)">.
2023-12-06Copy encoding flags when copying a regex [Bug #20039]Dustin Brown
* :bug: Fixes [Bug #20039](https://bugs.ruby-lang.org/issues/20039) When a Regexp is initialized with another Regexp, we simply copy the properties from the original. However, the flags on the original were not being copied correctly. This caused an issue when the original had multibyte characters and was being compared with an ASCII string. Without the forced encoding flag (`KCODE_FIXED`) transferred on to the new Regexp, the comparison would fail. See the included test for an example. Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
2023-11-08Improve error and memory handlingAdam Hess
Apply Nobu's suggestions which improve style, memory handling and error correction. Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
2023-10-30Optimize regexp matching for look-around and atomic groups (#7931)Hiroya Fujinami
2023-10-18Skip some timeout tests on s390xYusuke Endoh
They are too unstable on the machine. ``` 1) Failure: TestRegexp#test_timeout_shorter_than_global [/home/chkbuild/chkbuild/tmp/build/20231018T230003Z/ruby/test/ruby/test_regexp.rb:1788]: Expected |0.2 - 0.962938869| (0.7629388690000001) to be <= 0.15000000000000002. ``` https://rubyci.s3.amazonaws.com/s390x/ruby-master/log/20231018T230003Z.fail.html.gz ``` 1) Failure: TestRegexp#test_timeout_longer_than_global [/home/chkbuild/chkbuild/tmp/build/20231017T140006Z/ruby/test/ruby/test_regexp.rb:1788]: Expected |0.5 - 1.040696078| (0.5406960780000001) to be <= 0.375. ``` https://rubyci.s3.amazonaws.com/s390x/ruby-master/log/20231017T140006Z.fail.html.gz
2023-10-01Move repeating `matches` and `unmatches` to keyword argumentsNobuyoshi Nakada
And default to the corresponding instance variables.
2023-10-01Add tests for Unicode age property 15.0Nobuyoshi Nakada
2023-05-22Allow the match cache optimization for atomic groups (#7804)TSUYUSATO Kitsune
Notes: Merged-By: makenowjust <make.just.on@gmail.com>
2023-04-23Use UTF-8 encoding for literal extended regexps with UTF-8 characters in ↵Jeremy Evans
comments Fixes [Bug #19455] Notes: Merged: https://github.com/ruby/ruby/pull/7592
2023-04-19* remove trailing spaces. [ci skip]git
2023-04-19Refactor `Regexp#match` cache implementation (#7724)TSUYUSATO Kitsune
* Refactor Regexp#match cache implementation Improved variable and function names Fixed [Bug 19537] (Maybe fixed in https://github.com/ruby/ruby/pull/7694) * Add a comment of the glossary for "match cache" * Skip to reset match cache when no cache point on null check Notes: Merged-By: makenowjust <make.just.on@gmail.com>
2023-04-19MatchData#named_captures: add optional symbolize_names keyword (#6952)Vladimir Dementyev
Notes: Merged-By: ioquatix <samuel@codeotaku.com>
2023-04-12[Bug #19587] Fix `reset_match_cache` argumentsNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/7694
2023-03-18core_assertions.rb: Relax `assert_linear_performance`Nobuyoshi Nakada
* Use an `Enumerable` as factors, instead of three arguments. * Include `assert_operator` time in rehearsal time. * Round up max expected time. Notes: Merged: https://github.com/ruby/ruby/pull/7554
2023-03-16Revert "core_assertions.rb: Refine `assert_linear_performance`"Takashi Kokubun
This reverts commit cae4342dd559e34c1ce6219593f77f0ad80286da. This is failing a lot of CIs and nobody is actively looking into fixing it. Let me revert this until we have a solution to it.
2023-03-16core_assertions.rb: Refine `assert_linear_performance`Nobuyoshi Nakada
* Use an `Enumerable` as factors, instead of three arguments.
2023-03-13[Bug #19476]: correct cache index computation for repetition (#7457)TSUYUSATO Kitsune
Notes: Merged-By: makenowjust <make.just.on@gmail.com>
2023-03-13* remove trailing spaces. [ci skip]git
2023-03-13[Bug #19467] correct cache points and counting failure on ↵TSUYUSATO Kitsune
`OP_ANYCHAR_STAR_PEEK_NEXT` (#7454) Notes: Merged-By: makenowjust <make.just.on@gmail.com>
2023-03-12Add test for linear performanceNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/7506
2023-03-03[Bug #19471] `Regexp.compile` should handle keyword argumentsNobuyoshi Nakada
As well as `Regexp.new`, it should pass keyword arguments to the `Regexp#initialize` method. Notes: Merged: https://github.com/ruby/ruby/pull/7431
2023-03-01Remove support for the Regexp.new 3rd argumentJeremy Evans
This was deprecated in Ruby 3.2. Fixes [Bug #18797] Notes: Merged: https://github.com/ruby/ruby/pull/7039
2023-01-30Fix parsing of regexps that toggle extended mode on/off inside regexpJeremy Evans
This was broken in ec3542229b29ec93062e9d90e877ea29d3c19472. That commit didn't handle cases where extended mode was turned on/off inside the regexp. There are two ways to turn extended mode on/off: ``` /(?-x:#y)#z /x =~ '#y' /(?-x)#y(?x)#z /x =~ '#y' ``` These can be nested inside the same regexp: ``` /(?-x:(?x)#x (?-x)#y)#z /x =~ '#y' ``` As you can probably imagine, this makes handling these regexps somewhat complex. Due to the nesting inside portions of regexps, the unassign_nonascii function needs to be recursive. In recursive mode, it needs to track both opening and closing parentheses, similar to how it already tracked opening and closing brackets for character classes. When scanning the regexp and coming to `(?` not followed by `#`, scan for options, and use `x` and `i` to determine whether to turn on or off extended mode. For `:`, indicting only the current regexp section should have the extended mode switched, recurse with the extended mode set or unset. For `)`, indicating the remainder of the regexp (or current regexp portion if already recursing) should turn extended mode on or off, just change the extended mode flag and keep scanning. While testing this, I noticed that `a`, `d`, and `u` are accepted as options, in addition to `i`, `m`, and `x`, but I can't see where those options are documented. I'm not sure whether or not handling `a`, `d`, and `u` as options is a bug. Fixes [Bug #19379] Notes: Merged: https://github.com/ruby/ruby/pull/7192
2022-12-28Fix [Bug 19273], set correct value to `outer_repeat` on `OP_REPEAT` (#7035)TSUYUSATO Kitsune
Notes: Merged-By: makenowjust <make.just.on@gmail.com>
2022-12-22Always issue deprecation warning when calling Regexp.new with 3rd positional ↵Jeremy Evans
argument Previously, only certain values of the 3rd argument triggered a deprecation warning. First step for fix for bug #18797. Support for the 3rd argument will be removed after the release of Ruby 3.2. Fix minor fallout discovered by the tests. Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org> Notes: Merged: https://github.com/ruby/ruby/pull/6976
2022-12-22Share argument parsing in `Regexp#initialize` and `Regexp.linear_time?`Nobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/6988
2022-12-14Add `Regexp.linear_time?` (#6901)TSUYUSATO Kitsune
Notes: Merged-By: makenowjust <make.just.on@gmail.com>
2022-12-12Make absent operator work at the end of the input stringYusuke Endoh
https://bugs.ruby-lang.org/issues/19104#change-100542 Notes: Merged: https://github.com/ruby/ruby/pull/6902
2022-11-27Relax a too strict timeoutTakashi Kokubun
Regexp tests are flaky. http://rubyci.s3.amazonaws.com/s390x/ruby-master/log/20221128T050004Z.fail.html.gz
2022-11-24Relax the timeout of TestRegexp#test_cache_optimization_squareYusuke Endoh
It fails on riscv (QEmu) http://rubyci.s3.amazonaws.com/debian-riscv64/ruby-master/log/20221124T000021Z.fail.html.gz ``` 1) Error: TestRegexp#test_cache_optimization_square: Regexp::TimeoutError: regexp match timeout /home/rubyci/chkbuild/tmp/build/20221124T000021Z/ruby/test/ruby/test_regexp.rb:1693:in `<main>' /home/rubyci/chkbuild/tmp/build/20221124T000021Z/ruby/test/ruby/test_regexp.rb:1688:in `test_cache_optimization_square' ```
2022-11-19Avoid a timeout on test_cache_optimization_exponentialTakashi Kokubun
The timeout seems too short for some CIs. http://rubyci.s3.amazonaws.com/debian11-aarch64/ruby-master/log/20221120T012840Z.fail.html.gz
2022-11-11Allow a float error for Regexp.timeoutYusuke Endoh
The tests failed on windows https://github.com/ruby/ruby/actions/runs/3440997073/jobs/5740085169#step:18:62 ``` 1) Failure: TestRegexp#test_s_timeout [D:/a/ruby/ruby/src/test/ruby/test_regexp.rb:1586]: <0.30000000000000004> expected but was <0.3>. 2) Failure: TestRegexp#test_timeout_shorter_than_global [D:/a/ruby/ruby/src/test/ruby/test_regexp.rb:1631]: <0.30000000000000004> expected but was <0.3>. ```
2022-11-11Run EnvUtil.apply_timeout_scale outside of assert_separatelyYusuke Endoh
It does not work well in assert_separately
2022-11-09Update timeout seconds for square testTSUYUSATO Kitsune
generated by cgit v1.2.3 (git 2.25.1) at 2026-01-17 00:15:58 +0000