summaryrefslogtreecommitdiff
path: root/tool/lib/test
AgeCommit message (Collapse)Author
2024-05-08Show the caller location of assertion methodsNobuyoshi Nakada
Not only defined in `Test::Unit` or `CoreAssertions`, also show the caller location of assertion methods defined in the current class or ancestors.
2024-05-07[PRISM] Enable test_methoddef_endless_commandKevin Newton
2024-04-03Launchable: Add lineNumber fieldNaoto Ono
2024-04-02Launchable: Refactor the logic of JsonStreamWriterNaoto Ono
2024-02-29Inform failures in parallel tests before retryingNobuyoshi Nakada
Displays for each failure which test it actually occurred in. The output destination follows the --{stdout,stderr}-on-failure option.
2024-02-23Add Launchable into CINaoto Ono
2024-02-21Add `Test::Unit::TestCase#method_name`Nobuyoshi Nakada
For the compatibility with test-unit gem.
2024-02-15Show the method owner in backtracesYusuke Endoh
``` test.rb:1:in 'Object#toplevel_meth': unhandled exception from test.rb:4:in 'Foo.class_meth' from test.rb:6:in 'Foo#instance_meth' from test.rb:11:in 'singleton_meth' from test.rb:13:in '<main>' ``` [Feature #19117]
2024-01-31Add newline between end and defNaoto Ono
2024-01-31Add the ability to generate Launchable test reportsNaoto Ono
2023-12-22Get rid of obsoleted RJITFirstTakashi Kokubun
It was renamed from test_mjit, but we did not maintain it as test_rjit. We test RJIT very differently.
2023-11-15Refactor the settings of test-all outYusuke Endoh
test/runner.rb and tool/lib/test/unit/parallel.rb must use the same settings. However, some settings were copied and pasted, while some were added only to test/runner.rb. This changeset creates tool/test/init.rb for all settings of test-unit, which is loaded not only by test/runner.rb but also tool/lib/test/unit/parallel.rb. Background: the GEM_HOME environment variable was removed in test/runner.rb, which prohibit `require "rake"` (note that rake is a bundled gem). However the parallel mode didn't refrect this setting, i.e., `require "rake"` was allowed. This leads to an inconsistency, which actually affected a test test defines s test class *only when* `require "rake"` is successful. (test/rubygems/test_gem_package_task.rb) https://github.com/ruby/ruby/actions/runs/6807729617/job/18511055636#step:8:1714 ``` /home/runner/work/ruby/ruby/src/tool/lib/test/unit.rb:729:in `const_get': uninitialized constant TestGemPackageTask (NameError) suites.map! {|r| ::Object.const_get(r[:testcase])} ^^^^^^^^^^ ```
2023-10-26tool test/unit/testcase: rename vars @passed, @@currentlukeg
to @__passed__, @@__current__. @passed is redefined in a few test suites, and this could lead to bugs. Also rename @options (Runner#options) to @__runner_options__, which is only used in make test-tool anyway.
2023-10-21Fallback job status to normal if no ttyNobuyoshi Nakada
2023-10-18Handle `Timeout::Error` reported from workersNobuyoshi Nakada
2023-10-18Fix total count when filters givenNobuyoshi Nakada
2023-10-18Escape method names containing control charactersNobuyoshi Nakada
2023-10-18Manage parallel test workers after timeoutNobuyoshi Nakada
2023-08-29Expose Test.filter_backtrace for the default gems.Hiroshi SHIBATA
Notes: Merged: https://github.com/ruby/ruby/pull/8313
2023-08-08Add message for `--ruby` option [ci skip]Nobuyoshi Nakada
2023-06-03Exit with a failure if any test files failed to loadNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/7897
2023-03-06s/mjit/rjit/Takashi Kokubun
Notes: Merged: https://github.com/ruby/ruby/pull/7462
2023-03-06s/MJIT/RJIT/Takashi Kokubun
Notes: Merged: https://github.com/ruby/ruby/pull/7462
2023-02-28fix test/rubygems/test_gem_package_task.rb when in -j modelukeg
This test skipped sometimes due to failure to load 'rake/packagetask'. This is due to manipulation of $LOAD_PATH by other rubygems tests. If rake is loaded before any rubygems tests run, then it works fine. To reproduce the skipping behavior: $ make test-all TESTOPTS="-j6 --test-order=sorted test/rubygems/test_*.rb" Notes: Merged: https://github.com/ruby/ruby/pull/7394
2023-01-18Supressing warnings messages like:Hiroshi SHIBATA
Ignoring debug-1.7.1 because its extensions are not built. Try: gem pristine debug --version 1.7. Revert "Clear gem paths for each test" This reverts commit 6698b580ddad8cfa8c5c86df9328472820d3ee6a.
2023-01-15Clear gem paths for each testNobuyoshi Nakada
So that rubygems can find the bundled rake. Notes: Merged: https://github.com/ruby/ruby/pull/7122
2022-12-01Use class methods of `File` over `Kernel.open` and `IO.read`Nobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/6834
2022-11-07jobserver option may not be at the lastNobuyoshi Nakada
2022-11-07Set `autoclose:` for inherited FDs only [ci skip]Nobuyoshi Nakada
2022-11-06GNU make 4.4 now uses a fifo for the jobserverNobuyoshi Nakada
2022-11-02Fix crash in test runner on timeoutPeter Zhu
When a test worker hangs and timeouts, the test runner crashes with the following stack trace: ruby/tool/lib/test/unit.rb:1747:in `puke': undefined method `backtrace' for Timeout::Error:Class (NoMethodError) from ruby/tool/lib/test/unit.rb:790:in `block in _run_parallel' from ruby/tool/lib/test/unit.rb:788:in `each' This commit adds handling for Timeout::Error and outputs a message.
2022-09-10Enable deprecation warnings for test-allBenoit Daloze
* So deprecated methods/constants/functions are dealt with early, instead of many tests breaking suddenly when removing a deprecated method/constant/function. * Follows https://bugs.ruby-lang.org/issues/17591 Notes: Merged: https://github.com/ruby/ruby/pull/6321
2022-07-05Add `--stdout-on-failure`, the reverse of `--stderr-on-failure`Nobuyoshi Nakada
2022-07-05Separate failed output optionNobuyoshi Nakada
It is unrelated to `GlobOption` at all.
2022-06-25Prevent accidental use of assert_raisesNobuyoshi Nakada
2022-05-20Rename test_jit to test_mjitTakashi Kokubun
to avoid confusion with YJIT
2022-04-20Added assert_true and assert_false same as test-unit gemHiroshi SHIBATA
2022-04-06Copyedit parallel test harness crash messageAlan Wu
Mea culpa. Co-authored-by: Kaíque Kandy Koga <kaiquekandykoga@gmail.com>
2022-04-06Copyedit parallel test harness crash messageAlan Wu
Notes: Merged: https://github.com/ruby/ruby/pull/5772
2022-04-05Try to fix NoMethodError on slow environmentsKazuhiro NISHIYAMA
``` 1) Failure: TestParallel::TestParallel#test_hungup [/home/user/ruby/tool/test/testunit/test_parallel.rb:215]: Expected /^Retrying hung up testcases\.+$/ to match "Run options: \n" + " --seed=43403\n" + " --ruby\n" + " \"./miniruby -I../lib -I. -I.ext/common ../tool/runruby.rb --extout=.ext -- --disable-gems\"\n" + " -j\n" + " t1\n" + " --worker-timeout=1\n" + "\n" + "# Running tests:\n" + "\n" + "/home/user/ruby/tool/lib/test/unit.rb:687:in `block in _run_parallel': undefined method `<' for nil:NilClass (NoMethodError)\n" + "\tfrom /home/user/ruby/tool/lib/test/unit.rb:538:in `block in quit_workers'\n" + "\tfrom /home/user/ruby/tool/lib/test/unit.rb:537:in `reject!'\n" + "\tfrom /home/user/ruby/tool/lib/test/unit.rb:537:in `quit_workers'\n" + "\tfrom /home/user/ruby/tool/lib/test/unit.rb:687:in `_run_parallel'\n" + "\tfrom /home/user/ruby/tool/lib/test/unit.rb:810:in `_run_suites'\n" + "\tfrom /home/user/ruby/tool/lib/test/unit.rb:849:in `_run_suites'\n" + "\tfrom /home/user/ruby/tool/lib/test/unit.rb:1479:in `_run_anything'\n" + "\tfrom /home/user/ruby/tool/lib/test/unit.rb:1263:in `_run_anything'\n" + "\tfrom /home/user/ruby/tool/lib/test/unit.rb:1654:in `run_tests'\n" + "\tfrom /home/user/ruby/tool/lib/test/unit.rb:1641:in `block in _run'\n" + "\tfrom /home/user/ruby/tool/lib/test/unit.rb:1640:in `each'\n" + "\tfrom /home/user/ruby/tool/lib/test/unit.rb:1640:in `_run'\n" + "\tfrom /home/user/ruby/tool/lib/test/unit.rb:1682:in `run'\n" + "\tfrom /home/user/ruby/tool/lib/test/unit.rb:1034:in `run'\n" + "\tfrom /home/user/ruby/tool/lib/test/unit.rb:882:in `run'\n" + "\tfrom /home/user/ruby/tool/lib/test/unit.rb:154:in `run'\n" + "\tfrom /home/user/ruby/tool/lib/test/unit.rb:1761:in `run'\n" + "\tfrom /home/user/ruby/tool/lib/test/unit.rb:1765:in `run'\n" + "\tfrom /home/user/ruby/tool/test/testunit/tests_for_parallel/runner.rb:14:in `<main>'\n". ```
2022-02-16fix parallel test timeout retryingKoichi Sasada
On the parallel test, workers can be killed because of timeout and the information for the retrying can be inconsistent. This patch will skip if the inconsistency is found and report as an error. http://ci.rvm.jp/results/trunk-asserts@phosphorus-docker/3834082 Notes: Merged: https://github.com/ruby/ruby/pull/5559
2022-01-26add `--stderr-on-failure` option to test-allKoichi Sasada
Now all failure messages are printed to stdout. This option makes all failure messages printed into stderr. Notes: Merged: https://github.com/ruby/ruby/pull/5483
2022-01-17Removed skip alias in test suiteHiroshi SHIBATA
Notes: Merged: https://github.com/ruby/ruby/pull/5446
2021-12-13Prepare for removing RubyVM::JIT (#5262)Takashi Kokubun
Notes: Merged-By: k0kubun <takashikkbn@gmail.com>
2021-11-09Filter method names only if filtering method name onlyNobuyoshi Nakada
If sole `filter` option doesn't seem including test case name, match with method name only. And if the filter is a Regexp or String, it never matches method name symbols.
2021-10-27Properly exclude test cases.Vít Ondruch
Lets consider the following scenario: ~~~ irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):001:0> p suite OpenSSL::TestEC => OpenSSL::TestEC irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):002:0> p all_test_methods ["test_ECPrivateKey", "test_ECPrivateKey_encrypted", "test_PUBKEY", "test_check_key", "test_derive_key", "test_dh_compute_key", "test_dsa_sign_asn1_FIPS186_3", "test_ec_group", "test_ec_key", "test_ec_point", "test_ec_point_add", "test_ec_point_mul", "test_generate", "test_marshal", "test_sign_verify", "test_sign_verify_raw"] => ["test_ECPrivateKey", "test_ECPrivateKey_encrypted", "test_PUBKEY", "test_check_key", "test_derive_key", "test_dh_compute_key", "test_dsa_sign_asn1_FIPS186_3", "test_ec_group", "test_ec_key", "test_ec_point", "test_ec_point_add", "test_ec_point_mul", "test_generate", "test_marshal", "test_sign_verify", "test_sign_verify_raw"] irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):003:0> p filter /\A(?=.*)(?!.*(?-mix:(?-mix:memory_leak)|(?-mix:OpenSSL::TestEC.test_check_key)))/ => /\A(?=.*)(?!.*(?-mix:(?-mix:memory_leak)|(?-mix:OpenSSL::TestEC.test_check_key)))/ irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):004:0> method = "test_check_key" => "test_check_key" ~~~ The intention here is to exclude the `test_check_key` test case. Unfortunately this does not work as expected, because the negative filter is never checked: ~~~ irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):005:0> filter === method => true irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):006:0> filter === "#{suite}##{method}" => false irb(#<Test::Unit::AutoRunner::Runner:0x0000560f68afc3c8>):007:0> filter === method || filter === "#{suite}##{method}" => true ~~~ Therefore always filter against the fully qualified method name `#{suite}##{method}`, which should provide the expected result. However, if plain string filter is used, keep checking also only the method name. This resolves [Bug #16936]. Notes: Merged: https://github.com/ruby/ruby/pull/5026
2021-10-26Revert "introduce check code for mysterious EBADF"Koichi Sasada
This reverts commit 7864efa105921eb3900c843126f2e0db02b9c6ae.
2021-10-26Revert "check other IO#close calls"Koichi Sasada
This reverts commit a1c4cab11d613d7df037a81a770ee44a23a2e9be.
2021-10-25check other IO#close callsKoichi Sasada
http://ci.rvm.jp/results/trunk@ruby-iga/3690333 > tool/lib/test/unit/parallel.rb:68:in `close': Bad file descriptor (Errno::EBADF)
2021-10-25introduce check code for mysterious EBADFKoichi Sasada
parallel test randomly failed with EBADF. This patch checks wich suite causes this error. ex) http://ci.rvm.jp/results/trunk@ruby-iga/3690219 ``` /tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:88:in `close': Bad file descriptor (Errno::EBADF) /tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:88:in `ensure in _run_suite' /tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:89:in `_run_suite' /tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:30:in `block in _run_suites' /tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:29:in `map' /tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:29:in `_run_suites' /tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:128:in `run' /tmp/ruby/v3/src/trunk/tool/lib/test/unit/parallel.rb:211:in `<main>' ```
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-29 04:54:30 +0000