Backport lib/reline, ext/readline, and lib/irb for 3.0.1 (#4085)

* Get rid of inconsistent dll linkages against vcpkg readline

* [ruby/irb] Enhance colored inspect output

https://github.com/ruby/irb/commit/dffcdb5269

* [ruby/irb] Add color_printer.rb to gemspec

https://github.com/ruby/irb/commit/b4df0fd8b2

* [ruby/irb] Fix failing tests

https://github.com/ruby/irb/commit/7723ade899

* irb: add more syntax errors colorizing support (#3967)

* [ruby/irb] Do not colorize partially-correct inspect

This is to prevent a yellow-mixed output for ActiveSupport::TimeWithZone.

Follows up https://github.com/ruby/irb/pull/159 and https://github.com/ruby/ruby/pull/3967.

https://github.com/ruby/irb/commit/a5804c3560bb1de3ea8e40002635bff87f6a2825

* [ruby/irb] Remove unnecessary ignore_error in dispatch_seq

Just forgotten in https://github.com/ruby/irb/commit/a5804c3560bb1de3ea8e40002635bff87f6a2825

https://github.com/ruby/irb/commit/e42e548793

* Increase timeout for reline with --jit-wait

for failures like:
http://ci.rvm.jp/logfiles/brlog.trunk-mjit-wait.20201229-130509
http://ci.rvm.jp/logfiles/brlog.trunk-mjit-wait.20201229-165132
http://ci.rvm.jp/logfiles/brlog.trunk-mjit-wait.20201228-015519

* [ruby/irb] Stringify when a non-object is passed to PP#text

If a nested object is passed to #pp, it may be sometimes passed to the #text
method as an object without being stringified.

This is fixed on the Ruby main repository;
https://github.com/ruby/ruby/commit/433a3be86a811de0b4adbb92e054ee3a6fc6b4d8
but it was a bug of Ripper so still needs this workaround for using irb
as a gem on Ruby 3.0.0 or earlier.

Co-authored-by: k0kubun <takashikkbn@gmail.com>

https://github.com/ruby/irb/commit/8d13df22ee

* [ruby/irb] Newline in oneliner def doesn't reset indent

This closes ruby/irb#132.

https://github.com/ruby/irb/commit/43456dcf5e

* [ruby/irb] Escape invalid byte sequence in Exception

This fixes ruby/irb#141.

https://github.com/ruby/irb/commit/0815317d42

* [ruby/irb] Handle indentations related to keyword "do" correctly

This fixes ruby/irb#158.

https://github.com/ruby/irb/commit/964643400b

* [ruby/irb] Heredoc may contain multiple newlines in a single token

Use the start token as the indentation criteria so that it works properly in
heredoc.

ref. https://github.com/ruby/reline/pull/242

https://github.com/ruby/irb/commit/9704808dfd

* [ruby/irb] Use Ripper::Lexer#scan to take broken tokens

ref. https://github.com/ruby/reline/pull/242

https://github.com/ruby/irb/commit/54f90cb6c9

* [ruby/irb] Use error tokens if there are no correct tokens in the same place

For example, the broken code "%www" will result in only one error token.

https://github.com/ruby/irb/commit/9fa39a7cf3

* [ruby/irb] Ensure to restore $VERBOSE

https://github.com/ruby/irb/commit/cef474a76a

* 600x larger timeout for Reline

I didn't notice it's msec. 2.5s is too short.
http://ci.rvm.jp/results/trunk-mjit-wait@phosphorus-docker/3311385

* [ruby/irb] fix typo in `IRB::Irb#convert_invalid_byte_sequence`

https://github.com/ruby/irb/commit/d09d3c3d68

* [ruby/irb] do not escape a predicate method for doc namespace

* Fixes #88

https://github.com/ruby/irb/commit/d431a30af4

* [ruby/irb] refactoring an error handling in `IRB::Inspector`

* moved rescue clause to `#inspect_value` to catch all failures in inspectors
* test with all (currently five kind of) inspect modes
  - tweaked the input due to only `Marshal` can inspect(dump) a `BasicObject`

https://github.com/ruby/irb/commit/9d112fab8e

* [ruby/irb] Use Exception#full_message to show backtrace in the correct order

[Bug #17466]

https://github.com/ruby/irb/commit/1c76845cca

* [ruby/irb] Fix BACK_TRACE_LIMIT logic

https://github.com/ruby/irb/commit/30dc5d43fe

* irb: Drop lines from backtrace for tests in Ruby repository

* [ruby/reline] Update cursor correctly when just cursor moving

This fixes ruby/reline#236 and ruby/reline#239.

https://github.com/ruby/reline/commit/3e3c89d00b

* [ruby/reline] Correct var names in Reline were different from vi-*-mode-string

https://github.com/ruby/reline/commit/8255fc93b9

* [ruby/reline] Remove debug print

https://github.com/ruby/reline/commit/d7fbaedc6a

* [ruby/reline] Suppress crashing when auto_indent_proc returns broken indent info

Co-authored-by: Juanito Fatas <me@juanitofatas.com>

https://github.com/ruby/reline/commit/7c24276275

* [ruby/reline] Suppress crashing when dynamic_prompt_proc returns a broken prompt list

Co-authored-by: Juanito Fatas <me@juanitofatas.com>

https://github.com/ruby/reline/commit/558f7be168

* [ruby/reline] Suppress auto indent for adding newlines in pasting

Co-authored-by: Juanito Fatas <me@juanitofatas.com>

https://github.com/ruby/reline/commit/074bb017a7

* [ruby/reline] Add acknowledgments and license for rb-readline

https://github.com/ruby/reline/commit/19df59b916

* [ruby/irb] Fix comment, irb gem supports 2.5.0 or older

https://github.com/ruby/irb/commit/36118015ba

* should use `assert_include` here.

Random ordering test can introduce antoher candidate so it should be
`assert_include`.

* [ruby/irb] Add missing require

This is useful if you want to use IRB::ColorPrinter as a library like:

```
begin
  require 'irb/color_printer'
  IRB::ColorPrinter.pp(obj)
rescue LoadError
  pp(obj)
end
```

https://github.com/ruby/irb/commit/f8461691c7

* [ruby/irb] Make IRB::ColorPrinter.pp compatible with PP.pp

The incompatible interface is not helpful, again if you want to use it
as a standalone library, falling it back to PP.

Original PP.pp also ends with `out << "\n"`.

https://github.com/ruby/irb/commit/4c74c7d84c

* Suppress constant redefinition warnings

* Fix the failing test with XDG_CONFIG_HOME

* [ruby/irb] Version 1.3.1

https://github.com/ruby/irb/commit/c230d08911

* [ruby/reline] Handle ed_search_{prev,next}_history in multiline correctly

The current line was being handled incorrectly when displaying the hit
history, so it has been fixed to be correct.

https://github.com/ruby/reline/commit/a3df4343b3

* [ruby/reline] Move the cursor correctly when deleting at eol

This fixes ruby/reline#246.

https://github.com/ruby/reline/commit/07a73ba601

* [ruby/reline] Version 0.2.1

https://github.com/ruby/reline/commit/a3b3c6ee60

* [ruby/reline] Initialize a variable just in case

https://github.com/ruby/reline/commit/29b10f6e98

* [ruby/reline] Tests with yamatanooroti don't need chdir

Because of chdir, log files ware created in temporary directries on Windows.

https://github.com/ruby/reline/commit/200b469a68

* [ruby/reline] Windows needs more times to wait rendering

https://github.com/ruby/reline/commit/53ff2b09c7

* [ruby/reline] Support for change in Windows-specific behavior at eol

The behavior of automatically moving the cursor to the next line when
displaying a char at the eol on Windows suddenly disappeared.

https://github.com/ruby/reline/commit/cad4de6ee8

* [ruby/reline] Reline::Windows.erase_after_cursor erases attributes too

https://github.com/ruby/reline/commit/68b961dfc7

* [ruby/irb] [ruby/irb] [ruby/reline] Version 0.2.2

https://github.com/ruby/reline/commit/dfb710946f

https://github.com/ruby/irb/commit/1a1cdf9628

https://github.com/ruby/irb/commit/fe99faf8bd

* [ruby/irb] handle `__ENCODING__` as a keyword as well

https://github.com/ruby/irb/commit/a6a33d908f

* [ruby/irb] handle repeated exception separately

https://github.com/ruby/irb/commit/fcf6b34bc5

* [ruby/irb] skip a failling test on TruffleRuby

* due to the difference of backtrace pointed out by @aycabta

https://github.com/ruby/irb/commit/5e00a0ae61

* [ruby/irb] Version 1.3.2

https://github.com/ruby/irb/commit/a7699026cc

Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
Co-authored-by: Nobuhiro IMAI <nov@yo.rim.or.jp>
Co-authored-by: Koichi Sasada <ko1@atdot.net>
Co-authored-by: Hiroshi SHIBATA <hsbt@ruby-lang.org>

11 files changed, 520 insertions, 18 deletions

diff --git a/test/irb/test_color.rb b/test/irb/test_color.rb
index 03c4ab74a8..d035e443a8 100644
--- a/test/irb/test_color.rb
+++ b/test/irb/test_color.rb
@@ -1,6 +1,7 @@
 # frozen_string_literal: false
 require 'test/unit'
 require 'irb/color'
+require 'irb/color_printer'
 require 'rubygems'
 require 'stringio'
 
@@ -49,7 +50,7 @@ module TestIRB
         '"#{}"' => "#{RED}#{BOLD}\"#{CLEAR}#{RED}\#{#{CLEAR}#{RED}}#{CLEAR}#{RED}#{BOLD}\"#{CLEAR}",
         ':"a#{}b"' => "#{YELLOW}:\"#{CLEAR}#{YELLOW}a#{CLEAR}#{YELLOW}\#{#{CLEAR}#{YELLOW}}#{CLEAR}#{YELLOW}b#{CLEAR}#{YELLOW}\"#{CLEAR}",
         ':"a#{ def b; end; \'c\' + "#{ :d }" }e"' => "#{YELLOW}:\"#{CLEAR}#{YELLOW}a#{CLEAR}#{YELLOW}\#{#{CLEAR} #{GREEN}def#{CLEAR} #{BLUE}#{BOLD}b#{CLEAR}; #{GREEN}end#{CLEAR}; #{RED}#{BOLD}'#{CLEAR}#{RED}c#{CLEAR}#{RED}#{BOLD}'#{CLEAR} + #{RED}#{BOLD}\"#{CLEAR}#{RED}\#{#{CLEAR} #{YELLOW}:#{CLEAR}#{YELLOW}d#{CLEAR} #{RED}}#{CLEAR}#{RED}#{BOLD}\"#{CLEAR} #{YELLOW}}#{CLEAR}#{YELLOW}e#{CLEAR}#{YELLOW}\"#{CLEAR}",
-        "[__FILE__, __LINE__]" => "[#{CYAN}#{BOLD}__FILE__#{CLEAR}, #{CYAN}#{BOLD}__LINE__#{CLEAR}]",
+        "[__FILE__, __LINE__, __ENCODING__]" => "[#{CYAN}#{BOLD}__FILE__#{CLEAR}, #{CYAN}#{BOLD}__LINE__#{CLEAR}, #{CYAN}#{BOLD}__ENCODING__#{CLEAR}]",
         ":self" => "#{YELLOW}:#{CLEAR}#{YELLOW}self#{CLEAR}",
         ":class" => "#{YELLOW}:#{CLEAR}#{YELLOW}class#{CLEAR}",
         "[:end, 2]" => "[#{YELLOW}:#{CLEAR}#{YELLOW}end#{CLEAR}, #{BLUE}#{BOLD}2#{CLEAR}]",
@@ -82,9 +83,23 @@ module TestIRB
         tests.merge!({
           "[1]]]\u0013" => "[#{BLUE}#{BOLD}1#{CLEAR}]#{RED}#{REVERSE}]#{CLEAR}#{RED}#{REVERSE}]#{CLEAR}#{RED}#{REVERSE}^S#{CLEAR}",
         })
+        tests.merge!({
+          "def req(true) end" => "#{GREEN}def#{CLEAR} #{BLUE}#{BOLD}req#{CLEAR}(#{RED}#{REVERSE}true#{CLEAR}) #{RED}#{REVERSE}end#{CLEAR}",
+          "nil = 1" => "#{RED}#{REVERSE}nil#{CLEAR} = #{BLUE}#{BOLD}1#{CLEAR}",
+          "alias $x $1" => "#{GREEN}alias#{CLEAR} #{GREEN}#{BOLD}$x#{CLEAR} #{RED}#{REVERSE}$1#{CLEAR}",
+          "class bad; end" => "#{GREEN}class#{CLEAR} #{RED}#{REVERSE}bad#{CLEAR}; #{GREEN}end#{CLEAR}",
+          "def req(@a) end" => "#{GREEN}def#{CLEAR} #{BLUE}#{BOLD}req#{CLEAR}(#{RED}#{REVERSE}@a#{CLEAR}) #{GREEN}end#{CLEAR}",
+        })
       else
         tests.merge!({
           "[1]]]\u0013" => "[1]]]^S",
+          })
+        tests.merge!({
+          "def req(true) end" => "def req(true) end",
+          "nil = 1" => "#{CYAN}#{BOLD}nil#{CLEAR} = #{BLUE}#{BOLD}1#{CLEAR}",
+          "alias $x $1" => "#{GREEN}alias#{CLEAR} #{GREEN}#{BOLD}$x#{CLEAR} $1",
+          "class bad; end" => "#{GREEN}class#{CLEAR} bad; #{GREEN}end#{CLEAR}",
+          "def req(@a) end" => "#{GREEN}def#{CLEAR} #{BLUE}#{BOLD}req#{CLEAR}(@a) #{GREEN}end#{CLEAR}",
         })
       end
 
@@ -138,6 +153,23 @@ module TestIRB
       end
     end
 
+    IRBTestColorPrinter = Struct.new(:a)
+
+    def test_color_printer
+      unless ripper_lexer_scan_supported?
+        skip 'Ripper::Lexer#scan is supported in Ruby 2.7+'
+      end
+      {
+        1 => "#{BLUE}#{BOLD}1#{CLEAR}\n",
+        IRBTestColorPrinter.new('test') => "#{GREEN}#<struct TestIRB::TestColor::IRBTestColorPrinter#{CLEAR} a#{GREEN}=#{CLEAR}#{RED}#{BOLD}\"#{CLEAR}#{RED}test#{CLEAR}#{RED}#{BOLD}\"#{CLEAR}#{GREEN}>#{CLEAR}\n",
+        Ripper::Lexer.new('1').scan => "[#{GREEN}#<Ripper::Lexer::Elem:#{CLEAR} on_int@1:0 END token: #{RED}#{BOLD}\"#{CLEAR}#{RED}1#{CLEAR}#{RED}#{BOLD}\"#{CLEAR}#{GREEN}>#{CLEAR}]\n",
+        Class.new{define_method(:pretty_print){|q| q.text("[__FILE__, __LINE__, __ENCODING__]")}}.new => "[#{CYAN}#{BOLD}__FILE__#{CLEAR}, #{CYAN}#{BOLD}__LINE__#{CLEAR}, #{CYAN}#{BOLD}__ENCODING__#{CLEAR}]\n",
+      }.each do |object, result|
+        actual = with_term { IRB::ColorPrinter.pp(object, '') }
+        assert_equal(result, actual, "Case: IRB::ColorPrinter.pp(#{object.inspect}, '')")
+      end
+    end
+
     def test_inspect_colorable
       {
         1 => true,
@@ -170,6 +202,10 @@ module TestIRB
       Gem::Version.new(RUBY_VERSION) >= Gem::Version.new('2.7.0')
     end
 
+    def ripper_lexer_scan_supported?
+      Gem::Version.new(RUBY_VERSION) >= Gem::Version.new('2.7.0')
+    end
+
     def with_term
       stdout = $stdout
       io = StringIO.new
diff --git a/test/irb/test_completion.rb b/test/irb/test_completion.rb
index a765bbf3a5..984453d059 100644
--- a/test/irb/test_completion.rb
+++ b/test/irb/test_completion.rb
@@ -47,5 +47,13 @@ module TestIRB
         assert_include candidates, word
       end
     end
+
+    def test_complete_predicate?
+      candidates = IRB::InputCompletor.retrieve_completion_data("1.posi", bind: binding)
+      assert_include candidates, '1.positive?'
+
+      namespace = IRB::InputCompletor.retrieve_completion_data("1.positive?", bind: binding, doc_namespace: true)
+      assert_equal "Integer.positive?", namespace
+    end
   end
 end
diff --git a/test/irb/test_context.rb b/test/irb/test_context.rb
index ef7e2c5b69..f3d0626caa 100644
--- a/test/irb/test_context.rb
+++ b/test/irb/test_context.rb
@@ -83,6 +83,7 @@ module TestIRB
     end
 
     def test_eval_input
+      skip if RUBY_ENGINE == 'truffleruby'
       verbose, $VERBOSE = $VERBOSE, nil
       input = TestInputMethod.new([
         "raise 'Foo'\n",
@@ -95,7 +96,7 @@ module TestIRB
         irb.eval_input
       end
       assert_empty err
-      assert_pattern_list([:*, /RuntimeError \(.*Foo.*\).*\n/,
+      assert_pattern_list([:*, /\(irb\):1:in `<main>': Foo \(RuntimeError\)\n/,
                            :*, /#<RuntimeError: Foo>\n/,
                            :*, /0$/,
                            :*, /0$/,
@@ -104,17 +105,43 @@ module TestIRB
       $VERBOSE = verbose
     end
 
-    def test_eval_object_without_inspect_method
-      verbose, $VERBOSE = $VERBOSE, nil
+    def test_eval_input_raise2x
+      skip if RUBY_ENGINE == 'truffleruby'
       input = TestInputMethod.new([
-        "BasicObject.new\n",
+        "raise 'Foo'\n",
+        "raise 'Bar'\n",
+        "_\n",
       ])
       irb = IRB::Irb.new(IRB::WorkSpace.new(Object.new), input)
       out, err = capture_output do
         irb.eval_input
       end
       assert_empty err
-      assert(/\(Object doesn't support #inspect\)\n(=> )?\n/, out)
+      assert_pattern_list([
+          :*, /\(irb\):1:in `<main>': Foo \(RuntimeError\)\n/,
+          :*, /\(irb\):2:in `<main>': Bar \(RuntimeError\)\n/,
+          :*, /#<RuntimeError: Bar>\n/,
+        ], out)
+    end
+
+    def test_eval_object_without_inspect_method
+      verbose, $VERBOSE = $VERBOSE, nil
+      all_assertions do |all|
+        IRB::Inspector::INSPECTORS.invert.each_value do |mode|
+          all.for(mode) do
+            input = TestInputMethod.new([
+                "[BasicObject.new, Class.new]\n",
+              ])
+            irb = IRB::Irb.new(IRB::WorkSpace.new(Object.new), input)
+            irb.context.inspect_mode = mode
+            out, err = capture_output do
+              irb.eval_input
+            end
+            assert_empty err
+            assert_match(/\(Object doesn't support #inspect\)\n(=> )?\n/, out)
+          end
+        end
+      end
     ensure
       $VERBOSE = verbose
     end
@@ -234,7 +261,7 @@ module TestIRB
         irb.eval_input
       end
       assert_empty err
-      assert_equal("=> #{value.inspect}\n", out)
+      assert_equal("=> \n#{value.pretty_inspect}", out)
 
       input.reset
       irb.context.echo = true
@@ -243,7 +270,7 @@ module TestIRB
         irb.eval_input
       end
       assert_empty err
-      assert_equal("=> #{value.inspect[0..(input.winsize.last - 9)]}...\e[0m\n=> #{value.inspect}\n", out)
+      assert_equal("=> \n#{value.pretty_inspect[0..3]}...\n=> \n#{value.pretty_inspect}", out)
 
       input.reset
       irb.context.echo = true
@@ -252,7 +279,7 @@ module TestIRB
         irb.eval_input
       end
       assert_empty err
-      assert_equal("=> #{value.inspect}\n=> #{value.inspect}\n", out)
+      assert_equal("=> \n#{value.pretty_inspect}=> \n#{value.pretty_inspect}", out)
 
       input.reset
       irb.context.echo = false
@@ -408,5 +435,131 @@ module TestIRB
       assert_equal("=> abc\ndef\n",
                    out)
     end
+
+    def test_eval_input_with_exception
+      skip if RUBY_ENGINE == 'truffleruby'
+      verbose, $VERBOSE = $VERBOSE, nil
+      input = TestInputMethod.new([
+        "def hoge() fuga; end; def fuga() raise; end; hoge\n",
+      ])
+      irb = IRB::Irb.new(IRB::WorkSpace.new(Object.new), input)
+      out, err = capture_output do
+        irb.eval_input
+      end
+      assert_empty err
+      if '2.5.0' <= RUBY_VERSION && RUBY_VERSION < '3.0.0'
+        expected = [
+          :*, /Traceback \(most recent call last\):\n/,
+          :*, /\t 2: from \(irb\):1:in `<main>'\n/,
+          :*, /\t 1: from \(irb\):1:in `hoge'\n/,
+          :*, /\(irb\):1:in `fuga': unhandled exception\n/,
+        ]
+      else
+        expected = [
+          :*, /\(irb\):1:in `fuga': unhandled exception\n/,
+          :*, /\tfrom \(irb\):1:in `hoge'\n/,
+          :*, /\tfrom \(irb\):1:in `<main>'\n/,
+        ]
+      end
+      assert_pattern_list(expected, out)
+    ensure
+      $VERBOSE = verbose
+    end
+
+    def test_eval_input_with_invalid_byte_sequence_exception
+      skip if RUBY_ENGINE == 'truffleruby'
+      verbose, $VERBOSE = $VERBOSE, nil
+      input = TestInputMethod.new([
+        %Q{def hoge() fuga; end; def fuga() raise "A\\xF3B"; end; hoge\n},
+      ])
+      irb = IRB::Irb.new(IRB::WorkSpace.new(Object.new), input)
+      out, err = capture_output do
+        irb.eval_input
+      end
+      assert_empty err
+      if '2.5.0' <= RUBY_VERSION && RUBY_VERSION < '3.0.0'
+        expected = [
+          :*, /Traceback \(most recent call last\):\n/,
+          :*, /\t 2: from \(irb\):1:in `<main>'\n/,
+          :*, /\t 1: from \(irb\):1:in `hoge'\n/,
+          :*, /\(irb\):1:in `fuga': A\\xF3B \(RuntimeError\)\n/,
+        ]
+      else
+        expected = [
+          :*, /\(irb\):1:in `fuga': A\\xF3B \(RuntimeError\)\n/,
+          :*, /\tfrom \(irb\):1:in `hoge'\n/,
+          :*, /\tfrom \(irb\):1:in `<main>'\n/,
+        ]
+      end
+      assert_pattern_list(expected, out)
+    ensure
+      $VERBOSE = verbose
+    end
+
+    def test_eval_input_with_long_exception
+      skip if RUBY_ENGINE == 'truffleruby'
+      verbose, $VERBOSE = $VERBOSE, nil
+      nesting = 20
+      generated_code = ''
+      nesting.times do |i|
+        generated_code << "def a#{i}() a#{i + 1}; end; "
+      end
+      generated_code << "def a#{nesting}() raise; end; a0\n"
+      input = TestInputMethod.new([
+        generated_code
+      ])
+      irb = IRB::Irb.new(IRB::WorkSpace.new(Object.new), input)
+      out, err = capture_output do
+        irb.eval_input
+      end
+      assert_empty err
+      if '2.5.0' <= RUBY_VERSION && RUBY_VERSION < '3.0.0'
+        expected = [
+          :*, /Traceback \(most recent call last\):\n/,
+          :*, /\t... 5 levels...\n/,
+          :*, /\t16: from \(irb\):1:in `a4'\n/,
+          :*, /\t15: from \(irb\):1:in `a5'\n/,
+          :*, /\t14: from \(irb\):1:in `a6'\n/,
+          :*, /\t13: from \(irb\):1:in `a7'\n/,
+          :*, /\t12: from \(irb\):1:in `a8'\n/,
+          :*, /\t11: from \(irb\):1:in `a9'\n/,
+          :*, /\t10: from \(irb\):1:in `a10'\n/,
+          :*, /\t 9: from \(irb\):1:in `a11'\n/,
+          :*, /\t 8: from \(irb\):1:in `a12'\n/,
+          :*, /\t 7: from \(irb\):1:in `a13'\n/,
+          :*, /\t 6: from \(irb\):1:in `a14'\n/,
+          :*, /\t 5: from \(irb\):1:in `a15'\n/,
+          :*, /\t 4: from \(irb\):1:in `a16'\n/,
+          :*, /\t 3: from \(irb\):1:in `a17'\n/,
+          :*, /\t 2: from \(irb\):1:in `a18'\n/,
+          :*, /\t 1: from \(irb\):1:in `a19'\n/,
+          :*, /\(irb\):1:in `a20': unhandled exception\n/,
+        ]
+      else
+        expected = [
+          :*, /\(irb\):1:in `a20': unhandled exception\n/,
+          :*, /\tfrom \(irb\):1:in `a19'\n/,
+          :*, /\tfrom \(irb\):1:in `a18'\n/,
+          :*, /\tfrom \(irb\):1:in `a17'\n/,
+          :*, /\tfrom \(irb\):1:in `a16'\n/,
+          :*, /\tfrom \(irb\):1:in `a15'\n/,
+          :*, /\tfrom \(irb\):1:in `a14'\n/,
+          :*, /\tfrom \(irb\):1:in `a13'\n/,
+          :*, /\tfrom \(irb\):1:in `a12'\n/,
+          :*, /\tfrom \(irb\):1:in `a11'\n/,
+          :*, /\tfrom \(irb\):1:in `a10'\n/,
+          :*, /\tfrom \(irb\):1:in `a9'\n/,
+          :*, /\tfrom \(irb\):1:in `a8'\n/,
+          :*, /\tfrom \(irb\):1:in `a7'\n/,
+          :*, /\tfrom \(irb\):1:in `a6'\n/,
+          :*, /\tfrom \(irb\):1:in `a5'\n/,
+          :*, /\tfrom \(irb\):1:in `a4'\n/,
+          :*, /\t... 5 levels...\n/,
+        ]
+      end
+      assert_pattern_list(expected, out)
+    ensure
+      $VERBOSE = verbose
+    end
   end
 end
diff --git a/test/irb/test_init.rb b/test/irb/test_init.rb
index b51d01096d..83b4b5a543 100644
--- a/test/irb/test_init.rb
+++ b/test/irb/test_init.rb
@@ -21,6 +21,7 @@ module TestIRB
 
     def test_rc_file
       backup_irbrc = ENV.delete("IRBRC") # This is for RVM...
+      backup_xdg_config_home = ENV.delete("XDG_CONFIG_HOME")
       backup_home = ENV["HOME"]
       Dir.mktmpdir("test_irb_init_#{$$}") do |tmpdir|
         ENV["HOME"] = tmpdir
@@ -35,11 +36,13 @@ module TestIRB
       end
     ensure
       ENV["HOME"] = backup_home
+      ENV["XDG_CONFIG_HOME"] = backup_xdg_config_home
       ENV["IRBRC"] = backup_irbrc
     end
 
     def test_rc_file_in_subdir
       backup_irbrc = ENV.delete("IRBRC") # This is for RVM...
+      backup_xdg_config_home = ENV.delete("XDG_CONFIG_HOME")
       backup_home = ENV["HOME"]
       Dir.mktmpdir("test_irb_init_#{$$}") do |tmpdir|
         ENV["HOME"] = tmpdir
@@ -57,6 +60,7 @@ module TestIRB
       end
     ensure
       ENV["HOME"] = backup_home
+      ENV["XDG_CONFIG_HOME"] = backup_xdg_config_home
       ENV["IRBRC"] = backup_irbrc
     end
 
diff --git a/test/irb/test_raise_no_backtrace_exception.rb b/test/irb/test_raise_no_backtrace_exception.rb
index 699990f62d..40ee0c52bf 100644
--- a/test/irb/test_raise_no_backtrace_exception.rb
+++ b/test/irb/test_raise_no_backtrace_exception.rb
@@ -13,5 +13,13 @@ module TestIRB
       raise e
 IRB
     end
+
+    def test_raise_exception_with_invalid_byte_sequence
+      skip if RUBY_ENGINE == 'truffleruby'
+      bundle_exec = ENV.key?('BUNDLE_GEMFILE') ? ['-rbundler/setup'] : []
+      assert_in_out_err(bundle_exec + %w[-rirb -W0 -e IRB.start(__FILE__) -- -f --], <<~IRB, /A\\xF3B \(StandardError\)/, [])
+        raise StandardError, "A\\xf3B"
+      IRB
+    end
   end
 end
diff --git a/test/irb/test_ruby_lex.rb b/test/irb/test_ruby_lex.rb
index 41b5d49d1e..ed4944afc6 100644
--- a/test/irb/test_ruby_lex.rb
+++ b/test/irb/test_ruby_lex.rb
@@ -263,6 +263,129 @@ module TestIRB
       end
     end
 
+    def test_corresponding_syntax_to_keyword_do_in_class
+      input_with_correct_indents = [
+        Row.new(%q(class C), nil, 2, 1),
+        Row.new(%q(  while method_name do), nil, 4, 2),
+        Row.new(%q(    3), nil, 4, 2),
+        Row.new(%q(  end), 2, 2, 1),
+        Row.new(%q(  foo do), nil, 4, 2),
+        Row.new(%q(    3), nil, 4, 2),
+        Row.new(%q(  end), 2, 2, 1),
+        Row.new(%q(end), 0, 0, 0),
+      ]
+
+      lines = []
+      input_with_correct_indents.each do |row|
+        lines << row.content
+        assert_indenting(lines, row.current_line_spaces, false)
+        assert_indenting(lines, row.new_line_spaces, true)
+        assert_nesting_level(lines, row.nesting_level)
+      end
+    end
+
+    def test_corresponding_syntax_to_keyword_do
+      input_with_correct_indents = [
+        Row.new(%q(while i > 0), nil, 2, 1),
+        Row.new(%q(  3), nil, 2, 1),
+        Row.new(%q(end), 0, 0, 0),
+        Row.new(%q(while true), nil, 2, 1),
+        Row.new(%q(  3), nil, 2, 1),
+        Row.new(%q(end), 0, 0, 0),
+        Row.new(%q(while ->{i > 0}.call), nil, 2, 1),
+        Row.new(%q(  3), nil, 2, 1),
+        Row.new(%q(end), 0, 0, 0),
+        Row.new(%q(while ->{true}.call), nil, 2, 1),
+        Row.new(%q(  3), nil, 2, 1),
+        Row.new(%q(end), 0, 0, 0),
+        Row.new(%q(while i > 0 do), nil, 2, 1),
+        Row.new(%q(  3), nil, 2, 1),
+        Row.new(%q(end), 0, 0, 0),
+        Row.new(%q(while true do), nil, 2, 1),
+        Row.new(%q(  3), nil, 2, 1),
+        Row.new(%q(end), 0, 0, 0),
+        Row.new(%q(while ->{i > 0}.call do), nil, 2, 1),
+        Row.new(%q(  3), nil, 2, 1),
+        Row.new(%q(end), 0, 0, 0),
+        Row.new(%q(while ->{true}.call do), nil, 2, 1),
+        Row.new(%q(  3), nil, 2, 1),
+        Row.new(%q(end), 0, 0, 0),
+        Row.new(%q(foo do), nil, 2, 1),
+        Row.new(%q(  3), nil, 2, 1),
+        Row.new(%q(end), 0, 0, 0),
+        Row.new(%q(foo true do), nil, 2, 1),
+        Row.new(%q(  3), nil, 2, 1),
+        Row.new(%q(end), 0, 0, 0),
+        Row.new(%q(foo ->{true} do), nil, 2, 1),
+        Row.new(%q(  3), nil, 2, 1),
+        Row.new(%q(end), 0, 0, 0),
+        Row.new(%q(foo ->{i > 0} do), nil, 2, 1),
+        Row.new(%q(  3), nil, 2, 1),
+        Row.new(%q(end), 0, 0, 0),
+      ]
+
+      lines = []
+      input_with_correct_indents.each do |row|
+        lines << row.content
+        assert_indenting(lines, row.current_line_spaces, false)
+        assert_indenting(lines, row.new_line_spaces, true)
+        assert_nesting_level(lines, row.nesting_level)
+      end
+    end
+
+    def test_heredoc_with_indent
+      input_with_correct_indents = [
+        Row.new(%q(<<~Q), nil, 0, 0),
+        Row.new(%q({), nil, 0, 0),
+        Row.new(%q(  #), nil, 0, 0),
+        Row.new(%q(}), nil, 0, 0),
+      ]
+
+      lines = []
+      input_with_correct_indents.each do |row|
+        lines << row.content
+        assert_indenting(lines, row.current_line_spaces, false)
+        assert_indenting(lines, row.new_line_spaces, true)
+        assert_nesting_level(lines, row.nesting_level)
+      end
+    end
+
+    def test_oneliner_def_in_multiple_lines
+      input_with_correct_indents = [
+        Row.new(%q(def a()=[), nil, 4, 2),
+        Row.new(%q(  1,), nil, 4, 1),
+        Row.new(%q(].), 0, 0, 0),
+        Row.new(%q(to_s), nil, 0, 0),
+      ]
+
+      lines = []
+      input_with_correct_indents.each do |row|
+        lines << row.content
+        assert_indenting(lines, row.current_line_spaces, false)
+        assert_indenting(lines, row.new_line_spaces, true)
+        assert_nesting_level(lines, row.nesting_level)
+      end
+    end
+
+    def test_broken_heredoc
+      if Gem::Version.new(RUBY_VERSION) < Gem::Version.new('2.7.0')
+        skip 'This test needs Ripper::Lexer#scan to take broken tokens'
+      end
+      input_with_correct_indents = [
+        Row.new(%q(def foo), nil, 2, 1),
+        Row.new(%q(  <<~Q), nil, 2, 1),
+        Row.new(%q(  Qend), nil, 2, 1),
+      ]
+
+      lines = []
+      input_with_correct_indents.each do |row|
+        lines << row.content
+        assert_indenting(lines, row.current_line_spaces, false)
+        assert_indenting(lines, row.new_line_spaces, true)
+        assert_nesting_level(lines, row.nesting_level)
+      end
+    end
+
     PromptRow = Struct.new(:prompt, :content)
 
     class MockIO_DynamicPrompt
@@ -319,5 +442,37 @@ module TestIRB
       expected_prompt_list = input_with_prompt.map(&:prompt)
       assert_dynamic_prompt(lines, expected_prompt_list)
     end
+
+    def test_broken_percent_literal
+      if Gem::Version.new(RUBY_VERSION) < Gem::Version.new('2.7.0')
+        skip 'This test needs Ripper::Lexer#scan to take broken tokens'
+      end
+
+      ruby_lex = RubyLex.new
+      tokens = ruby_lex.ripper_lex_without_warning('%wwww')
+      pos_to_index = {}
+      tokens.each_with_index { |t, i|
+        assert_nil(pos_to_index[t[0]], "There is already another token in the position of #{t.inspect}.")
+        pos_to_index[t[0]] = i
+      }
+    end
+
+    def test_broken_percent_literal_in_method
+      if Gem::Version.new(RUBY_VERSION) < Gem::Version.new('2.7.0')
+        skip 'This test needs Ripper::Lexer#scan to take broken tokens'
+      end
+
+      ruby_lex = RubyLex.new
+      tokens = ruby_lex.ripper_lex_without_warning(<<~EOC.chomp)
+        def foo
+          %wwww
+        end
+      EOC
+      pos_to_index = {}
+      tokens.each_with_index { |t, i|
+        assert_nil(pos_to_index[t[0]], "There is already another token in the position of #{t.inspect}.")
+        pos_to_index[t[0]] = i
+      }
+    end
   end
 end
diff --git a/test/reline/helper.rb b/test/reline/helper.rb
index 5593b0a602..9712dde6c6 100644
--- a/test/reline/helper.rb
+++ b/test/reline/helper.rb
@@ -96,4 +96,18 @@ class Reline::TestCase < Test::Unit::TestCase
   def assert_cursor_max(expected)
     assert_equal(expected, @line_editor.instance_variable_get(:@cursor_max))
   end
+
+  def assert_line_index(expected)
+    assert_equal(expected, @line_editor.instance_variable_get(:@line_index))
+  end
+
+  def assert_whole_lines(expected)
+    previous_line_index = @line_editor.instance_variable_get(:@previous_line_index)
+    if previous_line_index
+      lines = @line_editor.whole_lines(index: previous_line_index)
+    else
+      lines = @line_editor.whole_lines
+    end
+    assert_equal(expected, lines)
+  end
 end
diff --git a/test/reline/test_key_actor_emacs.rb b/test/reline/test_key_actor_emacs.rb
index 84233a9bfd..b4dc3a1bcb 100644
--- a/test/reline/test_key_actor_emacs.rb
+++ b/test/reline/test_key_actor_emacs.rb
@@ -2233,6 +2233,53 @@ class Reline::KeyActor::Emacs::Test < Reline::TestCase
     assert_line('def hoge')
   end
 
+  def test_ed_search_prev_next_history_in_multibyte
+    Reline::HISTORY.concat([
+      "def hoge\n  67890\n  12345\nend", # old
+      "def aiu\n  0xDEADBEEF\nend",
+      "def foo\n  12345\nend" # new
+    ])
+    @line_editor.multiline_on
+    input_keys('  123')
+    # The ed_search_prev_history doesn't have default binding
+    @line_editor.__send__(:ed_search_prev_history, "\C-p".ord)
+    assert_whole_lines(['def foo', '  12345', 'end'])
+    assert_line_index(1)
+    assert_whole_lines(['def foo', '  12345', 'end'])
+    assert_byte_pointer_size('  123')
+    assert_cursor(5)
+    assert_cursor_max(7)
+    assert_line('  12345')
+    @line_editor.__send__(:ed_search_prev_history, "\C-p".ord)
+    assert_line_index(2)
+    assert_whole_lines(['def hoge', '  67890', '  12345', 'end'])
+    assert_byte_pointer_size('  123')
+    assert_cursor(5)
+    assert_cursor_max(7)
+    assert_line('  12345')
+    @line_editor.__send__(:ed_search_prev_history, "\C-p".ord)
+    assert_line_index(2)
+    assert_whole_lines(['def hoge', '  67890', '  12345', 'end'])
+    assert_byte_pointer_size('  123')
+    assert_cursor(5)
+    assert_cursor_max(7)
+    assert_line('  12345')
+    @line_editor.__send__(:ed_search_next_history, "\C-n".ord)
+    assert_line_index(1)
+    assert_whole_lines(['def foo', '  12345', 'end'])
+    assert_byte_pointer_size('  123')
+    assert_cursor(5)
+    assert_cursor_max(7)
+    assert_line('  12345')
+    @line_editor.__send__(:ed_search_next_history, "\C-n".ord)
+    assert_line_index(1)
+    assert_whole_lines(['def foo', '  12345', 'end'])
+    assert_byte_pointer_size('  123')
+    assert_cursor(5)
+    assert_cursor_max(7)
+    assert_line('  12345')
+  end
+
 =begin # TODO: move KeyStroke instance from Reline to LineEditor
   def test_key_delete
     input_keys('ab')
diff --git a/test/reline/test_key_actor_vi.rb b/test/reline/test_key_actor_vi.rb
index 6ac776fd9d..c6cd5eff48 100644
--- a/test/reline/test_key_actor_vi.rb
+++ b/test/reline/test_key_actor_vi.rb
@@ -1434,4 +1434,22 @@ class Reline::KeyActor::ViInsert::Test < Reline::TestCase
     assert_cursor(4)
     assert_cursor_max(4)
   end
+
+  def test_ed_delete_next_char_at_eol
+    input_keys('"あ"')
+    assert_line('"あ"')
+    assert_byte_pointer_size('"あ"')
+    assert_cursor(4)
+    assert_cursor_max(4)
+    input_keys("\C-[")
+    assert_line('"あ"')
+    assert_byte_pointer_size('"あ')
+    assert_cursor(3)
+    assert_cursor_max(4)
+    input_keys('xa"')
+    assert_line('"あ"')
+    assert_byte_pointer_size('"あ"')
+    assert_cursor(4)
+    assert_cursor_max(4)
+  end
 end
diff --git a/test/reline/test_within_pipe.rb b/test/reline/test_within_pipe.rb
index 53989a794f..70a0e0a5de 100644
--- a/test/reline/test_within_pipe.rb
+++ b/test/reline/test_within_pipe.rb
@@ -8,6 +8,7 @@ class Reline::WithinPipeTest < Reline::TestCase
     @reader, @output_writer = IO.pipe((RELINE_TEST_ENCODING rescue Encoding.default_external))
     @output = Reline.output = @output_writer
     @config = Reline.send(:core).config
+    @config.keyseq_timeout *= 600 if defined?(RubyVM::MJIT) && RubyVM::MJIT.enabled? # for --jit-wait CI
     @line_editor = Reline.send(:core).line_editor
   end
 
diff --git a/test/reline/yamatanooroti/test_rendering.rb b/test/reline/yamatanooroti/test_rendering.rb
index 0ec48c1896..b583f8ddac 100644
--- a/test/reline/yamatanooroti/test_rendering.rb
+++ b/test/reline/yamatanooroti/test_rendering.rb
@@ -14,14 +14,12 @@ begin
         FileUtils.rm_rf(@tmpdir)
         Dir.mkdir(@tmpdir)
       end
-      Dir.chdir(@tmpdir)
       @inputrc_backup = ENV['INPUTRC']
       @inputrc_file = ENV['INPUTRC'] = File.join(@tmpdir, 'temporaty_inputrc')
       File.unlink(@inputrc_file) if File.exist?(@inputrc_file)
     end
 
     def teardown
-      Dir.chdir(@pwd)
       FileUtils.rm_rf(@tmpdir)
       ENV['INPUTRC'] = @inputrc_backup
       ENV.delete('RELINE_TEST_PROMPT') if ENV['RELINE_TEST_PROMPT']
@@ -152,7 +150,7 @@ begin
       EOC
     end
 
-    def test_mode_icon_emacs
+    def test_mode_string_emacs
       write_inputrc <<~LINES
         set show-mode-in-prompt on
       LINES
@@ -164,7 +162,7 @@ begin
       EOC
     end
 
-    def test_mode_icon_vi
+    def test_mode_string_vi
       write_inputrc <<~LINES
         set editing-mode vi
         set show-mode-in-prompt on
@@ -180,7 +178,7 @@ begin
       EOC
     end
 
-    def test_original_mode_icon_emacs
+    def test_original_mode_string_emacs
       write_inputrc <<~LINES
         set show-mode-in-prompt on
         set emacs-mode-string [emacs]
@@ -193,7 +191,7 @@ begin
       EOC
     end
 
-    def test_original_mode_icon_with_quote
+    def test_original_mode_string_with_quote
       write_inputrc <<~LINES
         set show-mode-in-prompt on
         set emacs-mode-string "[emacs]"
@@ -206,7 +204,7 @@ begin
       EOC
     end
 
-    def test_original_mode_icon_vi
+    def test_original_mode_string_vi
       write_inputrc <<~LINES
         set editing-mode vi
         set show-mode-in-prompt on
@@ -224,7 +222,7 @@ begin
       EOC
     end
 
-    def test_mode_icon_vi_changing
+    def test_mode_string_vi_changing
       write_inputrc <<~LINES
         set editing-mode vi
         set show-mode-in-prompt on
@@ -450,6 +448,18 @@ begin
       EOC
     end
 
+    def test_broken_prompt_list
+      start_terminal(5, 30, %W{ruby -I#{@pwd}/lib #{@pwd}/bin/multiline_repl --broken-dynamic-prompt}, startup_message: 'Multiline REPL.')
+      write("def hoge\n  3\nend")
+      close
+      assert_screen(<<~EOC)
+        Multiline REPL.
+        [0000]> def hoge
+        [0001]>   3
+        [0001]> end
+      EOC
+    end
+
     def test_enable_bracketed_paste
       omit if Reline::IOGate.win?
       write_inputrc <<~LINES
@@ -515,6 +525,7 @@ begin
           end
         end
       EOC
+      sleep 1
       close
       assert_screen(<<~EOC)
         prompt>         prompt
@@ -561,6 +572,7 @@ begin
           end
         end
       EOC
+      sleep 1
       write("\C-p" * 6)
       close
       assert_screen(<<~EOC)
@@ -608,7 +620,7 @@ begin
           end
         end
       EOC
-      sleep 0.3
+      sleep 1
       write("\C-p" * 5)
       write("\C-n" * 3)
       close
@@ -620,6 +632,52 @@ begin
       EOC
     end
 
+    def test_update_cursor_correctly_when_just_cursor_moving
+      start_terminal(5, 30, %W{ruby -I#{@pwd}/lib #{@pwd}/bin/multiline_repl}, startup_message: 'Multiline REPL.')
+      write("def hoge\n  01234678")
+      write("\C-p")
+      write("\C-b")
+      write("\C-n")
+      write('5')
+      write("\C-e")
+      write('9')
+      close
+      assert_screen(<<~EOC)
+        Multiline REPL.
+        prompt> def hoge
+        prompt>   0123456789
+      EOC
+    end
+
+    def test_suppress_auto_indent_just_after_pasted
+      start_terminal(5, 30, %W{ruby -I#{@pwd}/lib #{@pwd}/bin/multiline_repl --auto-indent}, startup_message: 'Multiline REPL.')
+      write("def hoge\n  [[\n      3]]\ned")
+      write("\C-bn")
+      close
+      assert_screen(<<~EOC)
+        Multiline REPL.
+        prompt> def hoge
+        prompt>   [[
+        prompt>       3]]
+        prompt> end
+      EOC
+    end
+
+    def test_suppress_auto_indent_for_adding_newlines_in_pasting
+      start_terminal(5, 30, %W{ruby -I#{@pwd}/lib #{@pwd}/bin/multiline_repl --auto-indent}, startup_message: 'Multiline REPL.')
+      write("<<~Q\n")
+      write("{\n  #\n}")
+      write("#")
+      close
+      assert_screen(<<~EOC)
+        Multiline REPL.
+        prompt> <<~Q
+        prompt> {
+        prompt>   #
+        prompt> }#
+      EOC
+    end
+
     private def write_inputrc(content)
       File.open(@inputrc_file, 'w') do |f|
         f.write content