Backport lib/reline, ext/readline, and lib/irb for 3.0.1 (#4085)

* Get rid of inconsistent dll linkages against vcpkg readline

* [ruby/irb] Enhance colored inspect output

https://github.com/ruby/irb/commit/dffcdb5269

* [ruby/irb] Add color_printer.rb to gemspec

https://github.com/ruby/irb/commit/b4df0fd8b2

* [ruby/irb] Fix failing tests

https://github.com/ruby/irb/commit/7723ade899

* irb: add more syntax errors colorizing support (#3967)

* [ruby/irb] Do not colorize partially-correct inspect

This is to prevent a yellow-mixed output for ActiveSupport::TimeWithZone.

Follows up https://github.com/ruby/irb/pull/159 and https://github.com/ruby/ruby/pull/3967.

https://github.com/ruby/irb/commit/a5804c3560bb1de3ea8e40002635bff87f6a2825

* [ruby/irb] Remove unnecessary ignore_error in dispatch_seq

Just forgotten in https://github.com/ruby/irb/commit/a5804c3560bb1de3ea8e40002635bff87f6a2825

https://github.com/ruby/irb/commit/e42e548793

* Increase timeout for reline with --jit-wait

for failures like:
http://ci.rvm.jp/logfiles/brlog.trunk-mjit-wait.20201229-130509
http://ci.rvm.jp/logfiles/brlog.trunk-mjit-wait.20201229-165132
http://ci.rvm.jp/logfiles/brlog.trunk-mjit-wait.20201228-015519

* [ruby/irb] Stringify when a non-object is passed to PP#text

If a nested object is passed to #pp, it may be sometimes passed to the #text
method as an object without being stringified.

This is fixed on the Ruby main repository;
https://github.com/ruby/ruby/commit/433a3be86a811de0b4adbb92e054ee3a6fc6b4d8
but it was a bug of Ripper so still needs this workaround for using irb
as a gem on Ruby 3.0.0 or earlier.

Co-authored-by: k0kubun <takashikkbn@gmail.com>

https://github.com/ruby/irb/commit/8d13df22ee

* [ruby/irb] Newline in oneliner def doesn't reset indent

This closes ruby/irb#132.

https://github.com/ruby/irb/commit/43456dcf5e

* [ruby/irb] Escape invalid byte sequence in Exception

This fixes ruby/irb#141.

https://github.com/ruby/irb/commit/0815317d42

* [ruby/irb] Handle indentations related to keyword "do" correctly

This fixes ruby/irb#158.

https://github.com/ruby/irb/commit/964643400b

* [ruby/irb] Heredoc may contain multiple newlines in a single token

Use the start token as the indentation criteria so that it works properly in
heredoc.

ref. https://github.com/ruby/reline/pull/242

https://github.com/ruby/irb/commit/9704808dfd

* [ruby/irb] Use Ripper::Lexer#scan to take broken tokens

ref. https://github.com/ruby/reline/pull/242

https://github.com/ruby/irb/commit/54f90cb6c9

* [ruby/irb] Use error tokens if there are no correct tokens in the same place

For example, the broken code "%www" will result in only one error token.

https://github.com/ruby/irb/commit/9fa39a7cf3

* [ruby/irb] Ensure to restore $VERBOSE

https://github.com/ruby/irb/commit/cef474a76a

* 600x larger timeout for Reline

I didn't notice it's msec. 2.5s is too short.
http://ci.rvm.jp/results/trunk-mjit-wait@phosphorus-docker/3311385

* [ruby/irb] fix typo in `IRB::Irb#convert_invalid_byte_sequence`

https://github.com/ruby/irb/commit/d09d3c3d68

* [ruby/irb] do not escape a predicate method for doc namespace

* Fixes #88

https://github.com/ruby/irb/commit/d431a30af4

* [ruby/irb] refactoring an error handling in `IRB::Inspector`

* moved rescue clause to `#inspect_value` to catch all failures in inspectors
* test with all (currently five kind of) inspect modes
  - tweaked the input due to only `Marshal` can inspect(dump) a `BasicObject`

https://github.com/ruby/irb/commit/9d112fab8e

* [ruby/irb] Use Exception#full_message to show backtrace in the correct order

[Bug #17466]

https://github.com/ruby/irb/commit/1c76845cca

* [ruby/irb] Fix BACK_TRACE_LIMIT logic

https://github.com/ruby/irb/commit/30dc5d43fe

* irb: Drop lines from backtrace for tests in Ruby repository

* [ruby/reline] Update cursor correctly when just cursor moving

This fixes ruby/reline#236 and ruby/reline#239.

https://github.com/ruby/reline/commit/3e3c89d00b

* [ruby/reline] Correct var names in Reline were different from vi-*-mode-string

https://github.com/ruby/reline/commit/8255fc93b9

* [ruby/reline] Remove debug print

https://github.com/ruby/reline/commit/d7fbaedc6a

* [ruby/reline] Suppress crashing when auto_indent_proc returns broken indent info

Co-authored-by: Juanito Fatas <me@juanitofatas.com>

https://github.com/ruby/reline/commit/7c24276275

* [ruby/reline] Suppress crashing when dynamic_prompt_proc returns a broken prompt list

Co-authored-by: Juanito Fatas <me@juanitofatas.com>

https://github.com/ruby/reline/commit/558f7be168

* [ruby/reline] Suppress auto indent for adding newlines in pasting

Co-authored-by: Juanito Fatas <me@juanitofatas.com>

https://github.com/ruby/reline/commit/074bb017a7

* [ruby/reline] Add acknowledgments and license for rb-readline

https://github.com/ruby/reline/commit/19df59b916

* [ruby/irb] Fix comment, irb gem supports 2.5.0 or older

https://github.com/ruby/irb/commit/36118015ba

* should use `assert_include` here.

Random ordering test can introduce antoher candidate so it should be
`assert_include`.

* [ruby/irb] Add missing require

This is useful if you want to use IRB::ColorPrinter as a library like:

```
begin
  require 'irb/color_printer'
  IRB::ColorPrinter.pp(obj)
rescue LoadError
  pp(obj)
end
```

https://github.com/ruby/irb/commit/f8461691c7

* [ruby/irb] Make IRB::ColorPrinter.pp compatible with PP.pp

The incompatible interface is not helpful, again if you want to use it
as a standalone library, falling it back to PP.

Original PP.pp also ends with `out << "\n"`.

https://github.com/ruby/irb/commit/4c74c7d84c

* Suppress constant redefinition warnings

* Fix the failing test with XDG_CONFIG_HOME

* [ruby/irb] Version 1.3.1

https://github.com/ruby/irb/commit/c230d08911

* [ruby/reline] Handle ed_search_{prev,next}_history in multiline correctly

The current line was being handled incorrectly when displaying the hit
history, so it has been fixed to be correct.

https://github.com/ruby/reline/commit/a3df4343b3

* [ruby/reline] Move the cursor correctly when deleting at eol

This fixes ruby/reline#246.

https://github.com/ruby/reline/commit/07a73ba601

* [ruby/reline] Version 0.2.1

https://github.com/ruby/reline/commit/a3b3c6ee60

* [ruby/reline] Initialize a variable just in case

https://github.com/ruby/reline/commit/29b10f6e98

* [ruby/reline] Tests with yamatanooroti don't need chdir

Because of chdir, log files ware created in temporary directries on Windows.

https://github.com/ruby/reline/commit/200b469a68

* [ruby/reline] Windows needs more times to wait rendering

https://github.com/ruby/reline/commit/53ff2b09c7

* [ruby/reline] Support for change in Windows-specific behavior at eol

The behavior of automatically moving the cursor to the next line when
displaying a char at the eol on Windows suddenly disappeared.

https://github.com/ruby/reline/commit/cad4de6ee8

* [ruby/reline] Reline::Windows.erase_after_cursor erases attributes too

https://github.com/ruby/reline/commit/68b961dfc7

* [ruby/irb] [ruby/irb] [ruby/reline] Version 0.2.2

https://github.com/ruby/reline/commit/dfb710946f

https://github.com/ruby/irb/commit/1a1cdf9628

https://github.com/ruby/irb/commit/fe99faf8bd

* [ruby/irb] handle `__ENCODING__` as a keyword as well

https://github.com/ruby/irb/commit/a6a33d908f

* [ruby/irb] handle repeated exception separately

https://github.com/ruby/irb/commit/fcf6b34bc5

* [ruby/irb] skip a failling test on TruffleRuby

* due to the difference of backtrace pointed out by @aycabta

https://github.com/ruby/irb/commit/5e00a0ae61

* [ruby/irb] Version 1.3.2

https://github.com/ruby/irb/commit/a7699026cc

Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
Co-authored-by: Nobuhiro IMAI <nov@yo.rim.or.jp>
Co-authored-by: Koichi Sasada <ko1@atdot.net>
Co-authored-by: Hiroshi SHIBATA <hsbt@ruby-lang.org>

1 files changed, 59 insertions, 39 deletions

diff --git a/lib/irb.rb b/lib/irb.rb
index 4eef8be613..3f7f169c69 100644
--- a/lib/irb.rb
+++ b/lib/irb.rb
@@ -574,10 +574,35 @@ module IRB
             next
           end
           handle_exception(exc)
+          @context.workspace.local_variable_set(:_, exc)
+          exc = nil
         end
       end
     end
 
+    def convert_invalid_byte_sequence(str)
+      str = str.force_encoding(Encoding::ASCII_8BIT)
+      conv = Encoding::Converter.new(Encoding::ASCII_8BIT, Encoding::UTF_8)
+      dst = String.new
+      begin
+        ret = conv.primitive_convert(str, dst)
+        case ret
+        when :invalid_byte_sequence
+          conv.insert_output(conf.primitive_errinfo[3].dump[1..-2])
+          redo
+        when :undefined_conversion
+          c = conv.primitive_errinfo[3].dup.force_encoding(conv.primitive_errinfo[1])
+          conv.insert_output(c.dump[1..-2])
+          redo
+        when :incomplete_input
+          conv.insert_output(conv.primitive_errinfo[3].dump[1..-2])
+        when :finished
+        end
+        break
+      end while nil
+      dst
+    end
+
     def handle_exception(exc)
       if exc.backtrace && exc.backtrace[0] =~ /\/irb(2)?(\/.*|-.*|\.rb)?:/ && exc.class.to_s !~ /^IRB/ &&
          !(SyntaxError === exc) && !(EncodingError === exc)
@@ -587,49 +612,44 @@ module IRB
         irb_bug = false
       end
 
-      if STDOUT.tty?
-        attr = ATTR_TTY
-        print "#{attr[1]}Traceback#{attr[]} (most recent call last):\n"
-      else
-        attr = ATTR_PLAIN
-      end
-      messages = []
-      lasts = []
-      levels = 0
       if exc.backtrace
-        count = 0
-        exc.backtrace.each do |m|
-          m = @context.workspace.filter_backtrace(m) or next unless irb_bug
-          count += 1
-          if attr == ATTR_TTY
-            m = sprintf("%9d: from %s", count, m)
+        order = nil
+        if '2.5.0' == RUBY_VERSION
+          # Exception#full_message doesn't have keyword arguments.
+          message = exc.full_message # the same of (highlight: true, order: bottom)
+          order = :bottom
+        elsif '2.5.1' <= RUBY_VERSION && RUBY_VERSION < '3.0.0'
+          if STDOUT.tty?
+            message = exc.full_message(order: :bottom)
+            order = :bottom
           else
-            m = "\tfrom #{m}"
-          end
-          if messages.size < @context.back_trace_limit
-            messages.push(m)
-          elsif lasts.size < @context.back_trace_limit
-            lasts.push(m).shift
-            levels += 1
+            message = exc.full_message(order: :top)
+            order = :top
           end
+        else # '3.0.0' <= RUBY_VERSION
+          message = exc.full_message(order: :top)
+          order = :top
         end
-      end
-      if attr == ATTR_TTY
-        unless lasts.empty?
-          puts lasts.reverse
-          printf "... %d levels...\n", levels if levels > 0
-        end
-        puts messages.reverse
-      end
-      m = exc.to_s.split(/\n/)
-      print "#{attr[1]}#{exc.class} (#{attr[4]}#{m.shift}#{attr[0, 1]})#{attr[]}\n"
-      puts m.map {|s| "#{attr[1]}#{s}#{attr[]}\n"}
-      if attr == ATTR_PLAIN
-        puts messages
-        unless lasts.empty?
-          puts lasts
-          printf "... %d levels...\n", levels if levels > 0
-        end
+        message = convert_invalid_byte_sequence(message)
+        message = message.gsub(/((?:^\t.+$\n)+)/)  { |m|
+          case order
+          when :top
+            lines = m.split("\n")
+          when :bottom
+            lines = m.split("\n").reverse
+          end
+          unless irb_bug
+            lines = lines.map { |l| @context.workspace.filter_backtrace(l) }.compact
+            if lines.size > @context.back_trace_limit
+              omit = lines.size - @context.back_trace_limit
+              lines = lines[0..(@context.back_trace_limit - 1)]
+              lines << "\t... %d levels..." % omit
+            end
+          end
+          lines = lines.reverse if order == :bottom
+          lines.map{ |l| l + "\n" }.join
+        }
+        puts message
       end
       print "Maybe IRB bug!\n" if irb_bug
     end