Backport lib/reline, and lib/irb for 3.0.1 2nd (#4157)

* [ruby/irb] Stub a screen size for tests

https://github.com/ruby/irb/commit/6663057083

* [ruby/irb] Support GitHub Actions

https://github.com/ruby/irb/commit/8e9e6c4037

* [ruby/irb] Stub a screen size for test_context

http://ci.rvm.jp/logfiles/brlog.trunk-random1.20210119-074232

https://github.com/ruby/irb/commit/ea87592d4a

* [ruby/irb] Use a real screen size for pp by default

https://github.com/ruby/irb/commit/9b9300dec2

* [ruby/irb] Rescue Errno::EINVAL on IRB pp

http://rubyci.s3.amazonaws.com/solaris11-gcc/ruby-master/log/20210119T070008Z.log.html.gz
is caused by:

/export/home/chkbuild/chkbuild-gcc/tmp/build/20210119T150010Z/ruby/lib/reline/ansi.rb:157:in `winsize': Invalid argument - <STDIN> (Errno::EINVAL)
        from /export/home/chkbuild/chkbuild-gcc/tmp/build/20210119T150010Z/ruby/lib/reline/ansi.rb:157:in `get_screen_size'
        from /export/home/chkbuild/chkbuild-gcc/tmp/build/20210119T150010Z/ruby/lib/reline.rb:168:in `get_screen_size'
        from /export/home/chkbuild/chkbuild-gcc/tmp/build/20210119T150010Z/ruby/lib/forwardable.rb:238:in `get_screen_size'
        from /export/home/chkbuild/chkbuild-gcc/tmp/build/20210119T150010Z/ruby/lib/irb/color_printer.rb:7:in `pp'
        from -e:1:in `<main>'

https://github.com/ruby/irb/commit/1719514598

* [ruby/irb] Split test files for IRB::Color and IRB::ColorPrinter

https://github.com/ruby/irb/commit/d95e8daab3

* [ruby/irb] Undefine unused constants

https://github.com/ruby/irb/commit/eea9c16804

* [ruby/irb] Remove pp-specific stub from TestColor

because it was for TestColorPrinter

https://github.com/ruby/irb/commit/7569206fd4

* [ruby/irb] Delete a doodle-level memo comment...

https://github.com/ruby/irb/commit/fc3e1d9e0c

* [ruby/irb] Indent correctly with keyword "for" and "in"

https://github.com/ruby/irb/commit/47c83ea724

* [ruby/irb] Indent correctly with method calling with receiver

https://github.com/ruby/irb/commit/e7c68e74a0

* [ruby/irb] add `IRB::FileInputMethod.open` to ensure closing associated File

* tweak some methods not to raise exception after `#close`
* use it in `IRB::IrbLoader#{source_file,load_file}

https://github.com/ruby/irb/commit/ec2947acbd

* [ruby/irb] use `RubyLex::TerminateLineInput` appropriately [Bug #17564]

* using the appropriciate exception instead of `break` so that the session
  can be continue after the `irb_source` and `irb_load` commands
* suppress extra new line due to one more `#prompt` call

https://github.com/ruby/irb/commit/bdefaa7cfd

* [ruby/irb] specify the `VERBOSE` to `false` and fix tests to fit

https://github.com/ruby/irb/commit/502c590925

* In test, need to pass a context to IRB::WorkSpace.new explicitly

* Fix absolute path predicate on Windows

A path starts with '/' is not an absolute path on Windows, because
of drive letter or UNC.

* [ruby/irb] follow up the actual line number

https://github.com/ruby/irb/commit/7aed8fe3b1

* [ruby/irb] Add info.rb to gemspec

https://github.com/ruby/irb/commit/adbba19adf

* [ruby/irb] Allow "measure" command to take block

https://github.com/ruby/irb/commit/20f1ca23e9

* [ruby/irb] Enable to reassign a new block with "measure" command

https://github.com/ruby/irb/commit/b444573aa2

* [ruby/reline] Cache pasting state in processing a key

Because it's too slow.

The rendering time in IRB has been reduced as follows:

  start = Time.now

  def each_top_level_statement
    initialize_input
    catch(:TERM_INPUT) do
      loop do
        begin
          prompt
          unless l = lex
            throw :TERM_INPUT if @line == ''
          else
            @line_no += l.count("\n")
            next if l == "\n"
            @line.concat l
            if @code_block_open or @ltype or @continue or @indent > 0
              next
            end
          end
          if @line != "\n"
            @line.force_encoding(@io.encoding)
            yield @line, @exp_line_no
          end
          break if @io.eof?
          @line = ''
          @exp_line_no = @line_no

          @indent = 0
        rescue TerminateLineInput
          initialize_input
          prompt
        end
      end
    end
  end

  puts "Duration: #{Time.now - start} seconds"

0.22sec -> 0.14sec

https://github.com/ruby/reline/commit/b8b3dd52c0

* [ruby/reline] Initialize uninitialized variables in tests

https://github.com/ruby/reline/commit/25af4bb64b

* [ruby/reline] Remove an unused variable

https://github.com/ruby/reline/commit/123ea51166

* [ruby/reline] Scroll down when ^C is pressed

https://github.com/ruby/reline/commit/6877a7e3f5

* [ruby/reline] Show all lines higher than the screen when finished

On Unix-like OSes, logs prior to the screen are not editable. When the code
is higher than the screen, the code is only shown on the screen until input
is finished, but when it is finished, all lines are outputted.

https://github.com/ruby/reline/commit/8cd9132a39

* [ruby/reline] Handle past logs correctly when the code is higher than the screen

https://github.com/ruby/reline/commit/f197139b4a

* [ruby/reline] Update cursor info by inserting newline even if not in pasting

https://github.com/ruby/reline/commit/92d314f514

* [ruby/reline] Move cursor just after the last line when finished

https://github.com/ruby/reline/commit/ba06e4c480

* [ruby/reline] The vi_histedit supports multiline

This closes ruby/reline#253.

https://github.com/ruby/reline/commit/f131f86d71

* [ruby/reline] Autowrap correctly when inserting chars in the middle of a line

https://github.com/ruby/reline/commit/ebaf37255f

* [ruby/reline] Terminate correctly in the middle of lines higher than the screen

https://github.com/ruby/reline/commit/e1d9240ada

* [ruby/irb] Version 1.3.3

https://github.com/ruby/irb/commit/4c87035b7c

* [ruby/reline] Version 0.2.3

https://github.com/ruby/reline/commit/b26c7d60c8

Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
Co-authored-by: Nobuhiro IMAI <nov@yo.rim.or.jp>
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
Co-authored-by: ima1zumi <mariimaizumi5@gmail.com>

9 files changed, 162 insertions, 42 deletions

diff --git a/lib/irb/cmd/measure.rb b/lib/irb/cmd/measure.rb
index 5e0bef62af..58eaec2ded 100644
--- a/lib/irb/cmd/measure.rb
+++ b/lib/irb/cmd/measure.rb
@@ -8,7 +8,7 @@ module IRB
         super(*args)
       end
 
-      def execute(type = nil, arg = nil)
+      def execute(type = nil, arg = nil, &block)
         case type
         when :off
           IRB.conf[:MEASURE] = nil
@@ -22,9 +22,15 @@ module IRB
           added = IRB.set_measure_callback(type, arg)
           puts "#{added[0]} is added." if added
         else
-          IRB.conf[:MEASURE] = true
-          added = IRB.set_measure_callback(type, arg)
-          puts "#{added[0]} is added." if added
+          if block_given?
+            IRB.conf[:MEASURE] = true
+            added = IRB.set_measure_callback(&block)
+            puts "#{added[0]} is added." if added
+          else
+            IRB.conf[:MEASURE] = true
+            added = IRB.set_measure_callback(type, arg)
+            puts "#{added[0]} is added." if added
+          end
         end
         nil
       end
diff --git a/lib/irb/cmd/nop.rb b/lib/irb/cmd/nop.rb
index 9cf4337c28..fa3c011b5f 100644
--- a/lib/irb/cmd/nop.rb
+++ b/lib/irb/cmd/nop.rb
@@ -15,9 +15,9 @@ module IRB
     class Nop
 
 
-      def self.execute(conf, *opts)
+      def self.execute(conf, *opts, &block)
         command = new(conf)
-        command.execute(*opts)
+        command.execute(*opts, &block)
       end
 
       def initialize(conf)
diff --git a/lib/irb/color_printer.rb b/lib/irb/color_printer.rb
index 73a150f881..92afea51cd 100644
--- a/lib/irb/color_printer.rb
+++ b/lib/irb/color_printer.rb
@@ -4,11 +4,21 @@ require 'irb/color'
 
 module IRB
   class ColorPrinter < ::PP
-    def self.pp(obj, out = $>, width = 79)
-      q = ColorPrinter.new(out, width)
-      q.guard_inspect_key {q.pp obj}
-      q.flush
-      out << "\n"
+    class << self
+      def pp(obj, out = $>, width = screen_width)
+        q = ColorPrinter.new(out, width)
+        q.guard_inspect_key {q.pp obj}
+        q.flush
+        out << "\n"
+      end
+
+      private
+
+      def screen_width
+        Reline.get_screen_size.last
+      rescue Errno::EINVAL # in `winsize': Invalid argument - <STDIN>
+        79
+      end
     end
 
     def text(str, width = nil)
diff --git a/lib/irb/ext/loader.rb b/lib/irb/ext/loader.rb
index 1b683d88e5..90dcd70bd0 100644
--- a/lib/irb/ext/loader.rb
+++ b/lib/irb/ext/loader.rb
@@ -31,8 +31,31 @@ module IRB # :nodoc:
       load_file(path, priv)
     end
 
+    if File.respond_to?(:absolute_path?)
+      def absolute_path?(path)
+        File.absolute_path?(path)
+      end
+    else
+      separator =
+        if File::ALT_SEPARATOR
+          File::SEPARATOR
+        else
+          "[#{Regexp.quote(File::SEPARATOR + File::ALT_SEPARATOR)}]"
+        end
+      ABSOLUTE_PATH_PATTERN = # :nodoc:
+        case Dir.pwd
+        when /\A\w:/, /\A#{separator}{2}/
+          /\A(?:\w:|#{separator})#{separator}/
+        else
+          /\A#{separator}/
+        end
+      def absolute_path?(path)
+        ABSOLUTE_PATH_PATTERN =~ path
+      end
+    end
+
     def search_file_from_ruby_path(fn) # :nodoc:
-      if /^#{Regexp.quote(File::Separator)}/ =~ fn
+      if absolute_path?(fn)
         return fn if File.exist?(fn)
         return nil
       end
@@ -50,16 +73,18 @@ module IRB # :nodoc:
     # See Irb#suspend_input_method for more information.
     def source_file(path)
       irb.suspend_name(path, File.basename(path)) do
-        irb.suspend_input_method(FileInputMethod.new(path)) do
-          |back_io|
-          irb.signal_status(:IN_LOAD) do
-            if back_io.kind_of?(FileInputMethod)
-              irb.eval_input
-            else
-              begin
+        FileInputMethod.open(path) do |io|
+          irb.suspend_input_method(io) do
+            |back_io|
+            irb.signal_status(:IN_LOAD) do
+              if back_io.kind_of?(FileInputMethod)
                 irb.eval_input
-              rescue LoadAbort
-                print "load abort!!\n"
+              else
+                begin
+                  irb.eval_input
+                rescue LoadAbort
+                  print "load abort!!\n"
+                end
               end
             end
           end
@@ -79,16 +104,18 @@ module IRB # :nodoc:
           ws = WorkSpace.new
         end
         irb.suspend_workspace(ws) do
-          irb.suspend_input_method(FileInputMethod.new(path)) do
-            |back_io|
-            irb.signal_status(:IN_LOAD) do
-              if back_io.kind_of?(FileInputMethod)
-                irb.eval_input
-              else
-                begin
+          FileInputMethod.open(path) do |io|
+            irb.suspend_input_method(io) do
+              |back_io|
+              irb.signal_status(:IN_LOAD) do
+                if back_io.kind_of?(FileInputMethod)
                   irb.eval_input
-                rescue LoadAbort
-                  print "load abort!!\n"
+                else
+                  begin
+                    irb.eval_input
+                  rescue LoadAbort
+                    print "load abort!!\n"
+                  end
                 end
               end
             end
diff --git a/lib/irb/init.rb b/lib/irb/init.rb
index 8428a4278f..78ef2fa3c1 100644
--- a/lib/irb/init.rb
+++ b/lib/irb/init.rb
@@ -146,7 +146,7 @@ module IRB # :nodoc:
     @CONF[:AT_EXIT] = []
   end
 
-  def IRB.set_measure_callback(type = nil, arg = nil)
+  def IRB.set_measure_callback(type = nil, arg = nil, &block)
     added = nil
     if type
       type_sym = type.upcase.to_sym
@@ -155,6 +155,16 @@ module IRB # :nodoc:
       end
     elsif IRB.conf[:MEASURE_PROC][:CUSTOM]
       added = [:CUSTOM, IRB.conf[:MEASURE_PROC][:CUSTOM], arg]
+    elsif block_given?
+      added = [:BLOCK, block, arg]
+      found = IRB.conf[:MEASURE_CALLBACKS].find{ |m| m[0] == added[0] && m[2] == added[2] }
+      if found
+        found[1] = block
+        return added
+      else
+        IRB.conf[:MEASURE_CALLBACKS] << added
+        return added
+      end
     else
       added = [:TIME, IRB.conf[:MEASURE_PROC][:TIME], arg]
     end
diff --git a/lib/irb/input-method.rb b/lib/irb/input-method.rb
index 61540a106f..e223672985 100644
--- a/lib/irb/input-method.rb
+++ b/lib/irb/input-method.rb
@@ -124,10 +124,22 @@ module IRB
 
   # Use a File for IO with irb, see InputMethod
   class FileInputMethod < InputMethod
+    class << self
+      def open(file, &block)
+        begin
+          io = new(file)
+          block.call(io)
+        ensure
+          io&.close
+        end
+      end
+    end
+
     # Creates a new input method object
     def initialize(file)
       super
       @io = IRB::MagicFile.open(file)
+      @external_encoding = @io.external_encoding
     end
     # The file name of this input method, usually given during initialization.
     attr_reader :file_name
@@ -137,7 +149,7 @@ module IRB
     #
     # See IO#eof? for more information.
     def eof?
-      @io.eof?
+      @io.closed? || @io.eof?
     end
 
     # Reads the next line from this input method.
@@ -150,13 +162,17 @@ module IRB
 
     # The external encoding for standard input.
     def encoding
-      @io.external_encoding
+      @external_encoding
     end
 
     # For debug message
     def inspect
       'FileInputMethod'
     end
+
+    def close
+      @io.close
+    end
   end
 
   begin
diff --git a/lib/irb/irb.gemspec b/lib/irb/irb.gemspec
index 9d889dfbf6..9842b4bce1 100644
--- a/lib/irb/irb.gemspec
+++ b/lib/irb/irb.gemspec
@@ -32,6 +32,7 @@ Gem::Specification.new do |spec|
     "lib/irb/cmd/chws.rb",
     "lib/irb/cmd/fork.rb",
     "lib/irb/cmd/help.rb",
+    "lib/irb/cmd/info.rb",
     "lib/irb/cmd/load.rb",
     "lib/irb/cmd/measure.rb",
     "lib/irb/cmd/nop.rb",
diff --git a/lib/irb/ruby-lex.rb b/lib/irb/ruby-lex.rb
index 35af148d02..ce94797dad 100644
--- a/lib/irb/ruby-lex.rb
+++ b/lib/irb/ruby-lex.rb
@@ -223,7 +223,10 @@ class RubyLex
             throw :TERM_INPUT if @line == ''
           else
             @line_no += l.count("\n")
-            next if l == "\n"
+            if l == "\n"
+              @exp_line_no += 1
+              next
+            end
             @line.concat l
             if @code_block_open or @ltype or @continue or @indent > 0
               next
@@ -233,7 +236,7 @@ class RubyLex
             @line.force_encoding(@io.encoding)
             yield @line, @exp_line_no
           end
-          break if @io.eof?
+          raise TerminateLineInput if @io.eof?
           @line = ''
           @exp_line_no = @line_no
 
@@ -424,14 +427,30 @@ class RubyLex
     indent
   end
 
+  def is_method_calling?(tokens, index)
+    tk = tokens[index]
+    if tk[3].anybits?(Ripper::EXPR_CMDARG) and tk[1] == :on_ident
+      # The target method call to pass the block with "do".
+      return true
+    elsif tk[3].anybits?(Ripper::EXPR_ARG) and tk[1] == :on_ident
+      non_sp_index = tokens[0..(index - 1)].rindex{ |t| t[1] != :on_sp }
+      if non_sp_index
+        prev_tk = tokens[non_sp_index]
+        if prev_tk[3].anybits?(Ripper::EXPR_DOT) and prev_tk[1] == :on_period
+          # The target method call with receiver to pass the block with "do".
+          return true
+        end
+      end
+    end
+    false
+  end
+
   def take_corresponding_syntax_to_kw_do(tokens, index)
     syntax_of_do = nil
     # Finding a syntax correnponding to "do".
     index.downto(0) do |i|
       tk = tokens[i]
       # In "continue", the token isn't the corresponding syntax to "do".
-      #is_continue = process_continue(@tokens[0..(i - 1)])
-      # continue ではなく、直前に (:on_ignored_nl|:on_nl|:on_comment):on_sp* みたいなのがあるかどうかを調べる
       non_sp_index = tokens[0..(i - 1)].rindex{ |t| t[1] != :on_sp }
       first_in_fomula = false
       if non_sp_index.nil?
@@ -439,8 +458,7 @@ class RubyLex
       elsif [:on_ignored_nl, :on_nl, :on_comment].include?(tokens[non_sp_index][1])
         first_in_fomula = true
       end
-      if tk[3].anybits?(Ripper::EXPR_CMDARG) and tk[1] == :on_ident
-        # The target method call to pass the block with "do".
+      if is_method_calling?(tokens, i)
         syntax_of_do = :method_calling
         break if first_in_fomula
       elsif tk[1] == :on_kw && %w{while until for}.include?(tk[2])
@@ -458,6 +476,34 @@ class RubyLex
     syntax_of_do
   end
 
+  def is_the_in_correspond_to_a_for(tokens, index)
+    syntax_of_in = nil
+    # Finding a syntax correnponding to "do".
+    index.downto(0) do |i|
+      tk = tokens[i]
+      # In "continue", the token isn't the corresponding syntax to "do".
+      non_sp_index = tokens[0..(i - 1)].rindex{ |t| t[1] != :on_sp }
+      first_in_fomula = false
+      if non_sp_index.nil?
+        first_in_fomula = true
+      elsif [:on_ignored_nl, :on_nl, :on_comment].include?(tokens[non_sp_index][1])
+        first_in_fomula = true
+      end
+      if tk[1] == :on_kw && tk[2] == 'for'
+        # A loop syntax in front of "do" found.
+        #
+        #   while cond do # also "until" or "for"
+        #   end
+        #
+        # This "do" doesn't increment indent because the loop syntax already
+        # incremented.
+        syntax_of_in = :for
+      end
+      break if first_in_fomula
+    end
+    syntax_of_in
+  end
+
   def check_newline_depth_difference
     depth_difference = 0
     open_brace_on_line = 0
@@ -513,8 +559,12 @@ class RubyLex
           unless t[3].allbits?(Ripper::EXPR_LABEL)
             depth_difference += 1
           end
-        when 'else', 'elsif', 'ensure', 'when', 'in'
+        when 'else', 'elsif', 'ensure', 'when'
           depth_difference += 1
+        when 'in'
+          unless is_the_in_correspond_to_a_for(@tokens, index)
+            depth_difference += 1
+          end
         when 'end'
           depth_difference -= 1
         end
diff --git a/lib/irb/version.rb b/lib/irb/version.rb
index 51b55766a5..a715293b34 100644
--- a/lib/irb/version.rb
+++ b/lib/irb/version.rb
@@ -11,7 +11,7 @@
 #
 
 module IRB # :nodoc:
-  VERSION = "1.3.2"
+  VERSION = "1.3.3"
   @RELEASE_VERSION = VERSION
-  @LAST_UPDATE_DATE = "2021-01-18"
+  @LAST_UPDATE_DATE = "2021-02-07"
 end