Backport lib/reline, and lib/irb for 3.0.1 2nd (#4157)

* [ruby/irb] Stub a screen size for tests

https://github.com/ruby/irb/commit/6663057083

* [ruby/irb] Support GitHub Actions

https://github.com/ruby/irb/commit/8e9e6c4037

* [ruby/irb] Stub a screen size for test_context

http://ci.rvm.jp/logfiles/brlog.trunk-random1.20210119-074232

https://github.com/ruby/irb/commit/ea87592d4a

* [ruby/irb] Use a real screen size for pp by default

https://github.com/ruby/irb/commit/9b9300dec2

* [ruby/irb] Rescue Errno::EINVAL on IRB pp

http://rubyci.s3.amazonaws.com/solaris11-gcc/ruby-master/log/20210119T070008Z.log.html.gz
is caused by:

/export/home/chkbuild/chkbuild-gcc/tmp/build/20210119T150010Z/ruby/lib/reline/ansi.rb:157:in `winsize': Invalid argument - <STDIN> (Errno::EINVAL)
        from /export/home/chkbuild/chkbuild-gcc/tmp/build/20210119T150010Z/ruby/lib/reline/ansi.rb:157:in `get_screen_size'
        from /export/home/chkbuild/chkbuild-gcc/tmp/build/20210119T150010Z/ruby/lib/reline.rb:168:in `get_screen_size'
        from /export/home/chkbuild/chkbuild-gcc/tmp/build/20210119T150010Z/ruby/lib/forwardable.rb:238:in `get_screen_size'
        from /export/home/chkbuild/chkbuild-gcc/tmp/build/20210119T150010Z/ruby/lib/irb/color_printer.rb:7:in `pp'
        from -e:1:in `<main>'

https://github.com/ruby/irb/commit/1719514598

* [ruby/irb] Split test files for IRB::Color and IRB::ColorPrinter

https://github.com/ruby/irb/commit/d95e8daab3

* [ruby/irb] Undefine unused constants

https://github.com/ruby/irb/commit/eea9c16804

* [ruby/irb] Remove pp-specific stub from TestColor

because it was for TestColorPrinter

https://github.com/ruby/irb/commit/7569206fd4

* [ruby/irb] Delete a doodle-level memo comment...

https://github.com/ruby/irb/commit/fc3e1d9e0c

* [ruby/irb] Indent correctly with keyword "for" and "in"

https://github.com/ruby/irb/commit/47c83ea724

* [ruby/irb] Indent correctly with method calling with receiver

https://github.com/ruby/irb/commit/e7c68e74a0

* [ruby/irb] add `IRB::FileInputMethod.open` to ensure closing associated File

* tweak some methods not to raise exception after `#close`
* use it in `IRB::IrbLoader#{source_file,load_file}

https://github.com/ruby/irb/commit/ec2947acbd

* [ruby/irb] use `RubyLex::TerminateLineInput` appropriately [Bug #17564]

* using the appropriciate exception instead of `break` so that the session
  can be continue after the `irb_source` and `irb_load` commands
* suppress extra new line due to one more `#prompt` call

https://github.com/ruby/irb/commit/bdefaa7cfd

* [ruby/irb] specify the `VERBOSE` to `false` and fix tests to fit

https://github.com/ruby/irb/commit/502c590925

* In test, need to pass a context to IRB::WorkSpace.new explicitly

* Fix absolute path predicate on Windows

A path starts with '/' is not an absolute path on Windows, because
of drive letter or UNC.

* [ruby/irb] follow up the actual line number

https://github.com/ruby/irb/commit/7aed8fe3b1

* [ruby/irb] Add info.rb to gemspec

https://github.com/ruby/irb/commit/adbba19adf

* [ruby/irb] Allow "measure" command to take block

https://github.com/ruby/irb/commit/20f1ca23e9

* [ruby/irb] Enable to reassign a new block with "measure" command

https://github.com/ruby/irb/commit/b444573aa2

* [ruby/reline] Cache pasting state in processing a key

Because it's too slow.

The rendering time in IRB has been reduced as follows:

  start = Time.now

  def each_top_level_statement
    initialize_input
    catch(:TERM_INPUT) do
      loop do
        begin
          prompt
          unless l = lex
            throw :TERM_INPUT if @line == ''
          else
            @line_no += l.count("\n")
            next if l == "\n"
            @line.concat l
            if @code_block_open or @ltype or @continue or @indent > 0
              next
            end
          end
          if @line != "\n"
            @line.force_encoding(@io.encoding)
            yield @line, @exp_line_no
          end
          break if @io.eof?
          @line = ''
          @exp_line_no = @line_no

          @indent = 0
        rescue TerminateLineInput
          initialize_input
          prompt
        end
      end
    end
  end

  puts "Duration: #{Time.now - start} seconds"

0.22sec -> 0.14sec

https://github.com/ruby/reline/commit/b8b3dd52c0

* [ruby/reline] Initialize uninitialized variables in tests

https://github.com/ruby/reline/commit/25af4bb64b

* [ruby/reline] Remove an unused variable

https://github.com/ruby/reline/commit/123ea51166

* [ruby/reline] Scroll down when ^C is pressed

https://github.com/ruby/reline/commit/6877a7e3f5

* [ruby/reline] Show all lines higher than the screen when finished

On Unix-like OSes, logs prior to the screen are not editable. When the code
is higher than the screen, the code is only shown on the screen until input
is finished, but when it is finished, all lines are outputted.

https://github.com/ruby/reline/commit/8cd9132a39

* [ruby/reline] Handle past logs correctly when the code is higher than the screen

https://github.com/ruby/reline/commit/f197139b4a

* [ruby/reline] Update cursor info by inserting newline even if not in pasting

https://github.com/ruby/reline/commit/92d314f514

* [ruby/reline] Move cursor just after the last line when finished

https://github.com/ruby/reline/commit/ba06e4c480

* [ruby/reline] The vi_histedit supports multiline

This closes ruby/reline#253.

https://github.com/ruby/reline/commit/f131f86d71

* [ruby/reline] Autowrap correctly when inserting chars in the middle of a line

https://github.com/ruby/reline/commit/ebaf37255f

* [ruby/reline] Terminate correctly in the middle of lines higher than the screen

https://github.com/ruby/reline/commit/e1d9240ada

* [ruby/irb] Version 1.3.3

https://github.com/ruby/irb/commit/4c87035b7c

* [ruby/reline] Version 0.2.3

https://github.com/ruby/reline/commit/b26c7d60c8

Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
Co-authored-by: Nobuhiro IMAI <nov@yo.rim.or.jp>
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
Co-authored-by: ima1zumi <mariimaizumi5@gmail.com>

2 files changed, 84 insertions, 20 deletions

diff --git a/lib/reline/line_editor.rb b/lib/reline/line_editor.rb
index 92ea42fffb..557b5aa737 100644
--- a/lib/reline/line_editor.rb
+++ b/lib/reline/line_editor.rb
@@ -58,13 +58,17 @@ class Reline::LineEditor
     reset_variables(encoding: encoding)
   end
 
+  def set_pasting_state(in_pasting)
+    @in_pasting = in_pasting
+  end
+
   def simplified_rendering?
     if finished?
       false
     elsif @just_cursor_moving and not @rerender_all
       true
     else
-      not @rerender_all and not finished? and Reline::IOGate.in_pasting?
+      not @rerender_all and not finished? and @in_pasting
     end
   end
 
@@ -146,6 +150,13 @@ class Reline::LineEditor
     @screen_height = @screen_size.first
     reset_variables(prompt, encoding: encoding)
     @old_trap = Signal.trap('SIGINT') {
+      if @scroll_partial_screen
+        move_cursor_down(@screen_height - (@line_index - @scroll_partial_screen) - 1)
+      else
+        move_cursor_down(@highest_in_all - @line_index - 1)
+      end
+      Reline::IOGate.move_cursor_column(0)
+      scroll_down(1)
       @old_trap.call if @old_trap.respond_to?(:call) # can also be string, ex: "DEFAULT"
       raise Interrupt
     }
@@ -227,6 +238,8 @@ class Reline::LineEditor
     @scroll_partial_screen = nil
     @prev_mode_string = nil
     @drop_terminate_spaces = false
+    @in_pasting = false
+    @auto_indent_proc = nil
     reset_line
   end
 
@@ -375,11 +388,29 @@ class Reline::LineEditor
       @cleared = false
       return
     end
+    if @is_multiline and finished? and @scroll_partial_screen
+      # Re-output all code higher than the screen when finished.
+      Reline::IOGate.move_cursor_up(@first_line_started_from + @started_from - @scroll_partial_screen)
+      Reline::IOGate.move_cursor_column(0)
+      @scroll_partial_screen = nil
+      prompt, prompt_width, prompt_list = check_multiline_prompt(whole_lines, prompt)
+      if @previous_line_index
+        new_lines = whole_lines(index: @previous_line_index, line: @line)
+      else
+        new_lines = whole_lines
+      end
+      modify_lines(new_lines).each_with_index do |line, index|
+        @output.write "#{prompt_list ? prompt_list[index] : prompt}#{line}\n"
+        Reline::IOGate.erase_after_cursor
+      end
+      @output.flush
+      return
+    end
     new_highest_in_this = calculate_height_by_width(prompt_width + calculate_width(@line.nil? ? '' : @line))
     # FIXME: end of logical line sometimes breaks
     rendered = false
     if @add_newline_to_end_of_buffer
-      rerender_added_newline
+      rerender_added_newline(prompt, prompt_width)
       @add_newline_to_end_of_buffer = false
     else
       if @just_cursor_moving and not @rerender_all
@@ -397,20 +428,32 @@ class Reline::LineEditor
       else
       end
     end
-    line = modify_lines(whole_lines)[@line_index]
     if @is_multiline
-      prompt, prompt_width, prompt_list = check_multiline_prompt(whole_lines, prompt)
       if finished?
         # Always rerender on finish because output_modifier_proc may return a different output.
+        if @previous_line_index
+          new_lines = whole_lines(index: @previous_line_index, line: @line)
+        else
+          new_lines = whole_lines
+        end
+        line = modify_lines(new_lines)[@line_index]
+        prompt, prompt_width, prompt_list = check_multiline_prompt(new_lines, prompt)
         render_partial(prompt, prompt_width, line, @first_line_started_from)
+        move_cursor_down(@highest_in_all - (@first_line_started_from + @highest_in_this - 1) - 1)
         scroll_down(1)
         Reline::IOGate.move_cursor_column(0)
         Reline::IOGate.erase_after_cursor
       elsif not rendered
-        render_partial(prompt, prompt_width, line, @first_line_started_from)
+        unless @in_pasting
+          line = modify_lines(whole_lines)[@line_index]
+          prompt, prompt_width, prompt_list = check_multiline_prompt(whole_lines, prompt)
+          render_partial(prompt, prompt_width, line, @first_line_started_from)
+        end
       end
       @buffer_of_lines[@line_index] = @line
+      @rest_height = 0 if @scroll_partial_screen
     else
+      line = modify_lines(whole_lines)[@line_index]
       render_partial(prompt, prompt_width, line, 0)
       if finished?
         scroll_down(1)
@@ -453,13 +496,13 @@ class Reline::LineEditor
     end
   end
 
-  private def rerender_added_newline
+  private def rerender_added_newline(prompt, prompt_width)
     scroll_down(1)
-    new_lines = whole_lines(index: @previous_line_index, line: @line)
-    prompt, prompt_width, = check_multiline_prompt(new_lines, prompt)
     @buffer_of_lines[@previous_line_index] = @line
     @line = @buffer_of_lines[@line_index]
-    render_partial(prompt, prompt_width, @line, @first_line_started_from + @started_from + 1, with_control: false)
+    unless @in_pasting
+      render_partial(prompt, prompt_width, @line, @first_line_started_from + @started_from + 1, with_control: false)
+    end
     @cursor = @cursor_max = calculate_width(@line)
     @byte_pointer = @line.bytesize
     @highest_in_all += @highest_in_this
@@ -568,7 +611,13 @@ class Reline::LineEditor
       new_first_line_started_from = calculate_height_by_lines(new_buffer[0..(@line_index - 1)], prompt_list || prompt)
     end
     new_started_from = calculate_height_by_width(prompt_width + @cursor) - 1
-    if back > old_highest_in_all
+    calculate_scroll_partial_screen(back, new_first_line_started_from + new_started_from)
+    if @scroll_partial_screen
+      move_cursor_up(@first_line_started_from + @started_from)
+      scroll_down(@screen_height - 1)
+      move_cursor_up(@screen_height)
+      Reline::IOGate.move_cursor_column(0)
+    elsif back > old_highest_in_all
       scroll_down(back - 1)
       move_cursor_up(back - 1)
     elsif back < old_highest_in_all
@@ -580,7 +629,6 @@ class Reline::LineEditor
       end
       move_cursor_up(old_highest_in_all - 1)
     end
-    calculate_scroll_partial_screen(back, new_first_line_started_from + new_started_from)
     render_whole_lines(new_buffer, prompt_list || prompt, prompt_width)
     if @prompt_proc
       prompt = prompt_list[@line_index]
@@ -666,8 +714,8 @@ class Reline::LineEditor
         @highest_in_this = height
       end
       move_cursor_up(@started_from)
-      cursor_up_from_last_line = height - 1 - @started_from
       @started_from = calculate_height_by_width(prompt_width + @cursor) - 1
+      cursor_up_from_last_line = height - 1 - @started_from
     end
     if Reline::Unicode::CSI_REGEXP.match?(prompt + line_to_render)
       @output.write "\e[0m" # clear character decorations
@@ -1082,7 +1130,7 @@ class Reline::LineEditor
     unless completion_occurs
       @completion_state = CompletionState::NORMAL
     end
-    if not Reline::IOGate.in_pasting? and @just_cursor_moving.nil?
+    if not @in_pasting and @just_cursor_moving.nil?
       if @previous_line_index and @buffer_of_lines[@previous_line_index] == @line
         @just_cursor_moving = true
       elsif @previous_line_index.nil? and @buffer_of_lines[@line_index] == @line and old_line == @line
@@ -1286,7 +1334,11 @@ class Reline::LineEditor
     if @buffer_of_lines.size == 1 and @line.nil?
       nil
     else
-      whole_lines.join("\n")
+      if @previous_line_index
+        whole_lines(index: @previous_line_index, line: @line).join("\n")
+      else
+        whole_lines.join("\n")
+      end
     end
   end
 
@@ -1332,14 +1384,14 @@ class Reline::LineEditor
       cursor_line = @line.byteslice(0, @byte_pointer)
       insert_new_line(cursor_line, next_line)
       @cursor = 0
-      @check_new_auto_indent = true unless Reline::IOGate.in_pasting?
+      @check_new_auto_indent = true unless @in_pasting
     end
   end
 
   private def ed_unassigned(key) end # do nothing
 
   private def process_insert(force: false)
-    return if @continuous_insertion_buffer.empty? or (Reline::IOGate.in_pasting? and not force)
+    return if @continuous_insertion_buffer.empty? or (@in_pasting and not force)
     width = Reline::Unicode.calculate_width(@continuous_insertion_buffer)
     bytesize = @continuous_insertion_buffer.bytesize
     if @cursor == @cursor_max
@@ -1374,7 +1426,7 @@ class Reline::LineEditor
       str = key.chr
       bytesize = 1
     end
-    if Reline::IOGate.in_pasting?
+    if @in_pasting
       @continuous_insertion_buffer << str
       return
     elsif not @continuous_insertion_buffer.empty?
@@ -2424,11 +2476,23 @@ class Reline::LineEditor
 
   private def vi_histedit(key)
     path = Tempfile.open { |fp|
-      fp.write @line
+      if @is_multiline
+        fp.write whole_lines.join("\n")
+      else
+        fp.write @line
+      end
       fp.path
     }
     system("#{ENV['EDITOR']} #{path}")
-    @line = File.read(path)
+    if @is_multiline
+      @buffer_of_lines = File.read(path).split("\n")
+      @buffer_of_lines = [String.new(encoding: @encoding)] if @buffer_of_lines.empty?
+      @line_index = 0
+      @line = @buffer_of_lines[@line_index]
+      @rerender_all = true
+    else
+      @line = File.read(path)
+    end
     finish
   end
 
diff --git a/lib/reline/version.rb b/lib/reline/version.rb
index 9241aef5a9..5b20f6f3e7 100644
--- a/lib/reline/version.rb
+++ b/lib/reline/version.rb
@@ -1,3 +1,3 @@
 module Reline
-  VERSION = '0.2.2'
+  VERSION = '0.2.3'
 end