summaryrefslogtreecommitdiff
path: root/test/irb/test_ruby_lex.rb
AgeCommit message (Collapse)Author
2024-05-28[Bug #20511] Update reline-0.5.7 (#10848)Hiroshi SHIBATA
* Update reline-0.5.7 * Update irb-1.13.1
2023-10-04[ruby/irb] Clear all context usages in RubyLexStan Lo
(https://github.com/ruby/irb/pull/684) After this change, `RubyLex` will not interact with `Context` directly in any way. This decoupling has a few benefits: - It makes `RubyLex` easier to test as it no longer has a dependency on `Context`. We can see this from the removal of `build_context` from `test_ruby_lex.rb`. - It will make `RubyLex` easier to understand as it will not be affected by state changes in `Context` objects. - It allows `RubyLex` to be used in places where `Context` is not available. https://github.com/ruby/irb/commit/d5b262a076
2023-09-30[ruby/irb] Fix Reline's test failure running with `make test-alltomoya ishida
TESTS='reline irb'` (https://github.com/ruby/irb/pull/722) * Specify TestInputMethod in test to avoid RelineInputMethod to be used * Reset Reline in teardown to avoid test failure of `make test-all TESTS="irb reline"` https://github.com/ruby/irb/commit/5d67967eb1
2023-08-26[ruby/irb] Fix deprecation test when ran multiple timesPeter Zhu
(https://github.com/ruby/irb/pull/695) https://github.com/ruby/irb/commit/ae0e5bb80f
2023-08-24[ruby/irb] Deprecate RubyLex and warn about referencing to itStan Lo
(https://github.com/ruby/irb/pull/692) `RubyLex` has always been a private component of IRB, so we should explicitly discourage usages of it. Also, it should be placed under the `IRB` module like other components. https://github.com/ruby/irb/commit/069b5625f7
2023-08-13[ruby/irb] Move IO configuration to IRB::IrbStan Lo
(https://github.com/ruby/irb/pull/681) It shouldn't be `RubyLex`'s responsibility to handle IO. So this moves the configuration to `IRB::Irb`. https://github.com/ruby/irb/commit/daff750076
2023-08-11[ruby/irb] Remove needless removal of trailing whitespace intomoya ishida
check_code_state (https://github.com/ruby/irb/pull/678) https://github.com/ruby/irb/commit/4a6af7d1ed
2023-08-11[ruby/irb] Move assignment check to RubyLexStan Lo
(https://github.com/ruby/irb/pull/670) Since assignment check relies on tokenization with `Ripper`, it feels like the responsibility of `RubyLex`. `Irb#eval_input` should simply get the result when calling `each_top_level_statement` on `RubyLex`. https://github.com/ruby/irb/commit/89d1adb3fd
2023-07-18[ruby/irb] Reduce boilerplate code in RubyLexTestStan Lo
(https://github.com/ruby/irb/pull/644) * Avoid initialising Row at every test input * Extract common assertion patterns into methods * Remove unnecessary PromptRow strcut and boilerplate code https://github.com/ruby/irb/commit/1ba586c0c6
2023-07-15[ruby/irb] Indent multiline percent literalstomoya ishida
(https://github.com/ruby/irb/pull/643) https://github.com/ruby/irb/commit/18bb4022a9
2023-06-30[ruby/irb] Re-implement prev_spaces feature for pasted codetomoya ishida
(https://github.com/ruby/irb/pull/607) https://github.com/ruby/irb/commit/9d97a192a5
2023-06-28[ruby/irb] Reduce test pend trufflerubytomoya ishida
(https://github.com/ruby/irb/pull/619) https://github.com/ruby/irb/commit/b7b071774f
2023-06-27[ruby/irb] Always add \n at the end of the test input in RubyLextomoya ishida
test (https://github.com/ruby/irb/pull/614) https://github.com/ruby/irb/commit/e68c6128aa
2023-06-25[ruby/irb] Fix process_continue(rename to should_continue?) andtomoya ishida
check_code_block(rename to check_code_syntax) (https://github.com/ruby/irb/pull/611) https://github.com/ruby/irb/commit/b7f4bfaaa4
2023-06-24[ruby/irb] Omit nesting_level, use indent_level to build prompttomoya ishida
string (https://github.com/ruby/irb/pull/610) https://github.com/ruby/irb/commit/f01ff0811b
2023-06-20[ruby/irb] Improve indentation: bugfix, heredoc, embdoc, stringstomoya ishida
(https://github.com/ruby/irb/pull/515) * Implement heredoc embdoc and string indentation with bugfix * Fix test_ruby_lex's indentation value * Add embdoc indent test * Add workaround for lines==[nil] passed to auto_indent when exit IRB with CTRL+d
2023-06-19[ruby/irb] Add missing token that ignored by rippertomoya ishida
(https://github.com/ruby/irb/pull/608) https://github.com/ruby/irb/commit/1cd3b45402
2023-06-15[ruby/irb] Rewrite RubyLex to fix some bugs and make it possible totomoya ishida
add new features easily (https://github.com/ruby/irb/pull/500) * Add nesting level parser for multiple use (indent, prompt, termination check) * Rewrite RubyLex using NestingParser * Add nesting parser tests, fix some existing tests * Add description comment, rename method to NestingParser * Add comments and tweak code to RubyLex * Update NestingParser test * Extract list of ltype tokens to constants
2023-06-13[ruby/irb] Stanardise test class names with `Test` postfix insteadStan Lo
of prefix (https://github.com/ruby/irb/pull/603) https://github.com/ruby/irb/commit/359cb28def
2023-05-21[ruby/irb] Fix typo (https://github.com/ruby/irb/pull/587)ima1zumi
* Fix typo * s/braking/breaking/g Co-authored-by: Stan Lo <stan001212@gmail.com> --------- https://github.com/ruby/irb/commit/5f8e69f5f2 Co-authored-by: Stan Lo <stan001212@gmail.com>
2023-05-20[ruby/irb] Add assertion for dynamic_prompt's assertion executionStan Lo
(https://github.com/ruby/irb/pull/586) Because the assertions for `dynamic_prompt` lives inside a block given to `RubyLex`, they could be skipped unnoticed if the setup is not correct. This commit adds a simple assertion to check if the block was actually executed.
2023-05-20[ruby/irb] Fix dynamic_prompt test not executed, remove unnecessarytomoya ishida
set_input (https://github.com/ruby/irb/pull/585) https://github.com/ruby/irb/commit/b5f3efdcf0
2023-05-19[ruby/irb] Simplify each_top_level_statementtomoya ishida
(https://github.com/ruby/irb/pull/576) * Simplify each_top_level_statement, reduce instance vars * Update lib/irb/ruby-lex.rb Co-authored-by: Stan Lo <stan001212@gmail.com> * Remove unused ltype from TestRubyLex#check_state response * Remove unnecessary const path of TerminateLineInput * Combine duplicated code state check into method --------- https://github.com/ruby/irb/commit/172453cec4 Co-authored-by: Stan Lo <stan001212@gmail.com>
2023-05-18[ruby/irb] Refactor RubyLex's input/io methodsStan Lo
(https://github.com/ruby/irb/pull/583) 1. Make `RubyLex#set_input` simply assign the input block. This matches the behavior of `RubyLex#set_prompt`. 2. Merge `RubyLex#set_input`'s IO configuration logic with `#set_auto_indent` into `#configure_io`.
2023-04-07[ruby/irb] Fix RubyLex's heredoc_with_hembexpr test to avoid rippertomoya ishida
tokenizing issue (https://github.com/ruby/irb/pull/558) https://github.com/ruby/irb/commit/f68e891ed1
2023-04-05[ruby/irb] Drop Ruby 2.6 supportStan Lo
(https://github.com/ruby/irb/pull/555) * Remove all Ruby 2.6 support * Drop Ruby 2.6 specific testing conditions * Only run Ruby 2.7+ on CI * Bump Ruby requirement to 2.7+ https://github.com/ruby/irb/commit/3f714b616c
2023-02-21[ruby/irb] Improve RubyLex's testsStan Lo
(https://github.com/ruby/irb/pull/484) * Improve assert_indenting helper Instead of putting assertions inside the `auto_indent` block, we can just make `auto_indent` return the calculated space count, and use it for assertion outside of the `auto_indent` block call. This simplifies the setup code and makes the intention easier to understand. * Introduce assert_row_indenting helper 1. Helper users shouldn't need to write 2 assertions for the current and the next line's indentation. 2. With this new approach, we can generate clearer error message for both cases: When the current line's space count doesn't match ``` Incorrect spaces calculation for line: ``` > def each_top_level_statement ``` All lines: ``` def each_top_level_statement ``` <0> expected but was <nil> ``` When the next line's space count doesn't match ``` Incorrect spaces calculation for line after the current line: ``` def each_top_level_statement > ``` All lines: ``` def each_top_level_statement ``` <3> expected but was <2> ``` * Replace assert_indenting with assert_row_indenting
2023-01-14[ruby/irb] Store context in RubyLexStan Lo
Some background for this refactor: 1. Through a RubyLex instance's lifetime, the context passed to its methods should be the same. Given that `Context` is only initialised in `Irb#initialize`, this should be true. 2. When `RubyLex` is initialised, the context object should be accessible. This is also true in all 3 of `RubyLex.new`'s invocations. With the above observations, we should be able to store the context in `RubyLex` as an instance variable. And doing so will make `RubyLex`'s instance methods easier to use and maintain. https://github.com/ruby/irb/commit/5c8d3df2df
2023-01-11[ruby/irb] Drop unused arguments in `RubyLex`Stan Lo
(https://github.com/ruby/irb/pull/504) * Simplify `RubyLex#set_prompt` It's optional argument is never used by any caller. * Remove the optional `p` argument from `RubyLex#set_input` The argument is only used in a test case, which can be easily replaced by a block argument.
2023-01-02[ruby/irb] Fix prompt and code mismatchtomoya ishida
(https://github.com/ruby/irb/pull/386) * fix prompt and code mismatch * Add test for prompt and code mismatch bug https://github.com/ruby/irb/commit/a5765d8177 Co-authored-by: Stan Lo <stan001212@gmail.com>
2022-12-27[ruby/irb] Add dynamic prompt test case for quoted heredocStan Lo
(https://github.com/ruby/irb/pull/486) https://github.com/ruby/irb/commit/31cfbeacbb
2022-12-26Sync IRB master: tool/sync_default_gems.rb irbTakashi Kokubun
It looks like tool/sync_default_gems.rb is not capable of cherry-picking commits from ruby/irb. I just executed `tool/sync_default_gems.rb irb` to fix the sync status. I'm not sure if what's the cause. It could be related to some diff that doesn't exist in ruby/ruby, or it might be related to non-linear history due to merge commits. For next time, I'd like to at least exclude the second possibility, so I disabled merge commits in ruby/irb.
2022-11-17Drop Ruby 2.5 supportst0012
Because it has reached EOL for more than 1.5 years and it won't be supported by the next reline version either.
2022-11-04[ruby/irb] Silent the noise created when building Context in testsStan Lo
https://github.com/ruby/irb/commit/27e4274b3c
2022-11-03[ruby/irb] Provide a base test class and let tests restore encodingsStan Lo
conveniently (https://github.com/ruby/irb/pull/429) * Create a base TestIRB::TestCase class * Save/restore encodings for tests that initializes InputMethod classes Because `RelineInputMethod#initializes` calls `set_encoding`, which changes stdio/out/err and Encoding's default encoding values, we need to make sure any test that directly or indirectly (e.g. through Context) initializes `RelineInputMethod` restores encodings. `ReadlineInputMethod` also changes encodings but currently no tests cover it. * Remove unnecessary TestHelper module Since we now have a base TestCase, without_rdoc can just live there. https://github.com/ruby/irb/commit/c2874ec121
2022-11-03[ruby/irb] Require rubygems to run the test alone on ruby/rubyTakashi Kokubun
`Gem` is not undefined on test-all https://github.com/ruby/irb/commit/08ac803d61
2022-11-03[ruby/irb] Fix build_context for ruby/ruby CITakashi Kokubun
Co-Authored-By: Stan Lo <stan.lo@shopify.com> https://github.com/ruby/irb/commit/d1fe234a9a
2022-11-03[ruby/irb] Require the entire irb lib in RubyLex testStan Lo
(https://github.com/ruby/irb/pull/428) RubyLex is not designed to be used alone. It's usually used with an IRB context, which requires workspace. So its tests should have access to those components too. https://github.com/ruby/irb/commit/608f261da4
2022-11-03[ruby/irb] Refactor RubyLex and its testsStan Lo
(https://github.com/ruby/irb/pull/427) * Make sure `RubyLex#set_input`'s context is always present in tests In real-world scenarios, the context should always be non-nil: https://github.com/ruby/irb/blob/master/lib/irb.rb#L489 So we should make sure our test setup reflects that. * Make context a required keyword Since in practice, `set_input`'s context should always be non-nil, its parameters should reflect that. And since `RubyLex#check_state` is only called by `#lex` and `#set_input`, both of which now always require context, we can assume its context should be non-nil too. https://github.com/ruby/irb/commit/1aeeb86203
2022-10-18[ruby/irb] Always use local variables in current context to parse code ↵tomoya ishida
(https://github.com/ruby/irb/pull/397) * Use local_variables for colorize, code_block_open check, nesting_level and assignment_expression check * Check if expression is an assignment BEFORE evaluating it. evaluate might define new localvars and change result of assignment_expression? * Add local_variables dependent code test * pend local variable dependent test on truffleruby code_block_open is not working on truffleruby * Always pass context to RubyLex#lex * Rename local_variable_assign_code generator method name * Add assignment expression truncate test * Add Context#local_variables and make generate_local_variables_assign_code more simple * Update lib/irb/input-method.rb Co-authored-by: Stan Lo <stan001212@gmail.com> * Add a comment why assignment expression check should be done before evaluate https://github.com/ruby/irb/commit/c8b3877281 Co-authored-by: Stan Lo <stan001212@gmail.com> Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
2022-10-18[ruby/irb] Fix code terminated check with heredoc and backtick ↵tomoya ishida
(https://github.com/ruby/irb/pull/390) * Fix backtick method def method call handled as backtick open * Fix handling heredoc in check_string_literal * Sort result of lexer.parse by pos in ruby<2.7. It's not sorted when the given code includes heredoc. * Update lib/irb/ruby-lex.rb Co-authored-by: Stan Lo <stan001212@gmail.com> * Update lib/irb/ruby-lex.rb Co-authored-by: Stan Lo <stan001212@gmail.com> * Add check_string_literal test for heredoc code that does not end with newline https://github.com/ruby/irb/commit/44bc712460 Co-authored-by: Stan Lo <stan001212@gmail.com>
2022-10-01[ruby/irb] Fix ripper_lex_without_warning duplicated heredoc tokentompng
https://github.com/ruby/irb/commit/45b539af39
2021-12-03[ruby/irb] Examine indentation of in keyword when trying to type includeKaíque Kandy Koga
Use in_keyword_case_scope? Return fast https://github.com/ruby/irb/commit/8acc7f8dc7
2021-12-02Compatibility with IRBschneems
Instead of accessing the struct as an array, access it via methods. There are other places inside of this file already using this API (for example https://github.com/ruby/ruby/blob/e0a5c3d2b71dfad038d7562fdd33f02ffd79232d/lib/irb/ruby-lex.rb#L829-L830). This commit moves all struct array-ish calls to use their method calls instead. It is also ~1.23 faster accessing values via a method instead of as an array according to this microbenchmark: ```ruby Elem = Struct.new(:pos, :event, :tok, :state, :message) do def initialize(pos, event, tok, state, message = nil) super(pos, event, tok, State.new(state), message) end # ... def to_a a = super a.pop unless a.empty? a end end class ElemClass attr_accessor :pos, :event, :tok, :state, :message def initialize(pos, event, tok, state, message = nil) @pos = pos @event = event @tok = tok @state = State.new(state) @message = message end def to_a if @message [@pos, @event, @tok, @state, @message] else [@pos, @event, @tok, @state] end end end # stub state class creation for now class State; def initialize(val); end; end ``` ```ruby Benchmark.ips do |x| x.report("struct") { struct[1] } x.report("class ") { from_class.event } x.compare! end; nil ``` ``` Warming up -------------------------------------- struct 1.624M i/100ms class 1.958M i/100ms Calculating ------------------------------------- struct 17.139M (± 2.6%) i/s - 86.077M in 5.025801s class 21.104M (± 3.4%) i/s - 105.709M in 5.015193s Comparison: class : 21103826.3 i/s struct: 17139201.5 i/s - 1.23x (± 0.00) slower ``` Notes: Merged: https://github.com/ruby/ruby/pull/5093
2021-09-22[ruby/irb] Use typed spaces when the line is inside the here documentsKaíque Kandy Koga
Use first method instead of square brackets to support 2.5 and 2.6 versions Use tokens Clear check_newline_depth_difference https://github.com/ruby/irb/commit/6fec2a5d46
2021-09-10[ruby/irb] Support symbol with backtickaycabta
https://github.com/ruby/irb/commit/0aa2425883
2021-08-30[ruby/irb] Use pend instead of skipHiroshi SHIBATA
https://github.com/ruby/irb/commit/f441ce35bf
2021-03-24[ruby/irb] Change ripper_lex_without_warning to a class methodaycabta
https://github.com/ruby/irb/commit/d9f8abc17e
2021-03-05[ruby/irb] Add a test for not continuing when endless range at eolaycabta
https://github.com/ruby/irb/commit/1020ac9c65
2021-01-24[ruby/irb] Indent correctly with method calling with receiveraycabta
https://github.com/ruby/irb/commit/e7c68e74a0