diff options
Diffstat (limited to 'doc')
141 files changed, 7454 insertions, 3818 deletions
diff --git a/doc/.document b/doc/.document index 6e6caa8333..3a95c9617b 100644 --- a/doc/.document +++ b/doc/.document @@ -1,12 +1,13 @@ -*.md -*.rb +[^_]*.md +[^_]*.rb [^_]*.rdoc contributing +distribution NEWS syntax optparse date rdoc -regexp -yjit -ruby +jit +security +language diff --git a/doc/NEWS/NEWS-3.0.0.md b/doc/NEWS/NEWS-3.0.0.md index 004fa4bf67..9fbaf504b4 100644 --- a/doc/NEWS/NEWS-3.0.0.md +++ b/doc/NEWS/NEWS-3.0.0.md @@ -367,11 +367,11 @@ Outstanding ones only. * Fiber.blocking? tells whether the current execution context is blocking. [[Feature #16786]] +* Thread + * Thread#join invokes the scheduler hooks `block`/`unblock` in a non-blocking execution context. [[Feature #16786]] -* Thread - * Thread.ignore_deadlock accessor has been added for disabling the default deadlock detection, allowing the use of signal handlers to break deadlock. [[Bug #13768]] diff --git a/doc/NEWS/NEWS-4.0.0.md b/doc/NEWS/NEWS-4.0.0.md new file mode 100644 index 0000000000..5d932fbf5d --- /dev/null +++ b/doc/NEWS/NEWS-4.0.0.md @@ -0,0 +1,802 @@ +# NEWS for Ruby 4.0.0 + +This document is a list of user-visible feature changes +since the **3.4.0** release, except for bug fixes. + +Note that each entry is kept to a minimum, see links for details. + +## Language changes + +* `*nil` no longer calls `nil.to_a`, similar to how `**nil` does + not call `nil.to_hash`. [[Feature #21047]] + +* Logical binary operators (`||`, `&&`, `and` and `or`) at the + beginning of a line continue the previous line, like fluent dot. + The following code examples are equal: + + ```ruby + if condition1 + && condition2 + ... + end + ``` + + Previously: + + ```ruby + if condition1 && condition2 + ... + end + ``` + + ```ruby + if condition1 && + condition2 + ... + end + ``` + + [[Feature #20925]] + +## Core classes updates + +Note: We're only listing outstanding class updates. + +* Array + + * `Array#rfind` has been added as a more efficient alternative to `array.reverse_each.find` [[Feature #21678]] + * `Array#find` has been added as a more efficient override of `Enumerable#find` [[Feature #21678]] +* Binding + + * `Binding#local_variables` does no longer include numbered parameters. + Also, `Binding#local_variable_get`, `Binding#local_variable_set`, and + `Binding#local_variable_defined?` reject to handle numbered parameters. + [[Bug #21049]] + + * `Binding#implicit_parameters`, `Binding#implicit_parameter_get`, and + `Binding#implicit_parameter_defined?` have been added to access + numbered parameters and "it" parameter. [[Bug #21049]] + +* Enumerator + + * `Enumerator.produce` now accepts an optional `size` keyword argument + to specify the size of the enumerator. It can be an integer, + `Float::INFINITY`, a callable object (such as a lambda), or `nil` to + indicate unknown size. When not specified, the size defaults to + `Float::INFINITY`. + + ```ruby + # Infinite enumerator + enum = Enumerator.produce(1, size: Float::INFINITY, &:succ) + enum.size # => Float::INFINITY + + # Finite enumerator with known/computable size + abs_dir = File.expand_path("./baz") # => "/foo/bar/baz" + traverser = Enumerator.produce(abs_dir, size: -> { abs_dir.count("/") + 1 }) { + raise StopIteration if it == "/" + File.dirname(it) + } + traverser.size # => 4 + ``` + + [[Feature #21701]] + +* ErrorHighlight + + * When an ArgumentError is raised, it now displays code snippets for + both the method call (caller) and the method definition (callee). + [[Feature #21543]] + + ``` + test.rb:1:in 'Object#add': wrong number of arguments (given 1, expected 2) (ArgumentError) + + caller: test.rb:3 + | add(1) + ^^^ + callee: test.rb:1 + | def add(x, y) = x + y + ^^^ + from test.rb:3:in '<main>' + ``` + +* Fiber + + * Introduce support for `Fiber#raise(cause:)` argument similar to + `Kernel#raise`. [[Feature #21360]] + +* Fiber::Scheduler + + * Introduce `Fiber::Scheduler#fiber_interrupt` to interrupt a fiber with a + given exception. The initial use case is to interrupt a fiber that is + waiting on a blocking IO operation when the IO operation is closed. + [[Feature #21166]] + + * Introduce `Fiber::Scheduler#yield` to allow the fiber scheduler to + continue processing when signal exceptions are disabled. + [[Bug #21633]] + + * Reintroduce the `Fiber::Scheduler#io_close` hook for asynchronous `IO#close`. + + * Invoke `Fiber::Scheduler#io_write` when flushing the IO write buffer. + [[Bug #21789]] + +* File + + * `File::Stat#birthtime` is now available on Linux via the statx + system call when supported by the kernel and filesystem. + [[Feature #21205]] + +* IO + + * `IO.select` accepts `Float::INFINITY` as a timeout argument. + [[Feature #20610]] + + * A deprecated behavior, process creation by `IO` class methods + with a leading `|`, was removed. [[Feature #19630]] + +* Kernel + + * `Kernel#inspect` now checks for the existence of a `#instance_variables_to_inspect` method, + allowing control over which instance variables are displayed in the `#inspect` string: + + ```ruby + class DatabaseConfig + def initialize(host, user, password) + @host = host + @user = user + @password = password + end + + private def instance_variables_to_inspect = [:@host, :@user] + end + + conf = DatabaseConfig.new("localhost", "root", "hunter2") + conf.inspect #=> #<DatabaseConfig:0x0000000104def350 @host="localhost", @user="root"> + ``` + + [[Feature #21219]] + + * A deprecated behavior, process creation by `Kernel#open` with a + leading `|`, was removed. [[Feature #19630]] + +* Math + + * `Math.log1p` and `Math.expm1` are added. [[Feature #21527]] + +* Pathname + + * Pathname has been promoted from a default gem to a core class of Ruby. + [[Feature #17473]] + +* Proc + + * `Proc#parameters` now shows anonymous optional parameters as `[:opt]` + instead of `[:opt, nil]`, making the output consistent with when the + anonymous parameter is required. [[Bug #20974]] + +* Ractor + + * `Ractor::Port` class was added for a new synchronization mechanism + to communicate between Ractors. [[Feature #21262]] + + ```ruby + port1 = Ractor::Port.new + port2 = Ractor::Port.new + Ractor.new port1, port2 do |port1, port2| + port1 << 1 + port2 << 11 + port1 << 2 + port2 << 12 + end + 2.times{ p port1.receive } #=> 1, 2 + 2.times{ p port2.receive } #=> 11, 12 + ``` + + `Ractor::Port` provides the following methods: + + * `Ractor::Port#receive` + * `Ractor::Port#send` (or `Ractor::Port#<<`) + * `Ractor::Port#close` + * `Ractor::Port#closed?` + + As a result, `Ractor.yield` and `Ractor#take` were removed. + + * `Ractor#join` and `Ractor#value` were added to wait for the + termination of a Ractor. These are similar to `Thread#join` + and `Thread#value`. + + * `Ractor#monitor` and `Ractor#unmonitor` were added as low-level + interfaces used internally to implement `Ractor#join`. + + * `Ractor.select` now only accepts Ractors and Ports. If Ractors are given, + it returns when a Ractor terminates. + + * `Ractor#default_port` was added. Each `Ractor` has a default port, + which is used by `Ractor.send`, `Ractor.receive`. + + * `Ractor#close_incoming` and `Ractor#close_outgoing` were removed. + + * `Ractor.shareable_proc` and `Ractor.shareable_lambda` are introduced + to make shareable Proc or lambda. + [[Feature #21550]], [[Feature #21557]] + +* Range + + * `Range#to_set` now performs size checks to prevent issues with + endless ranges. [[Bug #21654]] + + * `Range#overlap?` now correctly handles infinite (unbounded) ranges. + [[Bug #21185]] + + * `Range#max` behavior on beginless integer ranges has been fixed. + [[Bug #21174]] [[Bug #21175]] + +* Ruby + + * A new toplevel module `Ruby` has been defined, which contains + Ruby-related constants. This module was reserved in Ruby 3.4 + and is now officially defined. [[Feature #20884]] + +* Ruby::Box + + * A new (experimental) feature to provide separation about definitions. + For the detail of "Ruby Box", see [doc/language/box.md](doc/language/box.md). + [[Feature #21311]] [[Misc #21385]] + +* Set + + * `Set` is now a core class, instead of an autoloaded stdlib class. + [[Feature #21216]] + + * `Set#inspect` now uses a simpler display, similar to literal arrays. + (e.g., `Set[1, 2, 3]` instead of `#<Set: {1, 2, 3}>`). [[Feature #21389]] + + * Passing arguments to `Set#to_set` and `Enumerable#to_set` is now deprecated. + [[Feature #21390]] + +* Socket + + * `Socket.tcp` & `TCPSocket.new` accepts an `open_timeout` keyword argument to specify + the timeout for the initial connection. [[Feature #21347]] + * When a user-specified timeout occurred in `TCPSocket.new`, either `Errno::ETIMEDOUT` + or `IO::TimeoutError` could previously be raised depending on the situation. + This behavior has been unified so that `IO::TimeoutError` is now consistently raised. + (Please note that, in `Socket.tcp`, there are still cases where `Errno::ETIMEDOUT` + may be raised in similar situations, and that in both cases `Errno::ETIMEDOUT` may be + raised when the timeout occurs at the OS level.) + +* String + + * Update Unicode to Version 17.0.0 and Emoji Version 17.0. + [[Feature #19908]][[Feature #20724]][[Feature #21275]] (also applies to Regexp) + + * `String#strip`, `strip!`, `lstrip`, `lstrip!`, `rstrip`, and `rstrip!` + are extended to accept `*selectors` arguments. [[Feature #21552]] + +* Thread + + * Introduce support for `Thread#raise(cause:)` argument similar to + `Kernel#raise`. [[Feature #21360]] + +## Stdlib updates + +We only list stdlib changes that are notable feature changes. + +Other changes are listed in the following sections. We also listed release +history from the previous bundled version that is Ruby 3.4.0 if it has GitHub +releases. + +The following bundled gems are promoted from default gems. + +* ostruct 0.6.3 + * 0.6.1 to [v0.6.2][ostruct-v0.6.2], [v0.6.3][ostruct-v0.6.3] +* pstore 0.2.0 + * 0.1.4 to [v0.2.0][pstore-v0.2.0] +* benchmark 0.5.0 + * 0.4.0 to [v0.4.1][benchmark-v0.4.1], [v0.5.0][benchmark-v0.5.0] +* logger 1.7.0 + * 1.6.4 to [v1.6.5][logger-v1.6.5], [v1.6.6][logger-v1.6.6], [v1.7.0][logger-v1.7.0] +* rdoc 7.0.3 + * 6.14.0 to [v6.14.1][rdoc-v6.14.1], [v6.14.2][rdoc-v6.14.2], [v6.15.0][rdoc-v6.15.0], [v6.15.1][rdoc-v6.15.1], [v6.16.0][rdoc-v6.16.0], [v6.16.1][rdoc-v6.16.1], [v6.17.0][rdoc-v6.17.0], [v7.0.0][rdoc-v7.0.0], [v7.0.1][rdoc-v7.0.1], [v7.0.2][rdoc-v7.0.2], [v7.0.3][rdoc-v7.0.3] +* win32ole 1.9.2 + * 1.9.1 to [v1.9.2][win32ole-v1.9.2] +* irb 1.16.0 + * 1.14.3 to [v1.15.0][irb-v1.15.0], [v1.15.1][irb-v1.15.1], [v1.15.2][irb-v1.15.2], [v1.15.3][irb-v1.15.3], [v1.16.0][irb-v1.16.0] +* reline 0.6.3 + * 0.6.0 to [v0.6.1][reline-v0.6.1], [v0.6.2][reline-v0.6.2], [v0.6.3][reline-v0.6.3] +* readline 0.0.4 +* fiddle 1.1.8 + * 1.1.6 to [v1.1.7][fiddle-v1.1.7], [v1.1.8][fiddle-v1.1.8] + +The following default gem is added. + +* win32-registry 0.1.2 + +The following default gems are updated. + +* RubyGems 4.0.3 +* bundler 4.0.3 +* date 3.5.1 + * 3.4.1 to [v3.5.0][date-v3.5.0], [v3.5.1][date-v3.5.1] +* delegate 0.6.1 + * 0.4.0 to [v0.5.0][delegate-v0.5.0], [v0.6.0][delegate-v0.6.0], [v0.6.1][delegate-v0.6.1] +* digest 3.2.1 + * 3.2.0 to [v3.2.1][digest-v3.2.1] +* english 0.8.1 + * 0.8.0 to [v0.8.1][english-v0.8.1] +* erb 6.0.1 + * 4.0.4 to [v5.1.2][erb-v5.1.2], [v5.1.3][erb-v5.1.3], [v6.0.0][erb-v6.0.0], [v6.0.1][erb-v6.0.1] +* error_highlight 0.7.1 +* etc 1.4.6 +* fcntl 1.3.0 + * 1.2.0 to [v1.3.0][fcntl-v1.3.0] +* fileutils 1.8.0 + * 1.7.3 to [v1.8.0][fileutils-v1.8.0] +* forwardable 1.4.0 + * 1.3.3 to [v1.4.0][forwardable-v1.4.0] +* io-console 0.8.2 + * 0.8.1 to [v0.8.2][io-console-v0.8.2] +* io-nonblock 0.3.2 +* io-wait 0.4.0 + * 0.3.2 to [v0.3.3][io-wait-v0.3.3], [v0.3.5.test1][io-wait-v0.3.5.test1], [v0.3.5][io-wait-v0.3.5], [v0.3.6][io-wait-v0.3.6], [v0.4.0][io-wait-v0.4.0] +* ipaddr 1.2.8 +* json 2.18.0 + * 2.9.1 to [v2.10.0][json-v2.10.0], [v2.10.1][json-v2.10.1], [v2.10.2][json-v2.10.2], [v2.11.0][json-v2.11.0], [v2.11.1][json-v2.11.1], [v2.11.2][json-v2.11.2], [v2.11.3][json-v2.11.3], [v2.12.0][json-v2.12.0], [v2.12.1][json-v2.12.1], [v2.12.2][json-v2.12.2], [v2.13.0][json-v2.13.0], [v2.13.1][json-v2.13.1], [v2.13.2][json-v2.13.2], [v2.14.0][json-v2.14.0], [v2.14.1][json-v2.14.1], [v2.15.0][json-v2.15.0], [v2.15.1][json-v2.15.1], [v2.15.2][json-v2.15.2], [v2.16.0][json-v2.16.0], [v2.17.0][json-v2.17.0], [v2.17.1][json-v2.17.1], [v2.18.0][json-v2.18.0] +* net-http 0.9.1 + * 0.6.0 to [v0.7.0][net-http-v0.7.0], [v0.8.0][net-http-v0.8.0], [v0.9.0][net-http-v0.9.0], [v0.9.1][net-http-v0.9.1] +* openssl 4.0.0 + * 3.3.1 to [v3.3.2][openssl-v3.3.2], [v4.0.0][openssl-v4.0.0] +* optparse 0.8.1 + * 0.6.0 to [v0.7.0][optparse-v0.7.0], [v0.8.0][optparse-v0.8.0], [v0.8.1][optparse-v0.8.1] +* pp 0.6.3 + * 0.6.2 to [v0.6.3][pp-v0.6.3] +* prism 1.7.0 + * 1.5.2 to [v1.6.0][prism-v1.6.0], [v1.7.0][prism-v1.7.0] +* psych 5.3.1 + * 5.2.2 to [v5.2.3][psych-v5.2.3], [v5.2.4][psych-v5.2.4], [v5.2.5][psych-v5.2.5], [v5.2.6][psych-v5.2.6], [v5.3.0][psych-v5.3.0], [v5.3.1][psych-v5.3.1] +* resolv 0.7.0 + * 0.6.2 to [v0.6.3][resolv-v0.6.3], [v0.7.0][resolv-v0.7.0] +* stringio 3.2.0 + * 3.1.2 to [v3.1.3][stringio-v3.1.3], [v3.1.4][stringio-v3.1.4], [v3.1.5][stringio-v3.1.5], [v3.1.6][stringio-v3.1.6], [v3.1.7][stringio-v3.1.7], [v3.1.8][stringio-v3.1.8], [v3.1.9][stringio-v3.1.9], [v3.2.0][stringio-v3.2.0] +* strscan 3.1.6 + * 3.1.2 to [v3.1.3][strscan-v3.1.3], [v3.1.4][strscan-v3.1.4], [v3.1.5][strscan-v3.1.5], [v3.1.6][strscan-v3.1.6] +* time 0.4.2 + * 0.4.1 to [v0.4.2][time-v0.4.2] +* timeout 0.6.0 + * 0.4.3 to [v0.4.4][timeout-v0.4.4], [v0.5.0][timeout-v0.5.0], [v0.6.0][timeout-v0.6.0] +* uri 1.1.1 + * 1.0.4 to [v1.1.0][uri-v1.1.0], [v1.1.1][uri-v1.1.1] +* weakref 0.1.4 + * 0.1.3 to [v0.1.4][weakref-v0.1.4] +* zlib 3.2.2 + * 3.2.1 to [v3.2.2][zlib-v3.2.2] + +The following bundled gems are updated. + +* minitest 6.0.0 +* power_assert 3.0.1 + * 2.0.5 to [v3.0.0][power_assert-v3.0.0], [v3.0.1][power_assert-v3.0.1] +* rake 13.3.1 + * 13.2.1 to [v13.3.0][rake-v13.3.0], [v13.3.1][rake-v13.3.1] +* test-unit 3.7.5 + * 3.6.7 to [3.6.8][test-unit-3.6.8], [3.6.9][test-unit-3.6.9], [3.7.0][test-unit-3.7.0], [3.7.1][test-unit-3.7.1], [3.7.2][test-unit-3.7.2], [3.7.3][test-unit-3.7.3], [3.7.4][test-unit-3.7.4], [3.7.5][test-unit-3.7.5] +* rexml 3.4.4 +* rss 0.3.2 + * 0.3.1 to [0.3.2][rss-0.3.2] +* net-ftp 0.3.9 + * 0.3.8 to [v0.3.9][net-ftp-v0.3.9] +* net-imap 0.6.2 + * 0.5.8 to [v0.5.9][net-imap-v0.5.9], [v0.5.10][net-imap-v0.5.10], [v0.5.11][net-imap-v0.5.11], [v0.5.12][net-imap-v0.5.12], [v0.5.13][net-imap-v0.5.13], [v0.6.0][net-imap-v0.6.0], [v0.6.1][net-imap-v0.6.1], [v0.6.2][net-imap-v0.6.2] +* net-smtp 0.5.1 + * 0.5.0 to [v0.5.1][net-smtp-v0.5.1] +* matrix 0.4.3 + * 0.4.2 to [v0.4.3][matrix-v0.4.3] +* prime 0.1.4 + * 0.1.3 to [v0.1.4][prime-v0.1.4] +* rbs 3.10.0 + * 3.8.0 to [v3.8.1][rbs-v3.8.1], [v3.9.0.dev.1][rbs-v3.9.0.dev.1], [v3.9.0.pre.1][rbs-v3.9.0.pre.1], [v3.9.0.pre.2][rbs-v3.9.0.pre.2], [v3.9.0][rbs-v3.9.0], [v3.9.1][rbs-v3.9.1], [v3.9.2][rbs-v3.9.2], [v3.9.3][rbs-v3.9.3], [v3.9.4][rbs-v3.9.4], [v3.9.5][rbs-v3.9.5], [v3.10.0.pre.1][rbs-v3.10.0.pre.1], [v3.10.0.pre.2][rbs-v3.10.0.pre.2], [v3.10.0][rbs-v3.10.0] +* typeprof 0.31.1 +* debug 1.11.1 + * 1.11.0 to [v1.11.1][debug-v1.11.1] +* base64 0.3.0 + * 0.2.0 to [v0.3.0][base64-v0.3.0] +* bigdecimal 4.0.1 + * 3.1.8 to [v3.2.0][bigdecimal-v3.2.0], [v3.2.1][bigdecimal-v3.2.1], [v3.2.2][bigdecimal-v3.2.2], [v3.2.3][bigdecimal-v3.2.3], [v3.3.0][bigdecimal-v3.3.0], [v3.3.1][bigdecimal-v3.3.1], [v4.0.0][bigdecimal-v4.0.0], [v4.0.1][bigdecimal-v4.0.1] +* drb 2.2.3 + * 2.2.1 to [v2.2.3][drb-v2.2.3] +* syslog 0.3.0 + * 0.2.0 to [v0.3.0][syslog-v0.3.0] +* csv 3.3.5 + * 3.3.2 to [v3.3.3][csv-v3.3.3], [v3.3.4][csv-v3.3.4], [v3.3.5][csv-v3.3.5] +* repl_type_completor 0.1.12 + +### RubyGems and Bundler + +Ruby 4.0 bundled RubyGems and Bundler version 4. see the following links for details. + +* [Upgrading to RubyGems/Bundler 4 - RubyGems Blog](https://blog.rubygems.org/2025/12/03/upgrade-to-rubygems-bundler-4.html) +* [4.0.0 Released - RubyGems Blog](https://blog.rubygems.org/2025/12/03/4.0.0-released.html) +* [4.0.1 Released - RubyGems Blog](https://blog.rubygems.org/2025/12/09/4.0.1-released.html) +* [4.0.2 Released - RubyGems Blog](https://blog.rubygems.org/2025/12/17/4.0.2-released.html) +* [4.0.3 Released - RubyGems Blog](https://blog.rubygems.org/2025/12/23/4.0.3-released.html) + +## Supported platforms + +* Windows + + * Dropped support for MSVC versions older than 14.0 (_MSC_VER 1900). + This means Visual Studio 2015 or later is now required. + +## Compatibility issues + +* The following methods were removed from Ractor due to the addition of `Ractor::Port`: + + * `Ractor.yield` + * `Ractor#take` + * `Ractor#close_incoming` + * `Ractor#close_outgoing` + + [[Feature #21262]] + +* `ObjectSpace._id2ref` is deprecated. [[Feature #15408]] + +* `Process::Status#&` and `Process::Status#>>` have been removed. + They were deprecated in Ruby 3.3. [[Bug #19868]] + +* `rb_path_check` has been removed. This function was used for + `$SAFE` path checking which was removed in Ruby 2.7, + and was already deprecated. + [[Feature #20971]] + +* A backtrace for `ArgumentError` of "wrong number of arguments" now + include the receiver's class or module name (e.g., in `Foo#bar` + instead of in `bar`). [[Bug #21698]] + +* Backtraces no longer display `internal` frames. + These methods now appear as if it is in the Ruby source file, + consistent with other C-implemented methods. [[Bug #20968]] + + Before: + ``` + ruby -e '[1].fetch_values(42)' + <internal:array>:211:in 'Array#fetch': index 42 outside of array bounds: -1...1 (IndexError) + from <internal:array>:211:in 'block in Array#fetch_values' + from <internal:array>:211:in 'Array#map!' + from <internal:array>:211:in 'Array#fetch_values' + from -e:1:in '<main>' + ``` + + After: + ``` + $ ruby -e '[1].fetch_values(42)' + -e:1:in 'Array#fetch_values': index 42 outside of array bounds: -1...1 (IndexError) + from -e:1:in '<main>' + ``` + +## Stdlib compatibility issues + +* CGI library is removed from the default gems. Now we only provide `cgi/escape` for + the following methods: + + * `CGI.escape` and `CGI.unescape` + * `CGI.escapeHTML` and `CGI.unescapeHTML` + * `CGI.escapeURIComponent` and `CGI.unescapeURIComponent` + * `CGI.escapeElement` and `CGI.unescapeElement` + + [[Feature #21258]] + +* With the move of `Set` from stdlib to core class, `set/sorted_set.rb` has + been removed, and `SortedSet` is no longer an autoloaded constant. Please + install the `sorted_set` gem and `require 'sorted_set'` to use `SortedSet`. + [[Feature #21287]] + +* Net::HTTP + + * The default behavior of automatically setting the `Content-Type` header + to `application/x-www-form-urlencoded` for requests with a body + (e.g., `POST`, `PUT`) when the header was not explicitly set has been + removed. If your application relied on this automatic default, your + requests will now be sent without a Content-Type header, potentially + breaking compatibility with certain servers. + [[GH-net-http #205]] + +## C API updates + +* IO + + * `rb_thread_fd_close` is deprecated and now a no-op. If you need to expose + file descriptors from C extensions to Ruby code, create an `IO` instance + using `RUBY_IO_MODE_EXTERNAL` and use `rb_io_close(io)` to close it (this + also interrupts and waits for all pending operations on the `IO` + instance). Directly closing file descriptors does not interrupt pending + operations, and may lead to undefined behaviour. In other words, if two + `IO` objects share the same file descriptor, closing one does not affect + the other. [[Feature #18455]] + +* GVL + + * `rb_thread_call_with_gvl` now works with or without the GVL. + This allows gems to avoid checking `ruby_thread_has_gvl_p`. + Please still be diligent about the GVL. [[Feature #20750]] + +* Set + + * A C API for `Set` has been added. The following methods are supported: + [[Feature #21459]] + + * `rb_set_foreach` + * `rb_set_new` + * `rb_set_new_capa` + * `rb_set_lookup` + * `rb_set_add` + * `rb_set_clear` + * `rb_set_delete` + * `rb_set_size` + +## Implementation improvements + +* `Class#new` (ex. `Object.new`) is faster in all cases, but especially when passing keyword arguments. This has also been integrated into YJIT and ZJIT. [[Feature #21254]] +* GC heaps of different size pools now grow independently, reducing memory usage when only some pools contain long-lived objects +* GC sweeping is faster on pages of large objects +* "Generic ivar" objects (String, Array, `TypedData`, etc.) now use a new internal "fields" object for faster instance variable access +* The GC avoids maintaining an internal `id2ref` table until it is first used, making `object_id` allocation and GC sweeping faster +* `object_id` and `hash` are faster on Class and Module objects +* Larger bignum Integers can remain embedded using variable width allocation +* `Random`, `Enumerator::Product`, `Enumerator::Chain`, `Addrinfo`, + `StringScanner`, and some internal objects are now write-barrier protected, + which reduces GC overhead. + +### Ractor + +A lot of work has gone into making Ractors more stable, performant, and usable. These improvements bring Ractor implementation closer to leaving experimental status. + +* Performance improvements + * Frozen strings and the symbol table internally use a lock-free hash set [[Feature #21268]] + * Method cache lookups avoid locking in most cases + * Class (and generic ivar) instance variable access is faster and avoids locking + * CPU cache contention is avoided in object allocation by using a per-ractor counter + * CPU cache contention is avoided in xmalloc/xfree by using a thread-local counter + * `object_id` avoids locking in most cases +* Bug fixes and stability + * Fixed possible deadlocks when combining Ractors and Threads + * Fixed issues with require and autoload in a Ractor + * Fixed encoding/transcoding issues across Ractors + * Fixed race conditions in GC operations and method invalidation + * Fixed issues with processes forking after starting a Ractor + * GC allocation counts are now accurate under Ractors + * Fixed TracePoints not working after GC [[Bug #19112]] + +## JIT + +* ZJIT + * Introduce an [experimental method-based JIT compiler](https://docs.ruby-lang.org/en/master/jit/zjit_md.html). + Where available, ZJIT can be enabled at runtime with the `--zjit` option or by calling `RubyVM::ZJIT.enable`. + When building Ruby, Rust 1.85.0 or later is required to include ZJIT support. + * As of Ruby 4.0.0, ZJIT is faster than the interpreter, but not yet as fast as YJIT. + We encourage experimentation with ZJIT, but advise against deploying it in production for now. + * Our goal is to make ZJIT faster than YJIT and production-ready in Ruby 4.1. +* YJIT + * `RubyVM::YJIT.runtime_stats` + * `ratio_in_yjit` no longer works in the default build. + Use `--enable-yjit=stats` on `configure` to enable it on `--yjit-stats`. + * Add `invalidate_everything` to default stats, which is + incremented when every code is invalidated by TracePoint. + * Add `mem_size:` and `call_threshold:` options to `RubyVM::YJIT.enable`. +* RJIT + * `--rjit` is removed. We will move the implementation of the third-party JIT API + to the [ruby/rjit](https://github.com/ruby/rjit) repository. + +[Feature #15408]: https://bugs.ruby-lang.org/issues/15408 +[Feature #17473]: https://bugs.ruby-lang.org/issues/17473 +[Feature #18455]: https://bugs.ruby-lang.org/issues/18455 +[Bug #19112]: https://bugs.ruby-lang.org/issues/19112 +[Feature #19630]: https://bugs.ruby-lang.org/issues/19630 +[Bug #19868]: https://bugs.ruby-lang.org/issues/19868 +[Feature #19908]: https://bugs.ruby-lang.org/issues/19908 +[Feature #20610]: https://bugs.ruby-lang.org/issues/20610 +[Feature #20724]: https://bugs.ruby-lang.org/issues/20724 +[Feature #20750]: https://bugs.ruby-lang.org/issues/20750 +[Feature #20884]: https://bugs.ruby-lang.org/issues/20884 +[Feature #20925]: https://bugs.ruby-lang.org/issues/20925 +[Bug #20968]: https://bugs.ruby-lang.org/issues/20968 +[Feature #20971]: https://bugs.ruby-lang.org/issues/20971 +[Bug #20974]: https://bugs.ruby-lang.org/issues/20974 +[Feature #21047]: https://bugs.ruby-lang.org/issues/21047 +[Bug #21049]: https://bugs.ruby-lang.org/issues/21049 +[Feature #21166]: https://bugs.ruby-lang.org/issues/21166 +[Bug #21174]: https://bugs.ruby-lang.org/issues/21174 +[Bug #21175]: https://bugs.ruby-lang.org/issues/21175 +[Bug #21185]: https://bugs.ruby-lang.org/issues/21185 +[Feature #21205]: https://bugs.ruby-lang.org/issues/21205 +[Feature #21216]: https://bugs.ruby-lang.org/issues/21216 +[Feature #21219]: https://bugs.ruby-lang.org/issues/21219 +[Feature #21254]: https://bugs.ruby-lang.org/issues/21254 +[Feature #21258]: https://bugs.ruby-lang.org/issues/21258 +[Feature #21268]: https://bugs.ruby-lang.org/issues/21268 +[Feature #21262]: https://bugs.ruby-lang.org/issues/21262 +[Feature #21275]: https://bugs.ruby-lang.org/issues/21275 +[Feature #21287]: https://bugs.ruby-lang.org/issues/21287 +[Feature #21311]: https://bugs.ruby-lang.org/issues/21311 +[Feature #21347]: https://bugs.ruby-lang.org/issues/21347 +[Feature #21360]: https://bugs.ruby-lang.org/issues/21360 +[Misc #21385]: https://bugs.ruby-lang.org/issues/21385 +[Feature #21389]: https://bugs.ruby-lang.org/issues/21389 +[Feature #21390]: https://bugs.ruby-lang.org/issues/21390 +[Feature #21459]: https://bugs.ruby-lang.org/issues/21459 +[Feature #21527]: https://bugs.ruby-lang.org/issues/21527 +[Feature #21543]: https://bugs.ruby-lang.org/issues/21543 +[Feature #21550]: https://bugs.ruby-lang.org/issues/21550 +[Feature #21552]: https://bugs.ruby-lang.org/issues/21552 +[Feature #21557]: https://bugs.ruby-lang.org/issues/21557 +[Bug #21633]: https://bugs.ruby-lang.org/issues/21633 +[Bug #21654]: https://bugs.ruby-lang.org/issues/21654 +[Feature #21678]: https://bugs.ruby-lang.org/issues/21678 +[Bug #21698]: https://bugs.ruby-lang.org/issues/21698 +[Feature #21701]: https://bugs.ruby-lang.org/issues/21701 +[Bug #21789]: https://bugs.ruby-lang.org/issues/21789 +[GH-net-http #205]: https://github.com/ruby/net-http/issues/205 +[ostruct-v0.6.2]: https://github.com/ruby/ostruct/releases/tag/v0.6.2 +[ostruct-v0.6.3]: https://github.com/ruby/ostruct/releases/tag/v0.6.3 +[pstore-v0.2.0]: https://github.com/ruby/pstore/releases/tag/v0.2.0 +[benchmark-v0.4.1]: https://github.com/ruby/benchmark/releases/tag/v0.4.1 +[benchmark-v0.5.0]: https://github.com/ruby/benchmark/releases/tag/v0.5.0 +[logger-v1.6.5]: https://github.com/ruby/logger/releases/tag/v1.6.5 +[logger-v1.6.6]: https://github.com/ruby/logger/releases/tag/v1.6.6 +[logger-v1.7.0]: https://github.com/ruby/logger/releases/tag/v1.7.0 +[rdoc-v6.14.1]: https://github.com/ruby/rdoc/releases/tag/v6.14.1 +[rdoc-v6.14.2]: https://github.com/ruby/rdoc/releases/tag/v6.14.2 +[rdoc-v6.15.0]: https://github.com/ruby/rdoc/releases/tag/v6.15.0 +[rdoc-v6.15.1]: https://github.com/ruby/rdoc/releases/tag/v6.15.1 +[rdoc-v6.16.0]: https://github.com/ruby/rdoc/releases/tag/v6.16.0 +[rdoc-v6.16.1]: https://github.com/ruby/rdoc/releases/tag/v6.16.1 +[rdoc-v6.17.0]: https://github.com/ruby/rdoc/releases/tag/v6.17.0 +[rdoc-v7.0.0]: https://github.com/ruby/rdoc/releases/tag/v7.0.0 +[rdoc-v7.0.1]: https://github.com/ruby/rdoc/releases/tag/v7.0.1 +[rdoc-v7.0.2]: https://github.com/ruby/rdoc/releases/tag/v7.0.2 +[rdoc-v7.0.3]: https://github.com/ruby/rdoc/releases/tag/v7.0.3 +[win32ole-v1.9.2]: https://github.com/ruby/win32ole/releases/tag/v1.9.2 +[irb-v1.15.0]: https://github.com/ruby/irb/releases/tag/v1.15.0 +[irb-v1.15.1]: https://github.com/ruby/irb/releases/tag/v1.15.1 +[irb-v1.15.2]: https://github.com/ruby/irb/releases/tag/v1.15.2 +[irb-v1.15.3]: https://github.com/ruby/irb/releases/tag/v1.15.3 +[irb-v1.16.0]: https://github.com/ruby/irb/releases/tag/v1.16.0 +[reline-v0.6.1]: https://github.com/ruby/reline/releases/tag/v0.6.1 +[reline-v0.6.2]: https://github.com/ruby/reline/releases/tag/v0.6.2 +[reline-v0.6.3]: https://github.com/ruby/reline/releases/tag/v0.6.3 +[fiddle-v1.1.7]: https://github.com/ruby/fiddle/releases/tag/v1.1.7 +[fiddle-v1.1.8]: https://github.com/ruby/fiddle/releases/tag/v1.1.8 +[date-v3.5.0]: https://github.com/ruby/date/releases/tag/v3.5.0 +[date-v3.5.1]: https://github.com/ruby/date/releases/tag/v3.5.1 +[delegate-v0.5.0]: https://github.com/ruby/delegate/releases/tag/v0.5.0 +[delegate-v0.6.0]: https://github.com/ruby/delegate/releases/tag/v0.6.0 +[delegate-v0.6.1]: https://github.com/ruby/delegate/releases/tag/v0.6.1 +[digest-v3.2.1]: https://github.com/ruby/digest/releases/tag/v3.2.1 +[english-v0.8.1]: https://github.com/ruby/english/releases/tag/v0.8.1 +[erb-v5.1.2]: https://github.com/ruby/erb/releases/tag/v5.1.2 +[erb-v5.1.3]: https://github.com/ruby/erb/releases/tag/v5.1.3 +[erb-v6.0.0]: https://github.com/ruby/erb/releases/tag/v6.0.0 +[erb-v6.0.1]: https://github.com/ruby/erb/releases/tag/v6.0.1 +[fcntl-v1.3.0]: https://github.com/ruby/fcntl/releases/tag/v1.3.0 +[fileutils-v1.8.0]: https://github.com/ruby/fileutils/releases/tag/v1.8.0 +[forwardable-v1.4.0]: https://github.com/ruby/forwardable/releases/tag/v1.4.0 +[io-console-v0.8.2]: https://github.com/ruby/io-console/releases/tag/v0.8.2 +[io-wait-v0.3.3]: https://github.com/ruby/io-wait/releases/tag/v0.3.3 +[io-wait-v0.3.5.test1]: https://github.com/ruby/io-wait/releases/tag/v0.3.5.test1 +[io-wait-v0.3.5]: https://github.com/ruby/io-wait/releases/tag/v0.3.5 +[io-wait-v0.3.6]: https://github.com/ruby/io-wait/releases/tag/v0.3.6 +[io-wait-v0.4.0]: https://github.com/ruby/io-wait/releases/tag/v0.4.0 +[json-v2.10.0]: https://github.com/ruby/json/releases/tag/v2.10.0 +[json-v2.10.1]: https://github.com/ruby/json/releases/tag/v2.10.1 +[json-v2.10.2]: https://github.com/ruby/json/releases/tag/v2.10.2 +[json-v2.11.0]: https://github.com/ruby/json/releases/tag/v2.11.0 +[json-v2.11.1]: https://github.com/ruby/json/releases/tag/v2.11.1 +[json-v2.11.2]: https://github.com/ruby/json/releases/tag/v2.11.2 +[json-v2.11.3]: https://github.com/ruby/json/releases/tag/v2.11.3 +[json-v2.12.0]: https://github.com/ruby/json/releases/tag/v2.12.0 +[json-v2.12.1]: https://github.com/ruby/json/releases/tag/v2.12.1 +[json-v2.12.2]: https://github.com/ruby/json/releases/tag/v2.12.2 +[json-v2.13.0]: https://github.com/ruby/json/releases/tag/v2.13.0 +[json-v2.13.1]: https://github.com/ruby/json/releases/tag/v2.13.1 +[json-v2.13.2]: https://github.com/ruby/json/releases/tag/v2.13.2 +[json-v2.14.0]: https://github.com/ruby/json/releases/tag/v2.14.0 +[json-v2.14.1]: https://github.com/ruby/json/releases/tag/v2.14.1 +[json-v2.15.0]: https://github.com/ruby/json/releases/tag/v2.15.0 +[json-v2.15.1]: https://github.com/ruby/json/releases/tag/v2.15.1 +[json-v2.15.2]: https://github.com/ruby/json/releases/tag/v2.15.2 +[json-v2.16.0]: https://github.com/ruby/json/releases/tag/v2.16.0 +[json-v2.17.0]: https://github.com/ruby/json/releases/tag/v2.17.0 +[json-v2.17.1]: https://github.com/ruby/json/releases/tag/v2.17.1 +[json-v2.18.0]: https://github.com/ruby/json/releases/tag/v2.18.0 +[net-http-v0.7.0]: https://github.com/ruby/net-http/releases/tag/v0.7.0 +[net-http-v0.8.0]: https://github.com/ruby/net-http/releases/tag/v0.8.0 +[net-http-v0.9.0]: https://github.com/ruby/net-http/releases/tag/v0.9.0 +[net-http-v0.9.1]: https://github.com/ruby/net-http/releases/tag/v0.9.1 +[openssl-v3.3.2]: https://github.com/ruby/openssl/releases/tag/v3.3.2 +[openssl-v4.0.0]: https://github.com/ruby/openssl/releases/tag/v4.0.0 +[optparse-v0.7.0]: https://github.com/ruby/optparse/releases/tag/v0.7.0 +[optparse-v0.8.0]: https://github.com/ruby/optparse/releases/tag/v0.8.0 +[optparse-v0.8.1]: https://github.com/ruby/optparse/releases/tag/v0.8.1 +[pp-v0.6.3]: https://github.com/ruby/pp/releases/tag/v0.6.3 +[prism-v1.6.0]: https://github.com/ruby/prism/releases/tag/v1.6.0 +[prism-v1.7.0]: https://github.com/ruby/prism/releases/tag/v1.7.0 +[psych-v5.2.3]: https://github.com/ruby/psych/releases/tag/v5.2.3 +[psych-v5.2.4]: https://github.com/ruby/psych/releases/tag/v5.2.4 +[psych-v5.2.5]: https://github.com/ruby/psych/releases/tag/v5.2.5 +[psych-v5.2.6]: https://github.com/ruby/psych/releases/tag/v5.2.6 +[psych-v5.3.0]: https://github.com/ruby/psych/releases/tag/v5.3.0 +[psych-v5.3.1]: https://github.com/ruby/psych/releases/tag/v5.3.1 +[resolv-v0.6.3]: https://github.com/ruby/resolv/releases/tag/v0.6.3 +[resolv-v0.7.0]: https://github.com/ruby/resolv/releases/tag/v0.7.0 +[stringio-v3.1.3]: https://github.com/ruby/stringio/releases/tag/v3.1.3 +[stringio-v3.1.4]: https://github.com/ruby/stringio/releases/tag/v3.1.4 +[stringio-v3.1.5]: https://github.com/ruby/stringio/releases/tag/v3.1.5 +[stringio-v3.1.6]: https://github.com/ruby/stringio/releases/tag/v3.1.6 +[stringio-v3.1.7]: https://github.com/ruby/stringio/releases/tag/v3.1.7 +[stringio-v3.1.8]: https://github.com/ruby/stringio/releases/tag/v3.1.8 +[stringio-v3.1.9]: https://github.com/ruby/stringio/releases/tag/v3.1.9 +[stringio-v3.2.0]: https://github.com/ruby/stringio/releases/tag/v3.2.0 +[strscan-v3.1.3]: https://github.com/ruby/strscan/releases/tag/v3.1.3 +[strscan-v3.1.4]: https://github.com/ruby/strscan/releases/tag/v3.1.4 +[strscan-v3.1.5]: https://github.com/ruby/strscan/releases/tag/v3.1.5 +[strscan-v3.1.6]: https://github.com/ruby/strscan/releases/tag/v3.1.6 +[time-v0.4.2]: https://github.com/ruby/time/releases/tag/v0.4.2 +[timeout-v0.4.4]: https://github.com/ruby/timeout/releases/tag/v0.4.4 +[timeout-v0.5.0]: https://github.com/ruby/timeout/releases/tag/v0.5.0 +[timeout-v0.6.0]: https://github.com/ruby/timeout/releases/tag/v0.6.0 +[uri-v1.1.0]: https://github.com/ruby/uri/releases/tag/v1.1.0 +[uri-v1.1.1]: https://github.com/ruby/uri/releases/tag/v1.1.1 +[weakref-v0.1.4]: https://github.com/ruby/weakref/releases/tag/v0.1.4 +[zlib-v3.2.2]: https://github.com/ruby/zlib/releases/tag/v3.2.2 +[power_assert-v3.0.0]: https://github.com/ruby/power_assert/releases/tag/v3.0.0 +[power_assert-v3.0.1]: https://github.com/ruby/power_assert/releases/tag/v3.0.1 +[rake-v13.3.0]: https://github.com/ruby/rake/releases/tag/v13.3.0 +[rake-v13.3.1]: https://github.com/ruby/rake/releases/tag/v13.3.1 +[test-unit-3.6.8]: https://github.com/test-unit/test-unit/releases/tag/3.6.8 +[test-unit-3.6.9]: https://github.com/test-unit/test-unit/releases/tag/3.6.9 +[test-unit-3.7.0]: https://github.com/test-unit/test-unit/releases/tag/3.7.0 +[test-unit-3.7.1]: https://github.com/test-unit/test-unit/releases/tag/3.7.1 +[test-unit-3.7.2]: https://github.com/test-unit/test-unit/releases/tag/3.7.2 +[test-unit-3.7.3]: https://github.com/test-unit/test-unit/releases/tag/3.7.3 +[test-unit-3.7.4]: https://github.com/test-unit/test-unit/releases/tag/3.7.4 +[test-unit-3.7.5]: https://github.com/test-unit/test-unit/releases/tag/3.7.5 +[rss-0.3.2]: https://github.com/ruby/rss/releases/tag/0.3.2 +[net-ftp-v0.3.9]: https://github.com/ruby/net-ftp/releases/tag/v0.3.9 +[net-imap-v0.5.9]: https://github.com/ruby/net-imap/releases/tag/v0.5.9 +[net-imap-v0.5.10]: https://github.com/ruby/net-imap/releases/tag/v0.5.10 +[net-imap-v0.5.11]: https://github.com/ruby/net-imap/releases/tag/v0.5.11 +[net-imap-v0.5.12]: https://github.com/ruby/net-imap/releases/tag/v0.5.12 +[net-imap-v0.5.13]: https://github.com/ruby/net-imap/releases/tag/v0.5.13 +[net-imap-v0.6.0]: https://github.com/ruby/net-imap/releases/tag/v0.6.0 +[net-imap-v0.6.1]: https://github.com/ruby/net-imap/releases/tag/v0.6.1 +[net-imap-v0.6.2]: https://github.com/ruby/net-imap/releases/tag/v0.6.2 +[net-smtp-v0.5.1]: https://github.com/ruby/net-smtp/releases/tag/v0.5.1 +[matrix-v0.4.3]: https://github.com/ruby/matrix/releases/tag/v0.4.3 +[prime-v0.1.4]: https://github.com/ruby/prime/releases/tag/v0.1.4 +[rbs-v3.8.1]: https://github.com/ruby/rbs/releases/tag/v3.8.1 +[rbs-v3.9.0.dev.1]: https://github.com/ruby/rbs/releases/tag/v3.9.0.dev.1 +[rbs-v3.9.0.pre.1]: https://github.com/ruby/rbs/releases/tag/v3.9.0.pre.1 +[rbs-v3.9.0.pre.2]: https://github.com/ruby/rbs/releases/tag/v3.9.0.pre.2 +[rbs-v3.9.0]: https://github.com/ruby/rbs/releases/tag/v3.9.0 +[rbs-v3.9.1]: https://github.com/ruby/rbs/releases/tag/v3.9.1 +[rbs-v3.9.2]: https://github.com/ruby/rbs/releases/tag/v3.9.2 +[rbs-v3.9.3]: https://github.com/ruby/rbs/releases/tag/v3.9.3 +[rbs-v3.9.4]: https://github.com/ruby/rbs/releases/tag/v3.9.4 +[rbs-v3.9.5]: https://github.com/ruby/rbs/releases/tag/v3.9.5 +[rbs-v3.10.0.pre.1]: https://github.com/ruby/rbs/releases/tag/v3.10.0.pre.1 +[rbs-v3.10.0.pre.2]: https://github.com/ruby/rbs/releases/tag/v3.10.0.pre.2 +[rbs-v3.10.0]: https://github.com/ruby/rbs/releases/tag/v3.10.0 +[debug-v1.11.1]: https://github.com/ruby/debug/releases/tag/v1.11.1 +[base64-v0.3.0]: https://github.com/ruby/base64/releases/tag/v0.3.0 +[bigdecimal-v3.2.0]: https://github.com/ruby/bigdecimal/releases/tag/v3.2.0 +[bigdecimal-v3.2.1]: https://github.com/ruby/bigdecimal/releases/tag/v3.2.1 +[bigdecimal-v3.2.2]: https://github.com/ruby/bigdecimal/releases/tag/v3.2.2 +[bigdecimal-v3.2.3]: https://github.com/ruby/bigdecimal/releases/tag/v3.2.3 +[bigdecimal-v3.3.0]: https://github.com/ruby/bigdecimal/releases/tag/v3.3.0 +[bigdecimal-v3.3.1]: https://github.com/ruby/bigdecimal/releases/tag/v3.3.1 +[bigdecimal-v4.0.0]: https://github.com/ruby/bigdecimal/releases/tag/v4.0.0 +[bigdecimal-v4.0.1]: https://github.com/ruby/bigdecimal/releases/tag/v4.0.1 +[drb-v2.2.3]: https://github.com/ruby/drb/releases/tag/v2.2.3 +[syslog-v0.3.0]: https://github.com/ruby/syslog/releases/tag/v0.3.0 +[csv-v3.3.3]: https://github.com/ruby/csv/releases/tag/v3.3.3 +[csv-v3.3.4]: https://github.com/ruby/csv/releases/tag/v3.3.4 +[csv-v3.3.5]: https://github.com/ruby/csv/releases/tag/v3.3.5 diff --git a/doc/_regexp.rdoc b/doc/_regexp.rdoc index 468827da15..aa55a7eebf 100644 --- a/doc/_regexp.rdoc +++ b/doc/_regexp.rdoc @@ -39,7 +39,7 @@ A regexp may be used: most such methods accept an argument that may be either a string or the (much more powerful) regexp. - See {Regexp Methods}[rdoc-ref:regexp/methods.rdoc]. + See {Regexp Methods}[rdoc-ref:language/regexp/methods.rdoc]. == \Regexp Objects @@ -414,21 +414,21 @@ Each of these anchors matches a boundary: Lookahead anchors: -- <tt>(?=_pat_)</tt>: Positive lookahead assertion: +- <tt>(?=pat)</tt>: Positive lookahead assertion: ensures that the following characters match _pat_, but doesn't include those characters in the matched substring. -- <tt>(?!_pat_)</tt>: Negative lookahead assertion: +- <tt>(?!pat)</tt>: Negative lookahead assertion: ensures that the following characters <i>do not</i> match _pat_, but doesn't include those characters in the matched substring. Lookbehind anchors: -- <tt>(?<=_pat_)</tt>: Positive lookbehind assertion: +- <tt>(?<=pat)</tt>: Positive lookbehind assertion: ensures that the preceding characters match _pat_, but doesn't include those characters in the matched substring. -- <tt>(?<!_pat_)</tt>: Negative lookbehind assertion: +- <tt>(?<!pat)</tt>: Negative lookbehind assertion: ensures that the preceding characters do not match _pat_, but doesn't include those characters in the matched substring. @@ -439,6 +439,10 @@ without including the tags in the match: /(?<=<b>)\w+(?=<\/b>)/.match("Fortune favors the <b>bold</b>.") # => #<MatchData "bold"> +The pattern in lookbehind must be fixed-width. +But top-level alternatives can be of various lengths. +ex. (?<=a|bc) is OK. (?<=aaa(?:b|cd)) is not allowed. + ==== Match-Reset Anchor - <tt>\K</tt>: Match reset: @@ -498,7 +502,7 @@ An added _quantifier_ specifies how many matches are required or allowed: /\w*/.match('x') # => #<MatchData "x"> /\w*/.match('xyz') - # => #<MatchData "yz"> + # => #<MatchData "xyz"> - <tt>+</tt> - Matches one or more times: @@ -570,7 +574,7 @@ A simple regexp has (at most) one match: re.match('1943-02-04').size # => 1 re.match('foo') # => nil -Adding one or more pairs of parentheses, <tt>(_subexpression_)</tt>, +Adding one or more pairs of parentheses, <tt>(subexpression)</tt>, defines _groups_, which may result in multiple matched substrings, called _captures_: @@ -643,8 +647,8 @@ A regexp may contain any number of groups: - For a large number of groups: - - The ordinary <tt>\\_n_</tt> notation applies only for _n_ in range (1..9). - - The <tt>MatchData[_n_]</tt> notation applies for any non-negative _n_. + - The ordinary <tt>\\n</tt> notation applies only for _n_ in range (1..9). + - The <tt>MatchData[n]</tt> notation applies for any non-negative _n_. - <tt>\0</tt> is a special backreference, referring to the entire matched string; it may not be used within the regexp itself, @@ -657,7 +661,7 @@ A regexp may contain any number of groups: As seen above, a capture can be referred to by its number. A capture can also have a name, -prefixed as <tt>?<_name_></tt> or <tt>?'_name_'</tt>, +prefixed as <tt>?<name></tt> or <tt>?'name'</tt>, and the name (symbolized) may be used as an index in <tt>MatchData[]</tt>: md = /\$(?<dollars>\d+)\.(?'cents'\d+)/.match("$3.67") @@ -672,7 +676,7 @@ When a regexp contains a named capture, there are no unnamed captures: /\$(?<dollars>\d+)\.(\d+)/.match("$3.67") # => #<MatchData "$3.67" dollars:"3"> -A named group may be backreferenced as <tt>\k<_name_></tt>: +A named group may be backreferenced as <tt>\k<name></tt>: /(?<vowel>[aeiou]).\k<vowel>.\k<vowel>/.match('ototomy') # => #<MatchData "ototo" vowel:"o"> @@ -709,7 +713,7 @@ Analysis: 1. The leading subexpression <tt>"</tt> in the pattern matches the first character <tt>"</tt> in the target string. -2. The next subexpression <tt>.*</tt> matches the next substring <tt>Quote“</tt> +2. The next subexpression <tt>.*</tt> matches the next substring <tt>Quote"</tt> (including the trailing double-quote). 3. Now there is nothing left in the target string to match the trailing subexpression <tt>"</tt> in the pattern; @@ -728,10 +732,10 @@ see {Atomic Group}[https://www.regular-expressions.info/atomic.html]. ==== Subexpression Calls -As seen above, a backreference number (<tt>\\_n_</tt>) or name (<tt>\k<_name_></tt>) +As seen above, a backreference number (<tt>\\n</tt>) or name (<tt>\k<name></tt>) gives access to a captured _substring_; the corresponding regexp _subexpression_ may also be accessed, -via the number (<tt>\\g<i>n</i></tt>) or name (<tt>\g<_name_></tt>): +via the number n (<tt>\\gn</tt>) or name (<tt>\g<name></tt>): /\A(?<paren>\(\g<paren>*\))*\z/.match('(())') # ^1 @@ -764,12 +768,12 @@ See {Subexpression calls}[https://learnbyexample.github.io/Ruby_Regexp/groupings ==== Conditionals -The conditional construct takes the form <tt>(?(_cond_)_yes_|_no_)</tt>, where: +The conditional construct takes the form <tt>(?(cond)yes|no)</tt>, where: - _cond_ may be a capture number or name. - The match to be applied is _yes_ if _cond_ is captured; otherwise the match to be applied is _no_. -- If not needed, <tt>|_no_</tt> may be omitted. +- If not needed, <tt>|no</tt> may be omitted. Examples: @@ -798,7 +802,7 @@ The absence operator is a special group that matches anything which does _not_ m ==== Unicode Properties -The <tt>/\p{_property_name_}/</tt> construct (with lowercase +p+) +The <tt>/\p{property_name}/</tt> construct (with lowercase +p+) matches characters using a Unicode property name, much like a character class; property +Alpha+ specifies alphabetic characters: @@ -817,7 +821,7 @@ Or by using <tt>\P</tt> (uppercase +P+): /\P{Alpha}/.match('1') # => #<MatchData "1"> /\P{Alpha}/.match('a') # => nil -See {Unicode Properties}[rdoc-ref:regexp/unicode_properties.rdoc] +See {Unicode Properties}[rdoc-ref:language/regexp/unicode_properties.rdoc] for regexps based on the numerous properties. Some commonly-used properties correspond to POSIX bracket expressions: @@ -1029,23 +1033,23 @@ See also {Extended Mode}[rdoc-ref:Regexp@Extended+Mode]. Each of these modifiers sets a mode for the regexp: -- +i+: <tt>/_pattern_/i</tt> sets +- +i+: <tt>/pattern/i</tt> sets {Case-Insensitive Mode}[rdoc-ref:Regexp@Case-Insensitive+Mode]. -- +m+: <tt>/_pattern_/m</tt> sets +- +m+: <tt>/pattern/m</tt> sets {Multiline Mode}[rdoc-ref:Regexp@Multiline+Mode]. -- +x+: <tt>/_pattern_/x</tt> sets +- +x+: <tt>/pattern/x</tt> sets {Extended Mode}[rdoc-ref:Regexp@Extended+Mode]. -- +o+: <tt>/_pattern_/o</tt> sets +- +o+: <tt>/pattern/o</tt> sets {Interpolation Mode}[rdoc-ref:Regexp@Interpolation+Mode]. Any, all, or none of these may be applied. Modifiers +i+, +m+, and +x+ may be applied to subexpressions: -- <tt>(?_modifier_)</tt> turns the mode "on" for ensuing subexpressions -- <tt>(?-_modifier_)</tt> turns the mode "off" for ensuing subexpressions -- <tt>(?_modifier_:_subexp_)</tt> turns the mode "on" for _subexp_ within the group -- <tt>(?-_modifier_:_subexp_)</tt> turns the mode "off" for _subexp_ within the group +- <tt>(?modifier)</tt> turns the mode "on" for ensuing subexpressions +- <tt>(?-modifier)</tt> turns the mode "off" for ensuing subexpressions +- <tt>(?modifier:subexp)</tt> turns the mode "on" for _subexp_ within the group +- <tt>(?-modifier:subexp)</tt> turns the mode "off" for _subexp_ within the group Example: @@ -1162,22 +1166,22 @@ A regular expression containing non-US-ASCII characters is assumed to use the source encoding. This can be overridden with one of the following modifiers. -- <tt>/_pat_/n</tt>: US-ASCII if only containing US-ASCII characters, +- <tt>/pat/n</tt>: US-ASCII if only containing US-ASCII characters, otherwise ASCII-8BIT: /foo/n.encoding # => #<Encoding:US-ASCII> /foo\xff/n.encoding # => #<Encoding:ASCII-8BIT> /foo\x7f/n.encoding # => #<Encoding:US-ASCII> -- <tt>/_pat_/u</tt>: UTF-8 +- <tt>/pat/u</tt>: UTF-8 /foo/u.encoding # => #<Encoding:UTF-8> -- <tt>/_pat_/e</tt>: EUC-JP +- <tt>/pat/e</tt>: EUC-JP /foo/e.encoding # => #<Encoding:EUC-JP> -- <tt>/_pat_/s</tt>: Windows-31J +- <tt>/pat/s</tt>: Windows-31J /foo/s.encoding # => #<Encoding:Windows-31J> @@ -1247,7 +1251,7 @@ the potential vulnerability arising from this is the {regular expression denial- \Regexp matching can apply an optimization to prevent ReDoS attacks. When the optimization is applied, matching time increases linearly (not polynomially or exponentially) -in relation to the input size, and a ReDoS attach is not possible. +in relation to the input size, and a ReDoS attack is not possible. This optimization is applied if the pattern meets these criteria: @@ -1268,13 +1272,13 @@ because the optimization uses memoization (which may invoke large memory consump == References -Read (online PDF books): +Read: -- {Mastering Regular Expressions}[https://ia902508.us.archive.org/10/items/allitebooks-02/Mastering%20Regular%20Expressions%2C%203rd%20Edition.pdf] +- <i>Mastering Regular Expressions</i> by Jeffrey E.F. Friedl. -- {Regular Expressions Cookbook}[https://doc.lagout.org/programmation/Regular%20Expressions/Regular%20Expressions%20Cookbook_%20Detailed%20Solutions%20in%20Eight%20Programming%20Languages%20%282nd%20ed.%29%20%5BGoyvaerts%20%26%20Levithan%202012-09-06%5D.pdf] +- <i>Regular Expressions Cookbook</i> by Jan Goyvaerts & Steven Levithan. -Explore, test (interactive online editor): +Explore, test: -- {Rubular}[https://rubular.com/]. +- {Rubular}[https://rubular.com/]: interactive online editor. diff --git a/doc/command_injection.rdoc b/doc/command_injection.rdoc deleted file mode 100644 index ee33d4a04e..0000000000 --- a/doc/command_injection.rdoc +++ /dev/null @@ -1,37 +0,0 @@ -= Command Injection - -Some Ruby core methods accept string data -that includes text to be executed as a system command. - -They should not be called with unknown or unsanitized commands. - -These methods include: - -- Kernel.exec -- Kernel.spawn -- Kernel.system -- {\`command` (backtick method)}[rdoc-ref:Kernel#`] - (also called by the expression <tt>%x[command]</tt>). -- IO.popen (when called with other than <tt>"-"</tt>). - -Some methods execute a system command only if the given path name starts -with a <tt>|</tt>: - -- Kernel.open(command). -- IO.read(command). -- IO.write(command). -- IO.binread(command). -- IO.binwrite(command). -- IO.readlines(command). -- IO.foreach(command). -- URI.open(command). - -Note that some of these methods do not execute commands when called -from subclass +File+: - -- File.read(path). -- File.write(path). -- File.binread(path). -- File.binwrite(path). -- File.readlines(path). -- File.foreach(path). diff --git a/doc/command_line/environment.md b/doc/command_line/environment.md deleted file mode 100644 index 8f6d595f6c..0000000000 --- a/doc/command_line/environment.md +++ /dev/null @@ -1,174 +0,0 @@ -## Environment - -Certain command-line options affect the execution environment -of the invoked Ruby program. - -### About the Examples - -The examples here use command-line option `-e`, -which passes the Ruby code to be executed on the command line itself: - -```console -$ ruby -e 'puts "Hello, World."' -``` - -### Option `-C` - -The argument to option `-C` specifies a working directory -for the invoked Ruby program; -does not change the working directory for the current process: - -```console -$ basename `pwd` -ruby -$ ruby -C lib -e 'puts File.basename(Dir.pwd)' -lib -$ basename `pwd` -ruby -``` - -Whitespace between the option and its argument may be omitted. - -### Option `-I` - -The argument to option `-I` specifies a directory -to be added to the array in global variable `$LOAD_PATH`; -the option may be given more than once: - -```console -$ pushd /tmp -$ ruby -e 'p $LOAD_PATH.size' -8 -$ ruby -I my_lib -I some_lib -e 'p $LOAD_PATH.size' -10 -$ ruby -I my_lib -I some_lib -e 'p $LOAD_PATH.take(2)' -["/tmp/my_lib", "/tmp/some_lib"] -$ popd -``` - -Whitespace between the option and its argument may be omitted. - -### Option `-r` - -The argument to option `-r` specifies a library to be required -before executing the Ruby program; -the option may be given more than once: - -```console -$ ruby -e 'p defined?(JSON); p defined?(CSV)' -nil -nil -$ ruby -r CSV -r JSON -e 'p defined?(JSON); p defined?(CSV)' -"constant" -"constant" -``` - -Whitespace between the option and its argument may be omitted. - -### Option `-0` - -Option `-0` defines the input record separator `$/` -for the invoked Ruby program. - -The optional argument to the option must be octal digits, -each in the range `0..7`; -these digits are prefixed with digit `0` to form an octal value: - -- If no argument is given, the input record separator is `0x00`. -- If the argument is `0`, the input record separator is `''`; - see {Special Line Separator Values}[rdoc-ref:IO@Special+Line+Separator+Values]. -- If the argument is in range `(1..0377)`, - it becomes the character value of the input record separator `$/`. -- Otherwise, the input record separator is `nil`. - -Examples: - -```console -$ ruby -0 -e 'p $/' -"\x00" -$ ruby -00 -e 'p $/' -"" -$ ruby -012 -e 'p $/' -"\n" -$ ruby -015 -e 'p $/' -"\r" -$ ruby -0377 -e 'p $/' -"\xFF" -$ ruby -0400 -e 'p $/' -nil -``` - -The option may not be separated from its argument by whitespace. - -### Option `-d` - -Some code in (or called by) the Ruby program may include statements or blocks -conditioned by the global variable `$DEBUG` (e.g., `if $DEBUG`); -these commonly write to `$stdout` or `$stderr`. - -The default value for `$DEBUG` is `false`; -option `-d` (or `--debug`) sets it to `true`: - -```console -$ ruby -e 'p $DEBUG' -false -$ ruby -d -e 'p $DEBUG' -true -``` - -### Option '-w' - -Option `-w` (lowercase letter) is equivalent to option `-W1` (uppercase letter). - -### Option `-W` - -Any Ruby code can create a <i>warning message</i> by calling method Kernel#warn; -methods in the Ruby core and standard libraries can also create warning messages. -Such a message may be printed on `$stderr` -(or not, depending on certain settings). - -Option `-W` helps determine whether a particular warning message -will be written, -by setting the initial value of global variable `$-W`: - -- `-W0`: Sets `$-W` to `0` (silent; no warnings). -- `-W1`: Sets `$-W` to `1` (moderate verbosity). -- `-W2`: Sets `$-W` to `2` (high verbosity). -- `-W`: Same as `-W2` (high verbosity). -- Option not given: Same as `-W1` (moderate verbosity). - -The value of `$-W`, in turn, determines which warning messages (if any) -are to be printed to `$stdout` (see Kernel#warn): - -```console -$ ruby -W1 -e 'p $foo' -nil -$ ruby -W2 -e 'p $foo' --e:1: warning: global variable '$foo' not initialized -nil -``` - -Ruby code may also define warnings for certain categories; -these are the default settings for the defined categories: - -```ruby -Warning[:experimental] # => true -Warning[:deprecated] # => false -Warning[:performance] # => false -``` - -They may also be set: - -```ruby -Warning[:experimental] = false -Warning[:deprecated] = true -Warning[:performance] = true -``` - -You can suppress a category by prefixing `no-` to the category name: - -```console -$ ruby -W:no-experimental -e 'p IO::Buffer.new' -#<IO::Buffer> -``` - diff --git a/doc/bug_triaging.rdoc b/doc/contributing/bug_triaging.rdoc index 83fe88cabe..83fe88cabe 100644 --- a/doc/bug_triaging.rdoc +++ b/doc/contributing/bug_triaging.rdoc diff --git a/doc/contributing/building_ruby.md b/doc/contributing/building_ruby.md index c0eafe182f..286bd1f116 100644 --- a/doc/contributing/building_ruby.md +++ b/doc/contributing/building_ruby.md @@ -17,7 +17,7 @@ * [autoconf] - 2.67 or later * [gperf] - 3.1 or later * Usually unneeded; only if you edit some source files using gperf - * ruby - 3.0 or later + * ruby - 3.1 or later * We can upgrade this version to system ruby version of the latest Ubuntu LTS. * git - 2.32 or later @@ -184,7 +184,7 @@ cause build failures. ## Building on Windows The documentation for building on Windows can be found in [the separated -file](../windows.md). +file](../distribution/windows.md). ## More details @@ -260,11 +260,25 @@ This will add launch configurations for debugging Ruby itself by running `test.r ### Compiling for Debugging -You should configure Ruby without optimization and other flags that may -interfere with debugging: +You can compile Ruby with the `RUBY_DEBUG` macro to enable debugging on some +features. One example is debugging object shapes in Ruby with +`RubyVM::Shape.of(object)`. + +Additionally Ruby can be compiled to support the `RUBY_DEBUG` environment +variable to enable debugging on some features. An example is using +`RUBY_DEBUG=gc_stress` to debug GC-related issues. + +There is also support for the `RUBY_DEBUG_LOG` environment variable to log a +lot of information about what the VM is doing, via the `USE_RUBY_DEBUG_LOG` +macro. + +You should also configure Ruby without optimization and other flags that may +interfere with debugging by changing the optimization flags. + +Bringing it all together: ```sh -./configure --enable-debug-env optflags="-O0 -fno-omit-frame-pointer" +./configure cppflags="-DRUBY_DEBUG=1 -DUSE_RUBY_DEBUG_LOG=1" --enable-debug-env optflags="-O0 -fno-omit-frame-pointer" ``` ### Building with Address Sanitizer @@ -293,9 +307,6 @@ RUBY_TEST_TIMEOUT_SCALE=5 SYNTAX_SUGGEST_TIMEOUT=600 make check Please note, however, the following caveats! -* ASAN will not work properly on any currently released version of Ruby; the - necessary support is currently only present on Ruby's master branch (and the - whole test suite passes only as of commit [Revision 9d0a5148]). * Due to [Bug #20243], Clang generates code for threadlocal variables which doesn't work with M:N threading. Thus, it's necessary to disable M:N threading support at build time for now (with the `-DUSE_MN_THREADS=0` diff --git a/doc/contributing/concurrency_guide.md b/doc/contributing/concurrency_guide.md new file mode 100644 index 0000000000..1fb58f7203 --- /dev/null +++ b/doc/contributing/concurrency_guide.md @@ -0,0 +1,154 @@ +# Concurrency Guide + +This is a guide to thinking about concurrency in the cruby source code, whether that's contributing to Ruby +by writing C or by contributing to one of the JITs. This does not touch on native extensions, only the core +language. It will go over: + +* What needs synchronizing? +* How to use the VM lock, and what you can and can't do when you've acquired this lock. +* What you can and can't do when you've acquired other native locks. +* The difference between the VM lock and the GVL. +* What a VM barrier is and when to use it. +* The lock ordering of some important locks. +* How ruby interrupt handling works. +* The timer thread and what it's responsible for. + +## What needs synchronizing? + +Before ractors, only one ruby thread could run at once. That didn't mean you could forget about concurrency issues, though. The timer thread +is a native thread that interacts with other ruby threads and changes some VM internals, so if these changes can be done in parallel by both the timer +thread and a ruby thread, they need to be synchronized. + +When you add ractors to the mix, it gets more complicated. However, ractors allow you to forget about synchronization for non-shareable objects because +they aren't used across ractors. Only one ruby thread can touch the object at once. For shareable objects, they are deeply frozen so there isn't any +mutation on the objects themselves. However, something like reading/writing constants across ractors does need to be synchronized. In this case, ruby threads need to +see a consistent view of the VM. If publishing the update takes 2 steps or even two separate instructions, like in this case, synchronization is required. + +Most synchronization is to protect VM internals. These internals include structures for the thread scheduler on each ractor, the global ractor scheduler, the +coordination between ruby threads and ractors, global tables (for `fstrings`, encodings, symbols and global vars), etc. Anything that can be mutated by a ractor +that can also be read or mutated by another ractor at the same time requires proper synchronization. + +## The VM Lock + +There's only one VM lock and it is for critical sections that can only be entered by one ractor at a time. +Without ractors, the VM lock is useless. It does not stop all ractors from running, as ractors can run +without trying to acquire this lock. If you're updating global (shared) data between ractors and aren't using +atomics, you need to use a lock and this is a convenient one to use. Unlike other locks, you can allocate ruby-managed +memory with it held. When you take the VM lock, there are things you can and can't do during your critical section: + +You can (as long as no other locks are also held before the VM lock): + +* Create ruby objects, call `ruby_xmalloc`, etc. + +You can't: + +* Context switch to another ruby thread or ractor. This is important, as many things can cause ruby-level context switches including: + + * Calling any ruby method through, for example, `rb_funcall`. If you execute ruby code, a context switch could happen. + This also applies to ruby methods defined in C, as they can be redefined in Ruby. Things that call ruby methods such as + `rb_obj_respond_to` are also disallowed. + + * Calling `rb_raise`. This will call `initialize` on the new exception object. With the VM lock + held, nothing you call should be able to raise an exception. `NoMemoryError` is allowed, however. + + * Calling `rb_nogvl` or a ruby-level mechanism that can context switch like `rb_mutex_lock`. + + * Enter any blocking operation managed by ruby. This will context switch to another ruby thread using `rb_nogvl` or + something equivalent. A blocking operation is one that blocks the thread's progress, such as `sleep` or `IO#read`. + +Internally, the VM lock is the `vm->ractor.sync.lock`. + +You need to be on a ruby thread to take the VM lock. You also can't take it inside any functions that could be called during sweeping, as MMTK sweeps +on another thread and you need a valid `ec` to grab the lock. For this same reason (among others), you can't take it from the timer thread either. + +## Other Locks + +All native locks that aren't the VM lock share a more strict set of rules for what's allowed during the critical section. By native locks, we mean +anything that uses `rb_native_mutex_lock`. Some important locks include the `interrupt_lock`, the ractor scheduling lock (protects global scheduling data structures), +the thread scheduling lock (local to each ractor, protects per-ractor scheduling data structures) and the ractor lock (local to each ractor, protects ractor data structures). + +When you acquire one of these locks, + +You can: + +* Allocate memory though non-ruby allocation such as raw `malloc` or the standard library. But be careful, some functions like `strdup` use +ruby allocation through the use of macros! + +* Use `ccan` lists, as they don't allocate. + +* Do the usual things like set variables or struct fields, manipulate linked lists, signal condition variables etc. + +You can't: + +* Allocate ruby-managed memory. This includes creating ruby objects or using `ruby_xmalloc` or `st_insert`. The reason this +is disallowed is if that allocation causes a GC, then all other ruby threads must join a VM barrier as soon as possible +(when they next check interrupts or acquire the VM lock). This is so that no other ractors are running during GC. If a ruby thread +is waiting (blocked) on this same native lock, it can't join the barrier and a deadlock occurs because the barrier will never finish. + +* Raise exceptions. You also can't use `EC_JUMP_TAG` if it jumps out of the critical section. + +* Context switch. See the `VM Lock` section for more info. + +## Difference Between VM Lock and GVL + +The VM Lock is a particular lock in the source code. There is only one VM Lock. The GVL, on the other hand, is more of a combination of locks. +It is "acquired" when a ruby thread is about to run or is running. Since many ruby threads can run at the same time if they're in different ractors, +there are many GVLs (1 per `SNT` + 1 for the main ractor). It can no longer be thought of as a "Global VM Lock" like it once was before ractors. + +## VM Barriers + +Sometimes, taking the VM Lock isn't enough and you need a guarantee that all ractors have stopped. This happens when running `GC`, for instance. +To get a barrier, you take the VM Lock and call `rb_vm_barrier()`. For the duration that the VM lock is held, no other ractors will be running. It's not used +often as taking a barrier slows ractor performance down considerably, but it's useful to know about and is sometimes the only solution. + +## Lock Orderings + +It's a good idea to not hold more than 2 locks at once on the same thread. Locking multiple locks can introduce deadlocks, so do it with care. When locking +multiple locks at once, follow an ordering that is consistent across the program, otherwise you can introduce deadlocks. Here are the orderings of some important locks: + +* VM lock before ractor_sched_lock +* thread_sched_lock before ractor_sched_lock +* interrupt_lock before timer_th.waiting_lock +* timer_th.waiting_lock before ractor_sched_lock + +These orderings are subject to change, so check the source if you're not sure. On top of this: + +* During each `ubf` (unblock) function, the VM lock can be taken around it in some circumstances. This happens during VM shutdown, for example. +See the "Interrupt Handling" section for more details. + +## Ruby Interrupt Handling + +When the VM runs ruby code, ruby's threads intermittently check ruby-level interrupts. These software interrupts +are for various things in ruby and they can be set by other ruby threads or the timer thread. + +* Ruby threads check when they should give up their timeslice. The native thread switches to another ruby thread when their time is up. +* The timer thread sends a "trap" interrupt to the main thread if any ruby-level signal handlers are pending. +* Ruby threads can have other ruby threads run tasks for them by sending them an interrupt. For instance, ractors send +the main thread an interrupt when they need to `require` a file so that it's done on the main thread. They wait for the +main thread's result. +* During VM shutdown, a "terminate" interrupt is sent to all ractor main threads top stop them asap. +* When calling `Thread#raise`, the caller sends an interrupt to that thread telling it which exception to raise. +* Unlocking a mutex sends the next waiter (if any) an interrupt telling it to grab the lock. +* Signalling or broadcasting on a condition variable tells the waiter(s) to wake up. + +This isn't a complete list. + +When sending an interrupt to a ruby thread, the ruby thread can be blocked. For example, it could be in the middle of a `TCPSocket#read` call. If so, +the receiving thread's `ubf` (unblock function) gets called from the thread (ruby thread or timer thread) that sent the interrupt. +Each ruby thread has a `ubf` that is set when it enters a blocking operation and is unset after returning from it. By default, this `ubf` function sends a +`SIGVTALRM` to the receiving thread to try to unblock it from the kernel so it can check its interrupts. There are other `ubfs` that +aren't associated with a syscall, such as when calling `Ractor#join` or `sleep`. All `ubfs` are called with the `interrupt_lock` held, +so take that into account when using locks inside `ubfs`. + +Remember, `ubfs` can be called from the timer thread so you cannot assume an `ec` inside them. The `ec` (execution context) is only set on ruby threads. + +## The Timer Thread + +The timer thread has a few functions. They are: + +* Send interrupts to ruby threads that have run for their whole timeslice. +* Wake up M:N ruby threads (threads in non-main ractors) blocked on IO or after a specified timeout. This +uses `kqueue` or `epoll`, depending on the OS, to receive IO events on behalf of the threads. +* Continue calling the `SIGVTARLM` signal if a thread is still blocked on a syscall after the first `ubf` call. +* Signal native threads (`SNT`) waiting on a ractor if there are ractors waiting in the global run queue. +* Create more `SNT`s if some are blocked, like on IO or on `Ractor#join`. diff --git a/doc/contributing/documentation_guide.md b/doc/contributing/documentation_guide.md index deb17a793a..4b1e2ac9ad 100644 --- a/doc/contributing/documentation_guide.md +++ b/doc/contributing/documentation_guide.md @@ -7,7 +7,7 @@ in the Ruby core and in the Ruby standard library. ## Generating documentation Most Ruby documentation lives in the source files and is written in -[RDoc format](rdoc-ref:RDoc::MarkupReference). +[RDoc format](https://ruby.github.io/rdoc/RDoc/MarkupReference.html). Some pages live under the `doc` folder and can be written in either `.rdoc` or `.md` format, determined by the file extension. @@ -44,14 +44,13 @@ Use your judgment about what the user needs to know. - Group sentences into (ideally short) paragraphs, each covering a single topic. - Organize material with - [headings](rdoc-ref:RDoc::MarkupReference@Headings). + [headings]. - Refer to authoritative and relevant sources using - [links](rdoc-ref:RDoc::MarkupReference@Links). + [links](https://ruby.github.io/rdoc/RDoc/MarkupReference.html#class-RDoc::MarkupReference-label-Links). - Use simple verb tenses: simple present, simple past, simple future. - Use simple sentence structure, not compound or complex structure. - Avoid: - - Excessive comma-separated phrases; - consider a [list](rdoc-ref:RDoc::MarkupReference@Lists). + - Excessive comma-separated phrases; consider a [list]. - Idioms and culture-specific references. - Overuse of headings. - Using US-ASCII-incompatible characters in C source files; @@ -62,7 +61,7 @@ Use your judgment about what the user needs to know. Use only US-ASCII-compatible characters in a C source file. (If you use other characters, the Ruby CI will gently let you know.) -If want to put ASCII-incompatible characters into the documentation +If you want to put ASCII-incompatible characters into the documentation for a C-coded class, module, or method, there are workarounds involving new files `doc/*.rdoc`: @@ -75,7 +74,7 @@ involving new files `doc/*.rdoc`: class Foo; end ``` -- Similarly, for module `Bar` (defined in file `bar.c`, +- Similarly, for module `Bar` (defined in file `bar.c`), create file `doc/bar.rdoc`, declare `module Bar; end`, and place the module documentation above that declaration: @@ -110,7 +109,7 @@ involving new files `doc/*.rdoc`: Ruby is documented using RDoc. For information on \RDoc syntax and features, see the -[RDoc Markup Reference](rdoc-ref:RDoc::MarkupReference). +[RDoc Markup Reference](https://ruby.github.io/rdoc/RDoc/MarkupReference.html). ### Output from `irb` @@ -129,22 +128,21 @@ a #=> [2, 3, 1] ### Headings -Organize a long discussion for a class or module with [headings](rdoc-ref:RDoc::MarkupReference@Headings). +Organize a long discussion for a class or module with [headings]. Do not use formal headings in the documentation for a method or constant. In the rare case where heading-like structures are needed within the documentation for a method or constant, use -[bold text](rdoc-ref:RDoc::MarkupReference@Bold) +[bold text](https://ruby.github.io/rdoc/RDoc/MarkupReference.html#class-RDoc::MarkupReference-label-Bold) as pseudo-headings. ### Blank Lines A blank line begins a new paragraph. -A [code block](rdoc-ref:RDoc::MarkupReference@Code+Blocks) -or [list](rdoc-ref:RDoc::MarkupReference@Lists) -should be preceded by and followed by a blank line. +A [code block](https://ruby.github.io/rdoc/RDoc/MarkupReference.html#class-RDoc::MarkupReference-label-Code+Blocks) +or [list] should be preceded by and followed by a blank line. This is unnecessary for the HTML output, but helps in the `ri` output. ### \Method Names @@ -164,7 +162,7 @@ For a method name in text: Code or commands embedded in running text (i.e., not in a code block) should marked up as -[monofont](rdoc-ref:RDoc::MarkupReference@Monofont). +[monofont]. Code that is a simple string should include the quote marks. @@ -214,7 +212,7 @@ refers to the current document (e.g., _Float_ in the documentation for class Float). In this case you may consider forcing the name to -[monofont](rdoc-ref:RDoc::MarkupReference@Monofont), +[monofont], which suppresses auto-linking, and also emphasizes that the word is a class name: ```rdoc @@ -267,16 +265,16 @@ and _never_ when referring to the class itself. When writing an explicit link, follow these guidelines. -#### +rdoc-ref+ Scheme +#### `rdoc-ref` Scheme -Use the +rdoc-ref+ scheme for: +Use the `rdoc-ref` scheme for: - A link in core documentation to other core documentation. - A link in core documentation to documentation in a standard library package. - A link in a standard library package to other documentation in that same standard library package. -See section "+rdoc-ref+ Scheme" in {Links}[rdoc-ref:RDoc::MarkupReference@Links]. +See section "`rdoc-ref` Scheme" in [links]. #### URL-Based Link @@ -286,7 +284,7 @@ Use a full URL-based link for: - A link in standard library documentation to documentation in a different standard library package. -Doing so ensures that the link will valid even when the package documentation +Doing so ensures that the link will be valid even when the package documentation is built independently (separately from the core documentation). The link should lead to a target in https://docs.ruby-lang.org/en/master/. @@ -296,10 +294,10 @@ Also use a full URL-based link for a link to an off-site document. ### Variable Names The name of a variable (as specified in its call-seq) should be marked up as -[monofont](rdoc-ref:RDoc::MarkupReference@Monofont). +[monofont]. Also, use monofont text for the name of a transient variable -(i.e., one defined and used only in the discussion, such as +n+). +(i.e., one defined and used only in the discussion, such as `n`). ### HTML Tags @@ -314,9 +312,9 @@ In particular, avoid building tables with HTML tags Alternatives: -- A {verbatim text block}[rdoc-ref:RDoc::MarkupReference@Verbatim+Text+Blocks], +- A {verbatim text block}[https://ruby.github.io/rdoc/RDoc/MarkupReference.html#class-RDoc::MarkupReference-label-Verbatim+Text+Blocks], using spaces and punctuation to format the text; - note that {text markup}[rdoc-ref:RDoc::MarkupReference@Text+Markup] + note that {text markup}[https://ruby.github.io/rdoc/RDoc/MarkupReference.html#class-RDoc::MarkupReference-label-Text+Markup] will not be honored: - Example {source}[https://github.com/ruby/ruby/blob/34d802f32f00df1ac0220b62f72605827c16bad8/file.c#L6570-L6596]. @@ -328,6 +326,16 @@ Alternatives: - Example {source}[https://github.com/ruby/ruby/blob/34d802f32f00df1ac0220b62f72605827c16bad8/doc/contributing/glossary.md?plain=1]. - Corresponding {output}[https://docs.ruby-lang.org/en/master/contributing/glossary_md.html]. +### Languages in Examples + +For symbols and strings in documentation examples: + +- Prefer \English in \English documentation: <tt>'Hello'</tt>. +- Prefer Japanese in Japanese documentation: <tt>'こんにちは'</tt>. +- If a second language is needed (as, for example, characters with different byte-sizes), + prefer Japanese in \English documentation and \English in Japanese documentation. +- Use other languages examples only as necessary: see String#capitalize. + ## Documenting Classes and Modules The general structure of the class or module documentation should be: @@ -356,8 +364,7 @@ Guidelines: - The section title is `What's Here`. - Consider listing the parent class and any included modules; consider - [links](rdoc-ref:RDoc::MarkupReference@Links) - to their "What's Here" sections if those exist. + [links] to their "What's Here" sections if those exist. - All methods mentioned in the left-pane table of contents should be listed (including any methods extended from another class). - Attributes (which are not included in the TOC) may also be listed. @@ -369,7 +376,7 @@ Guidelines: (and do not list the aliases separately). - Check the rendered documentation to determine whether \RDoc has recognized the method and linked to it; if not, manually insert a - [link](rdoc-ref:RDoc::MarkupReference@Links). + [link](https://ruby.github.io/rdoc/RDoc/MarkupReference.html#class-RDoc::MarkupReference-label-Links). - If there are numerous entries, consider grouping them into subsections with headings. - If there are more than a few such subsections, @@ -395,7 +402,7 @@ For methods written in Ruby, \RDoc documents the calling sequence automatically. For methods written in C, \RDoc cannot determine what arguments the method accepts, so those need to be documented using \RDoc directive -[`call-seq:`](rdoc-ref:RDoc::MarkupReference@Directives+for+Method+Documentation). +[`call-seq:`](https://ruby.github.io/rdoc/RDoc/MarkupReference.html#class-RDoc::MarkupReference-label-Directives+for+Method+Documentation). For a singleton method, use the form: @@ -484,10 +491,10 @@ Return types: - If the method can return multiple different types, separate the types with "or" and, if necessary, commas. -- If the method can return multiple types, use +object+. -- If the method returns the receiver, use +self+. +- If the method can return multiple types, use `object`. +- If the method returns the receiver, use `self`. - If the method returns an object of the same class, - prefix `new_` if an only if the object is not +self+; + prefix `new_` if and only if the object is not `self`; example: `new_array`. Aliases: @@ -570,7 +577,7 @@ argument passed if it is not obvious, not explicitly mentioned in the details, and not implicitly shown in the examples. If there is more than one argument or block argument, use a -[labeled list](rdoc-ref:RDoc::MarkupReference@Labeled+Lists). +[labeled list](https://ruby.github.io/rdoc/RDoc/MarkupReference.html#class-RDoc::MarkupReference-label-Labeled+Lists). ### Corner Cases and Exceptions @@ -591,7 +598,7 @@ mention `Hash#fetch` as a related method, and `Hash#merge` might mention `Hash#merge!` as a related method. - Consider which methods may be related - to the current method, and if you think the reader would benefit it, + to the current method, and if you think the reader would benefit from it, at the end of the method documentation, add a line starting with "Related: " (e.g. "Related: #fetch."). - Don't list more than three related methods. @@ -600,7 +607,7 @@ mention `Hash#fetch` as a related method, and `Hash#merge` might mention - Consider adding: - A phrase suggesting how the related method is similar to, - or different from,the current method. + or different from, the current method. See an example at Time#getutc. - Example code that illustrates the similarities and differences. See examples at Time#ctime, Time#inspect, Time#to_s. @@ -610,3 +617,7 @@ mention `Hash#fetch` as a related method, and `Hash#merge` might mention For methods that accept multiple argument types, in some cases it can be useful to document the different argument types separately. It's best to use a separate paragraph for each case you are discussing. + +[headings]: https://ruby.github.io/rdoc/RDoc/MarkupReference.html#class-RDoc::MarkupReference-label-Headings +[list]: https://ruby.github.io/rdoc/RDoc/MarkupReference.html#class-RDoc::MarkupReference-label-Lists +[monofont]: https://ruby.github.io/rdoc/RDoc/MarkupReference.html#class-RDoc::MarkupReference-label-Monofont diff --git a/doc/dtrace_probes.rdoc b/doc/contributing/dtrace_probes.rdoc index 1b20597ab4..1b20597ab4 100644 --- a/doc/dtrace_probes.rdoc +++ b/doc/contributing/dtrace_probes.rdoc diff --git a/doc/contributing/glossary.md b/doc/contributing/glossary.md index 86c6671fbd..3ec9796147 100644 --- a/doc/contributing/glossary.md +++ b/doc/contributing/glossary.md @@ -4,14 +4,17 @@ Just a list of acronyms I've run across in the Ruby source code and their meanin | Term | Definition | | --- | -----------| +| `bmethod` | Method defined by `define_method() {}` (a Block that runs as a Method). | | `BIN` | Basic Instruction Name. Used as a macro to reference the YARV instruction. Converts pop into YARVINSN_pop. | | `bop` | Basic Operator. Relates to methods like `Integer` plus and minus which can be optimized as long as they haven't been redefined. | | `cc` | Call Cache. An inline cache structure for the call site. Stored in the `cd` | | `cd` | Call Data. A data structure that points at the `ci` and the `cc`. `iseq` objects points at the `cd`, and access call information and call caches via this structure | +| CFG | Control Flow Graph. Representation of the program where all control-flow and data dependencies have been made explicit by unrolling the stack and local variables. | | `cfp`| Control Frame Pointer. Represents a Ruby stack frame. Calling a method pushes a new frame (cfp), returning pops a frame. Points at the `pc`, `sp`, `ep`, and the corresponding `iseq`| -| `ci` | Call Information. Refers to an `rb_callinfo` struct. Contains call information about the call site, including number of parameters to be passed, whether it they are keyword arguments or not, etc. Used in conjunction with the `cc` and `cd`. | +| `ci` | Call Information. Refers to an `rb_callinfo` struct. Contains call information about the call site, including number of parameters to be passed, whether they are keyword arguments or not, etc. Used in conjunction with the `cc` and `cd`. | +| `cme` | Callable Method Entry. Refers to the `rb_callable_method_entry_t` struct, the internal representation of a Ruby method that has `defined_class` and `owner` set and is ready for dispatch. | | `cref` | Class reference. A structure pointing to the class reference where `klass_or_self`, visibility scope, and refinements are stored. It also stores a pointer to the next class in the hierarchy referenced by `rb_cref_struct * next`. The Class reference is lexically scoped. | -| CRuby | Implementation of Ruby written in C | +| CRuby | Reference implementation of Ruby written in C | | `cvar` | Class Variable. Refers to a Ruby class variable like `@@foo` | | `dvar` | Dynamic Variable. Used by the parser to refer to local variables that are defined outside of the current lexical scope. For example `def foo; bar = 1; -> { p bar }; end` the "bar" inside the block is a `dvar` | | `ec` | Execution Context. The top level VM context, points at the current `cfp` | @@ -25,16 +28,20 @@ Just a list of acronyms I've run across in the Ruby source code and their meanin | `insns` | Instructions. Usually an array of YARV instructions. | | `ivar` | Instance Variable. Refers to a Ruby instance variable like `@foo` | | `imemo` | Internal Memo. A tagged struct whose memory is managed by Ruby's GC, but contains internal information and isn't meant to be exposed to Ruby programs. Contains various information depending on the type. See the `imemo_type` enum for different types. | +| `IVC` | Instance Variable Cache. Cache specifically for instance variable access | | JIT | Just In Time compiler | | `lep` | Local Environment Pointer. An `ep` which is tagged `VM_ENV_FLAG_LOCAL`. Usually this is the `ep` of a method (rather than a block, whose `ep` isn't "local") | | `local` | Local. Refers to a local variable. | -| `me` | Method Entry. Refers to an `rb_method_entry_t` struct, the internal representation of a Ruby method. +| `me` | Method Entry. Refers to an `rb_method_entry_t` struct, the internal representation of a Ruby method. | | MRI | Matz's Ruby Implementation | | `pc` | Program Counter. Usually the instruction that will be executed _next_ by the VM. Pointed to by the `cfp` and incremented by the VM | +| `snt` | Shared Native Thread. OS thread on which many ruby threads can run. Ruby threads from different ractors can even run on the same SNT. Ruby threads can switch SNTs when they context switch. SNTs are used in the M:N threading model. By default, non-main ractors use this model. +| `dnt` | Dedicated Native Thread. OS thread on which only one ruby thread can run. The ruby thread always runs on that same OS thread. DNTs are used in the 1:1 threading model. By default, the main ractor uses this model. | `sp` | Stack Pointer. The top of the stack. The VM executes instructions in the `iseq` and instructions will push and pop values on the stack. The VM updates the `sp` on the `cfp` to point at the top of the stack| +| ST table | ST table is the main C implementation of a hash (smaller Ruby hashes may be backed by AR tables). | | `svar` | Special Variable. Refers to special local variables like `$~` and `$_`. See the `getspecial` instruction in `insns.def` | | `VALUE` | VALUE is a pointer to a ruby object from the Ruby C code. | -| VM | Virtual Machine. In MRI's case YARV (Yet Another Ruby VM) +| VM | Virtual Machine. In MRI's case YARV (Yet Another Ruby VM) | WB | Write Barrier. To do with GC write barriers | | WC | Wild Card. As seen in instructions like `getlocal_WC_0`. It means this instruction takes a "wild card" for the parameter (in this case an index for a local) | | YARV | Yet Another Ruby VM. The virtual machine that CRuby uses | diff --git a/doc/contributing/making_changes_to_stdlibs.md b/doc/contributing/making_changes_to_stdlibs.md index 721ba84289..2ceb2e6075 100644 --- a/doc/contributing/making_changes_to_stdlibs.md +++ b/doc/contributing/making_changes_to_stdlibs.md @@ -4,7 +4,7 @@ Everything in the [lib](https://github.com/ruby/ruby/tree/master/lib) directory If you'd like to make contributions to standard libraries, do so in the standalone repositories, and the changes will be automatically mirrored into the Ruby repository. -For example, CSV lives in [a separate repository](https://github.com/ruby/csv) and is mirrored into [Ruby](https://github.com/ruby/ruby/tree/master/lib/csv). +For example, ERB lives in [a separate repository](https://github.com/ruby/erb) and is mirrored into [Ruby](https://github.com/ruby/ruby/tree/master/lib/erb). ## Maintainers @@ -45,5 +45,5 @@ bundle exec rake test TEST="test/test_foo.rb" To run a single test case: ```shell -bundle exec rake test TEST="test/test_foo.rb" TESTOPS="--name=/test_mytest/" +bundle exec rake test TEST="test/test_foo.rb" TESTOPTS="--name=/test_mytest/" ``` diff --git a/doc/memory_view.md b/doc/contributing/memory_view.md index 0b1369163d..0b1369163d 100644 --- a/doc/memory_view.md +++ b/doc/contributing/memory_view.md diff --git a/doc/contributing/vm_stack_and_frames.md b/doc/contributing/vm_stack_and_frames.md new file mode 100644 index 0000000000..c7dc59db16 --- /dev/null +++ b/doc/contributing/vm_stack_and_frames.md @@ -0,0 +1,163 @@ +# Ruby VM Stack and Frame Layout + +This document explains the Ruby VM stack architecture, including how the value +stack (SP) and control frames (CFP) share a single contiguous memory region, +and how individual frames are structured. + +## VM Stack Architecture + +The Ruby VM uses a single contiguous stack (`ec->vm_stack`) with two different +regions growing toward each other. Understanding this requires distinguishing +the overall architecture (how CFPs and values share one stack) from individual +frame internals (how values are organized for one single frame). + +```text +High addresses (ec->vm_stack + ec->vm_stack_size) + ↓ + [CFP region starts here] ← RUBY_VM_END_CONTROL_FRAME(ec) + [CFP - 1] New frame pushed here (grows downward) + [CFP - 2] Another frame + ... + + (Unused space - stack overflow when they meet) + + ... Value stack grows UP toward higher addresses + [SP + n] Values pushed here + [ec->cfp->sp] Current executing frame's stack pointer + ↑ +Low addresses (ec->vm_stack) +``` + +The "unused space" represents free space available for new frames and values. When this gap closes (CFP meets SP), stack overflow occurs. + +### Stack Growth Directions + +**Control Frames (CFP):** + +- Start at `ec->vm_stack + ec->vm_stack_size` (high addresses) +- Grow **downward** toward lower addresses as frames are pushed +- Each new frame is allocated at `cfp - 1` (lower address) +- The `rb_control_frame_t` structure itself moves downward + +**Value Stack (SP):** + +- Starts at `ec->vm_stack` (low addresses) +- Grows **upward** toward higher addresses as values are pushed +- Each frame's `cfp->sp` points to the top of its value stack + +### Stack Overflow + +When recursive calls push too many frames, CFP grows downward until it collides +with SP growing upward. The VM detects this with `CHECK_VM_STACK_OVERFLOW0`, +which computes `const rb_control_frame_struct *bound = (void *)&sp[margin];` +and raises if `cfp <= &bound[1]`. + +## Understanding Individual Frame Value Stacks + +Each frame has its own portion of the overall VM stack, called its "VM value stack" +or simply "value stack". This space is pre-allocated when the frame is created, +with size determined by: + +- `local_size` - space for local variables +- `stack_max` - maximum depth for temporary values during execution + +The frame's value stack grows upward from its base (where self/arguments/locals +live) toward `cfp->sp` (the current top of temporary values). + +## Visualizing How Frames Fit in the VM Stack + +The left side shows the overall VM stack with CFP metadata separated from frame +values. The right side zooms into one frame's value region, revealing its internal +structure. + +```text +Overall VM Stack (ec->vm_stack): Zooming into Frame 2's value stack: + +High addr (vm_stack + vm_stack_size) High addr (cfp->sp) + ↓ ┌ + [CFP 1 metadata] │ [Temporaries] + [CFP 2 metadata] ─────────┐ │ [Env: Flags/Block/CME] ← cfp->ep + [CFP 3 metadata] │ │ [Locals] + ──────────────── │ ┌─┤ [Arguments] + (unused space) │ │ │ [self] + ──────────────── │ │ └ + [Frame 3 values] │ │ Low addr (frame base) + [Frame 2 values] <────────┴───────┘ + [Frame 1 values] + ↑ +Low addr (vm_stack) +``` + +## Examining a Single Frame's Value Stack + +Now let's walk through a concrete Ruby program to see how a single frame's +value stack is structured internally: + +```ruby +def foo(x, y) + z = x.casecmp(y) +end + +foo(:one, :two) +``` + +First, after arguments are evaluated and right before the `send` to `foo`: + +```text + ┌────────────┐ + putself │ :two │ + putobject :one 0x2 ├────────────┤ + putobject :two │ :one │ +► send <:foo, argc:2> 0x1 ├────────────┤ + leave │ self │ + 0x0 └────────────┘ +``` + +The `put*` instructions have pushed 3 items onto the stack. It's now time to +add a new control frame for `foo`. The following is the shape of the stack +after one instruction in `foo`: + +```text + cfp->sp=0x8 at this point. + 0x8 ┌────────────┐◄──Stack space for temporaries + │ :one │ live above the environment. + 0x7 ├────────────┤ + getlocal x@0 │ < flags > │ foo's rb_control_frame_t +► getlocal y@1 0x6 ├────────────┤◄──has cfp->ep=0x6 + send <:casecmp, argc:1> │ <no block> │ + dup 0x5 ├────────────┤ The flags, block, and CME triple + setlocal z@2 │ <CME: foo> │ (VM_ENV_DATA_SIZE) form an + leave 0x4 ├────────────┤ environment. They can be used to + │ z (nil) │ figure out what local variables + 0x3 ├────────────┤ are below them. + │ :two │ + 0x2 ├────────────┤ Notice how the arguments, now + │ :one │ locals, never moved. This layout + 0x1 ├────────────┤ allows for argument transfer + │ self │ without copying. + 0x0 └────────────┘ +``` + +Given that locals have lower address than `cfp->ep`, it makes sense then that +`getlocal` in `insns.def` has `val = *(vm_get_ep(GET_EP(), level) - idx);`. +When accessing variables in the immediate scope, where `level=0`, it's +essentially `val = cfp->ep[-idx];`. + +Note that this EP-relative index has a different basis than the index that comes +after "@" in disassembly listings. The "@" index is relative to the 0th local +(`x` in this case). + +### Q&A + +Q: It seems that the receiver is always at an offset relative to EP, + like locals. Couldn't we use EP to access it instead of using `cfp->self`? + +A: Not all calls put the `self` in the callee on the stack. Two + examples are `Proc#call`, where the receiver is the Proc object, but `self` + inside the callee is `Proc#receiver`, and `yield`, where the receiver isn't + pushed onto the stack before the arguments. + +Q: Why have `cfp->ep` when it seems that everything is below `cfp->sp`? + +A: In the example, `cfp->ep` points to the stack, but it can also point to the + GC heap. Blocks can capture and evacuate their environment to the heap. diff --git a/doc/distribution.md b/doc/distribution/distribution.md index 5a4d51da6f..164e1b7109 100644 --- a/doc/distribution.md +++ b/doc/distribution/distribution.md @@ -8,7 +8,7 @@ This document outlines the expected way to distribute Ruby, with a specific focu The tarball for official releases is created by the release manager. The release manager uploads the tarball to the [Ruby website](https://www.ruby-lang.org/en/downloads/). -Downstream distributors should use the official release tarballs as part of their build process. This ensures that the tarball is created in a consistent way, and that the tarball is crytographically verified. +Downstream distributors should use the official release tarballs as part of their build process. This ensures that the tarball is created in a consistent way, and that the tarball is cryptographically verified. ### Using the nightly tarball for testing diff --git a/doc/windows.md b/doc/distribution/windows.md index cc0fd3f138..26a727d7ad 100644 --- a/doc/windows.md +++ b/doc/distribution/windows.md @@ -17,6 +17,7 @@ editor. Ruby core development can be done either in Windows `cmd` like: ```batch +ridk install ridk enable ucrt64 pacman -S --needed %MINGW_PACKAGE_PREFIX%-openssl %MINGW_PACKAGE_PREFIX%-libyaml %MINGW_PACKAGE_PREFIX%-libffi @@ -37,6 +38,7 @@ make or in MSYS2 `bash` like: ```bash +ridk install ridk enable ucrt64 bash @@ -76,14 +78,39 @@ sh ../../ruby/configure -C --disable-install-doc --with-opt-dir=C:\Users\usernam x64. The minimum requirement is here: - * VC++/MSVC on VS 2017/2019 version build tools. - * Visual Studio 2022 17.13.x is broken. see https://bugs.ruby-lang.org/issues/21167 + * VC++/MSVC on VS 2017/2019/2022 version build tools. * Windows 10/11 SDK - * 10.0.26100 is broken, 10.0.22621 is recommended. see https://bugs.ruby-lang.org/issues/21255 -3. Please set environment variable `INCLUDE`, `LIB`, `PATH` - to run required commands properly from the command line. - These are set properly by `vcvarall*.bat` usually. + You can install Visual Studio Build Tools with `winget`. + `win32\install-buildtools.cmd` is a batch file to install the + minimum requirements excluding the IDE etc. + +3. Please set environment variable `INCLUDE`, `LIB`, `PATH` to run + required commands properly from the command line. These are set + properly by `vsdevcmd.bat` or `vcvarall*.bat` usually. You can run + the following command to set them in your command line. + + To native build: + + ``` + cmd /k win32\vssetup.cmd + ``` + + To cross build arm64 binary: + + ``` + cmd /k win32\vssetup.cmd -arch=arm64 + ``` + + To cross build x64 binary: + + ``` + cmd /k win32\vssetup.cmd -arch=x64 + ``` + + This batch file is a wrapper of `vsdevcmd.bat` and options are + passed to it as-is. `win32\vssetup.cmd -help` for other command + line options. **Note** building ruby requires following commands. @@ -95,7 +122,7 @@ sh ../../ruby/configure -C --disable-install-doc --with-opt-dir=C:\Users\usernam 4. If you want to build from GIT source, following commands are required. * `git` - * `ruby` 3.0 or later + * `ruby` 3.1 or later You can use [scoop](https://scoop.sh/) to install them like: @@ -258,6 +285,7 @@ Any icon files(`*.ico`) in the build directory, directories specified with _icondirs_ make variable and `win32` directory under the ruby source directory will be included in DLL or executable files, according to their base names. + $(RUBY_INSTALL_NAME).ico or ruby.ico --> $(RUBY_INSTALL_NAME).exe $(RUBYW_INSTALL_NAME).ico or rubyw.ico --> $(RUBYW_INSTALL_NAME).exe the others --> $(RUBY_SO_NAME).dll diff --git a/doc/examples/files.rdoc b/doc/examples/files.rdoc index f736132770..cb400c81be 100644 --- a/doc/examples/files.rdoc +++ b/doc/examples/files.rdoc @@ -7,8 +7,8 @@ text = <<~EOT Fifth line EOT -# Russian text. -russian = "\u{442 435 441 442}" # => "тест" +# Japanese text. +japanese = 'こんにちは' # Binary data. data = "\u9990\u9991\u9992\u9993\u9994" @@ -16,8 +16,8 @@ data = "\u9990\u9991\u9992\u9993\u9994" # Text file. File.write('t.txt', text) -# File with Russian text. -File.write('t.rus', russian) +# File with Japanese text. +File.write('t.ja', japanese) # File with binary data. f = File.new('t.dat', 'wb:UTF-16') diff --git a/doc/extension.ja.rdoc b/doc/extension.ja.rdoc index 2f7856f3d4..381b94a230 100644 --- a/doc/extension.ja.rdoc +++ b/doc/extension.ja.rdoc @@ -1,5 +1,7 @@ # extension.ja.rdoc - -*- RDoc -*- created at: Mon Aug 7 16:45:54 JST 1995 +{English}[rdoc-ref:extension.rdoc] + = Rubyの拡張ライブラリの作り方 Rubyの拡張ライブラリの作り方を説明します． diff --git a/doc/extension.rdoc b/doc/extension.rdoc index ba59d107ab..18dc5817d4 100644 --- a/doc/extension.rdoc +++ b/doc/extension.rdoc @@ -1,5 +1,7 @@ # extension.rdoc - -*- RDoc -*- created at: Mon Aug 7 16:45:54 JST 1995 +{日本語}[rdoc-ref:extension.ja.rdoc] + = Creating extension libraries for Ruby This document explains how to make extension libraries for Ruby. @@ -2047,7 +2049,7 @@ the <code>*_kw</code> functions introduced in Ruby 2.7. #define rb_proc_call_with_block_kw(p, c, v, b, kw) rb_proc_call_with_block(p, c, v, b) #define rb_method_call_kw(c, v, m, kw) rb_method_call(c, v, m) #define rb_method_call_with_block_kw(c, v, m, b, kw) rb_method_call_with_block(c, v, m, b) - #define rb_eval_cmd_kwd(c, a, kw) rb_eval_cmd(c, a, 0) + #define rb_eval_cmd_kw(c, a, kw) rb_eval_cmd(c, a, 0) #endif == Appendix C. Functions available for use in extconf.rb diff --git a/doc/float.rb b/doc/float.rb new file mode 100644 index 0000000000..01668bfc6d --- /dev/null +++ b/doc/float.rb @@ -0,0 +1,128 @@ +# A \Float object stores a real number +# using the native architecture's double-precision floating-point representation. +# +# == \Float Imprecisions +# +# Some real numbers can be represented precisely as \Float objects: +# +# 37.5 # => 37.5 +# 98.75 # => 98.75 +# 12.3125 # => 12.3125 +# +# Others cannot; among these are the transcendental numbers, including: +# +# - Pi, <i>π</i>: in mathematics, a number of infinite precision: +# 3.1415926535897932384626433... (to 25 places); +# in Ruby, it is of limited precision (in this case, to 16 decimal places): +# +# Math::PI # => 3.141592653589793 +# +# - Euler's number, <i>e</i>: in mathematics, a number of infinite precision: +# 2.7182818284590452353602874... (to 25 places); +# in Ruby, it is of limited precision (in this case, to 15 decimal places): +# +# Math::E # => 2.718281828459045 +# +# Some floating-point computations in Ruby give precise results: +# +# 1.0/2 # => 0.5 +# 100.0/8 # => 12.5 +# +# Others do not: +# +# - In mathematics, 2/3 as a decimal number is an infinitely-repeating decimal: +# 0.666... (forever); +# in Ruby, +2.0/3+ is of limited precision (in this case, to 16 decimal places): +# +# 2.0/3 # => 0.6666666666666666 +# +# - In mathematics, the square root of 2 is an irrational number of infinite precision: +# 1.4142135623730950488016887... (to 25 decimal places); +# in Ruby, it is of limited precision (in this case, to 16 decimal places): +# +# Math.sqrt(2.0) # => 1.4142135623730951 +# +# - Even a simple computation can introduce imprecision: +# +# x = 0.1 + 0.2 # => 0.30000000000000004 +# y = 0.3 # => 0.3 +# x == y # => false +# +# See: +# +# - https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html +# - https://github.com/rdp/ruby_tutorials_core/wiki/Ruby-Talk-FAQ#-why-are-rubys-floats-imprecise +# - https://en.wikipedia.org/wiki/Floating_point#Accuracy_problems +# +# Note that precise storage and computation of rational numbers +# is possible using Rational objects. +# +# == Creating a \Float +# +# You can create a \Float object explicitly with: +# +# - A {floating-point literal}[rdoc-ref:syntax/literals.rdoc@Float+Literals]. +# +# You can convert certain objects to Floats with: +# +# - Method #Float. +# +# == What's Here +# +# First, what's elsewhere. Class \Float: +# +# - Inherits from +# {class Numeric}[rdoc-ref:Numeric@What-27s+Here] +# and {class Object}[rdoc-ref:Object@What-27s+Here]. +# - Includes {module Comparable}[rdoc-ref:Comparable@What-27s+Here]. +# +# Here, class \Float provides methods for: +# +# - {Querying}[rdoc-ref:Float@Querying] +# - {Comparing}[rdoc-ref:Float@Comparing] +# - {Converting}[rdoc-ref:Float@Converting] +# +# === Querying +# +# - #finite?: Returns whether +self+ is finite. +# - #hash: Returns the integer hash code for +self+. +# - #infinite?: Returns whether +self+ is infinite. +# - #nan?: Returns whether +self+ is a NaN (not-a-number). +# +# === Comparing +# +# - #<: Returns whether +self+ is less than the given value. +# - #<=: Returns whether +self+ is less than or equal to the given value. +# - #<=>: Returns a number indicating whether +self+ is less than, equal +# to, or greater than the given value. +# - #== (aliased as #=== and #eql?): Returns whether +self+ is equal to +# the given value. +# - #>: Returns whether +self+ is greater than the given value. +# - #>=: Returns whether +self+ is greater than or equal to the given value. +# +# === Converting +# +# - #% (aliased as #modulo): Returns +self+ modulo the given value. +# - #*: Returns the product of +self+ and the given value. +# - #**: Returns the value of +self+ raised to the power of the given value. +# - #+: Returns the sum of +self+ and the given value. +# - #-: Returns the difference of +self+ and the given value. +# - #/: Returns the quotient of +self+ and the given value. +# - #ceil: Returns the smallest number greater than or equal to +self+. +# - #coerce: Returns a 2-element array containing the given value converted to a \Float +# and +self+ +# - #divmod: Returns a 2-element array containing the quotient and remainder +# results of dividing +self+ by the given value. +# - #fdiv: Returns the \Float result of dividing +self+ by the given value. +# - #floor: Returns the greatest number smaller than or equal to +self+. +# - #next_float: Returns the next-larger representable \Float. +# - #prev_float: Returns the next-smaller representable \Float. +# - #quo: Returns the quotient from dividing +self+ by the given value. +# - #round: Returns +self+ rounded to the nearest value, to a given precision. +# - #to_i (aliased as #to_int): Returns +self+ truncated to an Integer. +# - #to_s (aliased as #inspect): Returns a string containing the place-value +# representation of +self+ in the given radix. +# - #truncate: Returns +self+ truncated to a given precision. +# + + class Float; end diff --git a/doc/globals.rdoc b/doc/globals.rdoc deleted file mode 100644 index 9d9fc57e6e..0000000000 --- a/doc/globals.rdoc +++ /dev/null @@ -1,416 +0,0 @@ -= Pre-Defined Global Variables - -Some of the pre-defined global variables have synonyms -that are available via module English. -For each of those, the \English synonym is given. - -To use the module: - - require 'English' - -== Exceptions - -=== <tt>$!</tt> (\Exception) - -Contains the Exception object set by Kernel#raise: - - begin - raise RuntimeError.new('Boo!') - rescue RuntimeError - p $! - end - -Output: - - #<RuntimeError: Boo!> - -English - <tt>$ERROR_INFO</tt> - -=== <tt>$@</tt> (Backtrace) - -Same as <tt>$!.backtrace</tt>; -returns an array of backtrace positions: - - begin - raise RuntimeError.new('Boo!') - rescue RuntimeError - pp $@.take(4) - end - -Output: - - ["(irb):338:in `<top (required)>'", - "/snap/ruby/317/lib/ruby/3.2.0/irb/workspace.rb:119:in `eval'", - "/snap/ruby/317/lib/ruby/3.2.0/irb/workspace.rb:119:in `evaluate'", - "/snap/ruby/317/lib/ruby/3.2.0/irb/context.rb:502:in `evaluate'"] - -English - <tt>$ERROR_POSITION</tt>. - -== Pattern Matching - -These global variables store information about the most recent -successful match in the current scope. - -For details and examples, -see {Regexp Global Variables}[rdoc-ref:Regexp@Global+Variables]. - -=== <tt>$~</tt> (\MatchData) - -MatchData object created from the match; -thread-local and frame-local. - -English - <tt>$LAST_MATCH_INFO</tt>. - -=== <tt>$&</tt> (Matched Substring) - -The matched string. - -English - <tt>$MATCH</tt>. - -=== <tt>$`</tt> (Pre-Match Substring) - -The string to the left of the match. - -English - <tt>$PREMATCH</tt>. - -=== <tt>$'</tt> (Post-Match Substring) - -The string to the right of the match. - -English - <tt>$POSTMATCH</tt>. - -=== <tt>$+</tt> (Last Matched Group) - -The last group matched. - -English - <tt>$LAST_PAREN_MATCH</tt>. - -=== <tt>$1</tt>, <tt>$2</tt>, \Etc. (Matched Group) - -For <tt>$_n_</tt> the _nth_ group of the match. - -No \English. - -== Separators - -=== <tt>$/</tt> (Input Record Separator) - -An input record separator, initially newline. - -English - <tt>$INPUT_RECORD_SEPARATOR</tt>, <tt>$RS</tt>. - -Aliased as <tt>$-0</tt>. - -=== <tt>$\\</tt> (Output Record Separator) - -An output record separator, initially +nil+. - -English - <tt>$OUTPUT_RECORD_SEPARATOR</tt>, <tt>$ORS</tt>. - -== Streams - -=== <tt>$stdin</tt> (Standard Input) - -The current standard input stream; initially: - - $stdin # => #<IO:<STDIN>> - -=== <tt>$stdout</tt> (Standard Output) - -The current standard output stream; initially: - - $stdout # => #<IO:<STDOUT>> - -=== <tt>$stderr</tt> (Standard Error) - -The current standard error stream; initially: - - $stderr # => #<IO:<STDERR>> - -=== <tt>$<</tt> (\ARGF or $stdin) - -Points to stream ARGF if not empty, else to stream $stdin; read-only. - -English - <tt>$DEFAULT_INPUT</tt>. - -=== <tt>$></tt> (Default Standard Output) - -An output stream, initially <tt>$stdout</tt>. - -English - <tt>$DEFAULT_OUTPUT - -=== <tt>$.</tt> (Input Position) - -The input position (line number) in the most recently read stream. - -English - <tt>$INPUT_LINE_NUMBER</tt>, <tt>$NR</tt> - -=== <tt>$_</tt> (Last Read Line) - -The line (string) from the most recently read stream. - -English - <tt>$LAST_READ_LINE</tt>. - -== Processes - -=== <tt>$0</tt> - -Initially, contains the name of the script being executed; -may be reassigned. - -=== <tt>$*</tt> (\ARGV) - -Points to ARGV. - -English - <tt>$ARGV</tt>. - -=== <tt>$$</tt> (Process ID) - -The process ID of the current process. Same as Process.pid. - -English - <tt>$PROCESS_ID</tt>, <tt>$PID</tt>. - -=== <tt>$?</tt> (Child Status) - -Initially +nil+, otherwise the Process::Status object -created for the most-recently exited child process; -thread-local. - -English - <tt>$CHILD_STATUS</tt>. - -=== <tt>$LOAD_PATH</tt> (Load Path) - -Contains the array of paths to be searched -by Kernel#load and Kernel#require. - -Singleton method <tt>$LOAD_PATH.resolve_feature_path(feature)</tt> -returns: - -- <tt>[:rb, _path_]</tt>, where +path+ is the path to the Ruby file - to be loaded for the given +feature+. -- <tt>[:so+ _path_]</tt>, where +path+ is the path to the shared object file - to be loaded for the given +feature+. -- +nil+ if there is no such +feature+ and +path+. - -Examples: - - $LOAD_PATH.resolve_feature_path('timeout') - # => [:rb, "/snap/ruby/317/lib/ruby/3.2.0/timeout.rb"] - $LOAD_PATH.resolve_feature_path('date_core') - # => [:so, "/snap/ruby/317/lib/ruby/3.2.0/x86_64-linux/date_core.so"] - $LOAD_PATH.resolve_feature_path('foo') - # => nil - -Aliased as <tt>$:</tt> and <tt>$-I</tt>. - -=== <tt>$LOADED_FEATURES</tt> - -Contains an array of the paths to the loaded files: - - $LOADED_FEATURES.take(10) - # => - ["enumerator.so", - "thread.rb", - "fiber.so", - "rational.so", - "complex.so", - "ruby2_keywords.rb", - "/snap/ruby/317/lib/ruby/3.2.0/x86_64-linux/enc/encdb.so", - "/snap/ruby/317/lib/ruby/3.2.0/x86_64-linux/enc/trans/transdb.so", - "/snap/ruby/317/lib/ruby/3.2.0/x86_64-linux/rbconfig.rb", - "/snap/ruby/317/lib/ruby/3.2.0/rubygems/compatibility.rb"] - -Aliased as <tt>$"</tt>. - -== Debugging - -=== <tt>$FILENAME</tt> - -The value returned by method ARGF.filename. - -=== <tt>$DEBUG</tt> - -Initially +true+ if command-line option <tt>-d</tt> or <tt>--debug</tt> is given, -otherwise initially +false+; -may be set to either value in the running program. - -When +true+, prints each raised exception to <tt>$stderr</tt>. - -Aliased as <tt>$-d</tt>. - -=== <tt>$VERBOSE</tt> - -Initially +true+ if command-line option <tt>-v</tt> or <tt>-w</tt> is given, -otherwise initially +false+; -may be set to either value, or to +nil+, in the running program. - -When +true+, enables Ruby warnings. - -When +nil+, disables warnings, including those from Kernel#warn. - -Aliased as <tt>$-v</tt> and <tt>$-w</tt>. - -== Other Variables - -=== <tt>$-a</tt> - -Whether command-line option <tt>-a</tt> was given; read-only. - -=== <tt>$-i</tt> - -Contains the extension given with command-line option <tt>-i</tt>, -or +nil+ if none. - -An alias of ARGF.inplace_mode. - -=== <tt>$-l</tt> - -Whether command-line option <tt>-l</tt> was set; read-only. - -=== <tt>$-p</tt> - -Whether command-line option <tt>-p</tt> was given; read-only. - -== Deprecated - -=== <tt>$=</tt> - -=== <tt>$,</tt> - -=== <tt>$;</tt> - -= Pre-Defined Global Constants - -= Streams - -=== <tt>STDIN</tt> - -The standard input stream (the default value for <tt>$stdin</tt>): - - STDIN # => #<IO:<STDIN>> - -=== <tt>STDOUT</tt> - -The standard output stream (the default value for <tt>$stdout</tt>): - - STDOUT # => #<IO:<STDOUT>> - -=== <tt>STDERR</tt> - -The standard error stream (the default value for <tt>$stderr</tt>): - - STDERR # => #<IO:<STDERR>> - -== Environment - -=== ENV - -A hash of the contains current environment variables names and values: - - ENV.take(5) - # => - [["COLORTERM", "truecolor"], - ["DBUS_SESSION_BUS_ADDRESS", "unix:path=/run/user/1000/bus"], - ["DESKTOP_SESSION", "ubuntu"], - ["DISPLAY", ":0"], - ["GDMSESSION", "ubuntu"]] - -=== ARGF - -The virtual concatenation of the files given on the command line, or from -<tt>$stdin</tt> if no files were given, <tt>"-"</tt> is given, or after -all files have been read. - -=== <tt>ARGV</tt> - -An array of the given command-line arguments. - -=== <tt>TOPLEVEL_BINDING</tt> - -The Binding of the top level scope: - - TOPLEVEL_BINDING # => #<Binding:0x00007f58da0da7c0> - -=== <tt>RUBY_VERSION</tt> - -The Ruby version: - - RUBY_VERSION # => "3.2.2" - -=== <tt>RUBY_RELEASE_DATE</tt> - -The release date string: - - RUBY_RELEASE_DATE # => "2023-03-30" - -=== <tt>RUBY_PLATFORM</tt> - -The platform identifier: - - RUBY_PLATFORM # => "x86_64-linux" - -=== <tt>RUBY_PATCHLEVEL</tt> - -The integer patch level for this Ruby: - - RUBY_PATCHLEVEL # => 53 - -For a development build the patch level will be -1. - -=== <tt>RUBY_REVISION</tt> - -The git commit hash for this Ruby: - - RUBY_REVISION # => "e51014f9c05aa65cbf203442d37fef7c12390015" - -=== <tt>RUBY_COPYRIGHT</tt> - -The copyright string: - - RUBY_COPYRIGHT - # => "ruby - Copyright (C) 1993-2023 Yukihiro Matsumoto" - -=== <tt>RUBY_ENGINE</tt> - -The name of the Ruby implementation: - - RUBY_ENGINE # => "ruby" - -=== <tt>RUBY_ENGINE_VERSION</tt> - -The version of the Ruby implementation: - - RUBY_ENGINE_VERSION # => "3.2.2" - -=== <tt>RUBY_DESCRIPTION</tt> - -The description of the Ruby implementation: - - RUBY_DESCRIPTION - # => "ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux]" - -== Embedded \Data - -=== <tt>DATA</tt> - -Defined if and only if the program has this line: - - __END__ - -When defined, <tt>DATA</tt> is a File object -containing the lines following the <tt>__END__</tt>, -positioned at the first of those lines: - - p DATA - DATA.each_line { |line| p line } - __END__ - Foo - Bar - Baz - -Output: - - #<File:t.rb> - "Foo\n" - "Bar\n" - "Baz\n" diff --git a/doc/yjit/yjit.md b/doc/jit/yjit.md index 0024c780b9..24aa163e60 100644 --- a/doc/yjit/yjit.md +++ b/doc/jit/yjit.md @@ -14,7 +14,7 @@ This project is open source and falls under the same license as CRuby. <p align="center"><b> If you're using YJIT in production, please - <a href="mailto:maxime.chevalierboisvert@shopify.com">share your success stories with us!</a> + <a href="mailto:ruby@shopify.com">share your success stories with us!</a> </b></p> If you wish to learn more about the approach taken, here are some conference talks and publications: @@ -388,7 +388,7 @@ There are multiple test suites: - `make test-all` - `make test-spec` - `make check` runs all of the above -- `make yjit-smoke-test` runs quick checks to see that YJIT is working correctly +- `make yjit-check` runs quick checks to see that YJIT is working correctly The tests can be run in parallel like this: diff --git a/doc/jit/zjit.md b/doc/jit/zjit.md new file mode 100644 index 0000000000..38124cb737 --- /dev/null +++ b/doc/jit/zjit.md @@ -0,0 +1,397 @@ +# ZJIT: ADVANCED RUBY JIT PROTOTYPE + +ZJIT is a method-based just-in-time (JIT) compiler for Ruby. It uses profile +information from the interpreter to guide optimization in the compiler. + +ZJIT is currently supported for macOS, Linux and BSD on x86-64 and arm64/aarch64 CPUs. +This project is open source and falls under the same license as CRuby. + +## Current Limitations + +ZJIT may not be suitable for certain applications. It currently only supports macOS, Linux and BSD on x86-64 and arm64/aarch64 CPUs. ZJIT will use more memory than the Ruby interpreter because the JIT compiler needs to generate machine code in memory and maintain additional state information. +You can change how much executable memory is allocated using [ZJIT's command-line options](rdoc-ref:@Command-Line+Options). + +## Contributing + +We welcome open source contributions. Feel free to open new issues to report +bugs or just to ask questions. Suggestions on how to make this document more +helpful for new contributors are most welcome. + +Bug fixes and bug reports are very valuable to us. If you find a bug in ZJIT, +it's very possible that nobody has reported it before, or that we don't have +a good reproduction for it, so please open a ticket on [the official Ruby bug +tracker][rubybugs] (or, if you don't want to make an account, [on +Shopify/ruby][shopifyruby]) and provide as much information as you can about +your configuration and a description of how you encountered the problem. List +the commands you used to run ZJIT so that we can easily reproduce the issue on +our end and investigate it. If you are able to produce a small program +reproducing the error to help us track it down, that is very much appreciated +as well. + +[rubybugs]: https://bugs.ruby-lang.org/projects/ruby-master +[shopifyruby]: https://github.com/Shopify/ruby/issues + +If you would like to contribute a large patch to ZJIT, we suggest [chatting on +Zulip][zulip] for a casual chat and then opening an issue on the [Shopify/ruby +repository][shopifyruby] so that we can have a technical discussion. A common +problem is that sometimes people submit large pull requests to open source +projects without prior communication, and we have to reject them because the +work they implemented does not fit within the design of the project. We want to +save you time and frustration, so please reach out so we can have a productive +discussion as to how you can contribute patches we will want to merge into +ZJIT. + +[zulip]: https://zjit.zulipchat.com/ + +## Build Instructions + +Refer to [Building Ruby](rdoc-ref:contributing/building_ruby.md) for general build prerequisites. +Additionally, ZJIT requires Rust 1.85.0 or later. Release builds need only `rustc`. Development +builds require `cargo` and may download dependencies. GNU Make is required. + +### For normal use + +To build ZJIT on macOS: + +```bash +./autogen.sh + +./configure \ + --enable-zjit \ + --prefix="$HOME"/.rubies/ruby-zjit \ + --disable-install-doc \ + --with-opt-dir="$(brew --prefix openssl):$(brew --prefix readline):$(brew --prefix libyaml)" + +make -j miniruby +``` + +To build ZJIT on Linux: + +```bash +./autogen.sh + +./configure \ + --enable-zjit \ + --prefix="$HOME"/.rubies/ruby-zjit \ + --disable-install-doc + +make -j miniruby +``` + +### For development + +To build ZJIT on macOS: + +```bash +./autogen.sh + +./configure \ + --enable-zjit=dev \ + --prefix="$HOME"/.rubies/ruby-zjit \ + --disable-install-doc \ + --with-opt-dir="$(brew --prefix openssl):$(brew --prefix readline):$(brew --prefix libyaml)" + +make -j miniruby +``` + +To build ZJIT on Linux: + +```bash +./autogen.sh + +./configure \ + --enable-zjit=dev \ + --prefix="$HOME"/.rubies/ruby-zjit \ + --disable-install-doc + +make -j miniruby +``` + +Note that `--enable-zjit=dev` does a lot of IR validation, which will help to catch errors early but mean compilation and warmup are significantly slower. + +The valid values for `--enable-zjit` are, from fastest to slowest: +* `--enable-zjit`: enable ZJIT in release mode for maximum performance +* `--enable-zjit=stats`: enable ZJIT in extended-stats mode +* `--enable-zjit=dev_nodebug`: enable ZJIT in development mode but without slow runtime checks +* `--enable-zjit=dev`: enable ZJIT in debug mode for development, also enables `RUBY_DEBUG` + +### Regenerate bindings + +When modifying `zjit/bindgen/src/main.rs` you need to regenerate bindings in `zjit/src/cruby_bindings.inc.rs` with: + +```bash +make zjit-bindgen +``` + +## Documentation + +### Command-Line Options + +See `ruby --help` for ZJIT-specific command-line options: + +``` +$ ruby --help +... +ZJIT options: + --zjit-mem-size=num + Max amount of memory that ZJIT can use in MiB (default: 128). + --zjit-call-threshold=num + Number of calls to trigger JIT (default: 30). + --zjit-num-profiles=num + Number of profiled calls before JIT (default: 5). + --zjit-stats[=quiet] + Enable collecting ZJIT statistics (=quiet to suppress output). + --zjit-disable Disable ZJIT for lazily enabling it with RubyVM::ZJIT.enable. + --zjit-perf Dump ISEQ symbols into /tmp/perf-{}.map for Linux perf. + --zjit-log-compiled-iseqs=path + Log compiled ISEQs to the file. The file will be truncated. + --zjit-trace-exits[=counter] + Record source on side-exit. `Counter` picks specific counter. + --zjit-trace-exits-sample-rate=num + Frequency at which to record side exits. Must be `usize`. +$ +``` + +### Source level documentation + +You can generate and open the source level documentation in your browser using: + +```bash +cargo doc --document-private-items -p zjit --open +``` + +### Graph of the Type System + +You can generate a graph of the ZJIT type hierarchy using: + +```bash +ruby zjit/src/hir_type/gen_hir_type.rb > zjit/src/hir_type/hir_type.inc.rs +dot -O -Tpdf zjit_types.dot +open zjit_types.dot.pdf +``` + +## Testing + +Note that tests link against CRuby, so directly calling `cargo test`, or `cargo nextest` should not build. All tests are instead accessed through `make`. + +### Setup + +First, ensure you have `cargo` installed. If you do not already have it, you can use [rustup.rs](https://rustup.rs/). + +Also install cargo-binstall with: + +```bash +cargo install cargo-binstall +``` + +Make sure to add `--enable-zjit=dev` when you run `configure`, then install the following tools: + +```bash +cargo binstall --secure cargo-nextest +cargo binstall --secure cargo-insta +``` + +`cargo-insta` is used for updating snapshots. `cargo-nextest` runs each test in its own process, which is valuable since CRuby only supports booting once per process, and most APIs are not thread safe. + +### Running unit tests + +For testing functionality within ZJIT, use: + +```bash +make zjit-test +``` + +You can also run a single test case by specifying the function name: + +```bash +make zjit-test ZJIT_TESTS=test_putobject +``` + +#### Snapshot Testing + +ZJIT uses [insta](https://insta.rs/) for snapshot testing within unit tests. When tests fail due to snapshot mismatches, pending snapshots are created. The test command will notify you if there are pending snapshots: + +``` +Pending snapshots found. Accept with: make zjit-test-update +``` + +To update/accept all the snapshot changes: + +```bash +make zjit-test-update +``` + +You can also review snapshot changes interactively one by one: + +```bash +cd zjit && cargo insta review +``` + +Test changes will be reviewed alongside code changes. + +### Running integration tests + +This command runs Ruby execution tests. + +```bash +make test-all TESTS="test/ruby/test_zjit.rb" +``` + +You can also run a single test case by matching the method name: + +```bash +make test-all TESTS="test/ruby/test_zjit.rb -n TestZJIT#test_putobject" +``` + +### Running all tests + +Runs both `make zjit-test` and `test/ruby/test_zjit.rb`: + +```bash +make zjit-check +``` + +## Statistics Collection + +ZJIT provides detailed statistics about JIT compilation and execution behavior. + +### Basic Stats + +Run with basic statistics printed on exit: + +```bash +./miniruby --zjit-stats script.rb +``` + +Collect stats without printing (access via `RubyVM::ZJIT.stats` in Ruby): + +```bash +./miniruby --zjit-stats=quiet script.rb +``` + +### Accessing Stats in Ruby + +```ruby +# Check if stats are enabled +if RubyVM::ZJIT.stats_enabled? + stats = RubyVM::ZJIT.stats + puts "Compiled ISEQs: #{stats[:compiled_iseq_count]}" + puts "Failed ISEQs: #{stats[:failed_iseq_count]}" + + # You can also reset stats during execution + RubyVM::ZJIT.reset_stats! +end +``` + +### Performance Ratio + +The `ratio_in_zjit` stat shows the percentage of Ruby instructions executed in JIT code vs interpreter. +This metric only appears when ZJIT is built with `--enable-zjit=stats` [or more](#build-instructions) (which enables `rb_vm_insn_count` tracking) and represents a key performance indicator for ZJIT effectiveness. + +### Tracing side exits + +Through [Stackprof](https://github.com/tmm1/stackprof), detailed information about the methods that the JIT side-exits from can be displayed after some execution of a program. Optionally, you can use `--zjit-trace-exits-sample-rate=N` to sample every N-th occurrence. Enabling `--zjit-trace-exits-sample-rate=N` will automatically enable `--zjit-trace-exits`. + +```bash +./miniruby --zjit-trace-exits script.rb +``` + +A file called `zjit_exits_{pid}.dump` will be created in the same directory as `script.rb`. Viewing the side exited methods can be done with Stackprof: + +```bash +stackprof path/to/zjit_exits_{pid}.dump +``` + +### Viewing HIR in Iongraph + +Using `--zjit-dump-hir-iongraph` will dump all compiled functions into a directory named `/tmp/zjit-iongraph-{PROCESS_PID}`. Each file will be named `func_{ZJIT_FUNC_NAME}.json`. In order to use them in the Iongraph viewer, you'll need to use `jq` to collate them to a single file. An example invocation of `jq` is shown below for reference. + +`jq --slurp --null-input '.functions=inputs | .version=1' /tmp/zjit-iongraph-{PROCESS_PID}/func*.json > ~/Downloads/ion.json` + +From there, you can use https://mozilla-spidermonkey.github.io/iongraph/ to view your trace. + +### Printing ZJIT Errors + +`--zjit-debug` prints ZJIT compilation errors and other diagnostics: + +```bash +./miniruby --zjit-debug script.rb +``` + +As you might guess from the name, this option is intended mostly for ZJIT developers. + +## Useful dev commands + +To view YARV output for code snippets: + +```bash +./miniruby --dump=insns -e0 +``` + +To run code snippets with ZJIT: + +```bash +./miniruby --zjit -e0 +``` + +You can also try https://www.rubyexplorer.xyz/ to view Ruby YARV disasm output with syntax highlighting +in a way that can be easily shared with other team members. + +## Understanding Ruby Stacks + +Ruby execution involves three distinct stacks and understanding them will help you understand ZJIT's implementation: + +### 1. Native Stack + +- **Purpose**: Return addresses and saved registers. ZJIT also uses it for some C functions' argument arrays +- **Management**: OS-managed, one per native thread +- **Growth**: Downward from high addresses +- **Constants**: `NATIVE_STACK_PTR`, `NATIVE_BASE_PTR` + +### 2. Ruby VM Stack + +The Ruby VM uses a single contiguous memory region (`ec->vm_stack`) containing two sub-stacks that grow toward each other. When they meet, stack overflow occurs. + +See [doc/contributing/vm_stack_and_frames.md](rdoc-ref:contributing/vm_stack_and_frames.md) for detailed architecture and frame layout. + +**Control Frame Stack:** + +- **Stores**: Frame metadata (`rb_control_frame_t` structures) +- **Growth**: Downward from `vm_stack + size` (high addresses) +- **Constants**: `CFP` + +**Value Stack:** + +- **Stores**: YARV bytecode operands (self, arguments, locals, temporaries) +- **Growth**: Upward from `vm_stack` (low addresses) +- **Constants**: `SP` + +## ZJIT Glossary + +This glossary contains terms that are helpful for understanding ZJIT. + +Please note that some terms may appear in CRuby internals too but with different meanings. + +| Term | Definition | +| ----------------- | ------------------------------------------------------------------------------------------------------------------------------- | +| HIR | High-level Intermediate Representation. High-level (Ruby semantics) graph representation in static single-assignment (SSA) form | +| LIR | Low-level Intermediate Representation. Low-level IR used in the backend for assembly generation | +| SSA | Static Single Assignment. A form where each variable is assigned exactly once | +| `opnd` | Operand. An operand to an IR instruction (can be register, memory, immediate, etc.) | +| `dst` | Destination. The output operand of an instruction where the result is stored | +| VReg | Virtual Register. A virtual register that gets lowered to physical register or memory | +| `insn_id` | Instruction ID. An index of an instruction in a function | +| `block_id` | The index of a basic block, which effectively acts like a pointer | +| `branch` | Control flow edge between basic blocks in the compiled code | +| `cb` | Code Block. Memory region for generated machine code | +| `entry` | The starting address of compiled code for an ISEQ | +| Patch Point | Location in generated code that can be modified later in case assumptions get invalidated | +| Frame State | Captured state of the Ruby stack frame at a specific point for deoptimization | +| Guard | A run-time check that ensures assumptions are still valid | +| `invariant` | An assumption that JIT code relies on, requiring invalidation if broken | +| Deopt | Deoptimization. Process of falling back from JIT code to interpreter | +| Side Exit | Exit from JIT code back to interpreter | +| Type Lattice | Hierarchy of types used for type inference and optimization | +| Constant Folding | Optimization that evaluates constant expressions at compile time | +| RSP | x86-64 stack pointer register used for native stack operations | +| Register Spilling | Process of moving register values to memory when running out of physical registers | diff --git a/doc/language/box.md b/doc/language/box.md new file mode 100644 index 0000000000..8c7fd20b20 --- /dev/null +++ b/doc/language/box.md @@ -0,0 +1,361 @@ +# Ruby Box - Ruby's in-process separation of Classes and Modules + +Ruby Box is designed to provide separated spaces in a Ruby process, to isolate application code, libraries and monkey patches. + +## Known issues + +* Experimental warning is shown when ruby starts with `RUBY_BOX=1` (specify `-W:no-experimental` option to hide it) +* Installing native extensions may fail under `RUBY_BOX=1` because of stack level too deep in extconf.rb +* `require 'active_support/core_ext'` may fail under `RUBY_BOX=1` +* Defined methods in a box may not be referred by built-in methods written in Ruby + +## TODOs + +* Add the loaded box on iseq to check if another box tries running the iseq (add a field only when VM_CHECK_MODE?) +* Assign its own TOPLEVEL_BINDING in boxes +* Fix calling `warn` in boxes to refer `$VERBOSE` and `Warning.warn` in the box +* Make an internal data container class `Ruby::Box::Entry` invisible +* More test cases about `$LOAD_PATH` and `$LOADED_FEATURES` + +## How to use + +### Enabling Ruby Box + +First, an environment variable should be set at the ruby process bootup: `RUBY_BOX=1`. +The only valid value is `1` to enable Ruby Box. Other values (or unset `RUBY_BOX`) means disabling Ruby Box. And setting the value after Ruby program starts doesn't work. + +### Using Ruby Box + +`Ruby::Box` class is the entrypoint of Ruby Box. + +```ruby +box = Ruby::Box.new +box.require('something') # or require_relative, load +``` + +The required file (either .rb or .so/.dll/.bundle) is loaded in the box (`box` here). The required/loaded files from `something` will be loaded in the box recursively. + +```ruby +# something.rb + +X = 1 + +class Something + def self.x = X + def x = ::X +end +``` + +Classes/modules, those methods and constants defined in the box can be accessed via `box` object. + +```ruby +X = 2 +p X # 2 +p ::X # 2 +p box::Something.x # 1 +p box::X # 1 +``` + +Instance methods defined in the box also run with definitions in the box. + +```ruby +s = box::Something.new + +p s.x # 1 +``` + +## Specifications + +### Ruby Box types + +There are two box types: + +* Root box +* User boxes + +There is the root box, just a single box in a Ruby process. Ruby bootstrap runs in the root box, and all builtin classes/modules are defined in the root box. (See "Builtin classes and modules".) + +User boxes are to run user-written programs and libraries loaded from user programs. The user's main program (specified by the `ruby` command line argument) is executed in the "main" box, which is a user box automatically created at the end of Ruby's bootstrap, copied from the root box. + +When `Ruby::Box.new` is called, an "optional" box (a user, non-main box) is created, copied from the root box. All user boxes are flat, copied from the root box. + +### Ruby Box class and instances + +`Ruby::Box` is a class, as a subclass of `Module`. `Ruby::Box` instances are a kind of `Module`. + +### Classes and modules defined in boxes + +The classes and modules, newly defined in a box `box`, are accessible via `box`. For example, if a class `A` is defined in `box`, it is accessible as `box::A` from outside of the box. + +In the box `box`, `A` can be referred to as `A` (and `::A`). + +### Built-in classes and modules reopened in boxes + +In boxes, builtin classes/modules are visible and can be reopened. Those classes/modules can be reopened using `class` or `module` clauses, and class/module definitions can be changed. + +The changed definitions are visible only in the box. In other boxes, builtin classes/modules and those instances work without changed definitions. + +```ruby +# in foo.rb +class String + BLANK_PATTERN = /\A\s*\z/ + def blank? + self.match?(BLANK_PATTERN) + end +end + +module Foo + def self.foo = "foo" + + def self.foo_is_blank? + foo.blank? + end +end + +Foo.foo.blank? #=> false +"foo".blank? #=> false + +# in main.rb +box = Ruby::Box.new +box.require_relative('foo') + +box::Foo.foo_is_blank? #=> false (#blank? called in box) + +"foo".blank? # NoMethodError +String::BLANK_PATTERN # NameError +``` + +The main box and `box` are different boxes, so monkey patches in main are also invisible in `box`. + +### Builtin classes and modules + +In the box context, "builtin" classes and modules are classes and modules: + +* Accessible without any `require` calls in user scripts +* Defined before any user program start running +* Including classes/modules loaded by `prelude.rb` (including RubyGems `Gem`, for example) + +Hereafter, "builtin classes and modules" will be referred to as just "builtin classes". + +### Builtin classes referred via box objects + +Builtin classes in a box `box` can be referred from other boxes. For example, `box::String` is a valid reference, and `String` and `box::String` are identical (`String == box::String`, `String.object_id == box::String.object_id`). + +`box::String`-like reference returns just a `String` in the current box, so its definition is `String` in the box, not in `box`. + +```ruby +# foo.rb +class String + def self.foo = "foo" +end + +# main.rb +box = Ruby::Box.new +box.require_relative('foo') + +box::String.foo # NoMethodError +``` + +### Class instance variables, class variables, constants + +Builtin classes can have different sets of class instance variables, class variables and constants between boxes. + +```ruby +# foo.rb +class Array + @v = "foo" + @@v = "_foo_" + V = "FOO" +end + +Array.instance_variable_get(:@v) #=> "foo" +Array.class_variable_get(:@@v) #=> "_foo_" +Array.const_get(:V) #=> "FOO" + +# main.rb +box = Ruby::Box.new +box.require_relative('foo') + +Array.instance_variable_get(:@v) #=> nil +Array.class_variable_get(:@@v) # NameError +Array.const_get(:V) # NameError +``` + +### Global variables + +In boxes, changes on global variables are also isolated in the boxes. Changes on global variables in a box are visible/applied only in the box. + +```ruby +# foo.rb +$foo = "foo" +$VERBOSE = nil + +puts "This appears: '#{$foo}'" + +# main.rb +p $foo #=> nil +p $VERBOSE #=> false + +box = Ruby::Box.new +box.require_relative('foo') # "This appears: 'foo'" + +p $foo #=> nil +p $VERBOSE #=> false +``` + +### Top level constants + +Usually, top level constants are defined as constants of `Object`. In boxes, top level constants are constants of `Object` in the box. And the box object `box`'s constants are strictly equal to constants of `Object`. + +```ruby +# foo.rb +FOO = 100 + +FOO #=> 100 +Object::FOO #=> 100 + +# main.rb +box = Ruby::Box.new +box.require_relative('foo') + +box::FOO #=> 100 + +FOO # NameError +Object::FOO # NameError +``` + +### Top level methods + +Top level methods are private instance methods of `Object`, in each box. + +```ruby +# foo.rb +def yay = "foo" + +class Foo + def self.say = yay +end + +Foo.say #=> "foo" +yay #=> "foo" + +# main.rb +box = Ruby::Box.new +box.require_relative('foo') + +box::Foo.say #=> "foo" + +yay # NoMethodError +``` + +There is no way to expose top level methods in boxes to others. +(See "Expose top level methods as a method of the box object" in "Discussions" section below) + +### Ruby Box scopes + +Ruby Box works in file scope. One `.rb` file runs in a single box. + +Once a file is loaded in a box `box`, all methods/procs defined/created in the file run in `box`. + +### Utility methods + +Several methods are available for trying/testing Ruby Box. + +* `Ruby::Box.current` returns the current box +* `Ruby::Box.enabled?` returns true/false to represent `RUBY_BOX=1` is specified or not +* `Ruby::Box.root` returns the root box +* `Ruby::Box.main` returns the main box +* `Ruby::Box#eval` evaluates a Ruby code (String) in the receiver box, just like calling `#load` with a file + +## Implementation details + +#### ISeq inline method/constant cache + +As described above in "Ruby Box scopes", an ".rb" file runs in a box. So method/constant resolution will be done in a box consistently. + +That means ISeq inline caches work well even with boxes. Otherwise, it's a bug. + +#### Method call global cache (gccct) + +`rb_funcall()` C function refers to the global cc cache table (gccct), and the cache key is calculated with the current box. + +So, `rb_funcall()` calls have a performance penalty when Ruby Box is enabled. + +#### Current box and loading box + +The current box is the box that the executing code is in. `Ruby::Box.current` returns the current box object. + +The loading box is an internally managed box to determine the box to load newly required/loaded files. For example, `box` is the loading box when `box.require("foo")` is called. + +## Discussions + +#### More builtin methods written in Ruby + +If Ruby Box is enabled by default, builtin methods can be written in Ruby because it can't be overridden by users' monkey patches. Builtin Ruby methods can be JIT-ed, and it could bring performance reward. + +#### Monkey patching methods called by builtin methods + +Builtin methods sometimes call other builtin methods. For example, `Hash#map` calls `Hash#each` to retrieve entries to be mapped. Without Ruby Box, Ruby users can overwrite `Hash#each` and expect the behavior change of `Hash#map` as a result. + +But with boxes, `Hash#map` runs in the root box. Ruby users can define `Hash#each` only in user boxes, so users cannot change `Hash#map`'s behavior in this case. To achieve it, users should override both`Hash#map` and `Hash#each` (or only `Hash#map`). + +It is a breaking change. + +Users can define methods using `Ruby::Box.root.eval(...)`, but it's clearly not ideal API. + +#### Assigning values to global variables used by builtin methods + +Similar to monkey patching methods, global variables assigned in a box is separated from the root box. Methods defined in the root box referring a global variable can't find the re-assigned one. + +#### Context of `$LOAD_PATH` and `$LOADED_FEATURES` + +Global variables `$LOAD_PATH` and `$LOADED_FEATURES` control `require` method behaviors. So those variables are determined by the loading box instead of the current box. + +This could potentially conflict with the user's expectations. We should find the solution. + +#### Expose top level methods as a method of the box object + +Currently, top level methods in boxes are not accessible from outside of the box. But there might be a use case to call other box's top level methods. + +#### Split root and builtin box + +Currently, the single "root" box is the source of classext CoW. And also, the "root" box can load additional files after starting main script evaluation by calling methods which contain lines like `require "openssl"`. + +That means, user boxes can have different sets of definitions according to when it is created. + +``` +[root] + | + |----[main] + | + |(require "openssl" called in root) + | + |----[box1] having OpenSSL + | + |(remove_const called for OpenSSL in root) + | + |----[box2] without OpenSSL +``` + +This could cause unexpected behavior differences between user boxes. It should NOT be a problem because user scripts which refer to `OpenSSL` should call `require "openssl"` by themselves. +But in the worst case, a script (without `require "openssl"`) runs well in `box1`, but doesn't run in `box2`. This situation looks like a "random failure" to users. + +An option possible to prevent this situation is to have "root" and "builtin" boxes. + +* root + * The box for the Ruby process bootstrap, then the source of CoW + * After starting the main box, no code runs in this box +* builtin + * The box copied from the root box at the same time with "main" + * Methods and procs defined in the "root" box run in this box + * Classes and modules required will be loaded in this box + +This design realizes a consistent source of box CoW. + +#### Separate `cc_tbl` and `callable_m_tbl`, `cvc_tbl` for less classext CoW + +The fields of `rb_classext_t` contains several cache(-like) data, `cc_tbl`(callcache table), `callable_m_tbl`(table of resolved complemented methods) and `cvc_tbl`(class variable cache table). + +The classext CoW is triggered when the contents of `rb_classext_t` are changed, including `cc_tbl`, `callable_m_tbl`, and `cvc_tbl`. But those three tables are changed by just calling methods or referring class variables. So, currently, classext CoW is triggered much more times than the original expectation. + +If we can move those three tables outside of `rb_classext_t`, the number of copied `rb_classext_t` will be much less than the current implementation. diff --git a/doc/bsearch.rdoc b/doc/language/bsearch.rdoc index 90705853d7..90705853d7 100644 --- a/doc/bsearch.rdoc +++ b/doc/language/bsearch.rdoc diff --git a/doc/date/calendars.rdoc b/doc/language/calendars.rdoc index 4e6fd8334b..a2540f1c43 100644 --- a/doc/date/calendars.rdoc +++ b/doc/language/calendars.rdoc @@ -31,7 +31,7 @@ See also {a concrete example here}[rdoc-ref:DateTime@When+should+you+use+DateTim === Argument +start+ Certain methods in class \Date handle differences in the -{Julian and Gregorian calendars}[rdoc-ref:date/calendars.rdoc@Julian+and+Gregorian+Calendars] +{Julian and Gregorian calendars}[rdoc-ref:@Julian+and+Gregorian+Calendars] by accepting an optional argument +start+, whose value may be: - Date::ITALY (the default): the created date is Julian diff --git a/doc/case_mapping.rdoc b/doc/language/case_mapping.rdoc index 57c037b9d8..d40155db03 100644 --- a/doc/case_mapping.rdoc +++ b/doc/language/case_mapping.rdoc @@ -37,7 +37,7 @@ Context-dependent case mapping as described in {Table 3-17 (Context Specification for Casing) of the Unicode standard}[https://www.unicode.org/versions/latest/ch03.pdf] is currently not supported. -In most cases, case conversions of a string have the same number of characters. +In most cases, the case conversion of a string has the same number of characters as before. There are exceptions (see also +:fold+ below): s = "\u00DF" # => "ß" @@ -58,25 +58,18 @@ Case changes may not be reversible: s.downcase.upcase # => "HELLO WORLD!" # Different from original s. Case changing methods may not maintain Unicode normalization. -See String#unicode_normalize). +See String#unicode_normalize. -== Options for Case Mapping +== Case Mappings Except for +casecmp+ and +casecmp?+, each of the case-mapping methods listed above -accepts optional arguments, <tt>*options</tt>. +accepts an optional argument, <tt>mapping</tt>. -The arguments may be: +The argument is one of: -- +:ascii+ only. -- +:fold+ only. -- +:turkic+ or +:lithuanian+ or both. - -The options: - -- +:ascii+: - ASCII-only mapping: - uppercase letters ('A'..'Z') are mapped to lowercase letters ('a'..'z); +- +:ascii+: ASCII-only mapping. + Uppercase letters ('A'..'Z') are mapped to lowercase letters ('a'..'z); other characters are not changed s = "Foo \u00D8 \u00F8 Bar" # => "Foo Ø ø Bar" @@ -85,8 +78,8 @@ The options: s.upcase(:ascii) # => "FOO Ø ø BAR" s.downcase(:ascii) # => "foo Ø ø bar" -- +:turkic+: - Full Unicode case mapping, adapted for the Turkic languages +- +:turkic+: Full Unicode case mapping. + For the Turkic languages that distinguish dotted and dotless I, for example Turkish and Azeri. s = 'Türkiye' # => "Türkiye" @@ -97,11 +90,8 @@ The options: s.downcase # => "türkiye" s.downcase(:turkic) # => "türkıye" # No dot above. -- +:lithuanian+: - Not yet implemented. - - +:fold+ (available only for String#downcase, String#downcase!, - and Symbol#downcase): + and Symbol#downcase). Unicode case folding, which is more far-reaching than Unicode case mapping. diff --git a/doc/character_selectors.rdoc b/doc/language/character_selectors.rdoc index 47cf242be7..8bfc9b719b 100644 --- a/doc/character_selectors.rdoc +++ b/doc/language/character_selectors.rdoc @@ -14,6 +14,8 @@ Each of these instance methods accepts one or more character selectors: - String#delete!(*selectors): returns +self+ or +nil+. - String#squeeze(*selectors): returns a new string. - String#squeeze!(*selectors): returns +self+ or +nil+. +- String#strip(*selectors): returns a new string. +- String#strip!(*selectors): returns +self+ or +nil+. A character selector identifies zero or more characters in +self+ that are to be operands for the method. @@ -29,7 +31,6 @@ contained in the selector itself: 'abracadabra'.delete('abc') # => "rdr" '0123456789'.delete('258') # => "0134679" '!@#$%&*()_+'.delete('+&#') # => "!@$%*()_" - 'тест'.delete('т') # => "ес" 'こんにちは'.delete('に') # => "こんちは" Note that order and repetitions do not matter: @@ -79,6 +80,8 @@ These instance methods accept multiple character selectors: - String#delete!(*selectors): returns +self+ or +nil+. - String#squeeze(*selectors): returns a new string. - String#squeeze!(*selectors): returns +self+ or +nil+. +- String#strip(*selectors): returns a new string. +- String#strip!(*selectors): returns +self+ or +nil+. In effect, the given selectors are formed into a single selector consisting of only those characters common to _all_ of the given selectors. diff --git a/doc/dig_methods.rdoc b/doc/language/dig_methods.rdoc index 366275d451..366275d451 100644 --- a/doc/dig_methods.rdoc +++ b/doc/language/dig_methods.rdoc diff --git a/doc/encodings.rdoc b/doc/language/encodings.rdoc index eaee2efd62..683842d3fb 100644 --- a/doc/encodings.rdoc +++ b/doc/language/encodings.rdoc @@ -138,7 +138,7 @@ A Ruby String object has an encoding that is an instance of class \Encoding. The encoding may be retrieved by method String#encoding. The default encoding for a string literal is the script encoding; -see {Script Encoding}[rdoc-ref:encodings.rdoc@Script+Encoding]. +see {Script Encoding}[rdoc-ref:@Script+Encoding]. 's'.encoding # => #<Encoding:UTF-8> @@ -147,7 +147,7 @@ The default encoding for a string created with method String.new is: - For no argument, ASCII-8BIT. - For a \String object argument, the encoding of that string. - For a string literal, the script encoding; - see {Script Encoding}[rdoc-ref:encodings.rdoc@Script+Encoding]. + see {Script Encoding}[rdoc-ref:@Script+Encoding]. In either case, any encoding may be specified: @@ -193,7 +193,7 @@ The default encoding for these, however, is: - US-ASCII, if all characters are US-ASCII. - The script encoding, otherwise; - see (Script Encoding)[rdoc-ref:encodings.rdoc@Script+Encoding]. + see (Script Encoding)[rdoc-ref:@Script+Encoding]. == Filesystem \Encoding @@ -393,7 +393,7 @@ These keyword-value pairs specify encoding options: - <tt>:replace: nil</tt> (default): Set replacement string to default value: <tt>"\uFFFD"</tt> ("�") for a Unicode encoding, <tt>'?'</tt> otherwise. - - <tt>:replace: _some_string_</tt>: Set replacement string to the given +some_string+; + - <tt>:replace: some_string</tt>: Set replacement string to the given +some_string+; overrides +:fallback+. Examples: @@ -407,12 +407,12 @@ These keyword-value pairs specify encoding options: One of these may be specified: - <tt>:fallback: nil</tt> (default): No replacement fallback. - - <tt>:fallback: _hash_like_object_</tt>: Set replacement fallback to the given - +hash_like_object+; the replacement string is <tt>_hash_like_object_[X]</tt>. - - <tt>:fallback: _method_</tt>: Set replacement fallback to the given - +method+; the replacement string is <tt>_method_(X)</tt>. - - <tt>:fallback: _proc_</tt>: Set replacement fallback to the given - +proc+; the replacement string is <tt>_proc_[X]</tt>. + - <tt>:fallback: hash_like_object</tt>: Set replacement fallback to the given + +hash_like_object+; the replacement string is <tt>hash_like_object[X]</tt>. + - <tt>:fallback: method</tt>: Set replacement fallback to the given + +method+; the replacement string is <tt>method(X)</tt>. + - <tt>:fallback: proc</tt>: Set replacement fallback to the given + +proc+; the replacement string is <tt>proc[X]</tt>. Examples: @@ -437,7 +437,7 @@ These keyword-value pairs specify encoding options: - <tt>:xml: nil</tt> (default): No handling for XML entities. - <tt>:xml: :text</tt>: Treat source text as XML; replace each undefined character - with its upper-case hexdecimal numeric character reference, + with its upper-case hexadecimal numeric character reference, except that: - <tt>&</tt> is replaced with <tt>&</tt>. @@ -446,7 +446,7 @@ These keyword-value pairs specify encoding options: - <tt>:xml: :attr</tt>: Treat source text as XML attribute value; replace each undefined character - with its upper-case hexdecimal numeric character reference, + with its upper-case hexadecimal numeric character reference, except that: - The replacement string <tt>r</tt> is double-quoted (<tt>"r"</tt>). diff --git a/doc/exceptions.md b/doc/language/exceptions.md index 2c47455911..5f8f0ece69 100644 --- a/doc/exceptions.md +++ b/doc/language/exceptions.md @@ -504,18 +504,18 @@ These methods return backtrace information: By default, Ruby sets the backtrace of the exception to the location where it was raised. -The developer might adjust this by either providing +backtrace+ argument +The developer might adjust this by either providing `backtrace` argument to Kernel#raise, or using Exception#set_backtrace. Note that: -- by default, both +backtrace+ and +backtrace_locations+ represent the same backtrace; +- by default, both `backtrace` and `backtrace_locations` represent the same backtrace; - if the developer sets the backtrace by one of the above methods to an array of Thread::Backtrace::Location, they still represent the same backtrace; - if the developer sets the backtrace to a string or an array of strings: - - by Kernel#raise: +backtrace_locations+ become +nil+; - - by Exception#set_backtrace: +backtrace_locations+ preserve the original + - by Kernel#raise: `backtrace_locations` become `nil`; + - by Exception#set_backtrace: `backtrace_locations` preserve the original value; -- if the developer sets the backtrace to +nil+ by Exception#set_backtrace, - +backtrace_locations+ preserve the original value; but if the exception is then - reraised, both +backtrace+ and +backtrace_locations+ become the location of reraise. +- if the developer sets the backtrace to `nil` by Exception#set_backtrace, + `backtrace_locations` preserve the original value; but if the exception is then + reraised, both `backtrace` and `backtrace_locations` become the location of reraise. diff --git a/doc/fiber.md b/doc/language/fiber.md index 2bc1ff96b2..d9011cce2f 100644 --- a/doc/fiber.md +++ b/doc/language/fiber.md @@ -212,6 +212,64 @@ I/O. Windows is a notable example where socket I/O can be non-blocking but pipe I/O is blocking. Provided that there *is* a scheduler and the current thread *is non-blocking*, the operation will invoke the scheduler. +##### `IO#close` + +Closing an IO interrupts all blocking operations on that IO. When a thread calls `IO#close`, it first attempts to interrupt any threads or fibers that are blocked on that IO. The closing thread waits until all blocked threads and fibers have been properly interrupted and removed from the IO's blocking list. Each interrupted thread or fiber receives an `IOError` and is cleanly removed from the blocking operation. Only after all blocking operations have been interrupted and cleaned up will the actual file descriptor be closed, ensuring proper resource cleanup and preventing potential race conditions. + +For fibers managed by a scheduler, the interruption process involves calling `rb_fiber_scheduler_fiber_interrupt` on the scheduler. This allows the scheduler to handle the interruption in a way that's appropriate for its event loop implementation. The scheduler can then notify the fiber, which will receive an `IOError` and be removed from the blocking operation. This mechanism ensures that fiber-based concurrency works correctly with IO operations, even when those operations are interrupted by `IO#close`. + +```mermaid +sequenceDiagram + participant ThreadB + participant ThreadA + participant Scheduler + participant IO + participant Fiber1 + participant Fiber2 + + Note over ThreadA: Thread A has a fiber scheduler + activate Scheduler + ThreadA->>Fiber1: Schedule Fiber 1 + activate Fiber1 + Fiber1->>IO: IO.read + IO->>Scheduler: rb_thread_io_blocking_region + deactivate Fiber1 + + ThreadA->>Fiber2: Schedule Fiber 2 + activate Fiber2 + Fiber2->>IO: IO.read + IO->>Scheduler: rb_thread_io_blocking_region + deactivate Fiber2 + + Note over Fiber1,Fiber2: Both fibers blocked on same IO + + Note over ThreadB: IO.close + activate ThreadB + ThreadB->>IO: thread_io_close_notify_all + Note over ThreadB: rb_mutex_sleep + + IO->>Scheduler: rb_fiber_scheduler_fiber_interrupt(Fiber1) + Scheduler->>Fiber1: fiber_interrupt with IOError + activate Fiber1 + Note over IO: fiber_interrupt causes removal from blocking list + Fiber1->>IO: rb_io_blocking_operation_exit() + IO-->>ThreadB: Wakeup thread + deactivate Fiber1 + + IO->>Scheduler: rb_fiber_scheduler_fiber_interrupt(Fiber2) + Scheduler->>Fiber2: fiber_interrupt with IOError + activate Fiber2 + Note over IO: fiber_interrupt causes removal from blocking list + Fiber2->>IO: rb_io_blocking_operation_exit() + IO-->>ThreadB: Wakeup thread + deactivate Fiber2 + deactivate Scheduler + + Note over ThreadB: Blocking operations list empty + ThreadB->>IO: close(fd) + deactivate ThreadB +``` + #### Mutex The `Mutex` class can be used in a non-blocking context and is fiber specific. diff --git a/doc/format_specifications.rdoc b/doc/language/format_specifications.rdoc index bdfdc24953..a59f210377 100644 --- a/doc/format_specifications.rdoc +++ b/doc/language/format_specifications.rdoc @@ -30,8 +30,9 @@ It consists of: - A leading percent character. - Zero or more _flags_ (each is a character). -- An optional _width_ _specifier_ (an integer). -- An optional _precision_ _specifier_ (a period followed by a non-negative integer). +- An optional _width_ _specifier_ (an integer, or <tt>*</tt>). +- An optional _precision_ _specifier_ (a period followed by a non-negative + integer, or <tt>*</tt>). - A _type_ _specifier_ (a character). Except for the leading percent character, @@ -45,42 +46,42 @@ The links lead to the details and examples. === \Integer Type Specifiers - +b+ or +B+: Format +argument+ as a binary integer. - See {Specifiers b and B}[rdoc-ref:format_specifications.rdoc@Specifiers+b+and+B]. + See {Specifiers b and B}[rdoc-ref:@Specifiers+b+and+B]. - +d+, +i+, or +u+ (all are identical): Format +argument+ as a decimal integer. - See {Specifier d}[rdoc-ref:format_specifications.rdoc@Specifier+d]. + See {Specifier d}[rdoc-ref:@Specifier+d]. - +o+: Format +argument+ as an octal integer. - See {Specifier o}[rdoc-ref:format_specifications.rdoc@Specifier+o]. + See {Specifier o}[rdoc-ref:@Specifier+o]. - +x+ or +X+: Format +argument+ as a hexadecimal integer. - See {Specifiers x and X}[rdoc-ref:format_specifications.rdoc@Specifiers+x+and+X]. + See {Specifiers x and X}[rdoc-ref:@Specifiers+x+and+X]. === Floating-Point Type Specifiers - +a+ or +A+: Format +argument+ as hexadecimal floating-point number. - See {Specifiers a and A}[rdoc-ref:format_specifications.rdoc@Specifiers+a+and+A]. + See {Specifiers a and A}[rdoc-ref:@Specifiers+a+and+A]. - +e+ or +E+: Format +argument+ in scientific notation. - See {Specifiers e and E}[rdoc-ref:format_specifications.rdoc@Specifiers+e+and+E]. + See {Specifiers e and E}[rdoc-ref:@Specifiers+e+and+E]. - +f+: Format +argument+ as a decimal floating-point number. - See {Specifier f}[rdoc-ref:format_specifications.rdoc@Specifier+f]. + See {Specifier f}[rdoc-ref:@Specifier+f]. - +g+ or +G+: Format +argument+ in a "general" format. - See {Specifiers g and G}[rdoc-ref:format_specifications.rdoc@Specifiers+g+and+G]. + See {Specifiers g and G}[rdoc-ref:@Specifiers+g+and+G]. === Other Type Specifiers - +c+: Format +argument+ as a character. - See {Specifier c}[rdoc-ref:format_specifications.rdoc@Specifier+c]. + See {Specifier c}[rdoc-ref:@Specifier+c]. - +p+: Format +argument+ as a string via <tt>argument.inspect</tt>. - See {Specifier p}[rdoc-ref:format_specifications.rdoc@Specifier+p]. + See {Specifier p}[rdoc-ref:@Specifier+p]. - +s+: Format +argument+ as a string via <tt>argument.to_s</tt>. - See {Specifier s}[rdoc-ref:format_specifications.rdoc@Specifier+s]. + See {Specifier s}[rdoc-ref:@Specifier+s]. - <tt>%</tt>: Format +argument+ (<tt>'%'</tt>) as a single percent character. - See {Specifier %}[rdoc-ref:format_specifications.rdoc@Specifier+-25]. + See {Specifier %}[rdoc-ref:@Specifier+-25]. == Flags The effect of a flag may vary greatly among type specifiers. These remarks are general in nature. -See {type-specific details}[rdoc-ref:format_specifications.rdoc@Type+Specifier+Details+and+Examples]. +See {type-specific details}[rdoc-ref:@Type+Specifier+Details+and+Examples]. Multiple flags may be given with single type specifier; order does not matter. @@ -125,13 +126,6 @@ Left-pad with zeros instead of spaces: sprintf('%6d', 100) # => " 100" sprintf('%06d', 100) # => "000100" -=== <tt>'*'</tt> Flag - -Use the next argument as the field width: - - sprintf('%d', 20, 14) # => "20" - sprintf('%*d', 20, 14) # => " 14" - === <tt>'n$'</tt> Flag Format the (1-based) <tt>n</tt>th argument into this field: @@ -152,6 +146,11 @@ of the formatted field: # Ignore if too small. sprintf('%1d', 100) # => "100" +If the width specifier is <tt>'*'</tt> instead of an integer, the actual minimum +width is taken from the argument list: + + sprintf('%*d', 20, 14) # => " 14" + == Precision Specifier A precision specifier is a decimal point followed by zero or more @@ -194,6 +193,11 @@ the number of characters to write: sprintf('%s', Time.now) # => "2022-05-04 11:59:16 -0400" sprintf('%.10s', Time.now) # => "2022-05-04" +If the precision specifier is <tt>'*'</tt> instead of a non-negative integer, +the actual precision is taken from the argument list: + + sprintf('%.*d', 20, 1) # => "00000000000000000001" + == Type Specifier Details and Examples === Specifiers +a+ and +A+ diff --git a/doc/language/globals.md b/doc/language/globals.md new file mode 100644 index 0000000000..ece950d3d8 --- /dev/null +++ b/doc/language/globals.md @@ -0,0 +1,610 @@ +# Pre-Defined Global Variables + +Some of the pre-defined global variables have synonyms +that are available via module English. +For each of those, the \English synonym is given. + +To use the module: + +```ruby +require 'English' +``` + +## In Brief + +### Exceptions + +| Variable | \English | Contains | Initially | Read-Only | Reset By | +|:--------:|:-----------------:|----------------------------------------|:---------:|:---------:|--------------| +| `$!` | `$ERROR_INFO` | \Exception object or `nil` | `nil` | Yes | Kernel#raise | +| `$@` | `$ERROR_POSITION` | \Array of backtrace positions or `nil` | `nil` | Yes | Kernel#raise | + +### Matched \Data + +| Variable | \English | Contains | Initially | Read-Only | Reset By | +|:---------:|:-------------------:|-----------------------------------|:---------:|:---------:|-----------------| +| `$~` | `$LAST_MATCH_INFO` | \MatchData object or `nil` | `nil` | No | Matcher methods | +| `$&` | `$MATCH` | Matched substring or `nil` | `nil` | No | Matcher methods | +| `` $` `` | `$PRE_MATCH` | Substring left of match or `nil` | `nil` | No | Matcher methods | +| `$'` | `$POST_MATCH` | Substring right of match or `nil` | `nil` | No | Matcher methods | +| `$+` | `$LAST_PAREN_MATCH` | Last group matched or `nil` | `nil` | No | Matcher methods | +| `$1` | | First group matched or `nil` | `nil` | Yes | Matcher methods | +| `$2` | | Second group matched or `nil` | `nil` | Yes | Matcher methods | +| `$n` | | <i>n</i>th group matched or `nil` | `nil` | Yes | Matcher methods | + +### Separators + +| Variable | \English | Contains | Initially | Read-Only | Reset By | +|:-----------:|:---------------------------:|-------------------------|:---------:|:---------:|----------| +| `$/`, `$-0` | `$INPUT_RECORD_SEPARATOR` | Input record separator | Newline | No | | +| `$\` | `$OUTPUT_RECORD_SEPARATOR` | Output record separator | `nil` | No | | + +### Streams + +| Variable | \English | Contains | Initially | Read-Only | Reset By | +|:---------:|:----------------------------:|---------------------------------------------|:---------:|:---------:|----------------------| +| `$stdin` | | Standard input stream | `STDIN` | No | | +| `$stdout` | | Standard output stream | `STDOUT` | No | | +| `$stderr` | | Standard error stream | `STDERR` | No | | +| `$<` | `$DEFAULT_INPUT` | Default standard input | `ARGF` | Yes | | +| `$>` | `$DEFAULT_OUTPUT` | Default standard output | `STDOUT` | No | | +| `$.` | `$INPUT_LINE_NUMBER`, `$NR` | Input position of most recently read stream | 0 | No | Certain read methods | +| `$_` | `$LAST_READ_LINE` | String from most recently read stream | `nil` | No | Certain read methods | + +### Processes + +| Variable | \English | Contains | Initially | Read-Only | Reset By | +|:-------------------------:|:----------------------:|---------------------------------|:-------------:|:---------:|----------| +| `$0`, `$PROGRAM_NAME` | | Program name | Program name | No | | +| `$*` | `$ARGV` | \ARGV array | `ARGV` | Yes | | +| `$$` | `$PROCESS_ID`, `$PID` | Process id | Process PID | Yes | | +| `$?` | `$CHILD_STATUS` | Status of recently exited child | `nil` | Yes | | +| `$LOAD_PATH`, `$:`, `$-I` | | \Array of search paths | Ruby defaults | Yes | | +| `$LOADED_FEATURES`, `$"` | | \Array of load paths | Ruby defaults | Yes | | + +### Debugging + +| Variable | \English | Contains | Initially | Read-Only | Reset By | +|:-----------:|:--------:|--------------------------------------------|:----------------------------:|:---------:|----------| +| `$FILENAME` | | Value returned by method `ARGF.filename` | Command-line argument or '-' | Yes | | +| `$DEBUG` | | Whether option `-d` or `--debug` was given | Command-line option | No | | +| `$VERBOSE` | | Whether option `-V` or `-W` was given | Command-line option | No | | + +### Other Variables + +| Variable | \English | Contains | Initially | Read-Only | Reset By | +|:-----------:|:--------:|-----------------------------------------------|:---------:|:---------:|----------| +| `$-F`, `$;` | | Separator given with command-line option `-F` | | | | +| `$-a` | | Whether option `-a` was given | | Yes | | +| `$-i` | | Extension given with command-line option `-i` | | No | | +| `$-l` | | Whether option `-l` was given | | Yes | | +| `$-p` | | Whether option `-p` was given | | Yes | | +| `$F` | | \Array of `$_` split by `$-F` | | | | + +## Exceptions + +### `$!` (\Exception) + +Contains the Exception object set by Kernel#raise: + +```ruby +begin + raise RuntimeError.new('Boo!') +rescue RuntimeError + p $! +end +``` + +Output: + +``` +#<RuntimeError: Boo!> +``` + +English - `$ERROR_INFO` + +### `$@` (Backtrace) + +Same as `$!.backtrace`; +returns an array of backtrace positions: + +```ruby +begin + raise RuntimeError.new('Boo!') +rescue RuntimeError + pp $@.take(4) +end +``` + +Output: + +``` +["(irb):338:in `<top (required)>'", + "/snap/ruby/317/lib/ruby/3.2.0/irb/workspace.rb:119:in `eval'", + "/snap/ruby/317/lib/ruby/3.2.0/irb/workspace.rb:119:in `evaluate'", + "/snap/ruby/317/lib/ruby/3.2.0/irb/context.rb:502:in `evaluate'"] +``` + +English - `$ERROR_POSITION`. + +## Matched \Data + +These global variables store information about the most recent +successful match in the current scope. + +For details and examples, +see {Regexp Global Variables}[rdoc-ref:Regexp@Global+Variables]. + +### `$~` (\MatchData) + +MatchData object created from the match; +thread-local and frame-local. + +English - `$LAST_MATCH_INFO`. + +### `$&` (Matched Substring) + +The matched string. + +English - `$MATCH`. + +### `` $` `` (Pre-Match Substring) +The string to the left of the match. + +English - `$PREMATCH`. + +### `$'` (Post-Match Substring) + +The string to the right of the match. + +English - `$POSTMATCH`. + +### `$+` (Last Matched Group) + +The last group matched. + +English - `$LAST_PAREN_MATCH`. + +### `$1`, `$2`, \Etc. (Matched Group) + +For <tt>$n</tt> the <i>n</i>th group of the match. + +No \English. + +## Separators + +### `$/` (Input Record Separator) + +An input record separator, initially newline. +Set by the [command-line option `-0`]. + +Setting to non-nil value by other than the command-line option is +deprecated. + +English - `$INPUT_RECORD_SEPARATOR`, `$RS`. + +Aliased as `$-0`. + +### `$\` (Output Record Separator) + +An output record separator, initially `nil`. + +Copied from `$/` when the [command-line option `-l`] is +given. + +Setting to non-nil value by other than the command-line option is +deprecated. + +English - `$OUTPUT_RECORD_SEPARATOR`, `$ORS`. + +## Streams + +### `$stdin` (Standard Input) + +The current standard input stream; initially: + +```ruby +$stdin # => #<IO:<STDIN>> +``` + +### `$stdout` (Standard Output) + +The current standard output stream; initially: + +```ruby +$stdout # => #<IO:<STDOUT>> +``` + +### `$stderr` (Standard Error) + +The current standard error stream; initially: + +```ruby +$stderr # => #<IO:<STDERR>> +``` + +### `$<` (\ARGF or $stdin) + +Points to stream ARGF if not empty, else to stream $stdin; read-only. + +English - `$DEFAULT_INPUT`. + +### `$>` (Default Standard Output) + +An output stream, initially `$stdout`. + +English - `$DEFAULT_OUTPUT` + +### `$.` (Input Position) + +The input position (line number) in the most recently read stream. + +English - `$INPUT_LINE_NUMBER`, `$NR` + +### `$_` (Last Read Line) + +The line (string) from the most recently read stream. + +English - `$LAST_READ_LINE`. + +## Processes + +### `$0` + +Initially, contains the name of the script being executed; +may be reassigned. + +### `$*` (\ARGV) + +Points to ARGV. + +English - `$ARGV`. + +### `$$` (Process ID) + +The process ID of the current process. Same as Process.pid. + +English - `$PROCESS_ID`, `$PID`. + +### `$?` (Child Status) + +Initially `nil`, otherwise the Process::Status object +created for the most-recently exited child process; +thread-local. + +English - `$CHILD_STATUS`. + +### `$LOAD_PATH` (Load Path) + +Contains the array of paths to be searched +by Kernel#load and Kernel#require. + +Singleton method `$LOAD_PATH.resolve_feature_path(feature)` +returns: + +- <tt>[:rb, path]</tt>, where `path` is the path to the Ruby file to be + loaded for the given `feature`. +- <tt>[:so, path]</tt>, where `path` is the path to the shared object file + to be loaded for the given `feature`. +- `nil` if there is no such `feature` and `path`. + +Examples: + +```ruby +$LOAD_PATH.resolve_feature_path('timeout') +# => [:rb, "/snap/ruby/317/lib/ruby/3.2.0/timeout.rb"] +$LOAD_PATH.resolve_feature_path('date_core') +# => [:so, "/snap/ruby/317/lib/ruby/3.2.0/x86_64-linux/date_core.so"] +$LOAD_PATH.resolve_feature_path('foo') +# => nil +``` + +Aliased as `$:` and `$-I`. + +### `$LOADED_FEATURES` + +Contains an array of the paths to the loaded files: + +```ruby +$LOADED_FEATURES.take(10) +# => +["enumerator.so", + "thread.rb", + "fiber.so", + "rational.so", + "complex.so", + "ruby2_keywords.rb", + "/snap/ruby/317/lib/ruby/3.2.0/x86_64-linux/enc/encdb.so", + "/snap/ruby/317/lib/ruby/3.2.0/x86_64-linux/enc/trans/transdb.so", + "/snap/ruby/317/lib/ruby/3.2.0/x86_64-linux/rbconfig.rb", + "/snap/ruby/317/lib/ruby/3.2.0/rubygems/compatibility.rb"] +``` + +Aliased as `$"`. + +## Debugging + +### `$FILENAME` + +The value returned by method ARGF.filename. + +### `$DEBUG` + +Initially `true` if [command-line option `-d`] or +[`--debug`][command-line option `-d`] is given, otherwise initially `false`; +may be set to either value in the running program. + +When `true`, prints each raised exception to `$stderr`. + +Aliased as `$-d`. + +### `$VERBOSE` + +Initially `true` if [command-line option `-v`] or +[`-w`][command-line option `-w`] is given, otherwise initially `false`; +may be set to either value, or to `nil`, in the running program. + +When `true`, enables Ruby warnings. + +When `nil`, disables warnings, including those from Kernel#warn. + +Aliased as `$-v` and `$-w`. + +## Other Variables + +### `$-F` + +The default field separator in String#split; must be a String or a +Regexp, and can be set with [command-line option `-F`]. + +Setting to non-nil value by other than the command-line option is +deprecated. + +Aliased as `$;`. + +### `$-a` + +Whether [command-line option `-a`] was given; read-only. + +### `$-i` + +Contains the extension given with [command-line option `-i`], +or `nil` if none. + +An alias of ARGF.inplace_mode. + +### `$-l` + +Whether [command-line option `-l`] was set; read-only. + +### `$-p` + +Whether [command-line option `-p`] was given; read-only. + +### `$F` + +If the [command-line option `-a`] is given, the array +obtained by splitting `$_` by `$-F` is assigned at the start of each +`-l`/`-p` loop. + +## Deprecated + +### `$=` + +### `$,` + +# Pre-Defined Global Constants + +## Summary + +### Streams + +| Constant | Contains | +|:--------:|-------------------------| +| `STDIN` | Standard input stream. | +| `STDOUT` | Standard output stream. | +| `STDERR` | Standard error stream. | + +### Environment + +| Constant | Contains | +|-----------------------|-------------------------------------------------------------------------------| +| `ENV` | Hash of current environment variable names and values. | +| `ARGF` | String concatenation of files given on the command line, or `$stdin` if none. | +| `ARGV` | Array of the given command-line arguments. | +| `TOPLEVEL_BINDING` | Binding of the top level scope. | +| `RUBY_VERSION` | String Ruby version. | +| `RUBY_RELEASE_DATE` | String Ruby release date. | +| `RUBY_PLATFORM` | String Ruby platform. | +| `RUBY_PATCH_LEVEL` | String Ruby patch level. | +| `RUBY_REVISION` | String Ruby revision. | +| `RUBY_COPYRIGHT` | String Ruby copyright. | +| `RUBY_ENGINE` | String Ruby engine. | +| `RUBY_ENGINE_VERSION` | String Ruby engine version. | +| `RUBY_DESCRIPTION` | String Ruby description. | + +### Embedded \Data + +| Constant | Contains | +|:---------------------:|-------------------------------------------------------------------------------| +| `DATA` | File containing embedded data (lines following `__END__`, if any). | + +## Streams + +### `STDIN` + +The standard input stream (the default value for `$stdin`): + +```ruby +STDIN # => #<IO:<STDIN>> +``` + +### `STDOUT` + +The standard output stream (the default value for `$stdout`): + +```ruby +STDOUT # => #<IO:<STDOUT>> +``` + +### `STDERR` + +The standard error stream (the default value for `$stderr`): + +```ruby +STDERR # => #<IO:<STDERR>> +``` + +## Environment + +### `ENV` + +A hash of the contains current environment variables names and values: + +```ruby +ENV.take(5) +# => +[["COLORTERM", "truecolor"], + ["DBUS_SESSION_BUS_ADDRESS", "unix:path=/run/user/1000/bus"], + ["DESKTOP_SESSION", "ubuntu"], + ["DISPLAY", ":0"], + ["GDMSESSION", "ubuntu"]] +``` + +### `ARGF` + +The virtual concatenation of the files given on the command line, or from +`$stdin` if no files were given, `"-"` is given, or after +all files have been read. + +### `ARGV` + +An array of the given command-line arguments. + +### `TOPLEVEL_BINDING` + +The Binding of the top level scope: + +```ruby +TOPLEVEL_BINDING # => #<Binding:0x00007f58da0da7c0> +``` + +### `RUBY_VERSION` + +The Ruby version: + +```ruby +RUBY_VERSION # => "3.2.2" +``` + +### `RUBY_RELEASE_DATE` + +The release date string: + +```ruby +RUBY_RELEASE_DATE # => "2023-03-30" +``` + +### `RUBY_PLATFORM` + +The platform identifier: + +```ruby +RUBY_PLATFORM # => "x86_64-linux" +``` + +### `RUBY_PATCHLEVEL` + +The integer patch level for this Ruby: + +```ruby +RUBY_PATCHLEVEL # => 53 +``` + +For a development build the patch level will be -1. + +### `RUBY_REVISION` + +The git commit hash for this Ruby: + +```ruby +RUBY_REVISION # => "e51014f9c05aa65cbf203442d37fef7c12390015" +``` + +### `RUBY_COPYRIGHT` + +The copyright string: + +```ruby +RUBY_COPYRIGHT +# => "ruby - Copyright (C) 1993-2023 Yukihiro Matsumoto" +``` + +### `RUBY_ENGINE` + +The name of the Ruby implementation: + +```ruby +RUBY_ENGINE # => "ruby" +``` + +### `RUBY_ENGINE_VERSION` + +The version of the Ruby implementation: + +```ruby +RUBY_ENGINE_VERSION # => "3.2.2" +``` + +### `RUBY_DESCRIPTION` + +The description of the Ruby implementation: + +```ruby +RUBY_DESCRIPTION +# => "ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux]" +``` + +## Embedded \Data + +### `DATA` + +Defined if and only if the program has this line: + +```ruby +__END__ +``` + +When defined, `DATA` is a File object +containing the lines following the `__END__`, +positioned at the first of those lines: + +```ruby +p DATA +DATA.each_line { |line| p line } +__END__ +Foo +Bar +Baz +``` + +Output: + +``` +#<File:t.rb> +"Foo\n" +"Bar\n" +"Baz\n" +``` + + +[command-line option `-0`]: rdoc-ref:language/options.md@0-3A+Set+-24-2F+-28Input+Record+Separator-29 +[command-line option `-F`]: rdoc-ref:language/options.md@F-3A+Set+Input+Field+Separator +[command-line option `-a`]: rdoc-ref:language/options.md@a-3A+Split+Input+Lines+into+Fields +[command-line option `-d`]: rdoc-ref:language/options.md@d-3A+Set+-24DEBUG+to+true +[command-line option `-i`]: rdoc-ref:language/options.md@i-3A+Set+ARGF+In-Place+Mode +[command-line option `-l`]: rdoc-ref:language/options.md@l-3A+Set+Output+Record+Separator-3B+Chop+Lines +[command-line option `-p`]: rdoc-ref:language/options.md@p-3A+-n-2C+with+Printing +[command-line option `-v`]: rdoc-ref:language/options.md@v-3A+Print+Version-3B+Set+-24VERBOSE +[command-line option `-w`]: rdoc-ref:language/options.md@w-3A+Synonym+for+-W1 + diff --git a/doc/hash_inclusion.rdoc b/doc/language/hash_inclusion.rdoc index 05c2b0932a..05c2b0932a 100644 --- a/doc/hash_inclusion.rdoc +++ b/doc/language/hash_inclusion.rdoc diff --git a/doc/implicit_conversion.rdoc b/doc/language/implicit_conversion.rdoc index e244096125..e244096125 100644 --- a/doc/implicit_conversion.rdoc +++ b/doc/language/implicit_conversion.rdoc diff --git a/doc/marshal.rdoc b/doc/language/marshal.rdoc index 740064ade6..740064ade6 100644 --- a/doc/marshal.rdoc +++ b/doc/language/marshal.rdoc diff --git a/doc/ruby/option_dump.md b/doc/language/option_dump.md index a156484bf6..a156484bf6 100644 --- a/doc/ruby/option_dump.md +++ b/doc/language/option_dump.md diff --git a/doc/ruby/options.md b/doc/language/options.md index bfbd2530de..3421d73f55 100644 --- a/doc/ruby/options.md +++ b/doc/language/options.md @@ -61,15 +61,15 @@ nil See also: -- {Option -a}[rdoc-ref:ruby/options.md@a-3A+Split+Input+Lines+into+Fields]: +- {Option -a}[rdoc-ref:@a-3A+Split+Input+Lines+into+Fields]: Split input lines into fields. -- {Option -F}[rdoc-ref:ruby/options.md@F-3A+Set+Input+Field+Separator]: +- {Option -F}[rdoc-ref:@F-3A+Set+Input+Field+Separator]: Set input field separator. -- {Option -l}[rdoc-ref:ruby/options.md@l-3A+Set+Output+Record+Separator-3B+Chop+Lines]: +- {Option -l}[rdoc-ref:@l-3A+Set+Output+Record+Separator-3B+Chop+Lines]: Set output record separator; chop lines. -- {Option -n}[rdoc-ref:ruby/options.md@n-3A+Run+Program+in+gets+Loop]: +- {Option -n}[rdoc-ref:@n-3A+Run+Program+in+gets+Loop]: Run program in `gets` loop. -- {Option -p}[rdoc-ref:ruby/options.md@p-3A+-n-2C+with+Printing]: +- {Option -p}[rdoc-ref:@p-3A+-n-2C+with+Printing]: `-n`, with printing. ### `-a`: Split Input Lines into Fields @@ -91,15 +91,15 @@ and the default field separator is `$;`. See also: -- {Option -0}[rdoc-ref:ruby/options.md@0-3A+Set+-24-2F+-28Input+Record+Separator-29]: +- {Option -0}[rdoc-ref:@0-3A+Set+-24-2F+-28Input+Record+Separator-29]: Set `$/` (input record separator). -- {Option -F}[rdoc-ref:ruby/options.md@F-3A+Set+Input+Field+Separator]: +- {Option -F}[rdoc-ref:@F-3A+Set+Input+Field+Separator]: Set input field separator. -- {Option -l}[rdoc-ref:ruby/options.md@l-3A+Set+Output+Record+Separator-3B+Chop+Lines]: +- {Option -l}[rdoc-ref:@l-3A+Set+Output+Record+Separator-3B+Chop+Lines]: Set output record separator; chop lines. -- {Option -n}[rdoc-ref:ruby/options.md@n-3A+Run+Program+in+gets+Loop]: +- {Option -n}[rdoc-ref:@n-3A+Run+Program+in+gets+Loop]: Run program in `gets` loop. -- {Option -p}[rdoc-ref:ruby/options.md@p-3A+-n-2C+with+Printing]: +- {Option -p}[rdoc-ref:@p-3A+-n-2C+with+Printing]: `-n`, with printing. ### `-c`: Check Syntax @@ -221,15 +221,15 @@ The argument must immediately follow the option See also: -- {Option -0}[rdoc-ref:ruby/options.md@0-3A+Set+-24-2F+-28Input+Record+Separator-29]: +- {Option -0}[rdoc-ref:@0-3A+Set+-24-2F+-28Input+Record+Separator-29]: Set `$/` (input record separator). -- {Option -a}[rdoc-ref:ruby/options.md@a-3A+Split+Input+Lines+into+Fields]: +- {Option -a}[rdoc-ref:@a-3A+Split+Input+Lines+into+Fields]: Split input lines into fields. -- {Option -l}[rdoc-ref:ruby/options.md@l-3A+Set+Output+Record+Separator-3B+Chop+Lines]: +- {Option -l}[rdoc-ref:@l-3A+Set+Output+Record+Separator-3B+Chop+Lines]: Set output record separator; chop lines. -- {Option -n}[rdoc-ref:ruby/options.md@n-3A+Run+Program+in+gets+Loop]: +- {Option -n}[rdoc-ref:@n-3A+Run+Program+in+gets+Loop]: Run program in `gets` loop. -- {Option -p}[rdoc-ref:ruby/options.md@p-3A+-n-2C+with+Printing]: +- {Option -p}[rdoc-ref:@p-3A+-n-2C+with+Printing]: `-n`, with printing. ### `-h`: Print Short Help Message @@ -295,7 +295,7 @@ $ ruby -n -e 'p $_' desiderata.txt "be on good terms with all persons.\n" ``` -With option `-l' (chopped): +With option `-l` (chopped): ```console $ ruby -ln -e 'p $_' desiderata.txt @@ -307,15 +307,15 @@ $ ruby -ln -e 'p $_' desiderata.txt See also: -- {Option -0}[rdoc-ref:ruby/options.md@0-3A+Set+-24-2F+-28Input+Record+Separator-29]: +- {Option -0}[rdoc-ref:@0-3A+Set+-24-2F+-28Input+Record+Separator-29]: Set `$/` (input record separator). -- {Option -a}[rdoc-ref:ruby/options.md@a-3A+Split+Input+Lines+into+Fields]: +- {Option -a}[rdoc-ref:@a-3A+Split+Input+Lines+into+Fields]: Split input lines into fields. -- {Option -F}[rdoc-ref:ruby/options.md@F-3A+Set+Input+Field+Separator]: +- {Option -F}[rdoc-ref:@F-3A+Set+Input+Field+Separator]: Set input field separator. -- {Option -n}[rdoc-ref:ruby/options.md@n-3A+Run+Program+in+gets+Loop]: +- {Option -n}[rdoc-ref:@n-3A+Run+Program+in+gets+Loop]: Run program in `gets` loop. -- {Option -p}[rdoc-ref:ruby/options.md@p-3A+-n-2C+with+Printing]: +- {Option -p}[rdoc-ref:@p-3A+-n-2C+with+Printing]: `-n`, with printing. ### `-n`: Run Program in `gets` Loop @@ -341,15 +341,15 @@ be on good terms with all persons. See also: -- {Option -0}[rdoc-ref:ruby/options.md@0-3A+Set+-24-2F+-28Input+Record+Separator-29]: +- {Option -0}[rdoc-ref:@0-3A+Set+-24-2F+-28Input+Record+Separator-29]: Set `$/` (input record separator). -- {Option -a}[rdoc-ref:ruby/options.md@a-3A+Split+Input+Lines+into+Fields]: +- {Option -a}[rdoc-ref:@a-3A+Split+Input+Lines+into+Fields]: Split input lines into fields. -- {Option -F}[rdoc-ref:ruby/options.md@F-3A+Set+Input+Field+Separator]: +- {Option -F}[rdoc-ref:@F-3A+Set+Input+Field+Separator]: Set input field separator. -- {Option -l}[rdoc-ref:ruby/options.md@l-3A+Set+Output+Record+Separator-3B+Chop+Lines]: +- {Option -l}[rdoc-ref:@l-3A+Set+Output+Record+Separator-3B+Chop+Lines]: Set output record separator; chop lines. -- {Option -p}[rdoc-ref:ruby/options.md@p-3A+-n-2C+with+Printing]: +- {Option -p}[rdoc-ref:@p-3A+-n-2C+with+Printing]: `-n`, with printing. ### `-p`: `-n`, with Printing @@ -370,15 +370,15 @@ be on good terms with all persons. See also: -- {Option -0}[rdoc-ref:ruby/options.md@0-3A+Set+-24-2F+-28Input+Record+Separator-29]: +- {Option -0}[rdoc-ref:@0-3A+Set+-24-2F+-28Input+Record+Separator-29]: Set `$/` (input record separator). -- {Option -a}[rdoc-ref:ruby/options.md@a-3A+Split+Input+Lines+into+Fields]: +- {Option -a}[rdoc-ref:@a-3A+Split+Input+Lines+into+Fields]: Split input lines into fields. -- {Option -F}[rdoc-ref:ruby/options.md@F-3A+Set+Input+Field+Separator]: +- {Option -F}[rdoc-ref:@F-3A+Set+Input+Field+Separator]: Set input field separator. -- {Option -l}[rdoc-ref:ruby/options.md@l-3A+Set+Output+Record+Separator-3B+Chop+Lines]: +- {Option -l}[rdoc-ref:@l-3A+Set+Output+Record+Separator-3B+Chop+Lines]: Set output record separator; chop lines. -- {Option -n}[rdoc-ref:ruby/options.md@n-3A+Run+Program+in+gets+Loop]: +- {Option -n}[rdoc-ref:@n-3A+Run+Program+in+gets+Loop]: Run program in `gets` loop. ### `-r`: Require Library @@ -427,7 +427,7 @@ $ ruby -s t.rb -foo=baz -bar=bat ``` The option may not be used with -{option -e}[rdoc-ref:ruby/options.md@e-3A+Execute+Given+Ruby+Code] +{option -e}[rdoc-ref:@e-3A+Execute+Given+Ruby+Code] ### `-S`: Search Directories in `ENV['PATH']` @@ -576,7 +576,7 @@ ruby - Copyright (C) 1993-2024 Yukihiro Matsumoto ### `--debug`: Alias for `-d` Option `--debug` is an alias for -{option -d}[rdoc-ref:ruby/options.md@d-3A+Set+-24DEBUG+to+true]. +{option -d}[rdoc-ref:@d-3A+Set+-24DEBUG+to+true]. ### `--disable`: Disable Features @@ -610,9 +610,9 @@ option was given: - `--dump=help`: Same as {option \-\-help}[options_md.html#label--help-3A+Print+Help+Message]. - `--dump=syntax`: - Same as {option -c}[rdoc-ref:ruby/options.md@c-3A+Check+Syntax]. + Same as {option -c}[rdoc-ref:@c-3A+Check+Syntax]. - `--dump=usage`: - Same as {option -h}[rdoc-ref:ruby/options.md@h-3A+Print+Short+Help+Message]. + Same as {option -h}[rdoc-ref:@h-3A+Print+Short+Help+Message]. - `--dump=version`: Same as {option \-\-version}[options_md.html#label--version-3A+Print+Ruby+Version]. @@ -634,13 +634,13 @@ see {option --disable}[options_md.html#label--disable-3A+Disable+Features]. ### `--encoding`: Alias for `-E`. Option `--encoding` is an alias for -{option -E}[rdoc-ref:ruby/options.md@E-3A+Set+Default+Encodings]. +{option -E}[rdoc-ref:@E-3A+Set+Default+Encodings]. ### `--external-encoding`: Set Default External \Encoding Option `--external-encoding` sets the default external encoding for the invoked Ruby program; -for values of +encoding+, +for values of `encoding`, see {Encoding: Names and Aliases}[rdoc-ref:encodings.rdoc@Names+and+Aliases]. ```console @@ -662,7 +662,7 @@ For a shorter help message, use option `-h`. Option `--internal-encoding` sets the default internal encoding for the invoked Ruby program; -for values of +encoding+, +for values of `encoding`, see {Encoding: Names and Aliases}[rdoc-ref:encodings.rdoc@Names+and+Aliases]. ```console @@ -672,6 +672,11 @@ $ ruby --internal-encoding=cesu-8 -e 'puts Encoding::default_internal' CESU-8 ``` +### `--jit` + +Option `--jit` is an alias for option `--yjit`, which enables YJIT; +see additional YJIT options in the [YJIT documentation](rdoc-ref:jit/yjit.md). + ### `--verbose`: Set `$VERBOSE` Option `--verbose` sets global variable `$VERBOSE` to `true` @@ -681,44 +686,3 @@ and disables input from `$stdin`. Option `--version` prints the version of the Ruby interpreter, then exits. -## Experimental Options - -These options are experimental in the current Ruby release, -and may be modified or withdrawn in later releases. - -### `--jit` - -Option `-jit` enables JIT compilation with the default option. - -#### `--jit-debug` - -Option `--jit-debug` enables JIT debugging (very slow); -adds compiler flags if given. - -#### `--jit-max-cache=num` - -Option `--jit-max-cache=num` sets the maximum number of methods -to be JIT-ed in a cache; default: 100). - -#### `--jit-min-calls=num` - -Option `jit-min-calls=num` sets the minimum number of calls to trigger JIT -(for testing); default: 10000). - -#### `--jit-save-temps` - -Option `--jit-save-temps` saves JIT temporary files in $TMP or /tmp (for testing). - -#### `--jit-verbose` - -Option `--jit-verbose` prints JIT logs of level `num` or less -to `$stderr`; default: 0. - -#### `--jit-wait` - -Option `--jit-wait` waits until JIT compilation finishes every time (for testing). - -#### `--jit-warnings` - -Option `--jit-warnings` enables printing of JIT warnings. - diff --git a/doc/packed_data.rdoc b/doc/language/packed_data.rdoc index b33eed58e7..597db5139f 100644 --- a/doc/packed_data.rdoc +++ b/doc/language/packed_data.rdoc @@ -151,7 +151,7 @@ Any directive may be followed by either of these modifiers: [65, 66].pack('C*') # => "AB" 'AB'.unpack('C*') # => [65, 66] -- Integer +count+ - The directive is to be applied +count+ times: +- \Integer +count+ - The directive is to be applied +count+ times: [65, 66].pack('C2') # => "AB" [65, 66].pack('C3') # Raises ArgumentError. @@ -159,7 +159,7 @@ Any directive may be followed by either of these modifiers: 'AB'.unpack('C3') # => [65, 66, nil] Note: Directives in <tt>%w[A a Z m]</tt> use +count+ differently; - see {String Directives}[rdoc-ref:packed_data.rdoc@String+Directives]. + see {\String Directives}[rdoc-ref:@String+Directives]. If elements don't fit the provided directive, only least significant bits are encoded: @@ -340,6 +340,22 @@ for one element in the input or output array. s.unpack('U*') # => [4194304] +- <tt>'r'</tt> - Signed LEB128-encoded integer + (see {Signed LEB128}[https://en.wikipedia.org/wiki/LEB128#Signed_LEB128]) + + s = [1, 127, -128, 16383, -16384].pack("r*") + # => "\x01\xFF\x00\x80\x7F\xFF\xFF\x00\x80\x80\x7F" + s.unpack('r*') + # => [1, 127, -128, 16383, -16384] + +- <tt>'R'</tt> - Unsigned LEB128-encoded integer + (see {Unsigned LEB128}[https://en.wikipedia.org/wiki/LEB128#Unsigned_LEB128]) + + s = [1, 127, 128, 16383, 16384].pack("R*") + # => "\x01\x7F\x80\x01\xFF\x7F\x80\x80\x01" + s.unpack('R*') + # => [1, 127, 128, 16383, 16384] + - <tt>'w'</tt> - BER-encoded integer (see {BER encoding}[https://en.wikipedia.org/wiki/X.690#BER_encoding]): @@ -441,15 +457,15 @@ for one byte in the input or output string. "foo ".unpack('A4') # => ["foo"] "foo".unpack('A4') # => ["foo"] - russian = "\u{442 435 441 442}" # => "тест" - russian.size # => 4 - russian.bytesize # => 8 - [russian].pack('A') # => "\xD1" - [russian].pack('A*') # => "\xD1\x82\xD0\xB5\xD1\x81\xD1\x82" - russian.unpack('A') # => ["\xD1"] - russian.unpack('A2') # => ["\xD1\x82"] - russian.unpack('A4') # => ["\xD1\x82\xD0\xB5"] - russian.unpack('A*') # => ["\xD1\x82\xD0\xB5\xD1\x81\xD1\x82"] + japanese = 'こんにちは' + japanese.size # => 5 + japanese.bytesize # => 15 + [japanese].pack('A') # => "\xE3" + [japanese].pack('A*') # => "\xE3\x81\x93\xE3\x82\x93\xE3\x81\xAB\xE3\x81\xA1\xE3\x81\xAF" + japanese.unpack('A') # => ["\xE3"] + japanese.unpack('A2') # => ["\xE3\x81"] + japanese.unpack('A4') # => ["\xE3\x81\x93\xE3"] + japanese.unpack('A*') # => ["\xE3\x81\x93\xE3\x82\x93\xE3\x81\xAB\xE3\x81\xA1\xE3\x81\xAF"] - <tt>'a'</tt> - Arbitrary binary string (null padded; count is width): diff --git a/doc/language/ractor.md b/doc/language/ractor.md new file mode 100644 index 0000000000..72fbde6e5a --- /dev/null +++ b/doc/language/ractor.md @@ -0,0 +1,797 @@ +# Ractor - Ruby's Actor-like concurrency abstraction + +Ractors are designed to provide parallel execution of Ruby code without thread-safety concerns. + +## Summary + +### Multiple Ractors in a ruby process + +You can create multiple Ractors which can run ruby code in parallel with each other. + +* `Ractor.new{ expr }` creates a new Ractor and `expr` can run in parallel with other ractors on a multi-core computer. +* Ruby processes start with one ractor (called the *main ractor*). +* If the main ractor terminates, all other ractors receive termination requests, similar to how threads behave. +* Each Ractor contains one or more `Thread`s. + * Threads within the same ractor share a ractor-wide global lock (GVL in MRI terminology), so they can't run in parallel wich each other (without releasing the GVL explicitly in C extensions). Threads in different ractors can run in parallel. + * The overhead of creating a ractor is slightly above the overhead of creating a thread. + +### Limited sharing between Ractors + +Ractors don't share all objects, unlike threads which can access any object other than objects stored in another thread's thread-locals. + +* Most objects are *unshareable objects*. Unshareable objects can only be used by the ractor that instantiated them, so you don't need to worry about thread-safety issues resulting from using the object concurrently across ractors. +* Some objects are *shareable objects*. Here is an incomplete list to give you an idea: + * `i = 123`: All `Integer`s are shareable. + * `s = "str".freeze`: Frozen strings are shareable if they have no instance variables that refer to unshareable objects. + * `a = [1, [2], 3].freeze`: `a` is not a shareable object because `a` refers to the unshareable object `[2]` (this Array is not frozen). + * `h = {c: Object}.freeze`: `h` is shareable because `Symbol`s and `Class`es are shareable, and the Hash is frozen. + * Class/Module objects are always shareable, even if they refer to unshareable objects. + * Special shareable objects + * Ractor objects themselves are shareable. + * And more... + +### Communication between Ractors with `Ractor::Port` + +Ractors communicate with each other and synchronize their execution by exchanging messages. The `Ractor::Port` class provides this communication mechanism. + +```ruby +port = Ractor::Port.new + +Ractor.new port do |port| + # Other ractors can send to the port + port << 42 +end + +port.receive # get a message from the port. Only the ractor that created the Port can receive from it. +#=> 42 +``` + +All Ractors have a default port, which `Ractor#send`, `Ractor.receive` (etc) will use. + +### Copy & Move semantics when sending objects + +To send unshareable objects to another ractor, objects are either copied or moved. + +* Copy: deep-copies the object to the other ractor. All unshareable objects will be `Kernel#clone`ed. +* Move: moves membership to another ractor. + * The sending ractor can not access the moved object after it moves. + * There is a guarantee that only one ractor can access an unshareable object at once. + +### Thread-safety + +Ractors help to write thread-safe, concurrent programs. They allow sharing of data only through explicit message passing for +unshareable objects. Shareable objects are guaranteed to work correctly across ractors, even if the ractors are running in parallel. +This guarantee, however, only applies across ractors. You still need to use `Mutex`es and other thread-safety tools within a ractor if +you're using multiple ruby `Thread`s. + + * Most objects are unshareable. You can't create data-races across ractors due to the inability to use these objects across ractors. + * Shareable objects are protected by locks (or otherwise don't need to be) so they can be used by more than one ractor at once. + +## Creation and termination + +### `Ractor.new` + +* `Ractor.new { expr }` creates a Ractor. + +```ruby +# Ractor.new with a block creates a new Ractor +r = Ractor.new do + # This block can run in parallel with other ractors +end + +# You can name a Ractor with a `name:` argument. +r = Ractor.new name: 'my-first-ractor' do +end + +r.name #=> 'my-first-ractor' +``` + +### Block isolation + +The Ractor executes `expr` in the given block. +The given block will be isolated from its outer scope. To prevent sharing objects between ractors, outer variables, `self` and other information is isolated from the block. + +This isolation occurs at Ractor creation time (when `Ractor.new` is called). If the given block is not able to be isolated because of outer variables or `self`, an error will be raised. + +```ruby +begin + a = true + r = Ractor.new do + a #=> ArgumentError because this block accesses outer variable `a`. + end + r.join # wait for ractor to finish +rescue ArgumentError +end +``` + +* The `self` of the given block is the `Ractor` object itself. + +```ruby +r = Ractor.new do + p self.class #=> Ractor + self.object_id +end +r.value == self.object_id #=> false +``` + +Arguments passed to `Ractor.new()` become block parameters for the given block. However, Ruby does not pass the objects themselves, but sends them as messages (see below for details). + +```ruby +r = Ractor.new 'ok' do |msg| + msg #=> 'ok' +end +r.value #=> 'ok' +``` + +```ruby +# similar to the last example +r = Ractor.new do + msg = Ractor.receive + msg +end +r.send 'ok' +r.value #=> 'ok' +``` + +### The execution result of the given block + +The return value of the given block becomes an outgoing message (see below for details). + +```ruby +r = Ractor.new do + 'ok' +end +r.value #=> `ok` +``` + +An error in the given block will be propagated to the consumer of the outgoing message. + +```ruby +r = Ractor.new do + raise 'ok' # exception will be transferred to the consumer +end + +begin + r.value +rescue Ractor::RemoteError => e + e.cause.class #=> RuntimeError + e.cause.message #=> 'ok' + e.ractor #=> r +end +``` + +## Communication between Ractors + +Communication between ractors is achieved by sending and receiving messages. There are two ways to communicate: + +* (1) Sending and receiving messages via `Ractor::Port` +* (2) Using shareable container objects. For example, the Ractor::TVar gem ([ko1/ractor-tvar](https://github.com/ko1/ractor-tvar)) + +Users can control program execution timing with (1), but should not control with (2) (only perform critical sections). + +For sending and receiving messages, these are the fundamental APIs: + +* send/receive via `Ractor::Port`. + * `Ractor::Port#send(obj)` (`Ractor::Port#<<(obj)` is an alias) sends a message to the port. Ports are connected to an infinite size incoming queue so sending will never block the caller. + * `Ractor::Port#receive` dequeues a message from its own incoming queue. If the incoming queue is empty, `Ractor::Port#receive` will block the execution of the current Thread until a message is sent. + * `Ractor#send` and `Ractor.receive` use ports (their default port) internally, so are conceptually similar to the above. +* You can close a `Ractor::Port` by `Ractor::Port#close`. A port can only be closed by the ractor that created it. + * If a port is closed, you can't `send` to it. Doing so raises an exception. + * When a ractor is terminated, the ractor's ports are automatically closed. +* You can wait for a ractor's termination and receive its return value with `Ractor#value`. This is similar to `Thread#value`. + +There are 3 ways to send an object as a message: + +1) Send a reference: sending a shareable object sends only a reference to the object (fast). + +2) Copy an object: sending an unshareable object through copying it deeply (can be slow). Note that you can not send an object this way which does not support deep copy. Some `T_DATA` objects (objects whose class is defined in a C extension, such as `StringIO`) are not supported. + +3) Move an object: sending an unshareable object across ractors with a membership change. The sending Ractor can not access the moved object after moving it, otherwise an exception will be raised. Implementation note: `T_DATA` objects are not supported. + +You can choose between "Copy" and "Move" by the `move:` keyword, `Ractor#send(obj, move: true/false)`. The default is `false` ("Copy"). However, if the object is shareable it will automatically use `move`. + +### Wait for multiple Ractors with `Ractor.select` + +You can wait for messages on multiple ports at once. +The return value of `Ractor.select()` is `[port, msg]` where `port` is a ready port and `msg` is the received message. + +To make it convenient, `Ractor.select` can also accept ractors. In this case, it waits for their termination. +The return value of `Ractor.select()` is `[r, msg]` where `r` is a terminated Ractor and `msg` is the value of the ractor's block. + +Wait for a single ractor (same as `Ractor#value`): + +```ruby +r1 = Ractor.new{'r1'} + +r, obj = Ractor.select(r1) +r == r1 and obj == 'r1' #=> true +``` + +Wait for two ractors: + +```ruby +r1 = Ractor.new{'r1'} +r2 = Ractor.new{'r2'} +rs = [r1, r2] +values = [] + +while rs.any? + r, obj = Ractor.select(*rs) + rs.delete(r) + values << obj +end + +values.sort == ['r1', 'r2'] #=> true +``` + +NOTE: Using `Ractor.select()` on a very large number of ractors has the same issue as `select(2)` currently. + +### Closing ports + +* `Ractor::Port#close` closes the port (similar to `Queue#close`). + * `port.send(obj)` will raise an exception when the port is closed. + * When the queue connected to the port is empty and port is closed, `Ractor::Port#receive` raises an exception. If the queue is not empty, it dequeues an object without exceptions. +* When a Ractor terminates, the ports are closed automatically. + +Example (try to get a result from closed ractor): + +```ruby +r = Ractor.new do + 'finish' +end +r.join # success (wait for the termination) +r.value # success (will return 'finish') + +# The ractor's termination value has already been given to another ractor +Ractor.new r do |r| + r.value #=> Ractor::Error +end.join +``` + +Example (try to send to closed port): + +```ruby +r = Ractor.new do +end + +r.join # wait for termination, closes default port + +begin + r.send(1) +rescue Ractor::ClosedError + 'ok' +end +``` + +### Send a message by copying + +`Ractor::Port#send(obj)` copies `obj` deeply if `obj` is an unshareable object. + +```ruby +obj = 'str'.dup +r = Ractor.new obj do |msg| + # return received msg's object_id + msg.object_id +end + +obj.object_id == r.value #=> false +``` + +Some objects do not support copying, and raise an exception. + +```ruby +obj = Thread.new{} +begin + Ractor.new obj do |msg| + msg + end +rescue TypeError => e + e.message #=> #<TypeError: allocator undefined for Thread> +end +``` + +### Send a message by moving + +`Ractor::Port#send(obj, move: true)` moves `obj` to the destination Ractor. +If the source ractor uses the moved object (for example, calls a method like `obj.foo()`), it will raise an error. + +```ruby +r = Ractor.new do + obj = Ractor.receive + obj << ' world' +end + +str = 'hello'.dup +r.send str, move: true +# str is now moved, and accessing str from this ractor is prohibited +modified = r.value #=> 'hello world' + + +begin + # Error because it uses moved str. + str << ' exception' # raise Ractor::MovedError +rescue Ractor::MovedError + modified #=> 'hello world' +end +``` + +Some objects do not support moving, and an exception will be raised. + +```ruby +r = Ractor.new do + Ractor.receive +end + +r.send(Thread.new{}, move: true) #=> allocator undefined for Thread (TypeError) +``` + +Once an object has been moved, the source object's class is changed to `Ractor::MovedObject`. + +### Shareable objects + +The following is an inexhaustive list of shareable objects: + +* `Integer`, `Float`, `Complex`, `Rational` +* `Symbol`, frozen `String` objects that don't refer to unshareables, `true`, `false`, `nil` +* `Regexp` objects, if they have no instance variables or their instance variables refer only to shareables +* `Class` and `Module` objects +* `Ractor` and other special objects which deal with synchronization + +To make objects shareable, `Ractor.make_shareable(obj)` is provided. It tries to make the object shareable by freezing `obj` and recursively traversing its references to freeze them all. This method accepts the `copy:` keyword (default value is false). `Ractor.make_shareable(obj, copy: true)` tries to make a deep copy of `obj` and make the copied object shareable. `Ractor.make_shareable(copy: false)` has no effect on an already shareable object. If the object cannot be made shareable, a `Ractor::Error` exception will be raised. + +## Language changes to limit sharing between Ractors + +To isolate unshareable objects across ractors, we introduced additional language semantics for multi-ractor Ruby programs. + +Note that when not using ractors, these additional semantics are not needed (100% compatible with Ruby 2). + +### Global variables + +Only the main Ractor can access global variables. + +```ruby +$gv = 1 +r = Ractor.new do + $gv +end + +begin + r.join +rescue Ractor::RemoteError => e + e.cause.message #=> 'can not access global variables from non-main Ractors' +end +``` + +Note that some special global variables, such as `$stdin`, `$stdout` and `$stderr` are local to each ractor. See [[Bug #17268]](https://bugs.ruby-lang.org/issues/17268) for more details. + +### Instance variables of shareable objects + +Instance variables of classes/modules can be accessed from non-main ractors only if their values are shareable objects. + +```ruby +class C + @iv = 1 +end + +p Ractor.new do + class C + @iv + end +end.value #=> 1 +``` + +Otherwise, only the main Ractor can access instance variables of shareable objects. + +```ruby +class C + @iv = [] # unshareable object +end + +Ractor.new do + class C + begin + p @iv + rescue Ractor::IsolationError + p $!.message + #=> "can not get unshareable values from instance variables of classes/modules from non-main Ractors" + end + + begin + @iv = 42 + rescue Ractor::IsolationError + p $!.message + #=> "can not set instance variables of classes/modules by non-main Ractors" + end + end +end.join +``` + +```ruby +shared = Ractor.new{} +shared.instance_variable_set(:@iv, 'str') + +r = Ractor.new shared do |shared| + p shared.instance_variable_get(:@iv) +end + +begin + r.join +rescue Ractor::RemoteError => e + e.cause.message #=> can not access instance variables of shareable objects from non-main Ractors (Ractor::IsolationError) +end +``` + +### Class variables + +Only the main Ractor can access class variables. + +```ruby +class C + @@cv = 'str' +end + +r = Ractor.new do + class C + p @@cv + end +end + + +begin + r.join +rescue => e + e.class #=> Ractor::IsolationError +end +``` + +### Constants + +Only the main Ractor can read constants which refer to an unshareable object. + +```ruby +class C + CONST = 'str'.dup +end +r = Ractor.new do + C::CONST +end +begin + r.join +rescue => e + e.class #=> Ractor::IsolationError +end +``` + +Only the main Ractor can define constants which refer to an unshareable object. + +```ruby +class C +end +r = Ractor.new do + C::CONST = 'str'.dup +end +begin + r.join +rescue => e + e.class #=> Ractor::IsolationError +end +``` + +When creating/updating a library to support ractors, constants should only refer to shareable objects if they are to be used by non-main ractors. + +```ruby +TABLE = {a: 'ko1', b: 'ko2', c: 'ko3'} +``` + +In this case, `TABLE` refers to an unshareable Hash object. In order for other ractors to use `TABLE`, we need to make it shareable. We can use `Ractor.make_shareable()` like so: + +```ruby +TABLE = Ractor.make_shareable( {a: 'ko1', b: 'ko2', c: 'ko3'} ) +``` + +To make it easy, Ruby 3.0 introduced a new `shareable_constant_value` file directive. + +```ruby +# shareable_constant_value: literal + +TABLE = {a: 'ko1', b: 'ko2', c: 'ko3'} +#=> Same as: TABLE = Ractor.make_shareable( {a: 'ko1', b: 'ko2', c: 'ko3'} ) +``` + +The `shareable_constant_value` directive accepts the following modes (descriptions use the example: `CONST = expr`): + +* none: Do nothing. Same as: `CONST = expr` +* literal: + * if `expr` consists of literals, replaced to `CONST = Ractor.make_shareable(expr)`. + * otherwise: replaced to `CONST = expr.tap{|o| raise unless Ractor.shareable?(o)}`. +* experimental_everything: replaced to `CONST = Ractor.make_shareable(expr)`. +* experimental_copy: replaced to `CONST = Ractor.make_shareable(expr, copy: true)`. + +Except for the `none` mode (default), it is guaranteed that these constants refer only to shareable objects. + +See [syntax/comments.rdoc](../syntax/comments.rdoc) for more details. + +### Shareable procs + +Procs and lambdas are unshareable objects, even when they are frozen. To create an unshareable Proc, you must use `Ractor.shareable_proc { expr }`. Much like during Ractor creation, the proc's block is isolated from its outer environment, so it cannot access variables from the outside scope. `self` is also changed within the Proc to be `nil` by default, although a `self:` keyword can be provided if you want to customize the value to a different shareable object. + +```ruby +p = Ractor.shareable_proc { p self } +p.call #=> nil +``` + +```ruby +begin + a = 1 + pr = Ractor.shareable_proc { p a } + pr.call # never gets here +rescue Ractor::IsolationError +end +``` + +In order to dynamically define a method with `Module#define_method` that can be used from different ractors, you must define it with a shareable proc. Alternatively, you can use `Module#class_eval` or `Module#module_eval` with a String. Even though the shareable proc's `self` is initially bound to `nil`, `define_method` will bind `self` to the correct value in the method. + +```ruby +class A + define_method :testing, &Ractor.shareable_proc do + p self + end +end +Ractor.new do + a = A.new + a.testing #=> #<A:0x0000000101acfe10> +end.join +``` + +This isolation must be done to prevent the method from accessing and assigning captured outer variables across ractors. + +### Ractor-local storage + +You can store any object (even unshareables) in ractor-local storage. + +```ruby +r = Ractor.new do + values = [] + Ractor[:threads] = [] + 3.times do |i| + Ractor[:threads] << Thread.new do + values << [Ractor.receive, i+1] # Ractor.receive blocks the current thread in the current ractor until it receives a message + end + end + Ractor[:threads].each(&:join) + values +end + +r << 1 +r << 2 +r << 3 +r.value #=> [[1,1],[2,2],[3,3]] (the order can change with each run) +``` + +## Examples + +### Traditional Ring example in Actor-model + +```ruby +RN = 1_000 +CR = Ractor.current + +r = Ractor.new do + p Ractor.receive + CR << :fin +end + +RN.times{ + r = Ractor.new r do |next_r| + next_r << Ractor.receive + end +} + +p :setup_ok +r << 1 +p Ractor.receive +``` + +### Fork-join + +```ruby +def fib n + if n < 2 + 1 + else + fib(n-2) + fib(n-1) + end +end + +RN = 10 +rs = (1..RN).map do |i| + Ractor.new i do |i| + [i, fib(i)] + end +end + +until rs.empty? + r, v = Ractor.select(*rs) + rs.delete r + p answer: v +end +``` + +### Worker pool + +(1) One ractor has a pool + +```ruby +require 'prime' + +N = 1000 +RN = 10 + +# make RN workers +workers = (1..RN).map do + Ractor.new do |; result_port| + loop do + n, result_port = Ractor.receive + result_port << [n, n.prime?, Ractor.current] + end + end +end + +result_port = Ractor::Port.new +results = [] + +(1..N).each do |i| + if workers.empty? + # receive a result + n, result, w = result_port.receive + results << [n, result] + else + w = workers.pop + end + + # send a task to the idle worker ractor + w << [i, result_port] +end + +# receive a result +while results.size != N + n, result, _w = result_port.receive + results << [n, result] +end + +pp results.sort_by{|n, result| n} +``` + +### Pipeline + +```ruby +# pipeline with send/receive + +r3 = Ractor.new Ractor.current do |cr| + cr.send Ractor.receive + 'r3' +end + +r2 = Ractor.new r3 do |r3| + r3.send Ractor.receive + 'r2' +end + +r1 = Ractor.new r2 do |r2| + r2.send Ractor.receive + 'r1' +end + +r1 << 'r0' +p Ractor.receive #=> "r0r1r2r3" +``` + +### Supervise + +```ruby +# ring example again + +r = Ractor.current +(1..10).map{|i| + r = Ractor.new r, i do |r, i| + r.send Ractor.receive + "r#{i}" + end +} + +r.send "r0" +p Ractor.receive #=> "r0r10r9r8r7r6r5r4r3r2r1" +``` + +```ruby +# ring example with an error + +r = Ractor.current +rs = (1..10).map{|i| + r = Ractor.new r, i do |r, i| + loop do + msg = Ractor.receive + raise if /e/ =~ msg + r.send msg + "r#{i}" + end + end +} + +r.send "r0" +p Ractor.receive #=> "r0r10r9r8r7r6r5r4r3r2r1" +r.send "r0" +p Ractor.select(*rs, Ractor.current) #=> [:receive, "r0r10r9r8r7r6r5r4r3r2r1"] +r.send "e0" +p Ractor.select(*rs, Ractor.current) +#=> +# <Thread:0x000056262de28bd8 run> terminated with exception (report_on_exception is true): +# Traceback (most recent call last): +# 2: from /home/ko1/src/ruby/trunk/test.rb:7:in `block (2 levels) in <main>' +# 1: from /home/ko1/src/ruby/trunk/test.rb:7:in `loop' +# /home/ko1/src/ruby/trunk/test.rb:9:in `block (3 levels) in <main>': unhandled exception +# Traceback (most recent call last): +# 2: from /home/ko1/src/ruby/trunk/test.rb:7:in `block (2 levels) in <main>' +# 1: from /home/ko1/src/ruby/trunk/test.rb:7:in `loop' +# /home/ko1/src/ruby/trunk/test.rb:9:in `block (3 levels) in <main>': unhandled exception +# 1: from /home/ko1/src/ruby/trunk/test.rb:21:in `<main>' +# <internal:ractor>:69:in `select': thrown by remote Ractor. (Ractor::RemoteError) +``` + +```ruby +# resend non-error message + +r = Ractor.current +rs = (1..10).map{|i| + r = Ractor.new r, i do |r, i| + loop do + msg = Ractor.receive + raise if /e/ =~ msg + r.send msg + "r#{i}" + end + end +} + +r.send "r0" +p Ractor.receive #=> "r0r10r9r8r7r6r5r4r3r2r1" +r.send "r0" +p Ractor.select(*rs, Ractor.current) +[:receive, "r0r10r9r8r7r6r5r4r3r2r1"] +msg = 'e0' +begin + r.send msg + p Ractor.select(*rs, Ractor.current) +rescue Ractor::RemoteError + msg = 'r0' + retry +end + +#=> <internal:ractor>:100:in `send': The incoming-port is already closed (Ractor::ClosedError) +# because r == r[-1] is terminated. +``` + +```ruby +# ring example with supervisor and re-start + +def make_ractor r, i + Ractor.new r, i do |r, i| + loop do + msg = Ractor.receive + raise if /e/ =~ msg + r.send msg + "r#{i}" + end + end +end + +r = Ractor.current +rs = (1..10).map{|i| + r = make_ractor(r, i) +} + +msg = 'e0' # error causing message +begin + r.send msg + p Ractor.select(*rs, Ractor.current) +rescue Ractor::RemoteError + r = rs[-1] = make_ractor(rs[-2], rs.size-1) + msg = 'x0' + retry +end + +#=> [:receive, "x0r9r9r8r7r6r5r4r3r2r1"] +``` diff --git a/doc/regexp/methods.rdoc b/doc/language/regexp/methods.rdoc index 356156ac9a..356156ac9a 100644 --- a/doc/regexp/methods.rdoc +++ b/doc/language/regexp/methods.rdoc diff --git a/doc/regexp/unicode_properties.rdoc b/doc/language/regexp/unicode_properties.rdoc index f1f1f9d6a9..94080f7199 100644 --- a/doc/regexp/unicode_properties.rdoc +++ b/doc/language/regexp/unicode_properties.rdoc @@ -146,6 +146,7 @@ Older versions may not support all of these. - <tt>\p{Bassa_Vah}</tt>, <tt>\p{Bass}</tt> - <tt>\p{Batak}</tt>, <tt>\p{Batk}</tt> - <tt>\p{Bengali}</tt>, <tt>\p{Beng}</tt> +- <tt>\p{Beria_Erfe}</tt>, <tt>\p{Berf}</tt> - <tt>\p{Bhaiksuki}</tt>, <tt>\p{Bhks}</tt> - <tt>\p{Bopomofo}</tt>, <tt>\p{Bopo}</tt> - <tt>\p{Brahmi}</tt>, <tt>\p{Brah}</tt> @@ -270,6 +271,7 @@ Older versions may not support all of these. - <tt>\p{Sharada}</tt>, <tt>\p{Shrd}</tt> - <tt>\p{Shavian}</tt>, <tt>\p{Shaw}</tt> - <tt>\p{Siddham}</tt>, <tt>\p{Sidd}</tt> +- <tt>\p{Sidetic}</tt>, <tt>\p{Sidt}</tt> - <tt>\p{SignWriting}</tt>, <tt>\p{Sgnw}</tt> - <tt>\p{Sinhala}</tt>, <tt>\p{Sinh}</tt> - <tt>\p{Sogdian}</tt>, <tt>\p{Sogd}</tt> @@ -284,6 +286,7 @@ Older versions may not support all of these. - <tt>\p{Tai_Le}</tt>, <tt>\p{Tale}</tt> - <tt>\p{Tai_Tham}</tt>, <tt>\p{Lana}</tt> - <tt>\p{Tai_Viet}</tt>, <tt>\p{Tavt}</tt> +- <tt>\p{Tai_Yo}</tt>, <tt>\p{Tayo}</tt> - <tt>\p{Takri}</tt>, <tt>\p{Takr}</tt> - <tt>\p{Tamil}</tt>, <tt>\p{Taml}</tt> - <tt>\p{Tangsa}</tt>, <tt>\p{Tnsa}</tt> @@ -295,6 +298,7 @@ Older versions may not support all of these. - <tt>\p{Tifinagh}</tt>, <tt>\p{Tfng}</tt> - <tt>\p{Tirhuta}</tt>, <tt>\p{Tirh}</tt> - <tt>\p{Todhri}</tt>, <tt>\p{Todr}</tt> +- <tt>\p{Tolong_Siki}</tt>, <tt>\p{Tols}</tt> - <tt>\p{Toto}</tt> - <tt>\p{Tulu_Tigalari}</tt>, <tt>\p{Tutg}</tt> - <tt>\p{Ugaritic}</tt>, <tt>\p{Ugar}</tt> @@ -336,6 +340,7 @@ Older versions may not support all of these. - <tt>\p{In_Bassa_Vah}</tt> - <tt>\p{In_Batak}</tt> - <tt>\p{In_Bengali}</tt> +- <tt>\p{In_Beria_Erfe}</tt> - <tt>\p{In_Bhaiksuki}</tt> - <tt>\p{In_Block_Elements}</tt> - <tt>\p{In_Bopomofo}</tt> @@ -363,6 +368,7 @@ Older versions may not support all of these. - <tt>\p{In_CJK_Unified_Ideographs_Extension_G}</tt> - <tt>\p{In_CJK_Unified_Ideographs_Extension_H}</tt> - <tt>\p{In_CJK_Unified_Ideographs_Extension_I}</tt> +- <tt>\p{In_CJK_Unified_Ideographs_Extension_J}</tt> - <tt>\p{In_Carian}</tt> - <tt>\p{In_Caucasian_Albanian}</tt> - <tt>\p{In_Chakma}</tt> @@ -516,6 +522,7 @@ Older versions may not support all of these. - <tt>\p{In_Miscellaneous_Mathematical_Symbols_A}</tt> - <tt>\p{In_Miscellaneous_Mathematical_Symbols_B}</tt> - <tt>\p{In_Miscellaneous_Symbols}</tt> +- <tt>\p{In_Miscellaneous_Symbols_Supplement}</tt> - <tt>\p{In_Miscellaneous_Symbols_and_Arrows}</tt> - <tt>\p{In_Miscellaneous_Symbols_and_Pictographs}</tt> - <tt>\p{In_Miscellaneous_Technical}</tt> @@ -575,9 +582,11 @@ Older versions may not support all of these. - <tt>\p{In_Samaritan}</tt> - <tt>\p{In_Saurashtra}</tt> - <tt>\p{In_Sharada}</tt> +- <tt>\p{In_Sharada_Supplement}</tt> - <tt>\p{In_Shavian}</tt> - <tt>\p{In_Shorthand_Format_Controls}</tt> - <tt>\p{In_Siddham}</tt> +- <tt>\p{In_Sidetic}</tt> - <tt>\p{In_Sinhala}</tt> - <tt>\p{In_Sinhala_Archaic_Numbers}</tt> - <tt>\p{In_Small_Form_Variants}</tt> @@ -613,12 +622,14 @@ Older versions may not support all of these. - <tt>\p{In_Tai_Tham}</tt> - <tt>\p{In_Tai_Viet}</tt> - <tt>\p{In_Tai_Xuan_Jing_Symbols}</tt> +- <tt>\p{In_Tai_Yo}</tt> - <tt>\p{In_Takri}</tt> - <tt>\p{In_Tamil}</tt> - <tt>\p{In_Tamil_Supplement}</tt> - <tt>\p{In_Tangsa}</tt> - <tt>\p{In_Tangut}</tt> - <tt>\p{In_Tangut_Components}</tt> +- <tt>\p{In_Tangut_Components_Supplement}</tt> - <tt>\p{In_Tangut_Supplement}</tt> - <tt>\p{In_Telugu}</tt> - <tt>\p{In_Thaana}</tt> @@ -627,6 +638,7 @@ Older versions may not support all of these. - <tt>\p{In_Tifinagh}</tt> - <tt>\p{In_Tirhuta}</tt> - <tt>\p{In_Todhri}</tt> +- <tt>\p{In_Tolong_Siki}</tt> - <tt>\p{In_Toto}</tt> - <tt>\p{In_Transport_and_Map_Symbols}</tt> - <tt>\p{In_Tulu_Tigalari}</tt> @@ -685,6 +697,7 @@ Older versions may not support all of these. - <tt>\p{Age_15_0}</tt> - <tt>\p{Age_15_1}</tt> - <tt>\p{Age_16_0}</tt> +- <tt>\p{Age_17_0}</tt> - <tt>\p{Age_1_1}</tt> - <tt>\p{Age_2_0}</tt> - <tt>\p{Age_2_1}</tt> diff --git a/doc/signals.rdoc b/doc/language/signals.rdoc index 403eb66549..a82dab81c6 100644 --- a/doc/signals.rdoc +++ b/doc/language/signals.rdoc @@ -17,7 +17,7 @@ for its internal data structures, but it does not know when it is safe for data structures in YOUR code. Ruby implements deferred signal handling by registering short C functions with only {async-signal-safe functions}[http://man7.org/linux/man-pages/man7/signal-safety.7.html] as -signal handlers. These short C functions only do enough tell the VM to +signal handlers. These short C functions only do enough to tell the VM to run callbacks registered via Signal.trap later in the main Ruby Thread. == Unsafe methods to call in Signal.trap blocks diff --git a/doc/strftime_formatting.rdoc b/doc/language/strftime_formatting.rdoc index 5c7b33155d..2bfa6b975e 100644 --- a/doc/strftime_formatting.rdoc +++ b/doc/language/strftime_formatting.rdoc @@ -136,7 +136,7 @@ the only required part is the conversion specifier, so we begin with that. t = Time.now # => 2022-06-29 07:10:20.3230914 -0500 t.strftime('%N') # => "323091400" # Default. - Use {width specifiers}[rdoc-ref:strftime_formatting.rdoc@Width+Specifiers] + Use {width specifiers}[rdoc-ref:@Width+Specifiers] to adjust units: t.strftime('%3N') # => "323" # Milliseconds. @@ -522,6 +522,4 @@ An ISO 8601 combined date and time representation may be any ISO 8601 date and any ISO 8601 time, separated by the letter +T+. -For the relevant +strftime+ formats, see -{Dates}[rdoc-ref:strftime_formatting.rdoc@Dates] -and {Times}[rdoc-ref:strftime_formatting.rdoc@Times] above. +For the relevant +strftime+ formats, see {Dates}[rdoc-ref:@Dates] and {Times}[rdoc-ref:@Times] above. diff --git a/doc/maintainers.md b/doc/maintainers.md index a216e564a5..46840343ca 100644 --- a/doc/maintainers.md +++ b/doc/maintainers.md @@ -28,6 +28,10 @@ not have authority to change/add a feature on his/her part. They need consensus on ruby-core/ruby-dev before changing/adding. Some of submaintainers have commit right, others don't. +No maintainer means that there is no specific maintainer for the part now. +The member of ruby core team can fix issues at anytime. But major changes need +consensus on ruby-core/ruby-dev. + ### Language core features including security * Yukihiro Matsumoto ([matz]) @@ -40,25 +44,22 @@ have commit right, others don't. * Yukihiro Matsumoto ([matz]) -## Standard Library Maintainers - -### Libraries +### Standard Library Maintainers #### lib/mkmf.rb -* *unmaintained* +* *No maintainer* #### lib/rubygems.rb, lib/rubygems/* -* Eric Hodel ([drbrain]) * Hiroshi SHIBATA ([hsbt]) -* https://github.com/rubygems/rubygems +* https://github.com/ruby/rubygems #### lib/unicode_normalize.rb, lib/unicode_normalize/* * Martin J. Dürst ([duerst]) -### Extensions +### Standard Library(Extensions) Maintainers #### ext/continuation @@ -78,15 +79,15 @@ have commit right, others don't. #### ext/objspace -* *unmaintained* +* *No maintainer* #### ext/pty -* *unmaintained* +* *No maintainer* #### ext/ripper -* *unmaintained* +* *No maintainer* #### ext/socket @@ -97,31 +98,27 @@ have commit right, others don't. * NAKAMURA Usaku ([unak]) -## Default gems Maintainers - -### Libraries +### Default gems(Libraries) Maintainers #### lib/bundler.rb, lib/bundler/* * Hiroshi SHIBATA ([hsbt]) -* https://github.com/rubygems/rubygems +* https://github.com/ruby/rubygems * https://rubygems.org/gems/bundler -#### lib/cgi.rb, lib/cgi/* +#### lib/cgi/escape.rb -* *unmaintained* -* https://github.com/ruby/cgi -* https://rubygems.org/gems/cgi +* *No maintainer* #### lib/English.rb -* *unmaintained* +* *No maintainer* * https://github.com/ruby/English * https://rubygems.org/gems/English #### lib/delegate.rb -* *unmaintained* +* *No maintainer* * https://github.com/ruby/delegate * https://rubygems.org/gems/delegate @@ -152,7 +149,7 @@ have commit right, others don't. #### lib/fileutils.rb -* *unmaintained* +* *No maintainer* * https://github.com/ruby/fileutils * https://rubygems.org/gems/fileutils @@ -178,6 +175,7 @@ have commit right, others don't. * Nobuyuki Nakada ([nobu]) * https://github.com/ruby/optparse +* https://rubygems.org/gems/optparse #### lib/net/http.rb, lib/net/https.rb @@ -187,13 +185,13 @@ have commit right, others don't. #### lib/net/protocol.rb -* *unmaintained* +* *No maintainer* * https://github.com/ruby/net-protocol * https://rubygems.org/gems/net-protocol #### lib/open3.rb -* *unmaintained* +* *No maintainer* * https://github.com/ruby/open3 * https://rubygems.org/gems/open3 @@ -201,6 +199,7 @@ have commit right, others don't. * Tanaka Akira ([akr]) * https://github.com/ruby/open-uri +* https://rubygems.org/gems/open-uri #### lib/pp.rb @@ -234,12 +233,6 @@ have commit right, others don't. * https://github.com/ruby/securerandom * https://rubygems.org/gems/securerandom -#### lib/set.rb - -* Akinori MUSHA ([knu]) -* https://github.com/ruby/set -* https://rubygems.org/gems/set - #### lib/shellwords.rb * Akinori MUSHA ([knu]) @@ -254,7 +247,7 @@ have commit right, others don't. #### lib/tempfile.rb -* *unmaintained* +* *No maintainer* * https://github.com/ruby/tempfile * https://rubygems.org/gems/tempfile @@ -270,24 +263,12 @@ have commit right, others don't. * https://github.com/ruby/timeout * https://rubygems.org/gems/timeout -#### lib/thwait.rb - -* Keiju ISHITSUKA ([keiju]) -* https://github.com/ruby/thwait -* https://rubygems.org/gems/thwait - #### lib/tmpdir.rb -* *unmaintained* +* *No maintainer* * https://github.com/ruby/tmpdir * https://rubygems.org/gems/tmpdir -#### lib/tsort.rb - -* Tanaka Akira ([akr]) -* https://github.com/ruby/tsort -* https://rubygems.org/gems/tsort - #### lib/un.rb * WATANABE Hirofumi ([eban]) @@ -309,33 +290,31 @@ have commit right, others don't. #### lib/weakref.rb -* *unmaintained* +* *No maintainer* * https://github.com/ruby/weakref * https://rubygems.org/gems/weakref -### Extensions +### Default gems(Extensions) Maintainers #### ext/cgi * Nobuyoshi Nakada ([nobu]) -* https://github.com/ruby/cgi -* https://rubygems.org/gems/cgi #### ext/date -* *unmaintained* +* *No maintainer* * https://github.com/ruby/date * https://rubygems.org/gems/date #### ext/etc -* *unmaintained* +* *No maintainer* * https://github.com/ruby/etc * https://rubygems.org/gems/etc #### ext/fcntl -* *unmaintained* +* *No maintainer* * https://github.com/ruby/fcntl * https://rubygems.org/gems/fcntl @@ -402,159 +381,252 @@ have commit right, others don't. * https://github.com/ruby/zlib * https://rubygems.org/gems/zlib -## Bundled gems upstream repositories +## Bundled gems upstream repositories and maintainers + +The maintanance policy of bundled gems is different from Module Maintainers above. +Please check the policies for each repository. + +The ruby core team tries to maintain the repositories with no maintainers. +It may needs to make consensus on ruby-core/ruby-dev before making major changes. ### minitest +* Ryan Davis ([zenspider]) * https://github.com/minitest/minitest +* https://rubygems.org/gems/minitest ### power_assert +* Tsujimoto Kenta ([k-tsj]) * https://github.com/ruby/power_assert +* https://rubygems.org/gems/power_assert ### rake +* Hiroshi SHIBATA ([hsbt]) * https://github.com/ruby/rake +* https://rubygems.org/gems/rake ### test-unit +* Kouhei Sutou ([kou]) * https://github.com/test-unit/test-unit +* https://rubygems.org/gems/test-unit ### rexml +* Kouhei Sutou ([kou]) * https://github.com/ruby/rexml +* https://rubygems.org/gems/rexml ### rss +* Kouhei Sutou ([kou]) * https://github.com/ruby/rss +* https://rubygems.org/gems/rss ### net-ftp +* Shugo Maeda ([shugo]) * https://github.com/ruby/net-ftp +* https://rubygems.org/gems/net-ftp ### net-imap +* Nicholas A. Evans ([nevans]) * https://github.com/ruby/net-imap +* https://rubygems.org/gems/net-imap ### net-pop * https://github.com/ruby/net-pop +* https://rubygems.org/gems/net-pop ### net-smtp +* TOMITA Masahiro ([tmtm]) * https://github.com/ruby/net-smtp +* https://rubygems.org/gems/net-smtp ### matrix +* Marc-André Lafortune ([marcandre]) * https://github.com/ruby/matrix +* https://rubygems.org/gems/matrix ### prime * https://github.com/ruby/prime +* https://rubygems.org/gems/prime ### rbs +* Soutaro Matsumoto ([soutaro]) * https://github.com/ruby/rbs +* https://rubygems.org/gems/rbs ### typeprof +* Yusuke Endoh ([mame]) * https://github.com/ruby/typeprof +* https://rubygems.org/gems/typeprof ### debug +* Koichi Sasada ([ko1]) * https://github.com/ruby/debug +* https://rubygems.org/gems/debug ### racc +* Yuichi Kaneko ([yui-knk]) * https://github.com/ruby/racc +* https://rubygems.org/gems/racc #### mutex_m * https://github.com/ruby/mutex_m +* https://rubygems.org/gems/mutex_m #### getoptlong * https://github.com/ruby/getoptlong +* https://rubygems.org/gems/getoptlong #### base64 +* Yusuke Endoh ([mame]) * https://github.com/ruby/base64 +* https://rubygems.org/gems/base64 #### bigdecimal +* Kenta Murata ([mrkn]) * https://github.com/ruby/bigdecimal +* https://rubygems.org/gems/bigdecimal #### observer * https://github.com/ruby/observer +* https://rubygems.org/gems/observer #### abbrev +* Akinori MUSHA ([knu]) * https://github.com/ruby/abbrev +* https://rubygems.org/gems/abbrev #### resolv-replace +* Akira TANAKA ([akr]) * https://github.com/ruby/resolv-replace +* https://rubygems.org/gems/resolv-replace #### rinda +* Masatoshi SEKI ([seki]) * https://github.com/ruby/rinda +* https://rubygems.org/gems/rinda #### drb +* Masatoshi SEKI ([seki]) * https://github.com/ruby/drb +* https://rubygems.org/gems/drb #### nkf +* Naruse Yusuke ([nurse]) * https://github.com/ruby/nkf +* https://rubygems.org/gems/nkf #### syslog +* Akinori Musha ([knu]) * https://github.com/ruby/syslog +* https://rubygems.org/gems/syslog #### csv +* Kouhei Sutou ([kou]) * https://github.com/ruby/csv +* https://rubygems.org/gems/csv #### ostruct +* Marc-André Lafortune ([marcandre]) * https://github.com/ruby/ostruct +* https://rubygems.org/gems/ostruct #### pstore * https://github.com/ruby/pstore +* https://rubygems.org/gems/pstore #### benchmark * https://github.com/ruby/benchmark +* https://rubygems.org/gems/benchmark #### logger +* Naotoshi Seo ([sonots]) * https://github.com/ruby/logger +* https://rubygems.org/gems/logger #### rdoc +* Stan Lo ([st0012]) +* Nobuyoshi Nakada ([nobu]) * https://github.com/ruby/rdoc +* https://rubygems.org/gems/rdoc #### win32ole +* Masaki Suketa ([suketa]) * https://github.com/ruby/win32ole +* https://rubygems.org/gems/win32ole #### irb +* Tomoya Ishida ([tompng]) +* Stan Lo ([st0012]) +* Mari Imaizumi ([ima1zumi]) +* HASUMI Hitoshi ([hasumikin]) * https://github.com/ruby/irb +* https://rubygems.org/gems/irb #### reline +* Tomoya Ishida ([tompng]) +* Stan Lo ([st0012]) +* Mari Imaizumi ([ima1zumi]) +* HASUMI Hitoshi ([hasumikin]) * https://github.com/ruby/reline +* https://rubygems.org/gems/reline #### readline * https://github.com/ruby/readline +* https://rubygems.org/gems/readline #### fiddle +* Kouhei Sutou ([kou]) * https://github.com/ruby/fiddle +* https://rubygems.org/gems/fiddle + +#### repl_type_completor + +* Tomoya Ishida ([tompng]) +* https://github.com/ruby/repl_type_completor +* https://rubygems.org/gems/repl_type_completor + +#### tsort + +* Tanaka Akira ([akr]) +* https://github.com/ruby/tsort +* https://rubygems.org/gems/tsort ## Platform Maintainers @@ -592,7 +664,7 @@ have commit right, others don't. ### cygwin, ... -* none. (Maintainer WANTED) +* **No maintainer** ### WebAssembly/WASI @@ -635,3 +707,12 @@ have commit right, others don't. [tompng]: https://github.com/tompng [unak]: https://github.com/unak [yuki24]: https://github.com/yuki24 +[zenspider]: https://github.com/zenspider +[k-tsj]: https://github.com/k-tsj +[nevans]: https://github.com/nevans +[tmtm]: https://github.com/tmtm +[shugo]: https://github.com/shugo +[soutaro]: https://github.com/soutaro +[yui-knk]: https://github.com/yui-knk +[hasumikin]: https://github.com/hasumikin +[suketa]: https://github.com/suketa diff --git a/doc/matchdata/begin.rdoc b/doc/matchdata/begin.rdoc index 8046dd9d55..6100617e19 100644 --- a/doc/matchdata/begin.rdoc +++ b/doc/matchdata/begin.rdoc @@ -10,12 +10,12 @@ returns the offset of the beginning of the <tt>n</tt>th match: m[3] # => "113" m.begin(3) # => 3 - m = /(т)(е)(с)/.match('тест') - # => #<MatchData "тес" 1:"т" 2:"е" 3:"с"> - m[0] # => "тес" - m.begin(0) # => 0 - m[3] # => "с" - m.begin(3) # => 2 + m = /(ん)(に)(ち)/.match('こんにちは') + # => #<MatchData "んにち" 1:"ん" 2:"に" 3:"ち"> + m[0] # => "んにち" + m.begin(0) # => 1 + m[3] # => "ち" + m.begin(3) # => 3 When string or symbol argument +name+ is given, returns the offset of the beginning for the named match: diff --git a/doc/matchdata/bytebegin.rdoc b/doc/matchdata/bytebegin.rdoc index 5b40a7ef73..54e417a7fc 100644 --- a/doc/matchdata/bytebegin.rdoc +++ b/doc/matchdata/bytebegin.rdoc @@ -10,12 +10,12 @@ returns the offset of the beginning of the <tt>n</tt>th match: m[3] # => "113" m.bytebegin(3) # => 3 - m = /(т)(е)(с)/.match('тест') - # => #<MatchData "тес" 1:"т" 2:"е" 3:"с"> - m[0] # => "тес" - m.bytebegin(0) # => 0 - m[3] # => "с" - m.bytebegin(3) # => 4 + m = /(ん)(に)(ち)/.match('こんにちは') + # => #<MatchData "んにち" 1:"ん" 2:"に" 3:"ち"> + m[0] # => "んにち" + m.bytebegin(0) # => 3 + m[3] # => "ち" + m.bytebegin(3) # => 9 When string or symbol argument +name+ is given, returns the offset of the beginning for the named match: diff --git a/doc/matchdata/byteend.rdoc b/doc/matchdata/byteend.rdoc index eb57664022..0a03f76208 100644 --- a/doc/matchdata/byteend.rdoc +++ b/doc/matchdata/byteend.rdoc @@ -10,12 +10,12 @@ returns the offset of the end of the <tt>n</tt>th match: m[3] # => "113" m.byteend(3) # => 6 - m = /(т)(е)(с)/.match('тест') - # => #<MatchData "тес" 1:"т" 2:"е" 3:"с"> - m[0] # => "тес" - m.byteend(0) # => 6 - m[3] # => "с" - m.byteend(3) # => 6 + m = /(ん)(に)(ち)/.match('こんにちは') + # => #<MatchData "んにち" 1:"ん" 2:"に" 3:"ち"> + m[0] # => "んにち" + m.byteend(0) # => 12 + m[3] # => "ち" + m.byteend(3) # => 12 When string or symbol argument +name+ is given, returns the offset of the end for the named match: diff --git a/doc/matchdata/end.rdoc b/doc/matchdata/end.rdoc index 0209b2d2fc..c43a5428f3 100644 --- a/doc/matchdata/end.rdoc +++ b/doc/matchdata/end.rdoc @@ -10,12 +10,12 @@ returns the offset of the end of the <tt>n</tt>th match: m[3] # => "113" m.end(3) # => 6 - m = /(т)(е)(с)/.match('тест') - # => #<MatchData "тес" 1:"т" 2:"е" 3:"с"> - m[0] # => "тес" - m.end(0) # => 3 - m[3] # => "с" - m.end(3) # => 3 + m = /(ん)(に)(ち)/.match('こんにちは') + # => #<MatchData "んにち" 1:"ん" 2:"に" 3:"ち"> + m[0] # => "んにち" + m.end(0) # => 4 + m[3] # => "ち" + m.end(3) # => 4 When string or symbol argument +name+ is given, returns the offset of the end for the named match: diff --git a/doc/matchdata/offset.rdoc b/doc/matchdata/offset.rdoc index 0985316d76..4194ef7ef9 100644 --- a/doc/matchdata/offset.rdoc +++ b/doc/matchdata/offset.rdoc @@ -11,12 +11,12 @@ returns the starting and ending offsets of the <tt>n</tt>th match: m[3] # => "113" m.offset(3) # => [3, 6] - m = /(т)(е)(с)/.match('тест') - # => #<MatchData "тес" 1:"т" 2:"е" 3:"с"> - m[0] # => "тес" - m.offset(0) # => [0, 3] - m[3] # => "с" - m.offset(3) # => [2, 3] + m = /(ん)(に)(ち)/.match('こんにちは') + # => #<MatchData "んにち" 1:"ん" 2:"に" 3:"ち"> + m[0] # => "んにち" + m.offset(0) # => [1, 4] + m[3] # => "ち" + m.offset(3) # => [3, 4] When string or symbol argument +name+ is given, returns the starting and ending offsets for the named match: diff --git a/doc/ractor.md b/doc/ractor.md deleted file mode 100644 index 6a1cff4e7b..0000000000 --- a/doc/ractor.md +++ /dev/null @@ -1,951 +0,0 @@ -# Ractor - Ruby's Actor-like concurrent abstraction - -Ractor is designed to provide a parallel execution feature of Ruby without thread-safety concerns. - -## Summary - -### Multiple Ractors in an interpreter process - -You can make multiple Ractors and they run in parallel. - -* `Ractor.new{ expr }` creates a new Ractor and `expr` is run in parallel on a parallel computer. -* Interpreter invokes with the first Ractor (called *main Ractor*). -* If main Ractor terminated, all Ractors receive terminate request like Threads (if main thread (first invoked Thread), Ruby interpreter sends all running threads to terminate execution). -* Each Ractor has 1 or more Threads. - * Threads in a Ractor shares a Ractor-wide global lock like GIL (GVL in MRI terminology), so they can't run in parallel (without releasing GVL explicitly in C-level). Threads in different ractors run in parallel. - * The overhead of creating a Ractor is similar to overhead of one Thread creation. - -### Limited sharing between multiple ractors - -Ractors don't share everything, unlike threads. - -* Most objects are *Unshareable objects*, so you don't need to care about thread-safety problems which are caused by sharing. -* Some objects are *Shareable objects*. - * Immutable objects: frozen objects which don't refer to unshareable-objects. - * `i = 123`: `i` is an immutable object. - * `s = "str".freeze`: `s` is an immutable object. - * `a = [1, [2], 3].freeze`: `a` is not an immutable object because `a` refers unshareable-object `[2]` (which is not frozen). - * `h = {c: Object}.freeze`: `h` is an immutable object because `h` refers Symbol `:c` and shareable `Object` class object which is not frozen. - * Class/Module objects - * Special shareable objects - * Ractor object itself. - * And more... - -### Two-types communication between Ractors - -Ractors communicate with each other and synchronize the execution by message exchanging between Ractors. There are two message exchange protocols: push type (message passing) and pull type. - -* Push type message passing: `Ractor#send(obj)` and `Ractor.receive()` pair. - * Sender ractor passes the `obj` to the ractor `r` by `r.send(obj)` and receiver ractor receives the message with `Ractor.receive`. - * Sender knows the destination Ractor `r` and the receiver does not know the sender (accept all messages from any ractors). - * Receiver has infinite queue and sender enqueues the message. Sender doesn't block to put message into this queue. - * This type of message exchanging is employed by many other Actor-based languages. - * `Ractor.receive_if{ filter_expr }` is a variant of `Ractor.receive` to select a message. -* Pull type communication: `Ractor.yield(obj)` and `Ractor#take()` pair. - * Sender ractor declare to yield the `obj` by `Ractor.yield(obj)` and receiver Ractor take it with `r.take`. - * Sender doesn't know a destination Ractor and receiver knows the sender Ractor `r`. - * Sender or receiver will block if there is no other side. - -### Copy & Move semantics to send messages - -To send unshareable objects as messages, objects are copied or moved. - -* Copy: use deep-copy. -* Move: move membership. - * Sender can not access the moved object after moving the object. - * Guarantee that at least only 1 Ractor can access the object. - -### Thread-safety - -Ractor helps to write a thread-safe concurrent program, but we can make thread-unsafe programs with Ractors. - -* GOOD: Sharing limitation - * Most objects are unshareable, so we can't make data-racy and race-conditional programs. - * Shareable objects are protected by an interpreter or locking mechanism. -* BAD: Class/Module can violate this assumption - * To make it compatible with old behavior, classes and modules can introduce data-race and so on. - * Ruby programmers should take care if they modify class/module objects on multi Ractor programs. -* BAD: Ractor can't solve all thread-safety problems - * There are several blocking operations (waiting send, waiting yield and waiting take) so you can make a program which has dead-lock and live-lock issues. - * Some kind of shareable objects can introduce transactions (STM, for example). However, misusing transactions will generate inconsistent state. - -Without Ractor, we need to trace all state-mutations to debug thread-safety issues. -With Ractor, you can concentrate on suspicious code which are shared with Ractors. - -## Creation and termination - -### `Ractor.new` - -* `Ractor.new{ expr }` generates another Ractor. - -```ruby -# Ractor.new with a block creates new Ractor -r = Ractor.new do - # This block will be run in parallel with other ractors -end - -# You can name a Ractor with `name:` argument. -r = Ractor.new name: 'test-name' do -end - -# and Ractor#name returns its name. -r.name #=> 'test-name' -``` - -### Given block isolation - -The Ractor executes given `expr` in a given block. -Given block will be isolated from outer scope by the `Proc#isolate` method (not exposed yet for Ruby users). To prevent sharing unshareable objects between ractors, block outer-variables, `self` and other information are isolated. - -`Proc#isolate` is called at Ractor creation time (when `Ractor.new` is called). If given Proc object is not able to isolate because of outer variables and so on, an error will be raised. - -```ruby -begin - a = true - r = Ractor.new do - a #=> ArgumentError because this block accesses `a`. - end - r.take # see later -rescue ArgumentError -end -``` - -* The `self` of the given block is the `Ractor` object itself. - -```ruby -r = Ractor.new do - p self.class #=> Ractor - self.object_id -end -r.take == self.object_id #=> false -``` - -Passed arguments to `Ractor.new()` becomes block parameters for the given block. However, an interpreter does not pass the parameter object references, but send them as messages (see below for details). - -```ruby -r = Ractor.new 'ok' do |msg| - msg #=> 'ok' -end -r.take #=> 'ok' -``` - -```ruby -# almost similar to the last example -r = Ractor.new do - msg = Ractor.receive - msg -end -r.send 'ok' -r.take #=> 'ok' -``` - -### An execution result of given block - -Return value of the given block becomes an outgoing message (see below for details). - -```ruby -r = Ractor.new do - 'ok' -end -r.take #=> `ok` -``` - -```ruby -# almost similar to the last example -r = Ractor.new do - Ractor.yield 'ok' -end -r.take #=> 'ok' -``` - -Error in the given block will be propagated to the receiver of an outgoing message. - -```ruby -r = Ractor.new do - raise 'ok' # exception will be transferred to the receiver -end - -begin - r.take -rescue Ractor::RemoteError => e - e.cause.class #=> RuntimeError - e.cause.message #=> 'ok' - e.ractor #=> r -end -``` - -## Communication between Ractors - -Communication between Ractors is achieved by sending and receiving messages. There are two ways to communicate with each other. - -* (1) Message sending/receiving - * (1-1) push type send/receive (sender knows receiver). Similar to the Actor model. - * (1-2) pull type yield/take (receiver knows sender). -* (2) Using shareable container objects - * Ractor::TVar gem ([ko1/ractor-tvar](https://github.com/ko1/ractor-tvar)) - * more? - -Users can control program execution timing with (1), but should not control with (2) (only manage as critical section). - -For message sending and receiving, there are two types of APIs: push type and pull type. - -* (1-1) send/receive (push type) - * `Ractor#send(obj)` (`Ractor#<<(obj)` is an alias) send a message to the Ractor's incoming port. Incoming port is connected to the infinite size incoming queue so `Ractor#send` will never block. - * `Ractor.receive` dequeue a message from its own incoming queue. If the incoming queue is empty, `Ractor.receive` calling will block. - * `Ractor.receive_if{|msg| filter_expr }` is variant of `Ractor.receive`. `receive_if` only receives a message which `filter_expr` is true (So `Ractor.receive` is the same as `Ractor.receive_if{ true }`. -* (1-2) yield/take (pull type) - * `Ractor.yield(obj)` send an message to a Ractor which are calling `Ractor#take` via outgoing port . If no Ractors are waiting for it, the `Ractor.yield(obj)` will block. If multiple Ractors are waiting for `Ractor.yield(obj)`, only one Ractor can receive the message. - * `Ractor#take` receives a message which is waiting by `Ractor.yield(obj)` method from the specified Ractor. If the Ractor does not call `Ractor.yield` yet, the `Ractor#take` call will block. -* `Ractor.select()` can wait for the success of `take`, `yield` and `receive`. -* You can close the incoming port or outgoing port. - * You can close then with `Ractor#close_incoming` and `Ractor#close_outgoing`. - * If the incoming port is closed for a Ractor, you can't `send` to the Ractor. If `Ractor.receive` is blocked for the closed incoming port, then it will raise an exception. - * If the outgoing port is closed for a Ractor, you can't call `Ractor#take` and `Ractor.yield` on the Ractor. If ractors are blocking by `Ractor#take` or `Ractor.yield`, closing outgoing port will raise an exception on these blocking ractors. - * When a Ractor is terminated, the Ractor's ports are closed. -* There are 3 ways to send an object as a message - * (1) Send a reference: Sending a shareable object, send only a reference to the object (fast) - * (2) Copy an object: Sending an unshareable object by copying an object deeply (slow). Note that you can not send an object which does not support deep copy. Some `T_DATA` objects (objects whose class is defined in a C extension, such as `StringIO`) are not supported. - * (3) Move an object: Sending an unshareable object reference with a membership. Sender Ractor can not access moved objects anymore (raise an exception) after moving it. Current implementation makes new object as a moved object for receiver Ractor and copies references of sending object to moved object. `T_DATA` objects are not supported. - * You can choose "Copy" and "Move" by the `move:` keyword, `Ractor#send(obj, move: true/false)` and `Ractor.yield(obj, move: true/false)` (default is `false` (COPY)). - -### Sending/Receiving ports - -Each Ractor has _incoming-port_ and _outgoing-port_. Incoming-port is connected to the infinite sized incoming queue. - -``` - Ractor r - +-------------------------------------------+ - | incoming outgoing | - | port port | - r.send(obj) ->*->[incoming queue] Ractor.yield(obj) ->*-> r.take - | | | - | v | - | Ractor.receive | - +-------------------------------------------+ - - -Connection example: r2.send obj on r1、Ractor.receive on r2 - +----+ +----+ - * r1 |---->* r2 * - +----+ +----+ - - -Connection example: Ractor.yield(obj) on r1, r1.take on r2 - +----+ +----+ - * r1 *---->- r2 * - +----+ +----+ - -Connection example: Ractor.yield(obj) on r1 and r2, - and waiting for both simultaneously by Ractor.select(r1, r2) - - +----+ - * r1 *------+ - +----+ | - +----> Ractor.select(r1, r2) - +----+ | - * r2 *------| - +----+ -``` - -```ruby -r = Ractor.new do - msg = Ractor.receive # Receive from r's incoming queue - msg # send back msg as block return value -end -r.send 'ok' # Send 'ok' to r's incoming port -> incoming queue -r.take # Receive from r's outgoing port -``` - -The last example shows the following ractor network. - -``` - +------+ +---+ - * main |------> * r *---+ - +------+ +---+ | - ^ | - +-------------------+ -``` - -And this code can be simplified by using an argument for `Ractor.new`. - -```ruby -# Actual argument 'ok' for `Ractor.new()` will be sent to created Ractor. -r = Ractor.new 'ok' do |msg| - # Values for formal parameters will be received from incoming queue. - # Similar to: msg = Ractor.receive - - msg # Return value of the given block will be sent via outgoing port -end - -# receive from the r's outgoing port. -r.take #=> `ok` -``` - -### Return value of a block for `Ractor.new` - -As already explained, the return value of `Ractor.new` (an evaluated value of `expr` in `Ractor.new{ expr }`) can be taken by `Ractor#take`. - -```ruby -Ractor.new{ 42 }.take #=> 42 -``` - -When the block return value is available, the Ractor is dead so that no ractors except taken Ractor can touch the return value, so any values can be sent with this communication path without any modification. - -```ruby -r = Ractor.new do - a = "hello" - binding -end - -r.take.eval("p a") #=> "hello" (other communication path can not send a Binding object directly) -``` - -### Wait for multiple Ractors with `Ractor.select` - -You can wait multiple Ractor's `yield` with `Ractor.select(*ractors)`. -The return value of `Ractor.select()` is `[r, msg]` where `r` is yielding Ractor and `msg` is yielded message. - -Wait for a single ractor (same as `Ractor.take`): - -```ruby -r1 = Ractor.new{'r1'} - -r, obj = Ractor.select(r1) -r == r1 and obj == 'r1' #=> true -``` - -Wait for two ractors: - -```ruby -r1 = Ractor.new{'r1'} -r2 = Ractor.new{'r2'} -rs = [r1, r2] -as = [] - -# Wait for r1 or r2's Ractor.yield -r, obj = Ractor.select(*rs) -rs.delete(r) -as << obj - -# Second try (rs only contain not-closed ractors) -r, obj = Ractor.select(*rs) -rs.delete(r) -as << obj -as.sort == ['r1', 'r2'] #=> true -``` - -\Complex example: - -```ruby -pipe = Ractor.new do - loop do - Ractor.yield Ractor.receive - end -end - -RN = 10 -rs = RN.times.map{|i| - Ractor.new pipe, i do |pipe, i| - msg = pipe.take - msg # ping-pong - end -} -RN.times{|i| - pipe << i -} -RN.times.map{ - r, n = Ractor.select(*rs) - rs.delete r - n -}.sort #=> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] -``` - -Multiple Ractors can send to one Ractor. - -```ruby -# Create 10 ractors and they send objects to pipe ractor. -# pipe ractor yield received objects - -pipe = Ractor.new do - loop do - Ractor.yield Ractor.receive - end -end - -RN = 10 -rs = RN.times.map{|i| - Ractor.new pipe, i do |pipe, i| - pipe << i - end -} - -RN.times.map{ - pipe.take -}.sort #=> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] -``` - -TODO: Current `Ractor.select()` has the same issue of `select(2)`, so this interface should be refined. - -TODO: `select` syntax of go-language uses round-robin technique to make fair scheduling. Now `Ractor.select()` doesn't use it. - -### Closing Ractor's ports - -* `Ractor#close_incoming/outgoing` close incoming/outgoing ports (similar to `Queue#close`). -* `Ractor#close_incoming` - * `r.send(obj)` where `r`'s incoming port is closed, will raise an exception. - * When the incoming queue is empty and incoming port is closed, `Ractor.receive` raises an exception. If the incoming queue is not empty, it dequeues an object without exceptions. -* `Ractor#close_outgoing` - * `Ractor.yield` on a Ractor which closed the outgoing port, it will raise an exception. - * `Ractor#take` for a Ractor which closed the outgoing port, it will raise an exception. If `Ractor#take` is blocking, it will raise an exception. -* When a Ractor terminates, the ports are closed automatically. - * Return value of the Ractor's block will be yielded as `Ractor.yield(ret_val)`, even if the implementation terminates the based native thread. - -Example (try to take from closed Ractor): - -```ruby -r = Ractor.new do - 'finish' -end -r.take # success (will return 'finish') -begin - o = r.take # try to take from closed Ractor -rescue Ractor::ClosedError - 'ok' -else - "ng: #{o}" -end -``` - -Example (try to send to closed (terminated) Ractor): - -```ruby -r = Ractor.new do -end - -r.take # wait terminate - -begin - r.send(1) -rescue Ractor::ClosedError - 'ok' -else - 'ng' -end -``` - -When multiple Ractors are waiting for `Ractor.yield()`, `Ractor#close_outgoing` will cancel all blocking by raising an exception (`ClosedError`). - -### Send a message by copying - -`Ractor#send(obj)` or `Ractor.yield(obj)` copy `obj` deeply if `obj` is an unshareable object. - -```ruby -obj = 'str'.dup -r = Ractor.new obj do |msg| - # return received msg's object_id - msg.object_id -end - -obj.object_id == r.take #=> false -``` - -Some objects are not supported to copy the value, and raise an exception. - -```ruby -obj = Thread.new{} -begin - Ractor.new obj do |msg| - msg - end -rescue TypeError => e - e.message #=> #<TypeError: allocator undefined for Thread> -else - 'ng' # unreachable here -end -``` - -### Send a message by moving - -`Ractor#send(obj, move: true)` or `Ractor.yield(obj, move: true)` move `obj` to the destination Ractor. -If the source Ractor touches the moved object (for example, call the method like `obj.foo()`), it will be an error. - -```ruby -# move with Ractor#send -r = Ractor.new do - obj = Ractor.receive - obj << ' world' -end - -str = 'hello' -r.send str, move: true -modified = r.take #=> 'hello world' - -# str is moved, and accessing str from this Ractor is prohibited - -begin - # Error because it touches moved str. - str << ' exception' # raise Ractor::MovedError -rescue Ractor::MovedError - modified #=> 'hello world' -else - raise 'unreachable' -end -``` - -```ruby -# move with Ractor.yield -r = Ractor.new do - obj = 'hello' - Ractor.yield obj, move: true - obj << 'world' # raise Ractor::MovedError -end - -str = r.take -begin - r.take -rescue Ractor::RemoteError - p str #=> "hello" -end -``` - -Some objects are not supported to move, and an exception will be raised. - -```ruby -r = Ractor.new do - Ractor.receive -end - -r.send(Thread.new{}, move: true) #=> allocator undefined for Thread (TypeError) -``` - -To achieve the access prohibition for moved objects, _class replacement_ technique is used to implement it. - -### Shareable objects - -The following objects are shareable. - -* Immutable objects - * Small integers, some symbols, `true`, `false`, `nil` (a.k.a. `SPECIAL_CONST_P()` objects in internal) - * Frozen native objects - * Numeric objects: `Float`, `Complex`, `Rational`, big integers (`T_BIGNUM` in internal) - * All Symbols. - * Frozen `String` and `Regexp` objects (their instance variables should refer only shareable objects) -* Class, Module objects (`T_CLASS`, `T_MODULE` and `T_ICLASS` in internal) -* `Ractor` and other special objects which care about synchronization. - -Implementation: Now shareable objects (`RVALUE`) have `FL_SHAREABLE` flag. This flag can be added lazily. - -To make shareable objects, `Ractor.make_shareable(obj)` method is provided. In this case, try to make sharaeble by freezing `obj` and recursively traversable objects. This method accepts `copy:` keyword (default value is false).`Ractor.make_shareable(obj, copy: true)` tries to make a deep copy of `obj` and make the copied object shareable. - -## Language changes to isolate unshareable objects between Ractors - -To isolate unshareable objects between Ractors, we introduced additional language semantics on multi-Ractor Ruby programs. - -Note that without using Ractors, these additional semantics is not needed (100% compatible with Ruby 2). - -### Global variables - -Only the main Ractor (a Ractor created at starting of interpreter) can access global variables. - -```ruby -$gv = 1 -r = Ractor.new do - $gv -end - -begin - r.take -rescue Ractor::RemoteError => e - e.cause.message #=> 'can not access global variables from non-main Ractors' -end -``` - -Note that some special global variables are ractor-local, like `$stdin`, `$stdout`, `$stderr`. See [[Bug #17268]](https://bugs.ruby-lang.org/issues/17268) for more details. - -### Instance variables of shareable objects - -Instance variables of classes/modules can be get from non-main Ractors if the referring values are shareable objects. - -```ruby -class C - @iv = 1 -end - -p Ractor.new do - class C - @iv - end -end.take #=> 1 -``` - -Otherwise, only the main Ractor can access instance variables of shareable objects. - -```ruby -class C - @iv = [] # unshareable object -end - -Ractor.new do - class C - begin - p @iv - rescue Ractor::IsolationError - p $!.message - #=> "can not get unshareable values from instance variables of classes/modules from non-main Ractors" - end - - begin - @iv = 42 - rescue Ractor::IsolationError - p $!.message - #=> "can not set instance variables of classes/modules by non-main Ractors" - end - end -end.take -``` - - - -```ruby -shared = Ractor.new{} -shared.instance_variable_set(:@iv, 'str') - -r = Ractor.new shared do |shared| - p shared.instance_variable_get(:@iv) -end - -begin - r.take -rescue Ractor::RemoteError => e - e.cause.message #=> can not access instance variables of shareable objects from non-main Ractors (Ractor::IsolationError) -end -``` - -Note that instance variables for class/module objects are also prohibited on Ractors. - -### Class variables - -Only the main Ractor can access class variables. - -```ruby -class C - @@cv = 'str' -end - -r = Ractor.new do - class C - p @@cv - end -end - - -begin - r.take -rescue => e - e.class #=> Ractor::IsolationError -end -``` - -### Constants - -Only the main Ractor can read constants which refer to the unshareable object. - -```ruby -class C - CONST = 'str' -end -r = Ractor.new do - C::CONST -end -begin - r.take -rescue => e - e.class #=> Ractor::IsolationError -end -``` - -Only the main Ractor can define constants which refer to the unshareable object. - -```ruby -class C -end -r = Ractor.new do - C::CONST = 'str' -end -begin - r.take -rescue => e - e.class #=> Ractor::IsolationError -end -``` - -To make multi-ractor supported library, the constants should only refer shareable objects. - -```ruby -TABLE = {a: 'ko1', b: 'ko2', c: 'ko3'} -``` - -In this case, `TABLE` references an unshareable Hash object. So that other ractors can not refer `TABLE` constant. To make it shareable, we can use `Ractor.make_shareable()` like that. - -```ruby -TABLE = Ractor.make_shareable( {a: 'ko1', b: 'ko2', c: 'ko3'} ) -``` - -To make it easy, Ruby 3.0 introduced new `shareable_constant_value` Directive. - -```ruby -# shareable_constant_value: literal - -TABLE = {a: 'ko1', b: 'ko2', c: 'ko3'} -#=> Same as: TABLE = Ractor.make_shareable( {a: 'ko1', b: 'ko2', c: 'ko3'} ) -``` - -`shareable_constant_value` directive accepts the following modes (descriptions use the example: `CONST = expr`): - -* none: Do nothing. Same as: `CONST = expr` -* literal: - * if `expr` consists of literals, replaced to `CONST = Ractor.make_shareable(expr)`. - * otherwise: replaced to `CONST = expr.tap{|o| raise unless Ractor.shareable?(o)}`. -* experimental_everything: replaced to `CONST = Ractor.make_shareable(expr)`. -* experimental_copy: replaced to `CONST = Ractor.make_shareable(expr, copy: true)`. - -Except the `none` mode (default), it is guaranteed that the assigned constants refer to only shareable objects. - -See [doc/syntax/comments.rdoc](syntax/comments.rdoc) for more details. - -## Implementation note - -* Each Ractor has its own thread, it means each Ractor has at least 1 native thread. -* Each Ractor has its own ID (`rb_ractor_t::pub::id`). - * On debug mode, all unshareable objects are labeled with current Ractor's id, and it is checked to detect unshareable object leak (access an object from different Ractor) in VM. - -## Examples - -### Traditional Ring example in Actor-model - -```ruby -RN = 1_000 -CR = Ractor.current - -r = Ractor.new do - p Ractor.receive - CR << :fin -end - -RN.times{ - r = Ractor.new r do |next_r| - next_r << Ractor.receive - end -} - -p :setup_ok -r << 1 -p Ractor.receive -``` - -### Fork-join - -```ruby -def fib n - if n < 2 - 1 - else - fib(n-2) + fib(n-1) - end -end - -RN = 10 -rs = (1..RN).map do |i| - Ractor.new i do |i| - [i, fib(i)] - end -end - -until rs.empty? - r, v = Ractor.select(*rs) - rs.delete r - p answer: v -end -``` - -### Worker pool - -```ruby -require 'prime' - -pipe = Ractor.new do - loop do - Ractor.yield Ractor.receive - end -end - -N = 1000 -RN = 10 -workers = (1..RN).map do - Ractor.new pipe do |pipe| - while n = pipe.take - Ractor.yield [n, n.prime?] - end - end -end - -(1..N).each{|i| - pipe << i -} - -pp (1..N).map{ - _r, (n, b) = Ractor.select(*workers) - [n, b] -}.sort_by{|(n, b)| n} -``` - -### Pipeline - -```ruby -# pipeline with yield/take -r1 = Ractor.new do - 'r1' -end - -r2 = Ractor.new r1 do |r1| - r1.take + 'r2' -end - -r3 = Ractor.new r2 do |r2| - r2.take + 'r3' -end - -p r3.take #=> 'r1r2r3' -``` - -```ruby -# pipeline with send/receive - -r3 = Ractor.new Ractor.current do |cr| - cr.send Ractor.receive + 'r3' -end - -r2 = Ractor.new r3 do |r3| - r3.send Ractor.receive + 'r2' -end - -r1 = Ractor.new r2 do |r2| - r2.send Ractor.receive + 'r1' -end - -r1 << 'r0' -p Ractor.receive #=> "r0r1r2r3" -``` - -### Supervise - -```ruby -# ring example again - -r = Ractor.current -(1..10).map{|i| - r = Ractor.new r, i do |r, i| - r.send Ractor.receive + "r#{i}" - end -} - -r.send "r0" -p Ractor.receive #=> "r0r10r9r8r7r6r5r4r3r2r1" -``` - -```ruby -# ring example with an error - -r = Ractor.current -rs = (1..10).map{|i| - r = Ractor.new r, i do |r, i| - loop do - msg = Ractor.receive - raise if /e/ =~ msg - r.send msg + "r#{i}" - end - end -} - -r.send "r0" -p Ractor.receive #=> "r0r10r9r8r7r6r5r4r3r2r1" -r.send "r0" -p Ractor.select(*rs, Ractor.current) #=> [:receive, "r0r10r9r8r7r6r5r4r3r2r1"] -r.send "e0" -p Ractor.select(*rs, Ractor.current) -#=> -# <Thread:0x000056262de28bd8 run> terminated with exception (report_on_exception is true): -# Traceback (most recent call last): -# 2: from /home/ko1/src/ruby/trunk/test.rb:7:in `block (2 levels) in <main>' -# 1: from /home/ko1/src/ruby/trunk/test.rb:7:in `loop' -# /home/ko1/src/ruby/trunk/test.rb:9:in `block (3 levels) in <main>': unhandled exception -# Traceback (most recent call last): -# 2: from /home/ko1/src/ruby/trunk/test.rb:7:in `block (2 levels) in <main>' -# 1: from /home/ko1/src/ruby/trunk/test.rb:7:in `loop' -# /home/ko1/src/ruby/trunk/test.rb:9:in `block (3 levels) in <main>': unhandled exception -# 1: from /home/ko1/src/ruby/trunk/test.rb:21:in `<main>' -# <internal:ractor>:69:in `select': thrown by remote Ractor. (Ractor::RemoteError) -``` - -```ruby -# resend non-error message - -r = Ractor.current -rs = (1..10).map{|i| - r = Ractor.new r, i do |r, i| - loop do - msg = Ractor.receive - raise if /e/ =~ msg - r.send msg + "r#{i}" - end - end -} - -r.send "r0" -p Ractor.receive #=> "r0r10r9r8r7r6r5r4r3r2r1" -r.send "r0" -p Ractor.select(*rs, Ractor.current) -[:receive, "r0r10r9r8r7r6r5r4r3r2r1"] -msg = 'e0' -begin - r.send msg - p Ractor.select(*rs, Ractor.current) -rescue Ractor::RemoteError - msg = 'r0' - retry -end - -#=> <internal:ractor>:100:in `send': The incoming-port is already closed (Ractor::ClosedError) -# because r == r[-1] is terminated. -``` - -```ruby -# ring example with supervisor and re-start - -def make_ractor r, i - Ractor.new r, i do |r, i| - loop do - msg = Ractor.receive - raise if /e/ =~ msg - r.send msg + "r#{i}" - end - end -end - -r = Ractor.current -rs = (1..10).map{|i| - r = make_ractor(r, i) -} - -msg = 'e0' # error causing message -begin - r.send msg - p Ractor.select(*rs, Ractor.current) -rescue Ractor::RemoteError - r = rs[-1] = make_ractor(rs[-2], rs.size-1) - msg = 'x0' - retry -end - -#=> [:receive, "x0r9r9r8r7r6r5r4r3r2r1"] -``` diff --git a/doc/rdoc/markup_reference.rb b/doc/rdoc/markup_reference.rb deleted file mode 100644 index ee585b2497..0000000000 --- a/doc/rdoc/markup_reference.rb +++ /dev/null @@ -1,1281 +0,0 @@ -require 'rdoc' - -# \Class \RDoc::MarkupReference exists only to provide a suitable home -# for a reference document for \RDoc markup. -# -# All objects defined in this class -- classes, modules, methods, aliases, -# attributes, and constants -- are solely for illustrating \RDoc markup, -# and have no other legitimate use. -# -# == About the Examples -# -# - Examples in this reference are Ruby code and comments; -# certain differences from other sources -# (such as C code and comments) are noted. -# - Almost all examples on this page are all RDoc-like; -# that is, they have no explicit comment markers like Ruby <tt>#</tt> -# or C <tt>/* ... */</tt>. -# - An example that shows rendered HTML output -# displays that output in a blockquote: -# -# >>> -# Some stuff -# -# == \RDoc Sources -# -# The sources of \RDoc documentation vary according to the type of file: -# -# - <tt>.rb</tt> (Ruby code file): -# -# - Markup may be found in Ruby comments: -# A comment that immediately precedes the definition -# of a Ruby class, module, method, alias, constant, or attribute -# becomes the documentation for that defined object. -# - An \RDoc directive may be found in: -# -# - A trailing comment (on the same line as code); -# see <tt>:nodoc:</tt>, <tt>:doc:</tt>, and <tt>:notnew:</tt>. -# - A single-line comment; -# see other {Directives}[rdoc-ref:RDoc::MarkupReference@Directives]. -# -# - Documentation may be derived from the Ruby code itself; -# see {Documentation Derived from Ruby Code}[rdoc-ref:RDoc::MarkupReference@Documentation+Derived+from+Ruby+Code]. -# -# - <tt>.c</tt> (C code file): markup is parsed from C comments. -# A comment that immediately precedes -# a function that implements a Ruby method, -# or otherwise immediately precedes the definition of a Ruby object, -# becomes the documentation for that object. -# - <tt>.rdoc</tt> (\RDoc markup text file) or <tt>.md</tt> (\RDoc markdown text file): -# markup is parsed from the entire file. -# The text is not associated with any code object, -# but may (depending on how the documentation is built) -# become a separate page. -# -# An <i>RDoc document</i>: -# -# - A (possibly multi-line) comment in a Ruby or C file -# that generates \RDoc documentation (as above). -# - The entire markup (<tt>.rdoc</tt>) file or markdown (<tt>.md</tt>) file -# (which is usually multi-line). -# -# === Blocks -# -# It's convenient to think of an \RDoc document as a sequence of _blocks_ -# of various types (details at the links): -# -# - {Paragraph}[rdoc-ref:RDoc::MarkupReference@Paragraphs]: -# an ordinary paragraph. -# - {Verbatim text block}[rdoc-ref:RDoc::MarkupReference@Verbatim+Text+Blocks]: -# a block of text to be rendered literally. -# - {Code block}[rdoc-ref:RDoc::MarkupReference@Code+Blocks]: -# a verbatim text block containing Ruby code, -# to be rendered with code highlighting. -# - {Block quote}[rdoc-ref:RDoc::MarkupReference@Block+Quotes]: -# a longish quoted passage, to be rendered with indentation -# instead of quote marks. -# - {List}[rdoc-ref:RDoc::MarkupReference@Lists]: items for -# a bullet list, numbered list, lettered list, or labeled list. -# - {Heading}[rdoc-ref:RDoc::MarkupReference@Headings]: -# a heading. -# - {Horizontal rule}[rdoc-ref:RDoc::MarkupReference@Horizontal+Rules]: -# a line across the rendered page. -# - {Directive}[rdoc-ref:RDoc::MarkupReference@Directives]: -# various special directions for the rendering. -# - {Text Markup}[rdoc-ref:RDoc:MarkupReference@Text+Markup]: -# text to be rendered in a special way. -# -# About the blocks: -# -# - Except for a paragraph, a block is distinguished by its indentation, -# or by unusual initial or embedded characters. -# - Any block may appear independently -# (that is, not nested in another block); -# some blocks may be nested, as detailed below. -# - In a multi-line block, -# \RDoc looks for the block's natural left margin, -# which becomes the <em>base margin</em> for the block -# and is the initial <em>current margin</em> for the block. -# -# ==== Paragraphs -# -# A paragraph consists of one or more non-empty lines of ordinary text, -# each beginning at the current margin. -# -# Note: Here, <em>ordinary text</em> means text that is <em>not identified</em> -# by indentation, or by unusual initial or embedded characters. -# See below. -# -# Paragraphs are separated by one or more empty lines. -# -# Example input: -# -# \RDoc produces HTML and command-line documentation for Ruby projects. -# \RDoc includes the rdoc and ri tools for generating and displaying -# documentation from the command-line. -# -# You'll love it. -# -# Rendered HTML: -# >>> -# \RDoc produces HTML and command-line documentation for Ruby projects. -# \RDoc includes the rdoc and ri tools for generating and displaying -# documentation from the command-line. -# -# You'll love it. -# -# A paragraph may contain nested blocks, including: -# -# - {Verbatim text blocks}[rdoc-ref:RDoc::MarkupReference@Verbatim+Text+Blocks]. -# - {Code blocks}[rdoc-ref:RDoc::MarkupReference@Code+Blocks]. -# - {Block quotes}[rdoc-ref:RDoc::MarkupReference@Block+Quotes]. -# - {Lists}[rdoc-ref:RDoc::MarkupReference@Lists]. -# - {Headings}[rdoc-ref:RDoc::MarkupReference@Headings]. -# - {Horizontal rules}[rdoc-ref:RDoc::MarkupReference@Horizontal+Rules]. -# - {Text Markup}[rdoc-ref:RDoc:MarkupReference@Text+Markup]. -# -# ==== Verbatim Text Blocks -# -# Text indented farther than the current margin becomes a <em>verbatim text block</em> -# (or a code block, described next). -# In the rendered HTML, such text: -# -# - Is indented. -# - Has a contrasting background color. -# -# The verbatim text block ends at the first line beginning at the current margin. -# -# Example input: -# -# This is not verbatim text. -# -# This is verbatim text. -# Whitespace is honored. # See? -# Whitespace is honored. # See? -# -# This is still the same verbatim text block. -# -# This is not verbatim text. -# -# Rendered HTML: -# >>> -# This is not verbatim text. -# -# This is verbatim text. -# Whitespace is honored. # See? -# Whitespace is honored. # See? -# -# This is still the same verbatim text block. -# -# This is not verbatim text. -# -# A verbatim text block may not contain nested blocks of any kind -# -- it's verbatim. -# -# ==== Code Blocks -# -# A special case of verbatim text is the <em>code block</em>, -# which is merely verbatim text that \RDoc recognizes as Ruby code: -# -# In the rendered HTML, the code block: -# -# - Is indented. -# - Has a contrasting background color. -# - Has syntax highlighting. -# -# Example input: -# -# Consider this method: -# -# def foo(name = '', value = 0) -# @name = name # Whitespace is still honored. -# @value = value -# end -# -# -# Rendered HTML: -# >>> -# Consider this method: -# -# def foo(name = '', value = 0) -# @name = name # Whitespace is still honored. -# @value = value -# end -# -# Pro tip: If your indented Ruby code does not get highlighted, -# it may contain a syntax error. -# -# A code block may not contain nested blocks of any kind -# -- it's verbatim. -# -# ==== Block Quotes -# -# You can use the characters <tt>>>></tt> (unindented), -# followed by indented text, to treat the text -# as a {block quote}[https://en.wikipedia.org/wiki/Block_quotation]: -# -# Example input: -# -# Here's a block quote: -# >>> -# Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer -# commodo quam iaculis massa posuere, dictum fringilla justo pulvinar. -# Quisque turpis erat, pharetra eu dui at, sollicitudin accumsan nulla. -# -# Aenean congue ligula eu ligula molestie, eu pellentesque purus -# faucibus. In id leo non ligula condimentum lobortis. Duis vestibulum, -# diam in pellentesque aliquet, mi tellus placerat sapien, id euismod -# purus magna ut tortor. -# -# Rendered HTML: -# -# >>> -# Here's a block quote: -# >>> -# Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer -# commodo quam iaculis massa posuere, dictum fringilla justo pulvinar. -# Quisque turpis erat, pharetra eu dui at, sollicitudin accumsan nulla. -# -# Aenean congue ligula eu ligula molestie, eu pellentesque purus -# faucibus. In id leo non ligula condimentum lobortis. Duis vestibulum, -# diam in pellentesque aliquet, mi tellus placerat sapien, id euismod -# purus magna ut tortor. -# -# Note that, unlike verbatim text, single newlines are not honored, -# but that a double newline begins a new paragraph in the block quote. -# -# A block quote may contain nested blocks, including: -# -# - Other block quotes. -# - {Paragraphs}[rdoc-ref:RDoc::MarkupReference@Paragraphs]. -# - {Verbatim text blocks}[rdoc-ref:RDoc::MarkupReference@Verbatim+Text+Blocks]. -# - {Code blocks}[rdoc-ref:RDoc::MarkupReference@Code+Blocks]. -# - {Lists}[rdoc-ref:RDoc::MarkupReference@Lists]. -# - {Headings}[rdoc-ref:RDoc::MarkupReference@Headings]. -# - {Horizontal rules}[rdoc-ref:RDoc::MarkupReference@Horizontal+Rules]. -# - {Text Markup}[rdoc-ref:RDoc:MarkupReference@Text+Markup]. -# -# ==== Lists -# -# Each type of list item is marked by a special beginning: -# -# - Bullet list item: Begins with a hyphen or asterisk. -# - Numbered list item: Begins with digits and a period. -# - Lettered list item: Begins with an alphabetic character and a period. -# - Labeled list item: Begins with one of: -# - Square-bracketed text. -# - A word followed by two colons. -# -# A list begins with a list item and continues, even across blank lines, -# as long as list items of the same type are found at the same indentation level. -# -# A new list resets the current margin inward. -# Additional lines of text aligned at that margin -# are part of the continuing list item. -# -# A list item may be continued on additional lines that are aligned -# with the first line. See examples below. -# -# A list item may contain nested blocks, including: -# -# - Other lists of any type. -# - {Paragraphs}[rdoc-ref:RDoc::MarkupReference@Paragraphs]. -# - {Verbatim text blocks}[rdoc-ref:RDoc::MarkupReference@Verbatim+Text+Blocks]. -# - {Code blocks}[rdoc-ref:RDoc::MarkupReference@Code+Blocks]. -# - {Block quotes}[rdoc-ref:RDoc::MarkupReference@Block+Quotes]. -# - {Headings}[rdoc-ref:RDoc::MarkupReference@Headings]. -# - {Horizontal rules}[rdoc-ref:RDoc::MarkupReference@Horizontal+Rules]. -# - {Text Markup}[rdoc-ref:RDoc:MarkupReference@Text+Markup]. -# -# ===== Bullet Lists -# -# A bullet list item begins with a hyphen or asterisk. -# -# Example input: -# -# - An item. -# - Another. -# - An item spanning -# multiple lines. -# -# * Yet another. -# - Last one. -# -# Rendered HTML: -# >>> -# - An item. -# - Another. -# - An item spanning -# multiple lines. -# -# * Yet another. -# - Last one. -# -# ===== Numbered Lists -# -# A numbered list item begins with digits and a period. -# -# The items are automatically re-numbered. -# -# Example input: -# -# 100. An item. -# 10. Another. -# 1. An item spanning -# multiple lines. -# -# 1. Yet another. -# 1000. Last one. -# -# Rendered HTML: -# >>> -# 100. An item. -# 10. Another. -# 1. An item spanning -# multiple lines. -# -# 1. Yet another. -# 1000. Last one. -# -# ===== Lettered Lists -# -# A lettered list item begins with letters and a period. -# -# The items are automatically "re-lettered." -# -# Example input: -# -# z. An item. -# y. Another. -# x. An item spanning -# multiple lines. -# -# x. Yet another. -# a. Last one. -# -# Rendered HTML: -# >>> -# z. An item. -# y. Another. -# -# x. Yet another. -# a. Last one. -# -# ===== Labeled Lists -# -# A labeled list item begins with one of: -# -# - Square-bracketed text: the label and text are on two lines. -# - A word followed by two colons: the label and text are on the same line. -# -# Example input: -# -# [foo] An item. -# bat:: Another. -# [bag] An item spanning -# multiple lines. -# -# [bar baz] Yet another. -# bam:: Last one. -# -# Rendered HTML: -# >>> -# [foo] An item. -# bat:: Another. -# [bag] An item spanning -# multiple lines. -# -# [bar baz] Yet another. -# bam:: Last one. -# -# ==== Headings -# -# A heading begins with up to six equal-signs, followed by heading text. -# Whitespace between those and the heading text is optional. -# -# Examples: -# -# = Section 1 -# == Section 1.1 -# === Section 1.1.1 -# === Section 1.1.2 -# == Section 1.2 -# = Section 2 -# = Foo -# == Bar -# === Baz -# ==== Bam -# ===== Bat -# ====== Bad -# ============Still a Heading (Level 6) -# \== Not a Heading -# -# A heading may contain only one type of nested block: -# -# - {Text Markup}[rdoc-ref:RDoc:MarkupReference@Text+Markup]. -# -# ==== Horizontal Rules -# -# A horizontal rule consists of a line with three or more hyphens -# and nothing more. -# -# Example input: -# -# --- -# --- Not a horizontal rule. -# -# -- Also not a horizontal rule. -# --- -# -# Rendered HTML: -# >>> -# --- -# --- Not a horizontal rule. -# -# -- Also not a horizontal rule. -# --- -# -# ==== Directives -# -# ===== Directives for Allowing or Suppressing Documentation -# -# - <tt># :stopdoc:</tt>: -# -# - Appears on a line by itself. -# - Specifies that \RDoc should ignore markup -# until next <tt>:startdoc:</tt> directive or end-of-file. -# -# - <tt># :startdoc:</tt>: -# -# - Appears on a line by itself. -# - Specifies that \RDoc should resume parsing markup. -# -# - <tt># :enddoc:</tt>: -# -# - Appears on a line by itself. -# - Specifies that \RDoc should ignore markup to end-of-file -# regardless of other directives. -# -# - <tt># :nodoc:</tt>: -# -# - Appended to a line of code -# that defines a class, module, method, alias, constant, or attribute. -# -# - Specifies that the defined object should not be documented. -# -# - For a method definition in C code, it the directive must be in the comment line -# immediately preceding the definition: -# -# /* :nodoc: */ -# static VALUE -# some_method(VALUE self) -# { -# return self; -# } -# -# Note that this directive has <em>no effect at all</em> -# when placed at the method declaration: -# -# /* :nodoc: */ -# rb_define_method(cMyClass, "do_something", something_func, 0); -# -# The above comment is just a comment and has nothing to do with \RDoc. -# Therefore, +do_something+ method will be reported as "undocumented" -# unless that method or function is documented elsewhere. -# -# - For a constant definition in C code, this directive <em>can not work</em> -# because there is no "implementation" place for a constant. -# -# - <tt># :nodoc: all</tt>: -# -# - Appended to a line of code -# that defines a class or module. -# - Specifies that the class or module should not be documented. -# By default, however, a nested class or module _will_ be documented. -# -# - <tt># :doc:</tt>: -# -# - Appended to a line of code -# that defines a class, module, method, alias, constant, or attribute. -# - Specifies the defined object should be documented, even if it otherwise -# would not be documented. -# -# - <tt># :notnew:</tt> (aliased as <tt>:not_new:</tt> and <tt>:not-new:</tt>): -# -# - Appended to a line of code -# that defines instance method +initialize+. -# - Specifies that singleton method +new+ should not be documented. -# By default, Ruby fakes a corresponding singleton method +new+, -# which \RDoc includes in the documentation. -# Note that instance method +initialize+ is private, and so by default -# is not documented. -# -# For Ruby code, but not for other \RDoc sources, -# there is a shorthand for <tt>:stopdoc:</tt> and <tt>:startdoc:</tt>: -# -# # Documented. -# #-- -# # Not documented. -# #++ -# # Documented. -# -# For C code, any of directives <tt>:startdoc:</tt>, <tt>:stopdoc:</tt>, -# and <tt>:enddoc:</tt> may appear in a stand-alone comment: -# -# /* :startdoc: */ -# /* :stopdoc: */ -# /* :enddoc: */ -# -# ===== Directive for Specifying \RDoc Source Format -# -# - <tt># :markup: _type_</tt>: -# -# - Appears on a line by itself. -# - Specifies the format for the \RDoc input; -# parameter +type+ is one of: +rdoc+ (the default), +markdown+, +rd+, +tomdoc+. -# See {Markup Formats}[rdoc-ref:RDoc::Markup@Markup+Formats]. -# -# ===== Directives for Method Documentation -# -# - <tt># :call-seq:</tt>: -# -# - Appears on a line by itself. -# - Specifies the calling sequence to be reported in the HTML, -# overriding the actual calling sequence in the code. -# See method #call_seq_directive. -# -# Note that \RDoc can build the calling sequence for a Ruby-coded method, -# but not for other languages. -# You may want to override that by explicitly giving a <tt>:call-seq:</tt> -# directive if you want to include: -# -# - A return type, which is not automatically inferred. -# - Multiple calling sequences. -# -# For C code, the directive may appear in a stand-alone comment. -# -# - <tt># :args: _arg_names_</tt> (aliased as <tt>:arg:</tt>): -# -# - Appears on a line by itself. -# - Specifies the arguments to be reported in the HTML, -# overriding the actual arguments in the code. -# See method #args_directive. -# -# - <tt># :yields: _arg_names_</tt> (aliased as <tt>:yield:</tt>): -# -# - Appears on a line by itself. -# - Specifies the yield arguments to be reported in the HTML, -# overriding the actual yield in the code. -# See method #yields_directive. -# -# ===== Directives for Organizing Documentation -# -# By default, \RDoc groups: -# -# - Singleton methods together in alphabetical order. -# - Instance methods and their aliases together in alphabetical order. -# - Attributes and their aliases together in alphabetical order. -# -# You can use directives to modify those behaviors. -# -# - <tt># :section: _section_title_</tt>: -# -# - Appears on a line by itself. -# - Specifies that following methods are to be grouped into the section -# with the given <em>section_title</em>, -# or into the default section if no title is given. -# The directive remains in effect until another such directive is given, -# but may be temporarily overridden by directive <tt>:category:</tt>. -# See below. -# -# The comment block containing this directive: -# -# - Must be separated by a blank line from the documentation for the next item. -# - May have one or more lines preceding the directive. -# These will be removed, along with any trailing lines that match them. -# Such lines may be visually helpful. -# - Lines of text that are not so removed become the descriptive text -# for the section. -# -# Example: -# -# # ---------------------------------------- -# # :section: My Section -# # This is the section that I wrote. -# # See it glisten in the noon-day sun. -# # ---------------------------------------- -# -# ## -# # Comment for some_method -# def some_method -# # ... -# end -# -# You can use directive <tt>:category:</tt> to temporarily -# override the current section. -# -# - <tt># :category: _section_title_</tt>: -# -# - Appears on a line by itself. -# - Specifies that just one following method is to be included -# in the given section, or in the default section if no title is given. -# Subsequent methods are to be grouped into the current section. -# -# ===== Directive for Including a File -# -# - <tt># :include: _filepath_</tt>: -# -# - Appears on a line by itself. -# - Specifies that the contents of the given file -# are to be included at this point. -# The file content is shifted to have the same indentation as the colon -# at the start of the directive. -# -# The file is searched for in the directory containing the current file, -# and then in each of the directories given with the <tt>--include</tt> -# command-line option. -# -# For C code, the directive may appear in a stand-alone comment -# -# ==== Text Markup -# -# Text markup is metatext that affects HTML rendering: -# -# - Typeface: italic, bold, monofont. -# - Character conversions: copyright, trademark, certain punctuation. -# - Links. -# - Escapes: marking text as "not markup." -# -# ===== Typeface Markup -# -# Typeface markup can specify that text is to be rendered -# as italic, bold, or monofont. -# -# Typeface markup may contain only one type of nested block: -# -# - More typeface markup: -# italic, bold, monofont. -# -# ====== Italic -# -# Text may be marked as italic via HTML tag <tt><i></tt> or <tt><em></tt>. -# -# Example input: -# -# <i>Italicized words</i> in a paragraph. -# -# >>> -# <i>Italicized words in a block quote</i>. -# -# - <i>Italicized words</i> in a list item. -# -# ====== <i>Italicized words</i> in a Heading -# -# <i>Italicized passage containing *bold* and +monofont+.</i> -# -# Rendered HTML: -# >>> -# <i>Italicized words</i> in a paragraph. -# -# >>> -# <i>Italicized words in a block quote</i>. -# -# - <i>Italicized words</i> in a list item. -# -# ====== <i>Italicized words</i> in a Heading -# -# <i>Italicized passage containing *bold* and +monofont+.</i> -# -# A single word may be italicized via a shorthand: -# prefixed and suffixed underscores. -# -# Example input: -# -# _Italic_ in a paragraph. -# -# >>> -# _Italic_ in a block quote. -# -# - _Italic_ in a list item. -# -# ====== _Italic_ in a Heading -# -# Rendered HTML: -# >>> -# _Italic_ in a paragraph. -# -# >>> -# _Italic_ in a block quote. -# -# - _Italic_ in a list item. -# -# ====== _Italic_ in a Heading -# -# ====== Bold -# -# Text may be marked as bold via HTML tag <tt><b></tt>. -# -# Example input: -# -# <b>Bold words</b> in a paragraph. -# -# >>> -# <b>Bold words</b> in a block quote. -# -# - <b>Bold words</b> in a list item. -# -# ====== <b>Bold words</b> in a Heading -# -# <b>Bold passage containing _italics_ and +monofont+.</b> -# -# Rendered HTML: -# -# >>> -# <b>Bold words</b> in a paragraph. -# -# >>> -# <b>Bold words</b> in a block quote. -# -# - <b>Bold words</b> in a list item. -# -# ====== <b>Bold words</b> in a Heading -# -# <b>Bold passage containing _italics_ and +monofont+.</b> -# -# A single word may be made bold via a shorthand: -# prefixed and suffixed asterisks. -# -# Example input: -# -# *Bold* in a paragraph. -# -# >>> -# *Bold* in a block quote. -# -# - *Bold* in a list item. -# -# ===== *Bold* in a Heading -# -# Rendered HTML: -# -# >>> -# *Bold* in a paragraph. -# -# >>> -# *Bold* in a block quote. -# -# - *Bold* in a list item. -# -# ===== *Bold* in a Heading -# -# ====== Monofont -# -# Text may be marked as monofont -# -- sometimes called 'typewriter font' -- -# via HTML tag <tt><tt></tt> or <tt><code></tt>. -# -# Example input: -# -# <tt>Monofont words</tt> in a paragraph. -# -# >>> -# <tt>Monofont words</tt> in a block quote. -# -# - <tt>Monofont words</tt> in a list item. -# -# ====== <tt>Monofont words</tt> in heading -# -# <tt>Monofont passage containing _italics_ and *bold*.</tt> -# -# Rendered HTML: -# -# >>> -# <tt>Monofont words</tt> in a paragraph. -# -# >>> -# <tt>Monofont words</tt> in a block quote. -# -# - <tt>Monofont words</tt> in a list item. -# -# ====== <tt>Monofont words</tt> in heading -# -# <tt>Monofont passage containing _italics_ and *bold*.</tt> -# -# A single word may be made monofont by a shorthand: -# prefixed and suffixed plus-signs. -# -# Example input: -# -# +Monofont+ in a paragraph. -# -# >>> -# +Monofont+ in a block quote. -# -# - +Monofont+ in a list item. -# -# ====== +Monofont+ in a Heading -# -# Rendered HTML: -# -# >>> -# +Monofont+ in a paragraph. -# -# >>> -# +Monofont+ in a block quote. -# -# - +Monofont+ in a list item. -# -# ====== +Monofont+ in a Heading -# -# ==== Character Conversions -# -# Certain combinations of characters may be converted to special characters; -# whether the conversion occurs depends on whether the special character -# is available in the current encoding. -# -# - <tt>(c)</tt> converts to (c) (copyright character); must be lowercase. -# -# - <tt>(r)</tt> converts to (r) (registered trademark character); must be lowercase. -# -# - <tt>'foo'</tt> converts to 'foo' (smart single-quotes). -# -# - <tt>"foo"</tt> converts to "foo" (smart double-quotes). -# -# - <tt>foo ... bar</tt> converts to foo ... bar (1-character ellipsis). -# -# - <tt>foo -- bar</tt> converts to foo -- bar (1-character en-dash). -# -# - <tt>foo --- bar</tt> converts to foo --- bar (1-character em-dash). -# -# ==== Links -# -# Certain strings in \RDoc text are converted to links. -# Any such link may be suppressed by prefixing a backslash. -# This section shows how to link to various -# targets. -# -# [Class] -# -# - On-page: <tt>DummyClass</tt> links to DummyClass. -# - Off-page: <tt>RDoc::Alias</tt> links to RDoc::Alias. -# -# Note: For poeple want to mark up code (such as class, module, -# constant, and method) as "<tt>+code+</tt>" (for interoperability -# with other MarkDown parsers mainly), such word that refers a known -# code object and is marked up entirely and separately as "monofont" -# is also converted to a link. -# -# - <tt>+DummyClass+</tt> links to DummyClass -# - <tt>+DummyClass-object+</tt> is not a link. -# -# [Module] -# -# - On-page: <tt>DummyModule</tt> links to DummyModule. -# - Off-page: <tt>RDoc</tt> links to RDoc. -# -# [Constant] -# -# - On-page: <tt>DUMMY_CONSTANT</tt> links to DUMMY_CONSTANT. -# - Off-page: <tt>RDoc::Text::MARKUP_FORMAT</tt> links to RDoc::Text::MARKUP_FORMAT. -# -# [Singleton Method] -# -# - On-page: <tt>::dummy_singleton_method</tt> links to ::dummy_singleton_method. -# - Off-page<tt>RDoc::TokenStream::to_html</tt> links to RDoc::TokenStream::to_html. -# -# Note: Occasionally \RDoc is not linked to a method whose name -# has only special characters. Check whether the links you were expecting -# are actually there. If not, you'll need to put in an explicit link; -# see below. -# -# Pro tip: The link to any method is available in the alphabetical table of contents -# at the top left of the page for the class or module. -# -# [Instance Method] -# -# - On-page: <tt>#dummy_instance_method</tt> links to #dummy_instance_method. -# - Off-page: <tt>RDoc::Alias#html_name</tt> links to RDoc::Alias#html_name. -# -# See the Note and Pro Tip immediately above. -# -# [Attribute] -# -# - On-page: <tt>#dummy_attribute</tt> links to #dummy_attribute. -# - Off-page: <tt>RDoc::Alias#name</tt> links to RDoc::Alias#name. -# -# [Alias] -# -# - On-page: <tt>#dummy_instance_alias</tt> links to #dummy_instance_alias. -# - Off-page: <tt>RDoc::Alias#new_name</tt> links to RDoc::Alias#new_name. -# -# [Protocol +http+] -# -# - Linked: <tt>http://yahoo.com</tt> links to http://yahoo.com. -# -# [Protocol +https+] -# -# - Linked: <tt>https://github.com</tt> links to https://github.com. -# -# [Protocol +ftp+] -# -# - Linked: <tt>ftp://nosuch.site</tt> links to ftp://nosuch.site. -# -# [Protocol +mailto+] -# -# - Linked: <tt>mailto:/foo@bar.com</tt> links to mailto://foo@bar.com. -# -# [Protocol +irc+] -# -# - link: <tt>irc://irc.freenode.net/ruby</tt> links to irc://irc.freenode.net/ruby. -# -# [Image Filename Extensions] -# -# - Link: <tt>https://www.ruby-lang.org/images/header-ruby-logo@2x.png</tt> is -# converted to an in-line HTML +img+ tag, which displays the image in the HTML: -# -# https://www.ruby-lang.org/images/header-ruby-logo@2x.png -# -# Also works for +bmp+, +gif+, +jpeg+, and +jpg+ files. -# -# Note: Works only for a fully qualified URL. -# -# [Heading] -# -# - Link: <tt>RDoc::RD@LICENSE</tt> links to RDoc::RDoc::RD@LICENSE. -# -# Note that spaces in the actual heading are represented by <tt>+</tt> characters -# in the linkable text. -# -# - Link: <tt>RDoc::Options@Saved+Options</tt> -# links to RDoc::Options@Saved+Options. -# -# Punctuation and other special characters must be escaped like CGI.escape. -# -# Pro tip: The link to any heading is available in the alphabetical table of contents -# at the top left of the page for the class or module. -# -# [Section] -# -# See {Directives for Organizing Documentation}[#class-RDoc::MarkupReference-label-Directives+for+Organizing+Documentation]. -# -# - Link: <tt>RDoc::Markup::ToHtml@Visitor</tt> links to RDoc::Markup::ToHtml@Visitor. -# -# If a section and a heading share the same name, the link target is the section. -# -# [Single-Word Text Link] -# -# Use square brackets to create single-word text link: -# -# - <tt>GitHub[https://github.com]</tt> links to GitHub[https://github.com]. -# -# [Multi-Word Text Link] -# -# Use square brackets and curly braces to create a multi-word text link. -# -# - <tt>{GitHub home page}[https://github.com]</tt> links to -# {GitHub home page}[https://github.com]. -# -# [<tt>rdoc-ref</tt> Scheme] -# -# A link with the <tt>rdoc-ref:</tt> scheme links to the referenced item, -# if that item exists. -# The referenced item may be a class, module, method, file, etc. -# -# - Class: <tt>Alias[rdoc-ref:RDoc::Alias]</tt> generates Alias[rdoc-ref:RDoc::Alias]. -# - Module: <tt>RDoc[rdoc-ref:RDoc]</tt> generates RDoc[rdoc-ref:RDoc]. -# - Method: <tt>foo[rdoc-ref:RDoc::MarkupReference#dummy_instance_method]</tt> -# generates foo[rdoc-ref:RDoc::MarkupReference#dummy_instance_method]. -# - Constant: <tt>bar[rdoc-ref:RDoc::MarkupReference::DUMMY_CONSTANT]</tt> -# generates bar[rdoc-ref:RDoc::MarkupReference::DUMMY_CONSTANT]. -# - Attribute: <tt>baz[rdoc-ref:RDoc::MarkupReference#dummy_attribute]</tt> -# generates baz[rdoc-ref:RDoc::MarkupReference#dummy_attribute]. -# - Alias: <tt>bad[rdoc-ref:RDoc::MarkupReference#dummy_instance_alias]</tt> -# generates bad[rdoc-ref:RDoc::MarkupReference#dummy_instance_alias]. -# -# If the referenced item does not exist, no link is generated -# and entire <tt>rdoc-ref:</tt> square-bracketed clause is removed -# from the resulting text. -# -# - <tt>Nosuch[rdoc-ref:RDoc::Nosuch]</tt> generates Nosuch. -# -# -# [<tt>rdoc-label</tt> Scheme] -# -# [Simple] -# -# You can specify a link target using this form, -# where the second part cites the id of an HTML element. -# -# This link refers to the constant +DUMMY_CONSTANT+ on this page: -# -# - <tt>{DUMMY_CONSTANT}[rdoc-label:DUMMY_CONSTANT]</tt> -# -# Thus: -# -# {DUMMY_CONSTANT}[rdoc-label:DUMMY_CONSTANT] -# -# [With Return] -# -# You can specify both a link target and a local label -# that can be used as the target for a return link. -# These two links refer to each other: -# -# - <tt>{go to addressee}[rdoc-label:addressee:sender]</tt> -# - <tt>{return to sender}[rdoc-label:sender:addressee]</tt> -# -# Thus: -# -# {go to addressee}[rdoc-label:addressee:sender] -# -# Some text. -# -# {return to sender}[rdoc-label:sender:addressee] -# -# [<tt>link:</tt> Scheme] -# -# - <tt>link:README_rdoc.html</tt> links to link:README_rdoc.html. -# -# [<tt>rdoc-image</tt> Scheme] -# -# Use the <tt>rdoc-image</tt> scheme to display an image that is also a link: -# -# # {rdoc-image:path/to/image}[link_target] -# -# - Link: <tt>{rdoc-image:https://www.ruby-lang.org/images/header-ruby-logo@2x.png}[https://www.ruby-lang.org]</tt> -# displays image <tt>https://www.ruby-lang.org/images/header-ruby-logo@2x.png</tt> -# as a link to <tt>https://www.ruby-lang.org</tt>. -# -# {rdoc-image:https://www.ruby-lang.org/images/header-ruby-logo@2x.png}[https://www.ruby-lang.org] -# -# A relative path as the target also works: -# -# - Link: <tt>{rdoc-image:https://www.ruby-lang.org/images/header-ruby-logo@2x.png}[./Alias.html]</tt> links to <tt>./Alias.html</tt> -# -# {rdoc-image:https://www.ruby-lang.org/images/header-ruby-logo@2x.png}[./Alias.html] -# -# === Escaping Text -# -# Text that would otherwise be interpreted as markup -# can be "escaped," so that it is not interpreted as markup; -# the escape character is the backslash (<tt>'\\'</tt>). -# -# In a verbatim text block or a code block, -# the escape character is always preserved: -# -# Example input: -# -# This is not verbatim text. -# -# This is verbatim text, with an escape character \. -# -# This is not a code block. -# -# def foo -# 'String with an escape character.' -# end -# -# Rendered HTML: -# -# >>> -# This is not verbatim text. -# -# This is verbatim text, with an escape character \. -# -# This is not a code block. -# -# def foo -# 'This is a code block with an escape character \.' -# end -# -# In typeface markup (italic, bold, or monofont), -# an escape character is preserved unless it is immediately -# followed by nested typeface markup. -# -# Example input: -# -# This list is about escapes; it contains: -# -# - <tt>Monofont text with unescaped nested _italic_</tt>. -# - <tt>Monofont text with escaped nested \_italic_</tt>. -# - <tt>Monofont text with an escape character \</tt>. -# -# Rendered HTML: -# -# >>> -# This list is about escapes; it contains: -# -# - <tt>Monofont text with unescaped nested _italic_</tt>. -# - <tt>Monofont text with escaped nested \_italic_</tt>. -# - <tt>Monofont text with an escape character \ </tt>. -# -# In other text-bearing blocks -# (paragraphs, block quotes, list items, headings): -# -# - A single escape character immediately followed by markup -# escapes the markup. -# - A single escape character followed by whitespace is preserved. -# - A single escape character anywhere else is ignored. -# - A double escape character is rendered as a single backslash. -# -# Example input: -# -# This list is about escapes; it contains: -# -# - An unescaped class name, RDoc, that will become a link. -# - An escaped class name, \RDoc, that will not become a link. -# - An escape character followed by whitespace \ . -# - An escape character \that is ignored. -# - A double escape character \\ that is rendered -# as a single backslash. -# -# Rendered HTML: -# -# >>> -# This list is about escapes; it contains: -# -# - An unescaped class name, RDoc, that will become a link. -# - An escaped class name, \RDoc, that will not become a link. -# - An escape character followed by whitespace \ . -# - An escape character \that is ignored. -# - A double escape character \\ that is rendered -# as a single backslash. -# -# == Documentation Derived from Ruby Code -# -# [Class] -# -# By default, \RDoc documents: -# -# - \Class name. -# - Parent class. -# - Singleton methods. -# - Instance methods. -# - Aliases. -# - Constants. -# - Attributes. -# -# [Module] -# -# By default, \RDoc documents: -# -# - \Module name. -# - \Singleton methods. -# - Instance methods. -# - Aliases. -# - Constants. -# - Attributes. -# -# [Method] -# -# By default, \RDoc documents: -# -# - \Method name. -# - Arguments. -# - Yielded values. -# -# See #method. -# -# [Alias] -# -# By default, \RDoc documents: -# -# - Alias name. -# - Aliased name. -# -# See #dummy_instance_alias and #dummy_instance_method. -# -# [Constant] -# -# By default, \RDoc documents: -# -# - \Constant name. -# -# See DUMMY_CONSTANT. -# -# [Attribute] -# -# By default, \RDoc documents: -# -# - Attribute name. -# - Attribute type (<tt>[R]</tt>, <tt>[W]</tt>, or <tt>[RW]</tt>) -# -# See #dummy_attribute. -# -class RDoc::MarkupReference - - # Example class. - class DummyClass; end - - # Example module. - module DummyModule; end - - # Example singleton method. - def self.dummy_singleton_method(foo, bar); end - - # Example instance method. - def dummy_instance_method(foo, bar); end; - - alias dummy_instance_alias dummy_instance_method - - # Example attribute. - attr_accessor :dummy_attribute - - alias dummy_attribute_alias dummy_attribute - - # Example constant. - DUMMY_CONSTANT = '' - - # :call-seq: - # call_seq_directive(foo, bar) - # Can be anything -> bar - # Also anything more -> baz or bat - # - # The <tt>:call-seq:</tt> directive overrides the actual calling sequence - # found in the Ruby code. - # - # - It can specify anything at all. - # - It can have multiple calling sequences. - # - # This one includes <tt>Can be anything -> foo</tt>, which is nonsense. - # - # Note that the "arrow" is two characters, hyphen and right angle-bracket, - # which is made into a single character in the HTML. - # - # Click on the calling sequence to see the code. - # - # Here is the <tt>:call-seq:</tt> directive given for the method: - # - # :call-seq: - # call_seq_directive(foo, bar) - # Can be anything -> bar - # Also anything more -> baz or bat - # - def call_seq_directive - nil - end - - # The <tt>:args:</tt> directive overrides the actual arguments found in the Ruby code. - # - # Click on the calling sequence to see the code. - # - def args_directive(foo, bar) # :args: baz - nil - end - - # The <tt>:yields:</tt> directive overrides the actual yield found in the Ruby code. - # - # Click on the calling sequence to see the code. - # - def yields_directive(foo, bar) # :yields: 'bat' - yield 'baz' - end - - # This method is documented only by \RDoc, except for these comments. - # - # Click on the calling sequence to see the code. - # - def method(foo, bar) - yield 'baz' - end - -end diff --git a/doc/reline/face.md b/doc/reline/face.md deleted file mode 100644 index 1fa916123b..0000000000 --- a/doc/reline/face.md +++ /dev/null @@ -1,111 +0,0 @@ -# Face - -With the `Reline::Face` class, you can modify the text color and text decorations in your terminal emulator. -This is primarily used to customize the appearance of the method completion dialog in IRB. - -## Usage - -### ex: Change the background color of the completion dialog cyan to blue - -```ruby -Reline::Face.config(:completion_dialog) do |conf| - conf.define :default, foreground: :white, background: :blue - # ^^^^^ `:cyan` by default - conf.define :enhanced, foreground: :white, background: :magenta - conf.define :scrollbar, foreground: :white, background: :blue -end -``` - -If you provide the above code to an IRB session in some way, you can apply the configuration. -It's generally done by writing it in `.irbrc`. - -Regarding `.irbrc`, please refer to the following link: [https://docs.ruby-lang.org/en/master/IRB.html](https://docs.ruby-lang.org/en/master/IRB.html) - -## Available parameters - -`Reline::Face` internally creates SGR (Select Graphic Rendition) code according to the block parameter of `Reline::Face.config` method. - -| Key | Value | SGR Code (numeric part following "\e[")| -|:------------|:------------------|-----:| -| :foreground | :black | 30 | -| | :red | 31 | -| | :green | 32 | -| | :yellow | 33 | -| | :blue | 34 | -| | :magenta | 35 | -| | :cyan | 36 | -| | :white | 37 | -| | :bright_black | 90 | -| | :gray | 90 | -| | :bright_red | 91 | -| | :bright_green | 92 | -| | :bright_yellow | 93 | -| | :bright_blue | 94 | -| | :bright_magenta | 95 | -| | :bright_cyan | 96 | -| | :bright_white | 97 | -| :background | :black | 40 | -| | :red | 41 | -| | :green | 42 | -| | :yellow | 43 | -| | :blue | 44 | -| | :magenta | 45 | -| | :cyan | 46 | -| | :white | 47 | -| | :bright_black | 100 | -| | :gray | 100 | -| | :bright_red | 101 | -| | :bright_green | 102 | -| | :bright_yellow | 103 | -| | :bright_blue | 104 | -| | :bright_magenta | 105 | -| | :bright_cyan | 106 | -| | :bright_white | 107 | -| :style | :reset | 0 | -| | :bold | 1 | -| | :faint | 2 | -| | :italicized | 3 | -| | :underlined | 4 | -| | :slowly_blinking | 5 | -| | :blinking | 5 | -| | :rapidly_blinking | 6 | -| | :negative | 7 | -| | :concealed | 8 | -| | :crossed_out | 9 | - -- The value for `:style` can be both a Symbol and an Array - ```ruby - # Single symbol - conf.define :default, style: :bold - # Array - conf.define :default, style: [:bold, :negative] - ``` -- The availability of specific SGR codes depends on your terminal emulator -- You can specify a hex color code to `:foreground` and `:background` color like `foreground: "#FF1020"`. Its availability also depends on your terminal emulator - -## Debugging - -You can see the current Face configuration by `Reline::Face.configs` method - -Example: - -```ruby -irb(main):001:0> Reline::Face.configs -=> -{:default=> - {:default=>{:style=>:reset, :escape_sequence=>"\e[0m"}, - :enhanced=>{:style=>:reset, :escape_sequence=>"\e[0m"}, - :scrollbar=>{:style=>:reset, :escape_sequence=>"\e[0m"}}, - :completion_dialog=> - {:default=>{:foreground=>:white, :background=>:cyan, :escape_sequence=>"\e[0m\e[37;46m"}, - :enhanced=>{:foreground=>:white, :background=>:magenta, :escape_sequence=>"\e[0m\e[37;45m"}, - :scrollbar=>{:foreground=>:white, :background=>:cyan, :escape_sequence=>"\e[0m\e[37;46m"}}} -``` - -## 256-Color and TrueColor - -Reline will automatically detect if your terminal emulator supports truecolor with `ENV['COLORTERM] in 'truecolor' | '24bit'`. When this env is not set, Reline will fallback to 256-color. -If your terminal emulator supports truecolor but does not set COLORTERM env, add this line to `.irbrc`. -```ruby -Reline::Face.force_truecolor -``` diff --git a/doc/security/command_injection.rdoc b/doc/security/command_injection.rdoc new file mode 100644 index 0000000000..d46e42f7be --- /dev/null +++ b/doc/security/command_injection.rdoc @@ -0,0 +1,15 @@ += Command Injection + +Some Ruby core methods accept string data +that includes text to be executed as a system command. + +They should not be called with unknown or unsanitized commands. + +These methods include: + +- Kernel.exec +- Kernel.spawn +- Kernel.system +- {\`command` (backtick method)}[rdoc-ref:Kernel#`] + (also called by the expression <tt>%x[command]</tt>). +- IO.popen (when called with other than <tt>"-"</tt>). diff --git a/doc/security.rdoc b/doc/security/security.rdoc index e428036cf5..af9970d336 100644 --- a/doc/security.rdoc +++ b/doc/security/security.rdoc @@ -37,7 +37,7 @@ programs for configuration and database persistence of Ruby object trees. Similar to +Marshal+, it is able to deserialize into arbitrary Ruby classes. For example, the following YAML data will create an +ERB+ object when -deserialized, using the `unsafe_load` method: +deserialized, using the +unsafe_load+ method: !ruby/object:ERB src: puts `uname` @@ -53,19 +53,16 @@ method, variable and constant names. The reason for this is that symbols are simply integers with names attached to them, so they are faster to look up in hashtables. -Starting in version 2.2, most symbols can be garbage collected; these are -called <i>mortal</i> symbols. Most symbols you create (e.g. by calling -+to_sym+) are mortal. +Most symbols can be garbage collected; these are called _mortal_ +symbols. Most symbols you create (e.g. by calling +to_sym+) are mortal. -<i>Immortal</i> symbols on the other hand will never be garbage collected. +_Immortal_ symbols on the other hand will never be garbage collected. They are created when modifying code: * defining a method (e.g. with +define_method+), * setting an instance variable (e.g. with +instance_variable_set+), * creating a variable or constant (e.g. with +const_set+) -C extensions that have not been updated and are still calling `SYM2ID` +C extensions that have not been updated and are still calling +SYM2ID+ will create immortal symbols. -Bugs in 2.2.0: +send+ and +__send__+ also created immortal symbols, -and calling methods with keyword arguments could also create some. Don't create immortal symbols from user inputs. Otherwise, this would allow a user to mount a denial of service attack against your application by @@ -128,12 +125,3 @@ Note that the use of +public_send+ is also dangerous, as +send+ itself is public: 1.public_send("send", "eval", "...ruby code to be executed...") - -== DRb - -As DRb allows remote clients to invoke arbitrary methods, it is not suitable to -expose to untrusted clients. - -When using DRb, try to avoid exposing it over the network if possible. If this -isn't possible and you need to expose DRb to the world, you *must* configure an -appropriate security policy with <code>DRb::ACL</code>. diff --git a/doc/standard_library.md b/doc/standard_library.md index a6702bb80f..7a477283a9 100644 --- a/doc/standard_library.md +++ b/doc/standard_library.md @@ -34,7 +34,6 @@ of each. ## Libraries - Bundler ([GitHub][bundler]): Manage your Ruby application's gem dependencies -- CGI ([GitHub][cgi]): Support for the Common Gateway Interface protocol - Delegator ([GitHub][delegate]): Provides three abilities to delegate method calls to an object - DidYouMean ([GitHub][did_you_mean]): "Did you mean?" experience in Ruby - English ([GitHub][English]): Provides references to special global variables with less cryptic names @@ -53,13 +52,12 @@ of each. - Prism ([GitHub][prism]): A portable, error-tolerant Ruby parser - Resolv ([GitHub][resolv]): Thread-aware DNS resolver library in Ruby - SecureRandom ([GitHub][securerandom]): Interface for a secure random number generator -- [Set](rdoc-ref:Set) ([GitHub][set]): Provides a class to deal with collections of unordered, unique values - Shellwords ([GitHub][shellwords]): Manipulates strings with the word parsing rules of the UNIX Bourne shell - Singleton ([GitHub][singleton]): Implementation of the Singleton pattern for Ruby - Tempfile ([GitHub][tempfile]): A utility class for managing temporary files - Time ([GitHub][time]): Extends the Time class with methods for parsing and conversion - Timeout ([GitHub][timeout]): Auto-terminate potentially long-running operations in Ruby -- TSort ([GitHub][tsort]): Topological sorting using Tarjan's algorithm +- TmpDir ([GitHub][tmpdir]): Extends the Dir class to manage the OS temporary file path - UN ([GitHub][un]): Utilities to replace common UNIX commands - URI ([GitHub][uri]): A Ruby module providing support for Uniform Resource Identifiers - YAML ([GitHub][yaml]): The Ruby client library for the Psych YAML implementation @@ -72,13 +70,14 @@ of each. - Etc ([GitHub][etc]): Provides access to information typically stored in the UNIX /etc directory - Fcntl ([GitHub][fcntl]): Loads constants defined in the OS fcntl.h C header file - IO.console ([GitHub][io-console]): Extensions for the IO class, including `IO.console`, `IO.winsize`, etc. +- IO#nonblock ([GitHub][io-nonblock]): Enable non-blocking mode with IO class. +- IO#wait ([GitHub][io-wait]): Provides the feature for waiting until IO is readable or writable without blocking. - JSON ([GitHub][json]): Implements JavaScript Object Notation for Ruby - OpenSSL ([GitHub][openssl]): Provides SSL, TLS, and general-purpose cryptography for Ruby - Pathname ([GitHub][pathname]): Representation of the name of a file or directory on the filesystem - Psych ([GitHub][psych]): A YAML parser and emitter for Ruby - StringIO ([GitHub][stringio]): Pseudo-I/O on String objects - StringScanner ([GitHub][strscan]): Provides lexical scanning operations on a String -- TmpDir ([GitHub][tmpdir]): Extends the Dir class to manage the OS temporary file path - Zlib ([GitHub][zlib]): Ruby interface for the zlib compression/decompression library # Bundled gems @@ -92,9 +91,9 @@ of each. - [minitest]: A test library supporting TDD, BDD, mocking, and benchmarking - [power_assert]: Power Assert for Ruby -- [rake]: Ruby build program with capabilities similar to make +- [rake][rake-doc] ([GitHub][rake]): Ruby build program with capabilities similar to make - [test-unit]: A compatibility layer for MiniTest -- [rexml]: An XML toolkit for Ruby +- [rexml][rexml-doc] ([GitHub][rexml]): An XML toolkit for Ruby - [rss]: A family of libraries supporting various XML-based "feeds" - [net-ftp]: Support for the File Transfer Protocol - [net-imap]: Ruby client API for the Internet Message Access Protocol @@ -105,7 +104,7 @@ of each. - [rbs]: RBS is a language to describe the structure of Ruby programs - [typeprof]: A type analysis tool for Ruby code based on abstract interpretation - [debug]: Debugging functionality for Ruby -- [racc]: A LALR(1) parser generator written in Ruby +- [racc][racc-doc] ([GitHub][racc]): A LALR(1) parser generator written in Ruby - [mutex_m]: Mixin to extend objects to be handled like a Mutex - [getoptlong]: Parse command line options similar to the GNU C getopt_long() - [base64]: Support for encoding and decoding binary data using a Base64 representation @@ -117,15 +116,16 @@ of each. - [drb]: Distributed object system for Ruby - [nkf]: Ruby extension for the Network Kanji Filter - [syslog]: Ruby interface for the POSIX system logging facility -- [csv]: Provides an interface to read and write CSV files and data +- [csv][csv-doc] ([GitHub][csv]): Provides an interface to read and write CSV files and data - [ostruct]: A class to build custom data structures, similar to a Hash - [benchmark]: Provides methods to measure and report the time used to execute code -- [logger]: Provides a simple logging utility for outputting messages +- [logger][logger-doc] ([GitHub][logger]): Provides a simple logging utility for outputting messages - [pstore]: Implements a file-based persistence mechanism based on a Hash - [win32ole]: Provides an interface for OLE Automation in Ruby -- [reline]: GNU Readline and Editline in a pure Ruby implementation +- [reline][reline-doc] ([GitHub][reline]): GNU Readline and Editline in a pure Ruby implementation - [readline]: Wrapper for the Readline extension and Reline - [fiddle]: A libffi wrapper for Ruby +- [tsort]: Topological sorting using Tarjan's algorithm ## Tools @@ -137,7 +137,6 @@ of each. [benchmark]: https://github.com/ruby/benchmark [bigdecimal]: https://github.com/ruby/bigdecimal [bundler]: https://github.com/rubygems/rubygems -[cgi]: https://github.com/ruby/cgi [csv]: https://github.com/ruby/csv [date]: https://github.com/ruby/date [debug]: https://github.com/ruby/debug @@ -156,6 +155,8 @@ of each. [forwardable]: https://github.com/ruby/forwardable [getoptlong]: https://github.com/ruby/getoptlong [io-console]: https://github.com/ruby/io-console +[io-nonblock]: https://github.com/ruby/io-nonblock +[io-wait]: https://github.com/ruby/io-wait [ipaddr]: https://github.com/ruby/ipaddr [irb]: https://github.com/ruby/irb [json]: https://github.com/ruby/json @@ -195,7 +196,6 @@ of each. [rinda]: https://github.com/ruby/rinda [rss]: https://github.com/ruby/rss [securerandom]: https://github.com/ruby/securerandom -[set]: https://github.com/ruby/set [shellwords]: https://github.com/ruby/shellwords [singleton]: https://github.com/ruby/singleton [stringio]: https://github.com/ruby/stringio @@ -215,5 +215,11 @@ of each. [yaml]: https://github.com/ruby/yaml [zlib]: https://github.com/ruby/zlib +[reline-doc]: https://ruby.github.io/reline/ +[rake-doc]: https://ruby.github.io/rake/ [irb-doc]: https://ruby.github.io/irb/ [rdoc-doc]: https://ruby.github.io/rdoc/ +[logger-doc]: https://ruby.github.io/logger/ +[racc-doc]: https://ruby.github.io/racc/ +[csv-doc]: https://ruby.github.io/csv/ +[rexml-doc]: https://ruby.github.io/rexml/ diff --git a/doc/string.rb b/doc/string.rb new file mode 100644 index 0000000000..304ab60c29 --- /dev/null +++ b/doc/string.rb @@ -0,0 +1,421 @@ +# A +String+ object has an arbitrary sequence of bytes, +# typically representing text or binary data. +# A +String+ object may be created using String::new or as literals. +# +# String objects differ from Symbol objects in that Symbol objects are +# designed to be used as identifiers, instead of text or data. +# +# You can create a +String+ object explicitly with: +# +# - A {string literal}[rdoc-ref:syntax/literals.rdoc@String+Literals]. +# - A {heredoc literal}[rdoc-ref:syntax/literals.rdoc@Here+Document+Literals]. +# +# You can convert certain objects to Strings with: +# +# - Method #String. +# +# Some +String+ methods modify +self+. +# Typically, a method whose name ends with <tt>!</tt> modifies +self+ +# and returns +self+; +# often, a similarly named method (without the <tt>!</tt>) +# returns a new string. +# +# In general, if both bang and non-bang versions of a method exist, +# the bang method mutates and the non-bang method does not. +# However, a method without a bang can also mutate, such as String#replace. +# +# == Substitution Methods +# +# These methods perform substitutions: +# +# - String#sub: One substitution (or none); returns a new string. +# - String#sub!: One substitution (or none); returns +self+ if any changes, +# +nil+ otherwise. +# - String#gsub: Zero or more substitutions; returns a new string. +# - String#gsub!: Zero or more substitutions; returns +self+ if any changes, +# +nil+ otherwise. +# +# Each of these methods takes: +# +# - A first argument, +pattern+ (String or Regexp), +# that specifies the substring(s) to be replaced. +# +# - Either of the following: +# +# - A second argument, +replacement+ (String or Hash), +# that determines the replacing string. +# - A block that will determine the replacing string. +# +# The examples in this section mostly use the String#sub and String#gsub methods; +# the principles illustrated apply to all four substitution methods. +# +# <b>Argument +pattern+</b> +# +# Argument +pattern+ is commonly a regular expression: +# +# s = 'hello' +# s.sub(/[aeiou]/, '*') # => "h*llo" +# s.gsub(/[aeiou]/, '*') # => "h*ll*" +# s.gsub(/[aeiou]/, '') # => "hll" +# s.sub(/ell/, 'al') # => "halo" +# s.gsub(/xyzzy/, '*') # => "hello" +# 'THX1138'.gsub(/\d+/, '00') # => "THX00" +# +# When +pattern+ is a string, all its characters are treated +# as ordinary characters (not as Regexp special characters): +# +# 'THX1138'.gsub('\d+', '00') # => "THX1138" +# +# <b>+String+ +replacement+</b> +# +# If +replacement+ is a string, that string determines +# the replacing string that is substituted for the matched text. +# +# Each of the examples above uses a simple string as the replacing string. +# +# +String+ +replacement+ may contain back-references to the pattern's captures: +# +# - <tt>\n</tt> (_n_ is a non-negative integer) refers to <tt>$n</tt>. +# - <tt>\k<name></tt> refers to the named capture +name+. +# +# See Regexp for details. +# +# Note that within the string +replacement+, a character combination +# such as <tt>$&</tt> is treated as ordinary text, not as +# a special match variable. +# However, you may refer to some special match variables using these +# combinations: +# +# - <tt>\&</tt> and <tt>\0</tt> correspond to <tt>$&</tt>, +# which contains the complete matched text. +# - <tt>\'</tt> corresponds to <tt>$'</tt>, +# which contains the string after the match. +# - <tt>\`</tt> corresponds to <tt>$`</tt>, +# which contains the string before the match. +# - <tt>\\+</tt> corresponds to <tt>$+</tt>, +# which contains the last capture group. +# +# See Regexp for details. +# +# Note that <tt>\\\\</tt> is interpreted as an escape, i.e., a single backslash. +# +# Note also that a string literal consumes backslashes. +# See {String Literals}[rdoc-ref:syntax/literals.rdoc@String+Literals] for details about string literals. +# +# A back-reference is typically preceded by an additional backslash. +# For example, if you want to write a back-reference <tt>\&</tt> in +# +replacement+ with a double-quoted string literal, you need to write +# <tt>"..\\\\&.."</tt>. +# +# If you want to write a non-back-reference string <tt>\&</tt> in +# +replacement+, you need to first escape the backslash to prevent +# this method from interpreting it as a back-reference, and then you +# need to escape the backslashes again to prevent a string literal from +# consuming them: <tt>"..\\\\\\\\&.."</tt>. +# +# You may want to use the block form to avoid excessive backslashes. +# +# <b>\Hash +replacement+</b> +# +# If the argument +replacement+ is a hash, and +pattern+ matches one of its keys, +# the replacing string is the value for that key: +# +# h = {'foo' => 'bar', 'baz' => 'bat'} +# 'food'.sub('foo', h) # => "bard" +# +# Note that a symbol key does not match: +# +# h = {foo: 'bar', baz: 'bat'} +# 'food'.sub('foo', h) # => "d" +# +# <b>Block</b> +# +# In the block form, the current match string is passed to the block; +# the block's return value becomes the replacing string: +# +# s = '@' +# '1234'.gsub(/\d/) { |match| s.succ! } # => "ABCD" +# +# Special match variables such as <tt>$1</tt>, <tt>$2</tt>, <tt>$`</tt>, +# <tt>$&</tt>, and <tt>$'</tt> are set appropriately. +# +# == Whitespace in Strings +# +# In the class +String+, _whitespace_ is defined as a contiguous sequence of characters +# consisting of any mixture of the following: +# +# - NL (null): <tt>"\x00"</tt>, <tt>"\u0000"</tt>. +# - HT (horizontal tab): <tt>"\x09"</tt>, <tt>"\t"</tt>. +# - LF (line feed): <tt>"\x0a"</tt>, <tt>"\n"</tt>. +# - VT (vertical tab): <tt>"\x0b"</tt>, <tt>"\v"</tt>. +# - FF (form feed): <tt>"\x0c"</tt>, <tt>"\f"</tt>. +# - CR (carriage return): <tt>"\x0d"</tt>, <tt>"\r"</tt>. +# - SP (space): <tt>"\x20"</tt>, <tt>" "</tt>. +# +# +# Whitespace is relevant for the following methods: +# +# - #lstrip, #lstrip!: Strip leading whitespace. +# - #rstrip, #rstrip!: Strip trailing whitespace. +# - #strip, #strip!: Strip leading and trailing whitespace. +# +# == What's Here +# +# First, what's elsewhere. Class +String+: +# +# - Inherits from the {Object class}[rdoc-ref:Object@What-27s+Here]. +# - Includes the {Comparable module}[rdoc-ref:Comparable@What-27s+Here]. +# +# Here, class +String+ provides methods that are useful for: +# +# - {Creating a \String}[rdoc-ref:String@Creating+a+String]. +# - {Freezing/Unfreezing a \String}[rdoc-ref:String@Freezing-2FUnfreezing]. +# - {Querying a \String}[rdoc-ref:String@Querying]. +# - {Comparing Strings}[rdoc-ref:String@Comparing]. +# - {Modifying a \String}[rdoc-ref:String@Modifying]. +# - {Converting to a new \String}[rdoc-ref:String@Converting+to+New+String]. +# - {Converting to a non-\String}[rdoc-ref:String@Converting+to+Non--5CString]. +# - {Iterating over a \String}[rdoc-ref:String@Iterating]. +# +# === Creating a \String +# +# - ::new: Returns a new string. +# - ::try_convert: Returns a new string created from a given object. +# +# === Freezing/Unfreezing +# +# - #+@: Returns a string that is not frozen: +self+ if not frozen; +# +self.dup+ otherwise. +# - #-@ (aliased as #dedup): Returns a string that is frozen: +self+ if already frozen; +# +self.freeze+ otherwise. +# - #freeze: Freezes +self+ if not already frozen; returns +self+. +# +# === Querying +# +# _Counts_ +# +# - #bytesize: Returns the count of bytes. +# - #count: Returns the count of substrings matching given strings. +# - #empty?: Returns whether the length of +self+ is zero. +# - #length (aliased as #size): Returns the count of characters (not bytes). +# +# _Substrings_ +# +# - #=~: Returns the index of the first substring that matches a given +# Regexp or other object; returns +nil+ if no match is found. +# - #byteindex: Returns the byte index of the first occurrence of a given substring. +# - #byterindex: Returns the byte index of the last occurrence of a given substring. +# - #index: Returns the index of the _first_ occurrence of a given substring; +# returns +nil+ if none found. +# - #rindex: Returns the index of the _last_ occurrence of a given substring; +# returns +nil+ if none found. +# - #include?: Returns +true+ if the string contains a given substring; +false+ otherwise. +# - #match: Returns a MatchData object if the string matches a given Regexp; +nil+ otherwise. +# - #match?: Returns +true+ if the string matches a given Regexp; +false+ otherwise. +# - #start_with?: Returns +true+ if the string begins with any of the given substrings. +# - #end_with?: Returns +true+ if the string ends with any of the given substrings. +# +# _Encodings_ +# +# - #encoding\: Returns the Encoding object that represents the encoding of the string. +# - #unicode_normalized?: Returns +true+ if the string is in Unicode normalized form; +false+ otherwise. +# - #valid_encoding?: Returns +true+ if the string contains only characters that are valid +# for its encoding. +# - #ascii_only?: Returns +true+ if the string has only ASCII characters; +false+ otherwise. +# +# _Other_ +# +# - #sum: Returns a basic checksum for the string: the sum of each byte. +# - #hash: Returns the integer hash code. +# +# === Comparing +# +# - #== (aliased as #===): Returns +true+ if a given other string has the same content as +self+. +# - #eql?: Returns +true+ if the content is the same as the given other string. +# - #<=>: Returns -1, 0, or 1 as a given other string is smaller than, +# equal to, or larger than +self+. +# - #casecmp: Ignoring case, returns -1, 0, or 1 as +# +self+ is smaller than, equal to, or larger than a given other string. +# - #casecmp?: Ignoring case, returns whether a given other string is equal to +self+. +# +# === Modifying +# +# Each of these methods modifies +self+. +# +# _Insertion_ +# +# - #insert: Returns +self+ with a given string inserted at a specified offset. +# - #<<: Returns +self+ concatenated with a given string or integer. +# - #append_as_bytes: Returns +self+ concatenated with strings without performing any +# encoding validation or conversion. +# - #prepend: Prefixes to +self+ the concatenation of given other strings. +# +# _Substitution_ +# +# - #bytesplice: Replaces bytes of +self+ with bytes from a given string; returns +self+. +# - #sub!: Replaces the first substring that matches a given pattern with a given replacement string; +# returns +self+ if any changes, +nil+ otherwise. +# - #gsub!: Replaces each substring that matches a given pattern with a given replacement string; +# returns +self+ if any changes, +nil+ otherwise. +# - #succ! (aliased as #next!): Returns +self+ modified to become its own successor. +# - #replace: Returns +self+ with its entire content replaced by a given string. +# - #reverse!: Returns +self+ with its characters in reverse order. +# - #setbyte: Sets the byte at a given integer offset to a given value; returns the argument. +# - #tr!: Replaces specified characters in +self+ with specified replacement characters; +# returns +self+ if any changes, +nil+ otherwise. +# - #tr_s!: Replaces specified characters in +self+ with specified replacement characters, +# removing duplicates from the substrings that were modified; +# returns +self+ if any changes, +nil+ otherwise. +# +# _Casing_ +# +# - #capitalize!: Upcases the initial character and downcases all others; +# returns +self+ if any changes, +nil+ otherwise. +# - #downcase!: Downcases all characters; returns +self+ if any changes, +nil+ otherwise. +# - #upcase!: Upcases all characters; returns +self+ if any changes, +nil+ otherwise. +# - #swapcase!: Upcases each downcase character and downcases each upcase character; +# returns +self+ if any changes, +nil+ otherwise. +# +# _Encoding_ +# +# - #encode!: Returns +self+ with all characters transcoded from one encoding to another. +# - #unicode_normalize!: Unicode-normalizes +self+; returns +self+. +# - #scrub!: Replaces each invalid byte with a given character; returns +self+. +# - #force_encoding: Changes the encoding to a given encoding; returns +self+. +# +# _Deletion_ +# +# - #clear: Removes all content, so that +self+ is empty; returns +self+. +# - #slice!, #[]=: Removes a substring determined by a given index, start/length, range, regexp, or substring. +# - #squeeze!: Removes contiguous duplicate characters; returns +self+. +# - #delete!: Removes characters as determined by the intersection of substring arguments. +# - #delete_prefix!: Removes leading prefix; returns +self+ if any changes, +nil+ otherwise. +# - #delete_suffix!: Removes trailing suffix; returns +self+ if any changes, +nil+ otherwise. +# - #lstrip!: Removes leading whitespace; returns +self+ if any changes, +nil+ otherwise. +# - #rstrip!: Removes trailing whitespace; returns +self+ if any changes, +nil+ otherwise. +# - #strip!: Removes leading and trailing whitespace; returns +self+ if any changes, +nil+ otherwise. +# - #chomp!: Removes the trailing record separator, if found; returns +self+ if any changes, +nil+ otherwise. +# - #chop!: Removes trailing newline characters if found; otherwise removes the last character; +# returns +self+ if any changes, +nil+ otherwise. +# +# === Converting to New \String +# +# Each of these methods returns a new +String+ based on +self+, +# often just a modified copy of +self+. +# +# _Extension_ +# +# - #*: Returns the concatenation of multiple copies of +self+. +# - #+: Returns the concatenation of +self+ and a given other string. +# - #center: Returns a copy of +self+, centered by specified padding. +# - #concat: Returns the concatenation of +self+ with given other strings. +# - #ljust: Returns a copy of +self+ of a given length, right-padded with a given other string. +# - #rjust: Returns a copy of +self+ of a given length, left-padded with a given other string. +# +# _Encoding_ +# +# - #b: Returns a copy of +self+ with ASCII-8BIT encoding. +# - #scrub: Returns a copy of +self+ with each invalid byte replaced with a given character. +# - #unicode_normalize: Returns a copy of +self+ with each character Unicode-normalized. +# - #encode: Returns a copy of +self+ with all characters transcoded from one encoding to another. +# +# _Substitution_ +# +# - #dump: Returns a printable version of +self+, enclosed in double-quotes. +# - #undump: Inverse of #dump; returns a copy of +self+ with changes of the kinds made by #dump "undone." +# - #sub: Returns a copy of +self+ with the first substring matching a given pattern +# replaced with a given replacement string. +# - #gsub: Returns a copy of +self+ with each substring that matches a given pattern +# replaced with a given replacement string. +# - #succ (aliased as #next): Returns the string that is the successor to +self+. +# - #reverse: Returns a copy of +self+ with its characters in reverse order. +# - #tr: Returns a copy of +self+ with specified characters replaced with specified replacement characters. +# - #tr_s: Returns a copy of +self+ with specified characters replaced with +# specified replacement characters, +# removing duplicates from the substrings that were modified. +# - #%: Returns the string resulting from formatting a given object into +self+. +# +# _Casing_ +# +# - #capitalize: Returns a copy of +self+ with the first character upcased +# and all other characters downcased. +# - #downcase: Returns a copy of +self+ with all characters downcased. +# - #upcase: Returns a copy of +self+ with all characters upcased. +# - #swapcase: Returns a copy of +self+ with all upcase characters downcased +# and all downcase characters upcased. +# +# _Deletion_ +# +# - #delete: Returns a copy of +self+ with characters removed. +# - #delete_prefix: Returns a copy of +self+ with a given prefix removed. +# - #delete_suffix: Returns a copy of +self+ with a given suffix removed. +# - #lstrip: Returns a copy of +self+ with leading whitespace removed. +# - #rstrip: Returns a copy of +self+ with trailing whitespace removed. +# - #strip: Returns a copy of +self+ with leading and trailing whitespace removed. +# - #chomp: Returns a copy of +self+ with a trailing record separator removed, if found. +# - #chop: Returns a copy of +self+ with trailing newline characters or the last character removed. +# - #squeeze: Returns a copy of +self+ with contiguous duplicate characters removed. +# - #[] (aliased as #slice): Returns a substring determined by a given index, start/length, range, regexp, or string. +# - #byteslice: Returns a substring determined by a given index, start/length, or range. +# - #chr: Returns the first character. +# +# _Duplication_ +# +# - #to_s (aliased as #to_str): If +self+ is a subclass of +String+, returns +self+ copied into a +String+; +# otherwise, returns +self+. +# +# === Converting to Non-\String +# +# Each of these methods converts the contents of +self+ to a non-+String+. +# +# <em>Characters, Bytes, and Clusters</em> +# +# - #bytes: Returns an array of the bytes in +self+. +# - #chars: Returns an array of the characters in +self+. +# - #codepoints: Returns an array of the integer ordinals in +self+. +# - #getbyte: Returns the integer byte at the given index in +self+. +# - #grapheme_clusters: Returns an array of the grapheme clusters in +self+. +# +# _Splitting_ +# +# - #lines: Returns an array of the lines in +self+, as determined by a given record separator. +# - #partition: Returns a 3-element array determined by the first substring that matches +# a given substring or regexp. +# - #rpartition: Returns a 3-element array determined by the last substring that matches +# a given substring or regexp. +# - #split: Returns an array of substrings determined by a given delimiter -- regexp or string -- +# or, if a block is given, passes those substrings to the block. +# +# _Matching_ +# +# - #scan: Returns an array of substrings matching a given regexp or string, or, +# if a block is given, passes each matching substring to the block. +# - #unpack: Returns an array of substrings extracted from +self+ according to a given format. +# - #unpack1: Returns the first substring extracted from +self+ according to a given format. +# +# _Numerics_ +# +# - #hex: Returns the integer value of the leading characters, interpreted as hexadecimal digits. +# - #oct: Returns the integer value of the leading characters, interpreted as octal digits. +# - #ord: Returns the integer ordinal of the first character in +self+. +# - #to_c: Returns the complex value of leading characters, interpreted as a complex number. +# - #to_i: Returns the integer value of leading characters, interpreted as an integer. +# - #to_f: Returns the floating-point value of leading characters, interpreted as a floating-point number. +# - #to_r: Returns the rational value of leading characters, interpreted as a rational. +# +# <em>Strings and Symbols</em> +# +# - #inspect: Returns a copy of +self+, enclosed in double quotes, with special characters escaped. +# - #intern (aliased as #to_sym): Returns the symbol corresponding to +self+. +# +# === Iterating +# +# - #each_byte: Calls the given block with each successive byte in +self+. +# - #each_char: Calls the given block with each successive character in +self+. +# - #each_codepoint: Calls the given block with each successive integer codepoint in +self+. +# - #each_grapheme_cluster: Calls the given block with each successive grapheme cluster in +self+. +# - #each_line: Calls the given block with each successive line in +self+, +# as determined by a given record separator. +# - #upto: Calls the given block with each string value returned by successive calls to #succ. + +class String; end diff --git a/doc/string/aref.rdoc b/doc/string/aref.rdoc new file mode 100644 index 0000000000..a9ab8857bc --- /dev/null +++ b/doc/string/aref.rdoc @@ -0,0 +1,96 @@ +Returns the substring of +self+ specified by the arguments. + +<b>Form <tt>self[offset]</tt></b> + +With non-negative integer argument +offset+ given, +returns the 1-character substring found in self at character offset +offset+: + + 'hello'[0] # => "h" + 'hello'[4] # => "o" + 'hello'[5] # => nil + 'こんにちは'[4] # => "は" + +With negative integer argument +offset+ given, +counts backward from the end of +self+: + + 'hello'[-1] # => "o" + 'hello'[-5] # => "h" + 'hello'[-6] # => nil + +<b>Form <tt>self[offset, size]</tt></b> + +With integer arguments +offset+ and +size+ given, +returns a substring of size +size+ characters (as available) +beginning at character offset specified by +offset+. + +If argument +offset+ is non-negative, +the offset is +offset+: + + 'hello'[0, 1] # => "h" + 'hello'[0, 5] # => "hello" + 'hello'[0, 6] # => "hello" + 'hello'[2, 3] # => "llo" + 'hello'[2, 0] # => "" + 'hello'[2, -1] # => nil + +If argument +offset+ is negative, +counts backward from the end of +self+: + + 'hello'[-1, 1] # => "o" + 'hello'[-5, 5] # => "hello" + 'hello'[-1, 0] # => "" + 'hello'[-6, 5] # => nil + +Special case: if +offset+ equals the size of +self+, +returns a new empty string: + + 'hello'[5, 3] # => "" + +<b>Form <tt>self[range]</tt></b> + +With Range argument +range+ given, +forms substring <tt>self[range.start, range.size]</tt>: + + 'hello'[0..2] # => "hel" + 'hello'[0, 3] # => "hel" + + 'hello'[0...2] # => "he" + 'hello'[0, 2] # => "he" + + 'hello'[0, 0] # => "" + 'hello'[0...0] # => "" + +<b>Form <tt>self[regexp, capture = 0]</tt></b> + +With Regexp argument +regexp+ given and +capture+ as zero, +searches for a matching substring in +self+; +updates {Regexp-related global variables}[rdoc-ref:Regexp@Global+Variables]: + + 'hello'[/ell/] # => "ell" + 'hello'[/l+/] # => "ll" + 'hello'[//] # => "" + 'hello'[/nosuch/] # => nil + +With +capture+ as a positive integer +n+, +returns the +n+th matched group: + + 'hello'[/(h)(e)(l+)(o)/] # => "hello" + 'hello'[/(h)(e)(l+)(o)/, 1] # => "h" + $1 # => "h" + 'hello'[/(h)(e)(l+)(o)/, 2] # => "e" + $2 # => "e" + 'hello'[/(h)(e)(l+)(o)/, 3] # => "ll" + 'hello'[/(h)(e)(l+)(o)/, 4] # => "o" + 'hello'[/(h)(e)(l+)(o)/, 5] # => nil + +<b>Form <tt>self[substring]</tt></b> + +With string argument +substring+ given, +returns the matching substring of +self+, if found: + + 'hello'['ell'] # => "ell" + 'hello'[''] # => "" + 'hello'['nosuch'] # => nil + 'こんにちは'['んにち'] # => "んにち" + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/aset.rdoc b/doc/string/aset.rdoc new file mode 100644 index 0000000000..98c58b59cc --- /dev/null +++ b/doc/string/aset.rdoc @@ -0,0 +1,179 @@ +Returns +self+ with all, a substring, or none of its contents replaced; +returns the argument +other_string+. + +<b>Form <tt>self[index] = other_string</tt></b> + +With non-negative integer argument +index+ given, +searches for the 1-character substring found in self at character offset index: + + s = 'hello' + s[0] = 'foo' # => "foo" + s # => "fooello" + + s = 'hello' + s[4] = 'foo' # => "foo" + s # => "hellfoo" + + s = 'hello' + s[5] = 'foo' # => "foo" + s # => "hellofoo" + + s = 'hello' + s[6] = 'foo' # Raises IndexError: index 6 out of string. + +With negative integer argument +index+ given, +counts backward from the end of +self+: + + s = 'hello' + s[-1] = 'foo' # => "foo" + s # => "hellfoo" + + s = 'hello' + s[-5] = 'foo' # => "foo" + s # => "fooello" + + s = 'hello' + s[-6] = 'foo' # Raises IndexError: index -6 out of string. + +<b>Form <tt>self[start, length] = other_string</tt></b> + +With integer arguments +start+ and +length+ given, +searches for a substring of size +length+ characters (as available) +beginning at character offset specified by +start+. + +If argument +start+ is non-negative, +the offset is +start+: + + s = 'hello' + s[0, 1] = 'foo' # => "foo" + s # => "fooello" + + s = 'hello' + s[0, 5] = 'foo' # => "foo" + s # => "foo" + + s = 'hello' + s[0, 9] = 'foo' # => "foo" + s # => "foo" + + s = 'hello' + s[2, 0] = 'foo' # => "foo" + s # => "hefoollo" + + s = 'hello' + s[2, -1] = 'foo' # Raises IndexError: negative length -1. + +If argument +start+ is negative, +counts backward from the end of +self+: + + s = 'hello' + s[-1, 1] = 'foo' # => "foo" + s # => "hellfoo" + + s = 'hello' + s[-1, 9] = 'foo' # => "foo" + s # => "hellfoo" + + s = 'hello' + s[-5, 2] = 'foo' # => "foo" + s # => "foollo" + + s = 'hello' + s[-3, 0] = 'foo' # => "foo" + s # => "hefoollo" + + s = 'hello' + s[-6, 2] = 'foo' # Raises IndexError: index -6 out of string. + +Special case: if +start+ equals the length of +self+, +the argument is appended to +self+: + + s = 'hello' + s[5, 3] = 'foo' # => "foo" + s # => "hellofoo" + +<b>Form <tt>self[range] = other_string</tt></b> + +With Range argument +range+ given, +equivalent to <tt>self[range.start, range.size] = other_string</tt>: + + s0 = 'hello' + s1 = 'hello' + s0[0..2] = 'foo' # => "foo" + s1[0, 3] = 'foo' # => "foo" + s0 # => "foolo" + s1 # => "foolo" + + s = 'hello' + s[0...2] = 'foo' # => "foo" + s # => "foollo" + + s = 'hello' + s[0...0] = 'foo' # => "foo" + s # => "foohello" + + s = 'hello' + s[9..10] = 'foo' # Raises RangeError: 9..10 out of range + +<b>Form <tt>self[regexp, capture = 0] = other_string</tt></b> + +With Regexp argument +regexp+ given and +capture+ as zero, +searches for a matching substring in +self+; +updates {Regexp-related global variables}[rdoc-ref:Regexp@Global+Variables]: + + s = 'hello' + s[/l/] = 'L' # => "L" + [$`, $&, $'] # => ["he", "l", "lo"] + s[/eLlo/] = 'owdy' # => "owdy" + [$`, $&, $'] # => ["h", "eLlo", ""] + s[/eLlo/] = 'owdy' # Raises IndexError: regexp not matched. + [$`, $&, $'] # => [nil, nil, nil] + +With +capture+ as a positive integer +n+, +searches for the +n+th matched group: + + s = 'hello' + s[/(h)(e)(l+)(o)/] = 'foo' # => "foo" + [$`, $&, $'] # => ["", "hello", ""] + + s = 'hello' + s[/(h)(e)(l+)(o)/, 1] = 'foo' # => "foo" + s # => "fooello" + [$`, $&, $'] # => ["", "hello", ""] + + s = 'hello' + s[/(h)(e)(l+)(o)/, 2] = 'foo' # => "foo" + s # => "hfoollo" + [$`, $&, $'] # => ["", "hello", ""] + + s = 'hello' + s[/(h)(e)(l+)(o)/, 4] = 'foo' # => "foo" + s # => "hellfoo" + [$`, $&, $'] # => ["", "hello", ""] + + s = 'hello' + # => "hello" + s[/(h)(e)(l+)(o)/, 5] = 'foo # Raises IndexError: index 5 out of regexp. + + s = 'hello' + s[/nosuch/] = 'foo' # Raises IndexError: regexp not matched. + +<b>Form <tt>self[substring] = other_string</tt></b> + +With string argument +substring+ given: + + s = 'hello' + s['l'] = 'foo' # => "foo" + s # => "hefoolo" + + s = 'hello' + s['ll'] = 'foo' # => "foo" + s # => "hefooo" + + s = 'こんにちは' + s['んにち'] = 'foo' # => "foo" + s # => "こfooは" + + s['nosuch'] = 'foo' # Raises IndexError: string not matched. + +Related: see {Modifying}[rdoc-ref:String@Modifying]. diff --git a/doc/string/b.rdoc b/doc/string/b.rdoc index f8ad2910b4..8abd6d9532 100644 --- a/doc/string/b.rdoc +++ b/doc/string/b.rdoc @@ -12,3 +12,5 @@ the underlying bytes are not modified: t = s.b # => "\xE4\x82\x95" t.encoding # => #<Encoding:ASCII-8BIT> t.bytes # => [228, 130, 149] + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/bytes.rdoc b/doc/string/bytes.rdoc index a9e89f1cd1..6dde0a745d 100644 --- a/doc/string/bytes.rdoc +++ b/doc/string/bytes.rdoc @@ -1,6 +1,7 @@ Returns an array of the bytes in +self+: - 'hello'.bytes # => [104, 101, 108, 108, 111] - 'тест'.bytes # => [209, 130, 208, 181, 209, 129, 209, 130] + 'hello'.bytes # => [104, 101, 108, 108, 111] 'こんにちは'.bytes # => [227, 129, 147, 227, 130, 147, 227, 129, 171, 227, 129, 161, 227, 129, 175] + +Related: see {Converting to Non-String}[rdoc-ref:String@Converting+to+Non--5CString]. diff --git a/doc/string/bytesize.rdoc b/doc/string/bytesize.rdoc index b0567ff67b..8d12a0d454 100644 --- a/doc/string/bytesize.rdoc +++ b/doc/string/bytesize.rdoc @@ -1,11 +1,12 @@ -Returns the count of bytes (not characters) in +self+: +Returns the count of bytes in +self+. - 'foo'.bytesize # => 3 - 'тест'.bytesize # => 8 - 'こんにちは'.bytesize # => 15 +Note that the byte count may be different from the character count (returned by #size): -Contrast with String#length: + s = 'foo' + s.bytesize # => 3 + s.size # => 3 + s = 'こんにちは' + s.bytesize # => 15 + s.size # => 5 - 'foo'.length # => 3 - 'тест'.length # => 4 - 'こんにちは'.length # => 5 +Related: see {Querying}[rdoc-ref:String@Querying]. diff --git a/doc/string/byteslice.rdoc b/doc/string/byteslice.rdoc new file mode 100644 index 0000000000..d70441fb2b --- /dev/null +++ b/doc/string/byteslice.rdoc @@ -0,0 +1,54 @@ +Returns a substring of +self+, or +nil+ if the substring cannot be constructed. + +With integer arguments +offset+ and +length+ given, +returns the substring beginning at the given +offset+ +and of the given +length+ (as available): + + s = '0123456789' # => "0123456789" + s.byteslice(2) # => "2" + s.byteslice(200) # => nil + s.byteslice(4, 3) # => "456" + s.byteslice(4, 30) # => "456789" + +Returns +nil+ if +length+ is negative or +offset+ falls outside of +self+: + + s.byteslice(4, -1) # => nil + s.byteslice(40, 2) # => nil + +Counts backwards from the end of +self+ +if +offset+ is negative: + + s = '0123456789' # => "0123456789" + s.byteslice(-4) # => "6" + s.byteslice(-4, 3) # => "678" + +With Range argument +range+ given, returns +<tt>byteslice(range.begin, range.size)</tt>: + + s = '0123456789' # => "0123456789" + s.byteslice(4..6) # => "456" + s.byteslice(-6..-4) # => "456" + s.byteslice(5..2) # => "" # range.size is zero. + s.byteslice(40..42) # => nil + +The starting and ending offsets need not be on character boundaries: + + s = 'こんにちは' + s.byteslice(0, 3) # => "こ" + s.byteslice(1, 3) # => "\x81\x93\xE3" + +The encodings of +self+ and the returned substring +are always the same: + + s.encoding # => #<Encoding:UTF-8> + s.byteslice(0, 3).encoding # => #<Encoding:UTF-8> + s.byteslice(1, 3).encoding # => #<Encoding:UTF-8> + +But, depending on the character boundaries, +the encoding of the returned substring may not be valid: + + s.valid_encoding? # => true + s.byteslice(0, 3).valid_encoding? # => true + s.byteslice(1, 3).valid_encoding? # => false + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/bytesplice.rdoc b/doc/string/bytesplice.rdoc new file mode 100644 index 0000000000..5689ef4a2b --- /dev/null +++ b/doc/string/bytesplice.rdoc @@ -0,0 +1,66 @@ +Replaces <i>target bytes</i> in +self+ with <i>source bytes</i> from the given string +str+; +returns +self+. + +In the first form, arguments +offset+ and +length+ determine the target bytes, +and the source bytes are all of the given +str+: + + '0123456789'.bytesplice(0, 3, 'abc') # => "abc3456789" + '0123456789'.bytesplice(3, 3, 'abc') # => "012abc6789" + '0123456789'.bytesplice(0, 50, 'abc') # => "abc" + '0123456789'.bytesplice(50, 3, 'abc') # Raises IndexError. + +The counts of the target bytes and source source bytes may be different: + + '0123456789'.bytesplice(0, 6, 'abc') # => "abc6789" # Shorter source. + '0123456789'.bytesplice(0, 1, 'abc') # => "abc123456789" # Shorter target. + +And either count may be zero (i.e., specifying an empty string): + + '0123456789'.bytesplice(0, 3, '') # => "3456789" # Empty source. + '0123456789'.bytesplice(0, 0, 'abc') # => "abc0123456789" # Empty target. + +In the second form, just as in the first, +arugments +offset+ and +length+ determine the target bytes; +argument +str+ _contains_ the source bytes, +and the additional arguments +str_offset+ and +str_length+ +determine the actual source bytes: + + '0123456789'.bytesplice(0, 3, 'abc', 0, 3) # => "abc3456789" + '0123456789'.bytesplice(0, 3, 'abc', 1, 1) # => "b3456789" # Shorter source. + '0123456789'.bytesplice(0, 1, 'abc', 0, 3) # => "abc123456789" # Shorter target. + '0123456789'.bytesplice(0, 3, 'abc', 1, 0) # => "3456789" # Empty source. + '0123456789'.bytesplice(0, 0, 'abc', 0, 3) # => "abc0123456789" # Empty target. + +In the third form, argument +range+ determines the target bytes +and the source bytes are all of the given +str+: + + '0123456789'.bytesplice(0..2, 'abc') # => "abc3456789" + '0123456789'.bytesplice(3..5, 'abc') # => "012abc6789" + '0123456789'.bytesplice(0..5, 'abc') # => "abc6789" # Shorter source. + '0123456789'.bytesplice(0..0, 'abc') # => "abc123456789" # Shorter target. + '0123456789'.bytesplice(0..2, '') # => "3456789" # Empty source. + '0123456789'.bytesplice(0...0, 'abc') # => "abc0123456789" # Empty target. + +In the fourth form, just as in the third, +arugment +range+ determines the target bytes; +argument +str+ _contains_ the source bytes, +and the additional argument +str_range+ +determines the actual source bytes: + + '0123456789'.bytesplice(0..2, 'abc', 0..2) # => "abc3456789" + '0123456789'.bytesplice(3..5, 'abc', 0..2) # => "012abc6789" + '0123456789'.bytesplice(0..2, 'abc', 0..1) # => "ab3456789" # Shorter source. + '0123456789'.bytesplice(0..1, 'abc', 0..2) # => "abc23456789" # Shorter target. + '0123456789'.bytesplice(0..2, 'abc', 0...0) # => "3456789" # Empty source. + '0123456789'.bytesplice(0...0, 'abc', 0..2) # => "abc0123456789" # Empty target. + +In any of the forms, the beginnings and endings of both source and target +must be on character boundaries. + +In these examples, +self+ has five 3-byte characters, +and so has character boundaries at offsets 0, 3, 6, 9, 12, and 15. + + 'こんにちは'.bytesplice(0, 3, 'abc') # => "abcんにちは" + 'こんにちは'.bytesplice(1, 3, 'abc') # Raises IndexError. + 'こんにちは'.bytesplice(0, 2, 'abc') # Raises IndexError. + diff --git a/doc/string/capitalize.rdoc b/doc/string/capitalize.rdoc new file mode 100644 index 0000000000..3a1a2dcb8b --- /dev/null +++ b/doc/string/capitalize.rdoc @@ -0,0 +1,26 @@ +Returns a string containing the characters in +self+, +each with possibly changed case: + +- The first character made uppercase. +- All other characters are made lowercase. + +Examples: + + 'hello'.capitalize # => "Hello" + 'HELLO'.capitalize # => "Hello" + 'straße'.capitalize # => "Straße" # Lowercase 'ß' not changed. + 'STRAẞE'.capitalize # => "Straße" # Uppercase 'ẞ' downcased to 'ß'. + +Some characters (and some character sets) do not have upcase and downcase versions; +see {Case Mapping}[rdoc-ref:case_mapping.rdoc]: + + s = '1, 2, 3, ...' + s.capitalize == s # => true + s = 'こんにちは' + s.capitalize == s # => true + +The casing is affected by the given +mapping+, +which may be +:ascii+, +:fold+, or +:turkic+; +see {Case Mappings}[rdoc-ref:case_mapping.rdoc@Case+Mappings]. + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/center.rdoc b/doc/string/center.rdoc index d53d921ad5..b86c8b5916 100644 --- a/doc/string/center.rdoc +++ b/doc/string/center.rdoc @@ -2,15 +2,18 @@ Returns a centered copy of +self+. If integer argument +size+ is greater than the size (in characters) of +self+, returns a new string of length +size+ that is a copy of +self+, -centered and padded on both ends with +pad_string+: +centered and padded on one or both ends with +pad_string+: - 'hello'.center(10) # => " hello " - ' hello'.center(10) # => " hello " - 'hello'.center(10, 'ab') # => "abhelloaba" - 'тест'.center(10) # => " тест " - 'こんにちは'.center(10) # => " こんにちは " + 'hello'.center(6) # => "hello " # Padded on one end. + 'hello'.center(10) # => " hello " # Padded on both ends. + 'hello'.center(20, '-|') # => "-|-|-|-hello-|-|-|-|" # Some padding repeated. + 'hello'.center(10, 'abcdefg') # => "abhelloabc" # Some padding not used. + ' hello '.center(13) # => " hello " + 'こんにちは'.center(10) # => " こんにちは " # Multi-byte characters. -If +size+ is not greater than the size of +self+, returns a copy of +self+: +If +size+ is less than or equal to the size of +self+, returns an unpadded copy of +self+: - 'hello'.center(5) # => "hello" - 'hello'.center(1) # => "hello" + 'hello'.center(5) # => "hello" + 'hello'.center(-10) # => "hello" + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/chars.rdoc b/doc/string/chars.rdoc index d24a1cc3a9..d4d15bf2ad 100644 --- a/doc/string/chars.rdoc +++ b/doc/string/chars.rdoc @@ -1,5 +1,7 @@ Returns an array of the characters in +self+: 'hello'.chars # => ["h", "e", "l", "l", "o"] - 'тест'.chars # => ["т", "е", "с", "т"] 'こんにちは'.chars # => ["こ", "ん", "に", "ち", "は"] + ''.chars # => [] + +Related: see {Converting to Non-String}[rdoc-ref:String@Converting+to+Non--5CString]. diff --git a/doc/string/chomp.rdoc b/doc/string/chomp.rdoc index b6fb9ff38c..4efff5c291 100644 --- a/doc/string/chomp.rdoc +++ b/doc/string/chomp.rdoc @@ -9,7 +9,6 @@ if they are <tt>"\r"</tt>, <tt>"\n"</tt>, or <tt>"\r\n"</tt> "abc\n".chomp # => "abc" "abc\r\n".chomp # => "abc" "abc\n\r".chomp # => "abc\n" - "тест\r\n".chomp # => "тест" "こんにちは\r\n".chomp # => "こんにちは" When +line_sep+ is <tt>''</tt> (an empty string), @@ -25,5 +24,8 @@ removes multiple trailing occurrences of <tt>"\n"</tt> or <tt>"\r\n"</tt> When +line_sep+ is neither <tt>"\n"</tt> nor <tt>''</tt>, removes a single trailing line separator if there is one: - 'abcd'.chomp('d') # => "abc" - 'abcdd'.chomp('d') # => "abcd" + 'abcd'.chomp('cd') # => "ab" + 'abcdcd'.chomp('cd') # => "abcd" + 'abcd'.chomp('xx') # => "abcd" + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/chop.rdoc b/doc/string/chop.rdoc index 8ef82f8a49..d818ba467a 100644 --- a/doc/string/chop.rdoc +++ b/doc/string/chop.rdoc @@ -3,14 +3,15 @@ Returns a new string copied from +self+, with trailing characters possibly remov Removes <tt>"\r\n"</tt> if those are the last two characters. "abc\r\n".chop # => "abc" - "тест\r\n".chop # => "тест" "こんにちは\r\n".chop # => "こんにちは" Otherwise removes the last character if it exists. 'abcd'.chop # => "abc" - 'тест'.chop # => "тес" 'こんにちは'.chop # => "こんにち" ''.chop # => "" -If you only need to remove the newline separator at the end of the string, String#chomp is a better alternative. +If you only need to remove the newline separator at the end of the string, +String#chomp is a better alternative. + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/chr.rdoc b/doc/string/chr.rdoc new file mode 100644 index 0000000000..153d5d71c3 --- /dev/null +++ b/doc/string/chr.rdoc @@ -0,0 +1,7 @@ +Returns a string containing the first character of +self+: + + 'hello'.chr # => "h" + 'こんにちは'.chr # => "こ" + ''.chr # => "" + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/codepoints.rdoc b/doc/string/codepoints.rdoc index 0c55d3f4b9..22cb22c889 100644 --- a/doc/string/codepoints.rdoc +++ b/doc/string/codepoints.rdoc @@ -2,5 +2,7 @@ Returns an array of the codepoints in +self+; each codepoint is the integer value for a character: 'hello'.codepoints # => [104, 101, 108, 108, 111] - 'тест'.codepoints # => [1090, 1077, 1089, 1090] 'こんにちは'.codepoints # => [12371, 12435, 12395, 12385, 12399] + ''.codepoints # => [] + +Related: see {Converting to Non-String}[rdoc-ref:String@Converting+to+Non--5CString]. diff --git a/doc/string/concat.rdoc b/doc/string/concat.rdoc new file mode 100644 index 0000000000..92ba664b8c --- /dev/null +++ b/doc/string/concat.rdoc @@ -0,0 +1,11 @@ +Concatenates each object in +objects+ to +self+; returns +self+: + + 'foo'.concat('bar', 'baz') # => "foobarbaz" + +For each given object +object+ that is an integer, +the value is considered a codepoint and converted to a character before concatenation: + + 'foo'.concat(32, 'bar', 32, 'baz') # => "foo bar baz" # Embeds spaces. + 'こん'.concat(12395, 12385, 12399) # => "こんにちは" + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/count.rdoc b/doc/string/count.rdoc new file mode 100644 index 0000000000..7a3b9f1e21 --- /dev/null +++ b/doc/string/count.rdoc @@ -0,0 +1,74 @@ +Returns the total number of characters in +self+ that are specified by the given selectors. + +For one 1-character selector, +returns the count of instances of that character: + + s = 'abracadabra' + s.count('a') # => 5 + s.count('b') # => 2 + s.count('x') # => 0 + s.count('') # => 0 + + s = 'よろしくお願いします' + s.count('よ') # => 1 + s.count('し') # => 2 + +For one multi-character selector, +returns the count of instances for all specified characters: + + s = 'abracadabra' + s.count('ab') # => 7 + s.count('abc') # => 8 + s.count('abcd') # => 9 + s.count('abcdr') # => 11 + s.count('abcdrx') # => 11 + +Order and repetition do not matter: + + s.count('ba') == s.count('ab') # => true + s.count('baab') == s.count('ab') # => true + +For multiple selectors, +forms a single selector that is the intersection of characters in all selectors +and returns the count of instances for that selector: + + s = 'abcdefg' + s.count('abcde', 'dcbfg') == s.count('bcd') # => true + s.count('abc', 'def') == s.count('') # => true + +In a character selector, three characters get special treatment: + +- A caret (<tt>'^'</tt>) functions as a _negation_ operator + for the immediately following characters: + + s = 'abracadabra' + s.count('^bc') # => 8 # Count of all except 'b' and 'c'. + +- A hyphen (<tt>'-'</tt>) between two other characters defines a _range_ of characters: + + s = 'abracadabra' + s.count('a-c') # => 8 # Count of all 'a', 'b', and 'c'. + +- A backslash (<tt>'\'</tt>) acts as an escape for a caret, a hyphen, + or another backslash: + + s = 'abracadabra' + s.count('\^bc') # => 3 # Count of '^', 'b', and 'c'. + s.count('a\-c') # => 6 # Count of 'a', '-', and 'c'. + 'foo\bar\baz'.count('\\') # => 2 # Count of '\'. + +These usages may be mixed: + + s = 'abracadabra' + s.count('a-cq-t') # => 10 # Multiple ranges. + s.count('ac-d') # => 7 # Range mixed with plain characters. + s.count('^a-c') # => 3 # Range mixed with negation. + +For multiple selectors, all forms may be used, including negations, ranges, and escapes. + + s = 'abracadabra' + s.count('^abc', '^def') == s.count('^abcdef') # => true + s.count('a-e', 'c-g') == s.count('cde') # => true + s.count('^abc', 'c-g') == s.count('defg') # => true + +Related: see {Querying}[rdoc-ref:String@Querying]. diff --git a/doc/string/delete.rdoc b/doc/string/delete.rdoc new file mode 100644 index 0000000000..1827f177e6 --- /dev/null +++ b/doc/string/delete.rdoc @@ -0,0 +1,75 @@ +Returns a new string that is a copy of +self+ with certain characters removed; +the removed characters are all instances of those specified by the given string +selectors+. + +For one 1-character selector, +removes all instances of that character: + + s = 'abracadabra' + s.delete('a') # => "brcdbr" + s.delete('b') # => "aracadara" + s.delete('x') # => "abracadabra" + s.delete('') # => "abracadabra" + + s = 'よろしくお願いします' + s.delete('よ') # => "ろしくお願いします" + s.delete('し') # => "よろくお願います" + +For one multi-character selector, +removes all instances of the specified characters: + + s = 'abracadabra' + s.delete('ab') # => "rcdr" + s.delete('abc') # => "rdr" + s.delete('abcd') # => "rr" + s.delete('abcdr') # => "" + s.delete('abcdrx') # => "" + +Order and repetition do not matter: + + s.delete('ba') == s.delete('ab') # => true + s.delete('baab') == s.delete('ab') # => true + +For multiple selectors, +forms a single selector that is the intersection of characters in all selectors +and removes all instances of characters specified by that selector: + + s = 'abcdefg' + s.delete('abcde', 'dcbfg') == s.delete('bcd') # => true + s.delete('abc', 'def') == s.delete('') # => true + +In a character selector, three characters get special treatment: + +- A caret (<tt>'^'</tt>) functions as a _negation_ operator + for the immediately following characters: + + s = 'abracadabra' + s.delete('^bc') # => "bcb" # Deletes all except 'b' and 'c'. + +- A hyphen (<tt>'-'</tt>) between two other characters defines a _range_ of characters: + + s = 'abracadabra' + s.delete('a-c') # => "rdr" # Deletes all 'a', 'b', and 'c'. + +- A backslash (<tt>'\'</tt>) acts as an escape for a caret, a hyphen, + or another backslash: + + s = 'abracadabra' + s.delete('\^bc') # => "araadara" # Deletes all '^', 'b', and 'c'. + s.delete('a\-c') # => "brdbr" # Deletes all 'a', '-', and 'c'. + 'foo\bar\baz'.delete('\\') # => "foobarbaz" # Deletes all '\'. + +These usages may be mixed: + + s = 'abracadabra' + s.delete('a-cq-t') # => "d" # Multiple ranges. + s.delete('ac-d') # => "brbr" # Range mixed with plain characters. + s.delete('^a-c') # => "abacaaba" # Range mixed with negation. + +For multiple selectors, all forms may be used, including negations, ranges, and escapes. + + s = 'abracadabra' + s.delete('^abc', '^def') == s.delete('^abcdef') # => true + s.delete('a-e', 'c-g') == s.delete('cde') # => true + s.delete('^abc', 'c-g') == s.delete('defg') # => true + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/delete_prefix.rdoc b/doc/string/delete_prefix.rdoc index fa9d8abd38..6255e300e3 100644 --- a/doc/string/delete_prefix.rdoc +++ b/doc/string/delete_prefix.rdoc @@ -1,8 +1,9 @@ -Returns a copy of +self+ with leading substring <tt>prefix</tt> removed: +Returns a copy of +self+ with leading substring +prefix+ removed: - 'hello'.delete_prefix('hel') # => "lo" - 'hello'.delete_prefix('llo') # => "hello" - 'тест'.delete_prefix('те') # => "ст" + 'oof'.delete_prefix('o') # => "of" + 'oof'.delete_prefix('oo') # => "f" + 'oof'.delete_prefix('oof') # => "" + 'oof'.delete_prefix('x') # => "oof" 'こんにちは'.delete_prefix('こん') # => "にちは" -Related: String#delete_prefix!, String#delete_suffix. +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/delete_suffix.rdoc b/doc/string/delete_suffix.rdoc index 4862b725cf..a4d9a80f85 100644 --- a/doc/string/delete_suffix.rdoc +++ b/doc/string/delete_suffix.rdoc @@ -1,8 +1,10 @@ Returns a copy of +self+ with trailing substring <tt>suffix</tt> removed: - 'hello'.delete_suffix('llo') # => "he" - 'hello'.delete_suffix('hel') # => "hello" - 'тест'.delete_suffix('ст') # => "те" + 'foo'.delete_suffix('o') # => "fo" + 'foo'.delete_suffix('oo') # => "f" + 'foo'.delete_suffix('foo') # => "" + 'foo'.delete_suffix('f') # => "foo" + 'foo'.delete_suffix('x') # => "foo" 'こんにちは'.delete_suffix('ちは') # => "こんに" -Related: String#delete_suffix!, String#delete_prefix. +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/downcase.rdoc b/doc/string/downcase.rdoc new file mode 100644 index 0000000000..d5fffa037b --- /dev/null +++ b/doc/string/downcase.rdoc @@ -0,0 +1,20 @@ +Returns a new string containing the downcased characters in +self+: + + 'HELLO'.downcase # => "hello" + 'STRAẞE'.downcase # => "straße" + 'ПРИВЕТ'.downcase # => "привет" + 'RubyGems.org'.downcase # => "rubygems.org" + +Some characters (and some character sets) do not have upcase and downcase versions; +see {Case Mapping}[rdoc-ref:case_mapping.rdoc]: + + s = '1, 2, 3, ...' + s.downcase == s # => true + s = 'こんにちは' + s.downcase == s # => true + +The casing is affected by the given +mapping+, +which may be +:ascii+, +:fold+, or +:turkic+; +see {Case Mappings}[rdoc-ref:case_mapping.rdoc@Case+Mappings]. + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/dump.rdoc b/doc/string/dump.rdoc new file mode 100644 index 0000000000..add3c35662 --- /dev/null +++ b/doc/string/dump.rdoc @@ -0,0 +1,89 @@ +For an ordinary string, this method, +String#dump+, +returns a printable ASCII-only version of +self+, enclosed in double-quotes. + +For a dumped string, method String#undump is the inverse of +String#dump+; +it returns a "restored" version of +self+, +where all the dumping changes have been undone. + +In the simplest case, the dumped string contains the original string, +enclosed in double-quotes; +this example is done in +irb+ (interactive Ruby), which uses method `inspect` to render the results: + + s = 'hello' # => "hello" + s.dump # => "\"hello\"" + s.dump.undump # => "hello" + +Keep in mind that in the second line above: + +- The outer double-quotes are put on by +inspect+, + and _are_ _not_ part of the output of #dump. +- The inner double-quotes _are_ part of the output of +dump+, + and are escaped by +inspect+ because they are within the outer double-quotes. + +To avoid confusion, we'll use this helper method to omit the outer double-quotes: + + def dump(s) + print "String: ", s, "\n" + print "Dumped: ", s.dump, "\n" + print "Undumped: ", s.dump.undump, "\n" + end + +So that for string <tt>'hello'</tt>, we'll see: + + String: hello + Dumped: "hello" + Undumped: hello + +In a dump, certain special characters are escaped: + + String: " + Dumped: "\"" + Undumped: " + + String: \ + Dumped: "\\" + Undumped: \ + +In a dump, unprintable characters are replaced by printable ones; +the unprintable characters are the whitespace characters (other than space itself); +here we see the ordinals for those characers, together with explanatory text: + + h = { + 7 => 'Alert (BEL)', + 8 => 'Backspace (BS)', + 9 => 'Horizontal tab (HT)', + 10 => 'Linefeed (LF)', + 11 => 'Vertical tab (VT)', + 12 => 'Formfeed (FF)', + 13 => 'Carriage return (CR)' + } + +In this example, the dumped output is printed by method #inspect, +and so contains both outer double-quotes and escaped inner double-quotes: + + s = '' + h.keys.each {|i| s << i } # => [7, 8, 9, 10, 11, 12, 13] + s # => "\a\b\t\n\v\f\r" + s.dump # => "\"\\a\\b\\t\\n\\v\\f\\r\"" + +If +self+ is encoded in UTF-8 and contains Unicode characters, +each Unicode character is dumped as a Unicode escape sequence: + + String: こんにちは + Dumped: "\u3053\u3093\u306B\u3061\u306F" + Undumped: こんにちは + +If the encoding of +self+ is not ASCII-compatible +(i.e., if <tt>self.encoding.ascii_compatible?</tt> returns +false+), +each ASCII-compatible byte is dumped as an ASCII character, +and all other bytes are dumped as hexadecimal; +also appends <tt>.dup.force_encoding(\"encoding\")</tt>, +where <tt><encoding></tt> is <tt>self.encoding.name</tt>: + + String: hello + Dumped: "\xFE\xFF\x00h\x00e\x00l\x00l\x00o".dup.force_encoding("UTF-16") + Undumped: hello + + String: こんにちは + Dumped: "\xFE\xFF0S0\x930k0a0o".dup.force_encoding("UTF-16") + Undumped: こんにちは diff --git a/doc/string/each_byte.rdoc b/doc/string/each_byte.rdoc index 643118fea3..642d71e84b 100644 --- a/doc/string/each_byte.rdoc +++ b/doc/string/each_byte.rdoc @@ -1,17 +1,15 @@ -Calls the given block with each successive byte from +self+; +With a block given, calls the block with each successive byte from +self+; returns +self+: - 'hello'.each_byte {|byte| print byte, ' ' } - print "\n" - 'тест'.each_byte {|byte| print byte, ' ' } - print "\n" - 'こんにちは'.each_byte {|byte| print byte, ' ' } - print "\n" + a = [] + 'hello'.each_byte {|byte| a.push(byte) } # Five 1-byte characters. + a # => [104, 101, 108, 108, 111] + a = [] + 'こんにちは'.each_byte {|byte| a.push(byte) } # Five 3-byte characters. + a # => [227, 129, 147, 227, 130, 147, 227, 129, 171, 227, 129, 161, 227, 129, 175] -Output: +With no block given, returns an enumerator. + +Related: see {Iterating}[rdoc-ref:String@Iterating]. - 104 101 108 108 111 - 209 130 208 181 209 129 209 130 - 227 129 147 227 130 147 227 129 171 227 129 161 227 129 175 -Returns an enumerator if no block is given. diff --git a/doc/string/each_char.rdoc b/doc/string/each_char.rdoc index e5ae5a1812..2dd56711d3 100644 --- a/doc/string/each_char.rdoc +++ b/doc/string/each_char.rdoc @@ -1,17 +1,17 @@ -Calls the given block with each successive character from +self+; +With a block given, calls the block with each successive character from +self+; returns +self+: - 'hello'.each_char {|char| print char, ' ' } - print "\n" - 'тест'.each_char {|char| print char, ' ' } - print "\n" - 'こんにちは'.each_char {|char| print char, ' ' } - print "\n" + a = [] + 'hello'.each_char do |char| + a.push(char) + end + a # => ["h", "e", "l", "l", "o"] + a = [] + 'こんにちは'.each_char do |char| + a.push(char) + end + a # => ["こ", "ん", "に", "ち", "は"] -Output: +With no block given, returns an enumerator. - h e l l o - т е с т - こ ん に ち は - -Returns an enumerator if no block is given. +Related: see {Iterating}[rdoc-ref:String@Iterating]. diff --git a/doc/string/each_codepoint.rdoc b/doc/string/each_codepoint.rdoc index 88bfcbd1c0..8e4e7545e6 100644 --- a/doc/string/each_codepoint.rdoc +++ b/doc/string/each_codepoint.rdoc @@ -1,18 +1,18 @@ -Calls the given block with each successive codepoint from +self+; -each codepoint is the integer value for a character; +With a block given, calls the block with each successive codepoint from +self+; +each {codepoint}[https://en.wikipedia.org/wiki/Code_point] is the integer value for a character; returns +self+: - 'hello'.each_codepoint {|codepoint| print codepoint, ' ' } - print "\n" - 'тест'.each_codepoint {|codepoint| print codepoint, ' ' } - print "\n" - 'こんにちは'.each_codepoint {|codepoint| print codepoint, ' ' } - print "\n" + a = [] + 'hello'.each_codepoint do |codepoint| + a.push(codepoint) + end + a # => [104, 101, 108, 108, 111] + a = [] + 'こんにちは'.each_codepoint do |codepoint| + a.push(codepoint) + end + a # => [12371, 12435, 12395, 12385, 12399] -Output: +With no block given, returns an enumerator. - 104 101 108 108 111 - 1090 1077 1089 1090 - 12371 12435 12395 12385 12399 - -Returns an enumerator if no block is given. +Related: see {Iterating}[rdoc-ref:String@Iterating]. diff --git a/doc/string/each_grapheme_cluster.rdoc b/doc/string/each_grapheme_cluster.rdoc index 40be95fcac..384cd6967d 100644 --- a/doc/string/each_grapheme_cluster.rdoc +++ b/doc/string/each_grapheme_cluster.rdoc @@ -1,12 +1,19 @@ -Calls the given block with each successive grapheme cluster from +self+ +With a block given, calls the given block with each successive grapheme cluster from +self+ (see {Unicode Grapheme Cluster Boundaries}[https://www.unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries]); returns +self+: - s = "\u0061\u0308-pqr-\u0062\u0308-xyz-\u0063\u0308" # => "ä-pqr-b̈-xyz-c̈" - s.each_grapheme_cluster {|gc| print gc, ' ' } + a = [] + 'hello'.each_grapheme_cluster do |grapheme_cluster| + a.push(grapheme_cluster) + end + a # => ["h", "e", "l", "l", "o"] -Output: + a = [] + 'こんにちは'.each_grapheme_cluster do |grapheme_cluster| + a.push(grapheme_cluster) + end + a # => ["こ", "ん", "に", "ち", "は"] - ä - p q r - b̈ - x y z - c̈ +With no block given, returns an enumerator. -Returns an enumerator if no block is given. +Related: see {Iterating}[rdoc-ref:String@Iterating]. diff --git a/doc/string/each_line.rdoc b/doc/string/each_line.rdoc index e254c22d40..217c188e35 100644 --- a/doc/string/each_line.rdoc +++ b/doc/string/each_line.rdoc @@ -1,9 +1,12 @@ -With a block given, forms the substrings ("lines") +With a block given, forms the substrings (lines) that are the result of splitting +self+ -at each occurrence of the given line separator +line_sep+; +at each occurrence of the given +record_separator+; passes each line to the block; -returns +self+: +returns +self+. +With the default +record_separator+: + + $/ # => "\n" s = <<~EOT This is the first line. This is line two. @@ -11,7 +14,6 @@ returns +self+: This is line four. This is line five. EOT - s.each_line {|line| p line } Output: @@ -22,9 +24,10 @@ Output: "This is line four.\n" "This is line five.\n" -With a different +line_sep+: +With a different +record_separator+: - s.each_line(' is ') {|line| p line } + record_separator = ' is ' + s.each_line(record_separator) {|line| p line } Output: @@ -34,7 +37,7 @@ Output: "line four.\nThis is " "line five.\n" -With +chomp+ as +true+, removes the trailing +line_sep+ from each line: +With +chomp+ as +true+, removes the trailing +record_separator+ from each line: s.each_line(chomp: true) {|line| p line } @@ -46,11 +49,12 @@ Output: "This is line four." "This is line five." -With an empty string as +line_sep+, +With an empty string as +record_separator+, forms and passes "paragraphs" by splitting at each occurrence of two or more newlines: - s.each_line('') {|line| p line } + record_separator = '' + s.each_line(record_separator) {|line| p line } Output: @@ -58,3 +62,5 @@ Output: "This is line four.\nThis is line five.\n" With no block given, returns an enumerator. + +Related: see {Iterating}[rdoc-ref:String@Iterating]. diff --git a/doc/string/encode.rdoc b/doc/string/encode.rdoc index 65872fdfd4..14b959ffff 100644 --- a/doc/string/encode.rdoc +++ b/doc/string/encode.rdoc @@ -1,4 +1,6 @@ -Returns a copy of +self+ transcoded as determined by +dst_encoding+. +Returns a copy of +self+ transcoded as determined by +dst_encoding+; +see {Encodings}[rdoc-ref:encodings.rdoc]. + By default, raises an exception if +self+ contains an invalid byte or a character not defined in +dst_encoding+; that behavior may be modified by encoding options; see below. @@ -45,3 +47,4 @@ given, conversion from an encoding +enc+ to the same encoding +enc+ no-op, i.e. the string is simply copied without any changes, and no exceptions are raised, even if there are invalid bytes. +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/end_with_p.rdoc b/doc/string/end_with_p.rdoc index f959cf7aaa..9a95d74fde 100644 --- a/doc/string/end_with_p.rdoc +++ b/doc/string/end_with_p.rdoc @@ -1,11 +1,9 @@ -Returns whether +self+ ends with any of the given +strings+. +Returns whether +self+ ends with any of the given +strings+: -Returns +true+ if any given string matches the end, +false+ otherwise: + 'foo'.end_with?('oo') # => true + 'foo'.end_with?('bar', 'oo') # => true + 'foo'.end_with?('bar', 'baz') # => false + 'foo'.end_with?('') # => true + 'こんにちは'.end_with?('は') # => true - 'hello'.end_with?('ello') #=> true - 'hello'.end_with?('heaven', 'ello') #=> true - 'hello'.end_with?('heaven', 'paradise') #=> false - 'тест'.end_with?('т') # => true - 'こんにちは'.end_with?('は') # => true - -Related: String#start_with?. +Related: see {Querying}[rdoc-ref:String@Querying]. diff --git a/doc/string/eql_p.rdoc b/doc/string/eql_p.rdoc new file mode 100644 index 0000000000..85409c5ed6 --- /dev/null +++ b/doc/string/eql_p.rdoc @@ -0,0 +1,18 @@ +Returns whether +self+ and +object+ have the same length and content: + + s = 'foo' + s.eql?('foo') # => true + s.eql?('food') # => false + s.eql?('FOO') # => false + +Returns +false+ if the two strings' encodings are not compatible: + + s0 = "äöü" # => "äöü" + s1 = s0.encode(Encoding::ISO_8859_1) # => "\xE4\xF6\xFC" + s0.encoding # => #<Encoding:UTF-8> + s1.encoding # => #<Encoding:ISO-8859-1> + s0.eql?(s1) # => false + +See {Encodings}[rdoc-ref:encodings.rdoc]. + +Related: see {Querying}[rdoc-ref:String@Querying]. diff --git a/doc/string/force_encoding.rdoc b/doc/string/force_encoding.rdoc index fd9615caaa..a509e67f80 100644 --- a/doc/string/force_encoding.rdoc +++ b/doc/string/force_encoding.rdoc @@ -1,5 +1,6 @@ -Changes the encoding of +self+ to +encoding+, +Changes the encoding of +self+ to the given +encoding+, which may be a string encoding name or an Encoding object; +does not change the underlying bytes; returns self: s = 'łał' @@ -7,14 +8,14 @@ returns self: s.encoding # => #<Encoding:UTF-8> s.force_encoding('ascii') # => "\xC5\x82a\xC5\x82" s.encoding # => #<Encoding:US-ASCII> - -Does not change the underlying bytes: - + s.valid_encoding? # => true s.bytes # => [197, 130, 97, 197, 130] Makes the change even if the given +encoding+ is invalid for +self+ (as is the change above): - s.valid_encoding? # => false - s.force_encoding(Encoding::UTF_8) # => "łał" - s.valid_encoding? # => true + s.valid_encoding? # => false + +See {Encodings}[rdoc-ref:encodings.rdoc]. + +Related: see {Modifying}[rdoc-ref:String@Modifying]. diff --git a/doc/string/getbyte.rdoc b/doc/string/getbyte.rdoc new file mode 100644 index 0000000000..1d0ed2a5a4 --- /dev/null +++ b/doc/string/getbyte.rdoc @@ -0,0 +1,23 @@ +Returns the byte at zero-based +index+ as an integer: + + s = 'foo' + s.getbyte(0) # => 102 + s.getbyte(1) # => 111 + s.getbyte(2) # => 111 + +Counts backward from the end if +index+ is negative: + + s.getbyte(-3) # => 102 + +Returns +nil+ if +index+ is out of range: + + s.getbyte(3) # => nil + s.getbyte(-4) # => nil + +More examples: + + s = 'こんにちは' + s.bytes # => [227, 129, 147, 227, 130, 147, 227, 129, 171, 227, 129, 161, 227, 129, 175] + s.getbyte(2) # => 147 + +Related: see {Converting to Non-String}[rdoc-ref:String@Converting+to+Non--5CString]. diff --git a/doc/string/grapheme_clusters.rdoc b/doc/string/grapheme_clusters.rdoc index 8c7f5a7259..07ea1e318b 100644 --- a/doc/string/grapheme_clusters.rdoc +++ b/doc/string/grapheme_clusters.rdoc @@ -1,6 +1,19 @@ Returns an array of the grapheme clusters in +self+ (see {Unicode Grapheme Cluster Boundaries}[https://www.unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries]): - s = "\u0061\u0308-pqr-\u0062\u0308-xyz-\u0063\u0308" # => "ä-pqr-b̈-xyz-c̈" + s = "ä-pqr-b̈-xyz-c̈" + s.size # => 16 + s.bytesize # => 19 + s.grapheme_clusters.size # => 13 s.grapheme_clusters # => ["ä", "-", "p", "q", "r", "-", "b̈", "-", "x", "y", "z", "-", "c̈"] + +Details: + + s = "ä" + s.grapheme_clusters # => ["ä"] # One grapheme cluster. + s.bytes # => [97, 204, 136] # Three bytes. + s.chars # => ["a", "̈"] # Two characters. + s.chars.map {|char| char.ord } # => [97, 776] # Their values. + +Related: see {Converting to Non-String}[rdoc-ref:String@Converting+to+Non--5CString]. diff --git a/doc/string/hash.rdoc b/doc/string/hash.rdoc new file mode 100644 index 0000000000..fe94770ed9 --- /dev/null +++ b/doc/string/hash.rdoc @@ -0,0 +1,19 @@ +Returns the integer hash value for +self+. + +Two \String objects that have identical content and compatible encodings +also have the same hash value; +see Object#hash and {Encodings}[rdoc-ref:encodings.rdoc]: + + s = 'foo' + h = s.hash # => -569050784 + h == 'foo'.hash # => true + h == 'food'.hash # => false + h == 'FOO'.hash # => false + + s0 = "äöü" + s1 = s0.encode(Encoding::ISO_8859_1) + s0.encoding # => #<Encoding:UTF-8> + s1.encoding # => #<Encoding:ISO-8859-1> + s0.hash == s1.hash # => false + +Related: see {Querying}[rdoc-ref:String@Querying]. diff --git a/doc/string/index.rdoc b/doc/string/index.rdoc index ce09a37bdf..c3cff24dac 100644 --- a/doc/string/index.rdoc +++ b/doc/string/index.rdoc @@ -1,31 +1,31 @@ -Returns the integer index of the first match for the given argument, -or +nil+ if none found; -the search of +self+ is forward, and begins at position +offset+ (in characters). +Returns the integer position of the first substring that matches the given argument +pattern+, +or +nil+ if none found. -With string argument +substring+, +When +pattern+ is a string, returns the index of the first matching substring in +self+: 'foo'.index('f') # => 0 'foo'.index('o') # => 1 'foo'.index('oo') # => 1 'foo'.index('ooo') # => nil - 'тест'.index('с') # => 2 - 'こんにちは'.index('ち') # => 3 + 'こんにちは'.index('ち') # => 3 -With Regexp argument +regexp+, returns the index of the first match in +self+: +When +pattern+ is a Regexp, returns the index of the first match in +self+: 'foo'.index(/o./) # => 1 'foo'.index(/.o/) # => 0 -With positive integer +offset+, begins the search at position +offset+: +When +offset+ is non-negative, begins the search at position +offset+; +the returned index is relative to the beginning of +self+: - 'foo'.index('o', 1) # => 1 - 'foo'.index('o', 2) # => 2 - 'foo'.index('o', 3) # => nil - 'тест'.index('с', 1) # => 2 - 'こんにちは'.index('ち', 2) # => 3 + 'bar'.index('r', 0) # => 2 + 'bar'.index('r', 1) # => 2 + 'bar'.index('r', 2) # => 2 + 'bar'.index('r', 3) # => nil + 'bar'.index(/[r-z]/, 0) # => 2 + 'こんにちは'.index('ち', 2) # => 3 -With negative integer +offset+, selects the search position by counting backward +With negative integer argument +offset+, selects the search position by counting backward from the end of +self+: 'foo'.index('o', -1) # => 2 @@ -35,4 +35,4 @@ from the end of +self+: 'foo'.index(/o./, -2) # => 1 'foo'.index(/.o/, -2) # => 1 -Related: String#rindex. +Related: see {Querying}[rdoc-ref:String@Querying]. diff --git a/doc/string/insert.rdoc b/doc/string/insert.rdoc new file mode 100644 index 0000000000..73205f2069 --- /dev/null +++ b/doc/string/insert.rdoc @@ -0,0 +1,15 @@ +Inserts the given +other_string+ into +self+; returns +self+. + +If the given +index+ is non-negative, inserts +other_string+ at offset +index+: + + 'foo'.insert(0, 'bar') # => "barfoo" + 'foo'.insert(1, 'bar') # => "fbaroo" + 'foo'.insert(3, 'bar') # => "foobar" + 'こんにちは'.insert(2, 'bar') # => "こんbarにちは" + +If the +index+ is negative, counts backward from the end of +self+ +and inserts +other_string+ _after_ the offset: + + 'foo'.insert(-2, 'bar') # => "fobaro" + +Related: see {Modifying}[rdoc-ref:String@Modifying]. diff --git a/doc/string/inspect.rdoc b/doc/string/inspect.rdoc new file mode 100644 index 0000000000..907828c2af --- /dev/null +++ b/doc/string/inspect.rdoc @@ -0,0 +1,38 @@ +Returns a printable version of +self+, enclosed in double-quotes. + +Most printable characters are rendered simply as themselves: + + 'abc'.inspect # => "\"abc\"" + '012'.inspect # => "\"012\"" + ''.inspect # => "\"\"" + "\u000012".inspect # => "\"\\u000012\"" + 'こんにちは'.inspect # => "\"こんにちは\"" + +But printable characters double-quote (<tt>'"'</tt>) and backslash and (<tt>'\\'</tt>) are escaped: + + '"'.inspect # => "\"\\\"\"" + '\\'.inspect # => "\"\\\\\"" + +Unprintable characters are the {ASCII characters}[https://en.wikipedia.org/wiki/ASCII] +whose values are in range <tt>0..31</tt>, +along with the character whose value is +127+. + +Most of these characters are rendered thus: + + 0.chr.inspect # => "\"\\x00\"" + 1.chr.inspect # => "\"\\x01\"" + 2.chr.inspect # => "\"\\x02\"" + # ... + +A few, however, have special renderings: + + 7.chr.inspect # => "\"\\a\"" # BEL + 8.chr.inspect # => "\"\\b\"" # BS + 9.chr.inspect # => "\"\\t\"" # TAB + 10.chr.inspect # => "\"\\n\"" # LF + 11.chr.inspect # => "\"\\v\"" # VT + 12.chr.inspect # => "\"\\f\"" # FF + 13.chr.inspect # => "\"\\r\"" # CR + 27.chr.inspect # => "\"\\e\"" # ESC + +Related: see {Converting to Non-String}[rdoc-ref:String@Converting+to+Non--5CString]. diff --git a/doc/string/intern.rdoc b/doc/string/intern.rdoc new file mode 100644 index 0000000000..eded6ac3d7 --- /dev/null +++ b/doc/string/intern.rdoc @@ -0,0 +1,8 @@ +Returns the Symbol object derived from +self+, +creating it if it did not already exist: + + 'foo'.intern # => :foo + 'こんにちは'.intern # => :こんにちは + +Related: see {Converting to Non-String}[rdoc-ref:String@Converting+to+Non--5CString]. + diff --git a/doc/string/length.rdoc b/doc/string/length.rdoc index 544bca269f..eb68edb10c 100644 --- a/doc/string/length.rdoc +++ b/doc/string/length.rdoc @@ -1,12 +1,11 @@ Returns the count of characters (not bytes) in +self+: 'foo'.length # => 3 - 'тест'.length # => 4 - 'こんにちは'.length # => 5 + 'こんにちは'.length # => 5 Contrast with String#bytesize: 'foo'.bytesize # => 3 - 'тест'.bytesize # => 8 - 'こんにちは'.bytesize # => 15 + 'こんにちは'.bytesize # => 15 +Related: see {Querying}[rdoc-ref:String@Querying]. diff --git a/doc/string/ljust.rdoc b/doc/string/ljust.rdoc index 8e23c1fc8f..a8ca62ee76 100644 --- a/doc/string/ljust.rdoc +++ b/doc/string/ljust.rdoc @@ -1,16 +1,13 @@ -Returns a left-justified copy of +self+. - -If integer argument +size+ is greater than the size (in characters) of +self+, -returns a new string of length +size+ that is a copy of +self+, -left justified and padded on the right with +pad_string+: +Returns a copy of +self+, left-justified and, if necessary, right-padded with the +pad_string+: 'hello'.ljust(10) # => "hello " ' hello'.ljust(10) # => " hello " 'hello'.ljust(10, 'ab') # => "helloababa" - 'тест'.ljust(10) # => "тест " - 'こんにちは'.ljust(10) # => "こんにちは " + 'こんにちは'.ljust(10) # => "こんにちは " -If +size+ is not greater than the size of +self+, returns a copy of +self+: +If <tt>width <= self.length</tt>, returns a copy of +self+: 'hello'.ljust(5) # => "hello" - 'hello'.ljust(1) # => "hello" + 'hello'.ljust(1) # => "hello" # Does not truncate to width. + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/new.rdoc b/doc/string/new.rdoc index 1d44291f76..e2752d6e1f 100644 --- a/doc/string/new.rdoc +++ b/doc/string/new.rdoc @@ -1,34 +1,38 @@ -Returns a new \String that is a copy of +string+. +Returns a new \String object containing the given +string+. -With no arguments, returns the empty string with the Encoding <tt>ASCII-8BIT</tt>: +The +options+ are optional keyword options (see below). - s = String.new - s # => "" - s.encoding # => #<Encoding:ASCII-8BIT> +With no argument given and keyword +encoding+ also not given, +returns an empty string with the Encoding <tt>ASCII-8BIT</tt>: -With optional argument +string+ and no keyword arguments, -returns a copy of +string+ with the same encoding: + s = String.new # => "" + s.encoding # => #<Encoding:ASCII-8BIT> - String.new('foo') # => "foo" - String.new('тест') # => "тест" - String.new('こんにちは') # => "こんにちは" +With argument +string+ given and keyword option +encoding+ not given, +returns a new string with the same encoding as +string+: + + s0 = 'foo'.encode(Encoding::UTF_16) + s1 = String.new(s0) + s1.encoding # => #<Encoding:UTF-16 (dummy)> (Unlike \String.new, a {string literal}[rdoc-ref:syntax/literals.rdoc@String+Literals] like <tt>''</tt> or a {here document literal}[rdoc-ref:syntax/literals.rdoc@Here+Document+Literals] always has {script encoding}[rdoc-ref:encodings.rdoc@Script+Encoding].) -With optional keyword argument +encoding+, returns a copy of +string+ -with the specified encoding; +With keyword option +encoding+ given, +returns a string with the specified encoding; the +encoding+ may be an Encoding object, an encoding name, or an encoding name alias: + String.new(encoding: Encoding::US_ASCII).encoding # => #<Encoding:US-ASCII> + String.new('', encoding: Encoding::US_ASCII).encoding # => #<Encoding:US-ASCII> String.new('foo', encoding: Encoding::US_ASCII).encoding # => #<Encoding:US-ASCII> String.new('foo', encoding: 'US-ASCII').encoding # => #<Encoding:US-ASCII> String.new('foo', encoding: 'ASCII').encoding # => #<Encoding:US-ASCII> The given encoding need not be valid for the string's content, -and that validity is not checked: +and its validity is not checked: s = String.new('こんにちは', encoding: 'ascii') s.valid_encoding? # => false @@ -37,19 +41,11 @@ But the given +encoding+ itself is checked: String.new('foo', encoding: 'bar') # Raises ArgumentError. -With optional keyword argument +capacity+, returns a copy of +string+ -(or an empty string, if +string+ is not given); -the given +capacity+ is advisory only, +With keyword option +capacity+ given, +the given value is advisory only, and may or may not set the size of the internal buffer, which may in turn affect performance: - String.new(capacity: 1) - String.new('foo', capacity: 4096) - -Note that Ruby strings are null-terminated internally, so the internal -buffer size will be one or more bytes larger than the requested capacity -depending on the encoding. - -The +string+, +encoding+, and +capacity+ arguments may all be used together: - - String.new('hello', encoding: 'UTF-8', capacity: 25) + String.new('foo', capacity: 1) # Buffer size is at least 4 (includes terminal null byte). + String.new('foo', capacity: 4096) # Buffer size is at least 4; + # may be equal to, greater than, or less than 4096. diff --git a/doc/string/ord.rdoc b/doc/string/ord.rdoc index d586363d44..87b469db02 100644 --- a/doc/string/ord.rdoc +++ b/doc/string/ord.rdoc @@ -2,5 +2,6 @@ Returns the integer ordinal of the first character of +self+: 'h'.ord # => 104 'hello'.ord # => 104 - 'тест'.ord # => 1090 'こんにちは'.ord # => 12371 + +Related: see {Converting to Non-String}[rdoc-ref:String@Converting+to+Non--5CString]. diff --git a/doc/string/partition.rdoc b/doc/string/partition.rdoc index ebe575e8eb..614ad029d4 100644 --- a/doc/string/partition.rdoc +++ b/doc/string/partition.rdoc @@ -1,24 +1,43 @@ Returns a 3-element array of substrings of +self+. -Matches a pattern against +self+, scanning from the beginning. -The pattern is: +If +pattern+ is matched, returns the array: -- +string_or_regexp+ itself, if it is a Regexp. -- <tt>Regexp.quote(string_or_regexp)</tt>, if +string_or_regexp+ is a string. + [pre_match, first_match, post_match] -If the pattern is matched, returns pre-match, first-match, post-match: +where: - 'hello'.partition('l') # => ["he", "l", "lo"] - 'hello'.partition('ll') # => ["he", "ll", "o"] - 'hello'.partition('h') # => ["", "h", "ello"] - 'hello'.partition('o') # => ["hell", "o", ""] - 'hello'.partition(/l+/) #=> ["he", "ll", "o"] - 'hello'.partition('') # => ["", "", "hello"] - 'тест'.partition('т') # => ["", "т", "ест"] - 'こんにちは'.partition('に') # => ["こん", "に", "ちは"] +- +first_match+ is the first-found matching substring. +- +pre_match+ and +post_match+ are the preceding and following substrings. -If the pattern is not matched, returns a copy of +self+ and two empty strings: +If +pattern+ is not matched, returns the array: - 'hello'.partition('x') # => ["hello", "", ""] + [self.dup, "", ""] -Related: String#rpartition, String#split. +Note that in the examples below, a returned string <tt>'hello'</tt> +is a copy of +self+, not +self+. + +If +pattern+ is a Regexp, performs the equivalent of <tt>self.match(pattern)</tt> +(also setting {matched-data variables}[rdoc-ref:language/globals.md@Matched+Data]): + + 'hello'.partition(/h/) # => ["", "h", "ello"] + 'hello'.partition(/l/) # => ["he", "l", "lo"] + 'hello'.partition(/l+/) # => ["he", "ll", "o"] + 'hello'.partition(/o/) # => ["hell", "o", ""] + 'hello'.partition(/^/) # => ["", "", "hello"] + 'hello'.partition(//) # => ["", "", "hello"] + 'hello'.partition(/$/) # => ["hello", "", ""] + 'hello'.partition(/x/) # => ["hello", "", ""] + +If +pattern+ is not a Regexp, converts it to a string (if it is not already one), +then performs the equivalent of <tt>self.index(pattern)</tt> +(and does _not_ set {matched-data global variables}[rdoc-ref:language/globals.md@Matched+Data]): + + 'hello'.partition('h') # => ["", "h", "ello"] + 'hello'.partition('l') # => ["he", "l", "lo"] + 'hello'.partition('ll') # => ["he", "ll", "o"] + 'hello'.partition('o') # => ["hell", "o", ""] + 'hello'.partition('') # => ["", "", "hello"] + 'hello'.partition('x') # => ["hello", "", ""] + 'こんにちは'.partition('に') # => ["こん", "に", "ちは"] + +Related: see {Converting to Non-String}[rdoc-ref:String@Converting+to+Non--5CString]. diff --git a/doc/string/rindex.rdoc b/doc/string/rindex.rdoc new file mode 100644 index 0000000000..2b81c3716d --- /dev/null +++ b/doc/string/rindex.rdoc @@ -0,0 +1,51 @@ +Returns the integer position of the _last_ substring that matches the given argument +pattern+, +or +nil+ if none found. + +When +pattern+ is a string, returns the index of the last matching substring in self: + + 'foo'.rindex('f') # => 0 + 'foo'.rindex('o') # => 2 + 'foo'.rindex('oo' # => 1 + 'foo'.rindex('ooo') # => nil + 'こんにちは'.rindex('ち') # => 3 + +When +pattern+ is a Regexp, returns the index of the last match in self: + + 'foo'.rindex(/f/) # => 0 + 'foo'.rindex(/o/) # => 2 + 'foo'.rindex(/oo/) # => 1 + 'foo'.rindex(/ooo/) # => nil + +When +offset+ is non-negative, it specifies the maximum starting position in the +string to end the search: + + 'foo'.rindex('o', 0) # => nil + 'foo'.rindex('o', 1) # => 1 + 'foo'.rindex('o', 2) # => 2 + 'foo'.rindex('o', 3) # => 2 + +With negative integer argument +offset+, +selects the search position by counting backward from the end of +self+: + + 'foo'.rindex('o', -1) # => 2 + 'foo'.rindex('o', -2) # => 1 + 'foo'.rindex('o', -3) # => nil + 'foo'.rindex('o', -4) # => nil + +The last match means starting at the possible last position, not +the last of longest matches: + + 'foo'.rindex(/o+/) # => 2 + $~ # => #<MatchData "o"> + +To get the last longest match, combine with negative lookbehind: + + 'foo'.rindex(/(?<!o)o+/) # => 1 + $~ # => #<MatchData "oo"> + +Or String#index with negative lookforward. + + 'foo'.index(/o+(?!.*o)/) # => 1 + $~ # => #<MatchData "oo"> + +Related: see {Querying}[rdoc-ref:String@Querying]. diff --git a/doc/string/rjust.rdoc b/doc/string/rjust.rdoc index 24e7bf3159..acd3f198d4 100644 --- a/doc/string/rjust.rdoc +++ b/doc/string/rjust.rdoc @@ -1,16 +1,17 @@ Returns a right-justified copy of +self+. -If integer argument +size+ is greater than the size (in characters) of +self+, -returns a new string of length +size+ that is a copy of +self+, +If integer argument +width+ is greater than the size (in characters) of +self+, +returns a new string of length +width+ that is a copy of +self+, right justified and padded on the left with +pad_string+: 'hello'.rjust(10) # => " hello" 'hello '.rjust(10) # => " hello " 'hello'.rjust(10, 'ab') # => "ababahello" - 'тест'.rjust(10) # => " тест" 'こんにちは'.rjust(10) # => " こんにちは" -If +size+ is not greater than the size of +self+, returns a copy of +self+: +If <tt>width <= self.size</tt>, returns a copy of +self+: 'hello'.rjust(5, 'ab') # => "hello" 'hello'.rjust(1, 'ab') # => "hello" + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/rpartition.rdoc b/doc/string/rpartition.rdoc index d24106fb9f..eed03949a5 100644 --- a/doc/string/rpartition.rdoc +++ b/doc/string/rpartition.rdoc @@ -1,24 +1,47 @@ Returns a 3-element array of substrings of +self+. -Matches a pattern against +self+, scanning backwards from the end. -The pattern is: +Searches +self+ for a match of +pattern+, seeking the _last_ match. -- +string_or_regexp+ itself, if it is a Regexp. -- <tt>Regexp.quote(string_or_regexp)</tt>, if +string_or_regexp+ is a string. +If +pattern+ is not matched, returns the array: -If the pattern is matched, returns pre-match, last-match, post-match: + ["", "", self.dup] - 'hello'.rpartition('l') # => ["hel", "l", "o"] - 'hello'.rpartition('ll') # => ["he", "ll", "o"] - 'hello'.rpartition('h') # => ["", "h", "ello"] - 'hello'.rpartition('o') # => ["hell", "o", ""] - 'hello'.rpartition(/l+/) # => ["hel", "l", "o"] - 'hello'.rpartition('') # => ["hello", "", ""] - 'тест'.rpartition('т') # => ["тес", "т", ""] - 'こんにちは'.rpartition('に') # => ["こん", "に", "ちは"] +If +pattern+ is matched, returns the array: -If the pattern is not matched, returns two empty strings and a copy of +self+: + [pre_match, last_match, post_match] - 'hello'.rpartition('x') # => ["", "", "hello"] +where: -Related: String#partition, String#split. +- +last_match+ is the last-found matching substring. +- +pre_match+ and +post_match+ are the preceding and following substrings. + +The pattern used is: + +- +pattern+ itself, if it is a Regexp. +- <tt>Regexp.quote(pattern)</tt>, if +pattern+ is a string. + +Note that in the examples below, a returned string <tt>'hello'</tt> is a copy of +self+, not +self+. + +If +pattern+ is a Regexp, searches for the last matching substring +(also setting {matched-data global variables}[rdoc-ref:language/globals.md@Matched+Data]): + + 'hello'.rpartition(/l/) # => ["hel", "l", "o"] + 'hello'.rpartition(/ll/) # => ["he", "ll", "o"] + 'hello'.rpartition(/h/) # => ["", "h", "ello"] + 'hello'.rpartition(/o/) # => ["hell", "o", ""] + 'hello'.rpartition(//) # => ["hello", "", ""] + 'hello'.rpartition(/x/) # => ["", "", "hello"] + 'こんにちは'.rpartition(/に/) # => ["こん", "に", "ちは"] + +If +pattern+ is not a Regexp, converts it to a string (if it is not already one), +then searches for the last matching substring +(and does _not_ set {matched-data global variables}[rdoc-ref:language/globals.md@Matched+Data]): + + 'hello'.rpartition('l') # => ["hel", "l", "o"] + 'hello'.rpartition('ll') # => ["he", "ll", "o"] + 'hello'.rpartition('h') # => ["", "h", "ello"] + 'hello'.rpartition('o') # => ["hell", "o", ""] + 'hello'.rpartition('') # => ["hello", "", ""] + 'こんにちは'.rpartition('に') # => ["こん", "に", "ちは"] + +Related: see {Converting to Non-String}[rdoc-ref:String@Converting+to+Non--5CString]. diff --git a/doc/string/scan.rdoc b/doc/string/scan.rdoc new file mode 100644 index 0000000000..04a2b02ff4 --- /dev/null +++ b/doc/string/scan.rdoc @@ -0,0 +1,35 @@ +Matches a pattern against +self+: + +- If +pattern+ is a Regexp, the pattern used is +pattern+ itself. +- If +pattern+ is a string, the pattern used is <tt>Regexp.quote(pattern)</tt>. + +Generates a collection of matching results +and updates {regexp-related global variables}[rdoc-ref:Regexp@Global+Variables]: + +- If the pattern contains no groups, each result is a matched substring. +- If the pattern contains groups, each result is an array + containing a matched substring for each group. + +With no block given, returns an array of the results: + + 'cruel world'.scan(/\w+/) # => ["cruel", "world"] + 'cruel world'.scan(/.../) # => ["cru", "el ", "wor"] + 'cruel world'.scan(/(...)/) # => [["cru"], ["el "], ["wor"]] + 'cruel world'.scan(/(..)(..)/) # => [["cr", "ue"], ["l ", "wo"]] + 'こんにちは'.scan(/../) # => ["こん", "にち"] + 'abracadabra'.scan('ab') # => ["ab", "ab"] + 'abracadabra'.scan('nosuch') # => [] + +With a block given, calls the block with each result; returns +self+: + + 'cruel world'.scan(/\w+/) {|w| p w } + # => "cruel" + # => "world" + 'cruel world'.scan(/(.)(.)/) {|x, y| p [x, y] } + # => ["c", "r"] + # => ["u", "e"] + # => ["l", " "] + # => ["w", "o"] + # => ["r", "l"] + +Related: see {Converting to Non-String}[rdoc-ref:String@Converting+to+Non--5CString]. diff --git a/doc/string/scrub.rdoc b/doc/string/scrub.rdoc index 1a5b1c79d0..5ace376cdb 100644 --- a/doc/string/scrub.rdoc +++ b/doc/string/scrub.rdoc @@ -1,25 +1,22 @@ Returns a copy of +self+ with each invalid byte sequence replaced by the given +replacement_string+. -With no block given and no argument, replaces each invalid sequence -with the default replacement string -(<tt>"�"</tt> for a Unicode encoding, <tt>'?'</tt> otherwise): +With no block given, replaces each invalid sequence +with the given +default_replacement_string+ +(by default, <tt>"�"</tt> for a Unicode encoding, <tt>'?'</tt> otherwise): - s = "foo\x81\x81bar" - s.scrub # => "foo��bar" + "foo\x81\x81bar"scrub # => "foo��bar" + "foo\x81\x81bar".force_encoding('US-ASCII').scrub # => "foo??bar" + "foo\x81\x81bar".scrub('xyzzy') # => "fooxyzzyxyzzybar" -With no block given and argument +replacement_string+ given, -replaces each invalid sequence with that string: +With a block given, calls the block with each invalid sequence, +and replaces that sequence with the return value of the block: - "foo\x81\x81bar".scrub('xyzzy') # => "fooxyzzyxyzzybar" + "foo\x81\x81bar".scrub {|sequence| p sequence; 'XYZZY' } # => "fooXYZZYXYZZYbar" -With a block given, replaces each invalid sequence with the value -of the block: - - "foo\x81\x81bar".scrub {|bytes| p bytes; 'XYZZY' } - # => "fooXYZZYXYZZYbar" - -Output: +Output : "\x81" "\x81" + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/split.rdoc b/doc/string/split.rdoc index 131c14b83f..1aee1de0a4 100644 --- a/doc/string/split.rdoc +++ b/doc/string/split.rdoc @@ -1,99 +1,101 @@ -Returns an array of substrings of +self+ -that are the result of splitting +self+ +Creates an array of substrings by splitting +self+ at each occurrence of the given field separator +field_sep+. -When +field_sep+ is <tt>$;</tt>: +With no arguments given, +splits using the field separator <tt>$;</tt>, +whose default value is +nil+. -- If <tt>$;</tt> is +nil+ (its default value), - the split occurs just as if +field_sep+ were given as a space character - (see below). +With no block given, returns the array of substrings: -- If <tt>$;</tt> is a string, - the split occurs just as if +field_sep+ were given as that string - (see below). + 'abracadabra'.split('a') # => ["", "br", "c", "d", "br"] -When +field_sep+ is <tt>' '</tt> and +limit+ is +0+ (its default value), -the split occurs at each sequence of whitespace: +When +field_sep+ is +nil+ or <tt>' '</tt> (a single space), +splits at each sequence of whitespace: - 'abc def ghi'.split(' ') # => ["abc", "def", "ghi"] - "abc \n\tdef\t\n ghi".split(' ') # => ["abc", "def", "ghi"] - 'abc def ghi'.split(' ') # => ["abc", "def", "ghi"] + 'foo bar baz'.split(nil) # => ["foo", "bar", "baz"] + 'foo bar baz'.split(' ') # => ["foo", "bar", "baz"] + "foo \n\tbar\t\n baz".split(' ') # => ["foo", "bar", "baz"] + 'foo bar baz'.split(' ') # => ["foo", "bar", "baz"] ''.split(' ') # => [] -When +field_sep+ is a string different from <tt>' '</tt> -and +limit+ is +0+, -the split occurs at each occurrence of +field_sep+; -trailing empty substrings are not returned: +When +field_sep+ is an empty string, +splits at every character: - 'abracadabra'.split('ab') # => ["", "racad", "ra"] - 'aaabcdaaa'.split('a') # => ["", "", "", "bcd"] - ''.split('a') # => [] - '3.14159'.split('1') # => ["3.", "4", "59"] - '!@#$%^$&*($)_+'.split('$') # => ["!@#", "%^", "&*(", ")_+"] - 'тест'.split('т') # => ["", "ес"] - 'こんにちは'.split('に') # => ["こん", "ちは"] + 'abracadabra'.split('') # => ["a", "b", "r", "a", "c", "a", "d", "a", "b", "r", "a"] + ''.split('') # => [] + 'こんにちは'.split('') # => ["こ", "ん", "に", "ち", "は"] -When +field_sep+ is a Regexp and +limit+ is +0+, -the split occurs at each occurrence of a match; -trailing empty substrings are not returned: +When +field_sep+ is a non-empty string and different from <tt>' '</tt> (a single space), +uses that string as the separator: + + 'abracadabra'.split('a') # => ["", "br", "c", "d", "br"] + 'abracadabra'.split('ab') # => ["", "racad", "ra"] + ''.split('a') # => [] + 'こんにちは'.split('に') # => ["こん", "ちは"] + +When +field_sep+ is a Regexp, +splits at each occurrence of a matching substring: 'abracadabra'.split(/ab/) # => ["", "racad", "ra"] - 'aaabcdaaa'.split(/a/) # => ["", "", "", "bcd"] - 'aaabcdaaa'.split(//) # => ["a", "a", "a", "b", "c", "d", "a", "a", "a"] '1 + 1 == 2'.split(/\W+/) # => ["1", "1", "2"] + 'abracadabra'.split(//) # => ["a", "b", "r", "a", "c", "a", "d", "a", "b", "r", "a"] -If the \Regexp contains groups, their matches are also included +If the \Regexp contains groups, their matches are included in the returned array: '1:2:3'.split(/(:)()()/, 2) # => ["1", ":", "", "", "2:3"] -As seen above, if +limit+ is +0+, -trailing empty substrings are not returned: +Argument +limit+ sets a limit on the size of the returned array; +it also determines whether trailing empty strings are included in the returned array. - 'aaabcdaaa'.split('a') # => ["", "", "", "bcd"] +When +limit+ is zero, +there is no limit on the size of the array, +but trailing empty strings are omitted: -If +limit+ is positive integer +n+, no more than <tt>n - 1-</tt> -splits occur, so that at most +n+ substrings are returned, -and trailing empty substrings are included: + 'abracadabra'.split('', 0) # => ["a", "b", "r", "a", "c", "a", "d", "a", "b", "r", "a"] + 'abracadabra'.split('a', 0) # => ["", "br", "c", "d", "br"] # Empty string after last 'a' omitted. - 'aaabcdaaa'.split('a', 1) # => ["aaabcdaaa"] - 'aaabcdaaa'.split('a', 2) # => ["", "aabcdaaa"] - 'aaabcdaaa'.split('a', 5) # => ["", "", "", "bcd", "aa"] - 'aaabcdaaa'.split('a', 7) # => ["", "", "", "bcd", "", "", ""] - 'aaabcdaaa'.split('a', 8) # => ["", "", "", "bcd", "", "", ""] +When +limit+ is a positive integer, +there is a limit on the size of the array (no more than <tt>n - 1</tt> splits occur), +and trailing empty strings are included: -Note that if +field_sep+ is a \Regexp containing groups, -their matches are in the returned array, but do not count toward the limit. + 'abracadabra'.split('', 3) # => ["a", "b", "racadabra"] + 'abracadabra'.split('a', 3) # => ["", "br", "cadabra"] + 'abracadabra'.split('', 30) # => ["a", "b", "r", "a", "c", "a", "d", "a", "b", "r", "a", ""] + 'abracadabra'.split('a', 30) # => ["", "br", "c", "d", "br", ""] + 'abracadabra'.split('', 1) # => ["abracadabra"] + 'abracadabra'.split('a', 1) # => ["abracadabra"] -If +limit+ is negative, it behaves the same as if +limit+ was zero, -meaning that there is no limit, -and trailing empty substrings are included: +When +limit+ is negative, +there is no limit on the size of the array, +and trailing empty strings are omitted: - 'aaabcdaaa'.split('a', -1) # => ["", "", "", "bcd", "", "", ""] + 'abracadabra'.split('', -1) # => ["a", "b", "r", "a", "c", "a", "d", "a", "b", "r", "a", ""] + 'abracadabra'.split('a', -1) # => ["", "br", "c", "d", "br", ""] If a block is given, it is called with each substring and returns +self+: - 'abc def ghi'.split(' ') {|substring| p substring } + 'foo bar baz'.split(' ') {|substring| p substring } + +Output : + + "foo" + "bar" + "baz" -Output: +Note that the above example is functionally equivalent to: - "abc" - "def" - "ghi" - => "abc def ghi" + 'foo bar baz'.split(' ').each {|substring| p substring } -Note that the above example is functionally the same as calling +#each+ after -+#split+ and giving the same block. However, the above example has better -performance because it avoids the creation of an intermediate array. Also, -note the different return values. +Output : - 'abc def ghi'.split(' ').each {|substring| p substring } + "foo" + "bar" + "baz" -Output: +But the latter: - "abc" - "def" - "ghi" - => ["abc", "def", "ghi"] +- Has poorer performance because it creates an intermediate array. +- Returns an array (instead of +self+). -Related: String#partition, String#rpartition. +Related: see {Converting to Non-String}[rdoc-ref:String@Converting+to+Non--5CString]. diff --git a/doc/string/squeeze.rdoc b/doc/string/squeeze.rdoc new file mode 100644 index 0000000000..1a38c08b32 --- /dev/null +++ b/doc/string/squeeze.rdoc @@ -0,0 +1,33 @@ +Returns a copy of +self+ with each tuple (doubling, tripling, etc.) of specified characters +"squeezed" down to a single character. + +The tuples to be squeezed are specified by arguments +selectors+, +each of which is a string; +see {Character Selectors}[rdoc-ref:character_selectors.rdoc@Character+Selectors]. + +A single argument may be a single character: + + 'Noooooo!'.squeeze('o') # => "No!" + 'foo bar baz'.squeeze(' ') # => "foo bar baz" + 'Mississippi'.squeeze('s') # => "Misisippi" + 'Mississippi'.squeeze('p') # => "Mississipi" + 'Mississippi'.squeeze('x') # => "Mississippi" # Unused selector character is ignored. + 'бессонница'.squeeze('с') # => "бесонница" + 'бессонница'.squeeze('н') # => "бессоница" + +A single argument may be a string of characters: + + 'Mississippi'.squeeze('sp') # => "Misisipi" + 'Mississippi'.squeeze('ps') # => "Misisipi" # Order doesn't matter. + 'Mississippi'.squeeze('nonsense') # => "Misisippi" # Unused selector characters are ignored. + +A single argument may be a range of characters: + + 'Mississippi'.squeeze('a-p') # => "Mississipi" + 'Mississippi'.squeeze('q-z') # => "Misisippi" + 'Mississippi'.squeeze('a-z') # => "Misisipi" + +Multiple arguments are allowed; +see {Multiple Character Selectors}[rdoc-ref:character_selectors.rdoc@Multiple+Character+Selectors]. + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/start_with_p.rdoc b/doc/string/start_with_p.rdoc index 5d1f9f9543..f78edc7fa3 100644 --- a/doc/string/start_with_p.rdoc +++ b/doc/string/start_with_p.rdoc @@ -1,10 +1,9 @@ -Returns whether +self+ starts with any of the given +string_or_regexp+. +Returns whether +self+ starts with any of the given +patterns+. -Matches patterns against the beginning of +self+. -For each given +string_or_regexp+, the pattern is: +For each argument, the pattern used is: -- +string_or_regexp+ itself, if it is a Regexp. -- <tt>Regexp.quote(string_or_regexp)</tt>, if +string_or_regexp+ is a string. +- The pattern itself, if it is a Regexp. +- <tt>Regexp.quote(pattern)</tt>, if it is a string. Returns +true+ if any pattern matches the beginning, +false+ otherwise: @@ -12,7 +11,6 @@ Returns +true+ if any pattern matches the beginning, +false+ otherwise: 'hello'.start_with?(/H/i) # => true 'hello'.start_with?('heaven', 'hell') # => true 'hello'.start_with?('heaven', 'paradise') # => false - 'тест'.start_with?('т') # => true 'こんにちは'.start_with?('こ') # => true -Related: String#end_with?. +Related: see {Querying}[rdoc-ref:String@Querying]. diff --git a/doc/string/sub.rdoc b/doc/string/sub.rdoc new file mode 100644 index 0000000000..ff051ea177 --- /dev/null +++ b/doc/string/sub.rdoc @@ -0,0 +1,33 @@ +Returns a copy of self, possibly with a substring replaced. + +Argument +pattern+ may be a string or a Regexp; +argument +replacement+ may be a string or a Hash. + +Varying types for the argument values makes this method very versatile. + +Below are some simple examples; for many more examples, +see {Substitution Methods}[rdoc-ref:String@Substitution+Methods]. + +With arguments +pattern+ and string +replacement+ given, +replaces the first matching substring with the given replacement string: + + s = 'abracadabra' # => "abracadabra" + s.sub('bra', 'xyzzy') # => "axyzzycadabra" + s.sub(/bra/, 'xyzzy') # => "axyzzycadabra" + s.sub('nope', 'xyzzy') # => "abracadabra" + +With arguments +pattern+ and hash +replacement+ given, +replaces the first matching substring with a value from the given replacement hash, or removes it: + + h = {'a' => 'A', 'b' => 'B', 'c' => 'C'} + s.sub('b', h) # => "aBracadabra" + s.sub(/b/, h) # => "aBracadabra" + s.sub(/d/, h) # => "abracaabra" # 'd' removed. + +With argument +pattern+ and a block given, +calls the block with each matching substring; +replaces that substring with the block’s return value: + + s.sub('b') {|match| match.upcase } # => "aBracadabra" + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/succ.rdoc b/doc/string/succ.rdoc new file mode 100644 index 0000000000..1b4b936a8e --- /dev/null +++ b/doc/string/succ.rdoc @@ -0,0 +1,52 @@ +Returns the successor to +self+. The successor is calculated by +incrementing characters. + +The first character to be incremented is the rightmost alphanumeric: +or, if no alphanumerics, the rightmost character: + + 'THX1138'.succ # => "THX1139" + '<<koala>>'.succ # => "<<koalb>>" + '***'.succ # => '**+' + 'こんにちは'.succ # => "こんにちば" + +The successor to a digit is another digit, "carrying" to the next-left +character for a "rollover" from 9 to 0, and prepending another digit +if necessary: + + '00'.succ # => "01" + '09'.succ # => "10" + '99'.succ # => "100" + +The successor to a letter is another letter of the same case, +carrying to the next-left character for a rollover, +and prepending another same-case letter if necessary: + + 'aa'.succ # => "ab" + 'az'.succ # => "ba" + 'zz'.succ # => "aaa" + 'AA'.succ # => "AB" + 'AZ'.succ # => "BA" + 'ZZ'.succ # => "AAA" + +The successor to a non-alphanumeric character is the next character +in the underlying character set's collating sequence, +carrying to the next-left character for a rollover, +and prepending another character if necessary: + + s = 0.chr * 3 # => "\x00\x00\x00" + s.succ # => "\x00\x00\x01" + s = 255.chr * 3 # => "\xFF\xFF\xFF" + s.succ # => "\x01\x00\x00\x00" + +Carrying can occur between and among mixtures of alphanumeric characters: + + s = 'zz99zz99' # => "zz99zz99" + s.succ # => "aaa00aa00" + s = '99zz99zz' # => "99zz99zz" + s.succ # => "100aa00aa" + +The successor to an empty +String+ is a new empty +String+: + + ''.succ # => "" + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/sum.rdoc b/doc/string/sum.rdoc index 5de24e6402..22045e5f4d 100644 --- a/doc/string/sum.rdoc +++ b/doc/string/sum.rdoc @@ -1,11 +1,12 @@ -Returns a basic +n+-bit checksum of the characters in +self+; +Returns a basic +n+-bit {checksum}[https://en.wikipedia.org/wiki/Checksum] of the characters in +self+; the checksum is the sum of the binary value of each byte in +self+, modulo <tt>2**n - 1</tt>: 'hello'.sum # => 532 'hello'.sum(4) # => 4 'hello'.sum(64) # => 532 - 'тест'.sum # => 1405 'こんにちは'.sum # => 2582 This is not a particularly strong checksum. + +Related: see {Querying}[rdoc-ref:String@Querying]. diff --git a/doc/string/swapcase.rdoc b/doc/string/swapcase.rdoc new file mode 100644 index 0000000000..4353c8528a --- /dev/null +++ b/doc/string/swapcase.rdoc @@ -0,0 +1,31 @@ +Returns a string containing the characters in +self+, with cases reversed: + +- Each uppercase character is downcased. +- Each lowercase character is upcased. + +Examples: + + 'Hello'.swapcase # => "hELLO" + 'Straße'.swapcase # => "sTRASSE" + 'RubyGems.org'.swapcase # => "rUBYgEMS.ORG" + +The sizes of +self+ and the upcased result may differ: + + s = 'Straße' + s.size # => 6 + s.swapcase # => "sTRASSE" + s.swapcase.size # => 7 + +Some characters (and some character sets) do not have upcase and downcase versions; +see {Case Mapping}[rdoc-ref:case_mapping.rdoc]: + + s = '1, 2, 3, ...' + s.swapcase == s # => true + s = 'こんにちは' + s.swapcase == s # => true + +The casing is affected by the given +mapping+, +which may be +:ascii+, +:fold+, or +:turkic+; +see {Case Mappings}[rdoc-ref:case_mapping.rdoc@Case+Mappings]. + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/unicode_normalize.rdoc b/doc/string/unicode_normalize.rdoc new file mode 100644 index 0000000000..5f733c0fb8 --- /dev/null +++ b/doc/string/unicode_normalize.rdoc @@ -0,0 +1,28 @@ +Returns a copy of +self+ with +{Unicode normalization}[https://unicode.org/reports/tr15] applied. + +Argument +form+ must be one of the following symbols +(see {Unicode normalization forms}[https://unicode.org/reports/tr15/#Norm_Forms]): + +- +:nfc+: Canonical decomposition, followed by canonical composition. +- +:nfd+: Canonical decomposition. +- +:nfkc+: Compatibility decomposition, followed by canonical composition. +- +:nfkd+: Compatibility decomposition. + +The encoding of +self+ must be one of: + +- <tt>Encoding::UTF_8</tt>. +- <tt>Encoding::UTF_16BE</tt>. +- <tt>Encoding::UTF_16LE</tt>. +- <tt>Encoding::UTF_32BE</tt>. +- <tt>Encoding::UTF_32LE</tt>. +- <tt>Encoding::GB18030</tt>. +- <tt>Encoding::UCS_2BE</tt>. +- <tt>Encoding::UCS_4BE</tt>. + +Examples: + + "a\u0300".unicode_normalize # => "à" # Lowercase 'a' with grave accens. + "a\u0300".unicode_normalize(:nfd) # => "à" # Same. + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/upcase.rdoc b/doc/string/upcase.rdoc new file mode 100644 index 0000000000..ad859e8973 --- /dev/null +++ b/doc/string/upcase.rdoc @@ -0,0 +1,27 @@ +Returns a new string containing the upcased characters in +self+: + + 'hello'.upcase # => "HELLO" + 'straße'.upcase # => "STRASSE" + 'привет'.upcase # => "ПРИВЕТ" + 'RubyGems.org'.upcase # => "RUBYGEMS.ORG" + +The sizes of +self+ and the upcased result may differ: + + s = 'Straße' + s.size # => 6 + s.upcase # => "STRASSE" + s.upcase.size # => 7 + +Some characters (and some character sets) do not have upcase and downcase versions; +see {Case Mapping}[rdoc-ref:case_mapping.rdoc]: + + s = '1, 2, 3, ...' + s.upcase == s # => true + s = 'こんにちは' + s.upcase == s # => true + +The casing is affected by the given +mapping+, +which may be +:ascii+, +:fold+, or +:turkic+; +see {Case Mappings}[rdoc-ref:case_mapping.rdoc@Case+Mappings]. + +Related: see {Converting to New String}[rdoc-ref:String@Converting+to+New+String]. diff --git a/doc/string/upto.rdoc b/doc/string/upto.rdoc new file mode 100644 index 0000000000..f860fe84fe --- /dev/null +++ b/doc/string/upto.rdoc @@ -0,0 +1,38 @@ +With a block given, calls the block with each +String+ value +returned by successive calls to String#succ; +the first value is +self+, the next is <tt>self.succ</tt>, and so on; +the sequence terminates when value +other_string+ is reached; +returns +self+: + + a = [] + 'a'.upto('f') {|c| a.push(c) } + a # => ["a", "b", "c", "d", "e", "f"] + + a = [] + 'Ж'.upto('П') {|c| a.push(c) } + a # => ["Ж", "З", "И", "Й", "К", "Л", "М", "Н", "О", "П"] + + a = [] + 'よ'.upto('ろ') {|c| a.push(c) } + a # => ["よ", "ら", "り", "る", "れ", "ろ"] + + a = [] + 'a8'.upto('b6') {|c| a.push(c) } + a # => ["a8", "a9", "b0", "b1", "b2", "b3", "b4", "b5", "b6"] + +If argument +exclusive+ is given as a truthy object, the last value is omitted: + + a = [] + 'a'.upto('f', true) {|c| a.push(c) } + a # => ["a", "b", "c", "d", "e"] + +If +other_string+ would not be reached, does not call the block: + + '25'.upto('5') {|s| fail s } + 'aa'.upto('a') {|s| fail s } + +With no block given, returns a new Enumerator: + + 'a8'.upto('b6') # => #<Enumerator: "a8":upto("b6")> + +Related: see {Iterating}[rdoc-ref:String@Iterating]. diff --git a/doc/string/valid_encoding_p.rdoc b/doc/string/valid_encoding_p.rdoc new file mode 100644 index 0000000000..e1db55174a --- /dev/null +++ b/doc/string/valid_encoding_p.rdoc @@ -0,0 +1,8 @@ +Returns whether +self+ is encoded correctly: + + s = 'Straße' + s.valid_encoding? # => true + s.encoding # => #<Encoding:UTF-8> + s.force_encoding(Encoding::ASCII).valid_encoding? # => false + +Related: see {Querying}[rdoc-ref:String@Querying]. diff --git a/doc/stringio/each_byte.rdoc b/doc/stringio/each_byte.rdoc new file mode 100644 index 0000000000..65e81c53a7 --- /dev/null +++ b/doc/stringio/each_byte.rdoc @@ -0,0 +1,34 @@ +With a block given, calls the block with each remaining byte in the stream; +positions the stream at end-of-file; +returns +self+: + + bytes = [] + strio = StringIO.new('hello') # Five 1-byte characters. + strio.each_byte {|byte| bytes.push(byte) } + strio.eof? # => true + bytes # => [104, 101, 108, 108, 111] + bytes = [] + strio = StringIO.new('тест') # Four 2-byte characters. + strio.each_byte {|byte| bytes.push(byte) } + bytes # => [209, 130, 208, 181, 209, 129, 209, 130] + bytes = [] + strio = StringIO.new('こんにちは') # Five 3-byte characters. + strio.each_byte {|byte| bytes.push(byte) } + bytes # => [227, 129, 147, 227, 130, 147, 227, 129, 171, 227, 129, 161, 227, 129, 175] + +The position in the stream matters: + + bytes = [] + strio = StringIO.new('こんにちは') + strio.getc # => "こ" + strio.pos # => 3 # 3-byte character was read. + strio.each_byte {|byte| bytes.push(byte) } + bytes # => [227, 130, 147, 227, 129, 171, 227, 129, 161, 227, 129, 175] + +If at end-of-file, does not call the block: + + strio.eof? # => true + strio.each_byte {|byte| fail 'Boo!' } + strio.eof? # => true + +With no block given, returns a new {Enumerator}[rdoc-ref:Enumerator]. diff --git a/doc/stringio/each_char.rdoc b/doc/stringio/each_char.rdoc new file mode 100644 index 0000000000..d0b5e4082c --- /dev/null +++ b/doc/stringio/each_char.rdoc @@ -0,0 +1,34 @@ +With a block given, calls the block with each remaining character in the stream; +positions the stream at end-of-file; +returns +self+: + + chars = [] + strio = StringIO.new('hello') + strio.each_char {|char| chars.push(char) } + strio.eof? # => true + chars # => ["h", "e", "l", "l", "o"] + chars = [] + strio = StringIO.new('тест') + strio.each_char {|char| chars.push(char) } + chars # => ["т", "е", "с", "т"] + chars = [] + strio = StringIO.new('こんにちは') + strio.each_char {|char| chars.push(char) } + chars # => ["こ", "ん", "に", "ち", "は"] + +Stream position matters: + + chars = [] + strio = StringIO.new('こんにちは') + strio.getc # => "こ" + strio.pos # => 3 # 3-byte character was read. + strio.each_char {|char| chars.push(char) } + chars # => ["ん", "に", "ち", "は"] + +When at end-of-stream does not call the block: + + strio.eof? # => true + strio.each_char {|char| fail 'Boo!' } + strio.eof? # => true + +With no block given, returns a new {Enumerator}[rdoc-ref:Enumerator]. diff --git a/doc/stringio/each_codepoint.rdoc b/doc/stringio/each_codepoint.rdoc new file mode 100644 index 0000000000..ede16de599 --- /dev/null +++ b/doc/stringio/each_codepoint.rdoc @@ -0,0 +1,36 @@ +With a block given, calls the block with each successive codepoint from self; +sets the position to end-of-stream; +returns +self+. + +Each codepoint is the integer value for a character; returns self: + + codepoints = [] + strio = StringIO.new('hello') + strio.each_codepoint {|codepoint| codepoints.push(codepoint) } + strio.eof? # => true + codepoints # => [104, 101, 108, 108, 111] + codepoints = [] + strio = StringIO.new('тест') + strio.each_codepoint {|codepoint| codepoints.push(codepoint) } + codepoints # => [1090, 1077, 1089, 1090] + codepoints = [] + strio = StringIO.new('こんにちは') + strio.each_codepoint {|codepoint| codepoints.push(codepoint) } + codepoints # => [12371, 12435, 12395, 12385, 12399] + +Position in the stream matters: + + codepoints = [] + strio = StringIO.new('こんにちは') + strio.getc # => "こ" + strio.pos # => 3 + strio.each_codepoint {|codepoint| codepoints.push(codepoint) } + codepoints # => [12435, 12395, 12385, 12399] + +When at end-of-stream, the block is not called: + + strio.eof? # => true + strio.each_codepoint {|codepoint| fail 'Boo!' } + strio.eof? # => true + +With no block given, returns a new {Enumerator}[rdoc-ref:Enumerator]. diff --git a/doc/stringio/each_line.md b/doc/stringio/each_line.md new file mode 100644 index 0000000000..e29640a12a --- /dev/null +++ b/doc/stringio/each_line.md @@ -0,0 +1,189 @@ +With a block given calls the block with each remaining line (see "Position" below) in the stream; +returns `self`. + +Leaves stream position at end-of-stream. + +**No Arguments** + +With no arguments given, +reads lines using the default record separator +(global variable `$/`, whose initial value is `"\n"`). + +```ruby +strio = StringIO.new(TEXT) +strio.each_line {|line| p line } +strio.eof? # => true +``` + +Output: + +``` +"First line\n" +"Second line\n" +"\n" +"Fourth line\n" +"Fifth line\n" +``` + +**Argument `sep`** + +With only string argument `sep` given, +reads lines using that string as the record separator: + +```ruby +strio = StringIO.new(TEXT) +strio.each_line(' ') {|line| p line } +``` + +Output: + +``` +"First " +"line\nSecond " +"line\n\nFourth " +"line\nFifth " +"line\n" +``` + +**Argument `limit`** + +With only integer argument `limit` given, +reads lines using the default record separator; +also limits the size (in characters) of each line to the given limit: + +```ruby +strio = StringIO.new(TEXT) +strio.each_line(10) {|line| p line } +``` + +Output: + +``` +"First line" +"\n" +"Second lin" +"e\n" +"\n" +"Fourth lin" +"e\n" +"Fifth line" +"\n" +``` + +**Arguments `sep` and `limit`** + +With arguments `sep` and `limit` both given, +honors both: + +```ruby +strio = StringIO.new(TEXT) +strio.each_line(' ', 10) {|line| p line } +``` + +Output: + +``` +"First " +"line\nSecon" +"d " +"line\n\nFour" +"th " +"line\nFifth" +" " +"line\n" +``` + +**Position** + +As stated above, method `each` _remaining_ line in the stream. + +In the examples above each `strio` object starts with its position at beginning-of-stream; +but in other cases the position may be anywhere (see StringIO#pos): + +```ruby +strio = StringIO.new(TEXT) +strio.pos = 30 # Set stream position to character 30. +strio.each_line {|line| p line } +``` + +Output: + +``` +" line\n" +"Fifth line\n" +``` + +In all the examples above, the stream position is at the beginning of a character; +in other cases, that need not be so: + +```ruby +s = 'こんにちは' # Five 3-byte characters. +strio = StringIO.new(s) +strio.pos = 3 # At beginning of second character. +strio.each_line {|line| p line } +strio.pos = 4 # At second byte of second character. +strio.each_line {|line| p line } +strio.pos = 5 # At third byte of second character. +strio.each_line {|line| p line } +``` + +Output: + +``` +"んにちは" +"\x82\x93にちは" +"\x93にちは" +``` + +**Special Record Separators** + +Like some methods in class `IO`, StringIO.each honors two special record separators; +see {Special Line Separators}[https://docs.ruby-lang.org/en/master/IO.html#class-IO-label-Special+Line+Separator+Values]. + +```ruby +strio = StringIO.new(TEXT) +strio.each_line('') {|line| p line } # Read as paragraphs (separated by blank lines). +``` + +Output: + +``` +"First line\nSecond line\n\n" +"Fourth line\nFifth line\n" +``` + +```ruby +strio = StringIO.new(TEXT) +strio.each_line(nil) {|line| p line } # "Slurp"; read it all. +``` + +Output: + +``` +"First line\nSecond line\n\nFourth line\nFifth line\n" +``` + +**Keyword Argument `chomp`** + +With keyword argument `chomp` given as `true` (the default is `false`), +removes trailing newline (if any) from each line: + +```ruby +strio = StringIO.new(TEXT) +strio.each_line(chomp: true) {|line| p line } +``` + +Output: + +``` +"First line" +"Second line" +"" +"Fourth line" +"Fifth line" +``` + +With no block given, returns a new {Enumerator}[https://docs.ruby-lang.org/en/master/Enumerator.html]. + + +Related: StringIO.each_byte, StringIO.each_char, StringIO.each_codepoint. diff --git a/doc/stringio/getbyte.rdoc b/doc/stringio/getbyte.rdoc new file mode 100644 index 0000000000..5e524941bc --- /dev/null +++ b/doc/stringio/getbyte.rdoc @@ -0,0 +1,31 @@ +Reads and returns the next integer byte (not character) from the stream: + + s = 'foo' + s.bytes # => [102, 111, 111] + strio = StringIO.new(s) + strio.getbyte # => 102 + strio.getbyte # => 111 + strio.getbyte # => 111 + +Returns +nil+ if at end-of-stream: + + strio.eof? # => true + strio.getbyte # => nil + +Returns a byte, not a character: + + s = 'Привет' + s.bytes + # => [208, 159, 209, 128, 208, 184, 208, 178, 208, 181, 209, 130] + strio = StringIO.new(s) + strio.getbyte # => 208 + strio.getbyte # => 159 + + s = 'こんにちは' + s.bytes + # => [227, 129, 147, 227, 130, 147, 227, 129, 171, 227, 129, 161, 227, 129, 175] + strio = StringIO.new(s) + strio.getbyte # => 227 + strio.getbyte # => 129 + +Related: #each_byte, #ungetbyte, #getc. diff --git a/doc/stringio/getc.rdoc b/doc/stringio/getc.rdoc new file mode 100644 index 0000000000..b2ab46843c --- /dev/null +++ b/doc/stringio/getc.rdoc @@ -0,0 +1,34 @@ +Reads and returns the next character (or byte; see below) from the stream: + + strio = StringIO.new('foo') + strio.getc # => "f" + strio.getc # => "o" + strio.getc # => "o" + +Returns +nil+ if at end-of-stream: + + strio.eof? # => true + strio.getc # => nil + +Returns characters, not bytes: + + strio = StringIO.new('Привет') + strio.getc # => "П" + strio.getc # => "р" + + strio = StringIO.new('こんにちは') + strio.getc # => "こ" + strio.getc # => "ん" + +In each of the examples above, the stream is positioned at the beginning of a character; +in other cases that need not be true: + + strio = StringIO.new('こんにちは') # Five 3-byte characters. + strio.pos = 3 # => 3 # At beginning of second character; returns character. + strio.getc # => "ん" + strio.pos = 4 # => 4 # At second byte of second character; returns byte. + strio.getc # => "\x82" + strio.pos = 5 # => 5 # At third byte of second character; returns byte. + strio.getc # => "\x93" + +Related: #getbyte, #putc, #ungetc. diff --git a/doc/stringio/gets.rdoc b/doc/stringio/gets.rdoc new file mode 100644 index 0000000000..bbefeb008a --- /dev/null +++ b/doc/stringio/gets.rdoc @@ -0,0 +1,99 @@ +Reads and returns a line from the stream; +returns +nil+ if at end-of-stream. + +Side effects: + +- Increments stream position by the number of bytes read. +- Assigns the return value to global variable <tt>$_</tt>. + +With no arguments given, reads a line using the default record separator +(global variable <tt>$/</tt>,* whose initial value is <tt>"\n"</tt>): + + strio = StringIO.new(TEXT) + strio.pos # => 0 + strio.gets # => "First line\n" + strio.pos # => 11 + $_ # => "First line\n" + strio.gets # => "Second line\n" + strio.read # => "\nFourth line\nFifth line\n" + strio.eof? # => true + strio.gets # => nil + + strio = StringIO.new('Привет') # Six 2-byte characters + strio.pos # => 0 + strio.gets # => "Привет" + strio.pos # => 12 + +<b>Argument +sep+</b> + +With only string argument +sep+ given, reads a line using that string as the record separator: + + strio = StringIO.new(TEXT) + strio.gets(' ') # => "First " + strio.gets(' ') # => "line\nSecond " + strio.gets(' ') # => "line\n\nFourth " + +<b>Argument +limit+</b> + +With only integer argument +limit+ given, +reads a line using the default record separator; +limits the size (in characters) of each line to the given limit: + + strio = StringIO.new(TEXT) + strio.gets(10) # => "First line" + strio.gets(10) # => "\n" + strio.gets(10) # => "Second lin" + strio.gets(10) # => "e\n" + +<b>Arguments +sep+ and +limit+</b> + +With arguments +sep+ and +limit+ both given, honors both: + + strio = StringIO.new(TEXT) + strio.gets(' ', 10) # => "First " + strio.gets(' ', 10) # => "line\nSecon" + strio.gets(' ', 10) # => "d " + +<b>Position</b> + +As stated above, method +gets+ reads and returns the next line in the stream. + +In the examples above each +strio+ object starts with its position at beginning-of-stream; +but in other cases the position may be anywhere: + + strio = StringIO.new(TEXT) + strio.pos = 12 + strio.gets # => "econd line\n" + +The position need not be at a character boundary: + + strio = StringIO.new('Привет') # Six 2-byte characters. + strio.pos = 2 # At beginning of second character. + strio.gets # => "ривет" + strio.pos = 3 # In middle of second character. + strio.gets # => "\x80ивет" + +<b>Special Record Separators</b> + +Like some methods in class IO, method +gets+ honors two special record separators; +see {Special Line Separators}[https://docs.ruby-lang.org/en/master/IO.html#class-IO-label-Special+Line+Separator+Values]: + + strio = StringIO.new(TEXT) + strio.gets('') # Read "paragraph" (up to empty line). + # => "First line\nSecond line\n\n" + + strio = StringIO.new(TEXT) + strio.gets(nil) # "Slurp": read all. + # => "First line\nSecond line\n\nFourth line\nFifth line\n" + +<b>Keyword Argument +chomp+</b> + +With keyword argument +chomp+ given as +true+ (the default is +false+), +removes the trailing newline (if any) from the returned line: + + strio = StringIO.new(TEXT) + strio.gets # => "First line\n" + strio.gets(chomp: true) # => "Second line" + +Related: #each_line, #readlines, +{Kernel#puts}[rdoc-ref:Kernel#puts]. diff --git a/doc/stringio/pread.rdoc b/doc/stringio/pread.rdoc new file mode 100644 index 0000000000..2dcbc18ad8 --- /dev/null +++ b/doc/stringio/pread.rdoc @@ -0,0 +1,65 @@ +**Note**: \Method +pread+ is different from other reading methods +in that it does not modify +self+ in any way; +thus, multiple threads may read safely from the same stream. + +Reads up to +maxlen+ bytes from the stream, +beginning at 0-based byte offset +offset+; +returns a string containing the read bytes. + +The returned string: + +- Contains +maxlen+ bytes from the stream, if available; + otherwise contains all available bytes. +- Has encoding +Encoding::ASCII_8BIT+. + +With only arguments +maxlen+ and +offset+ given, +returns a new string: + + english = 'Hello' # Five 1-byte characters. + strio = StringIO.new(english) + strio.pread(3, 0) # => "Hel" + strio.pread(3, 2) # => "llo" + strio.pread(0, 0) # => "" + strio.pread(50, 0) # => "Hello" + strio.pread(50, 2) # => "llo" + strio.pread(50, 4) # => "o" + strio.pread(0, 0).encoding + # => #<Encoding:BINARY (ASCII-8BIT)> + + russian = 'Привет' # Six 2-byte characters. + strio = StringIO.new(russian) + strio.pread(50, 0) # All 12 bytes. + # => "\xD0\x9F\xD1\x80\xD0\xB8\xD0\xB2\xD0\xB5\xD1\x82" + strio.pread(3, 0) # => "\xD0\x9F\xD1" + strio.pread(3, 3) # => "\x80\xD0\xB8" + strio.pread(0, 0).encoding + # => #<Encoding:BINARY (ASCII-8BIT)> + + japanese = 'こんにちは' # Five 3-byte characters. + strio = StringIO.new(japanese) + strio.pread(50, 0) # All 15 bytes. + # => "\xE3\x81\x93\xE3\x82\x93\xE3\x81\xAB\xE3\x81\xA1\xE3\x81\xAF" + strio.pread(6, 0) # => "\xE3\x81\x93\xE3\x82\x93" + strio.pread(1, 2) # => "\x93" + strio.pread(0, 0).encoding + # => #<Encoding:BINARY (ASCII-8BIT)> + +Raises an exception if +offset+ is out-of-range: + + strio = StringIO.new(english) + strio.pread(5, 50) # Raises EOFError: end of file reached + +With string argument +out_string+ given: + +- Reads as above. +- Overwrites the content of +out_string+ with the read bytes. + +Examples: + + out_string = 'Will be overwritten' + out_string.encoding # => #<Encoding:UTF-8> + result = StringIO.new(english).pread(50, 0, out_string) + result.__id__ == out_string.__id__ # => true + out_string # => "Hello" + out_string.encoding # => #<Encoding:BINARY (ASCII-8BIT)> + diff --git a/doc/stringio/putc.rdoc b/doc/stringio/putc.rdoc new file mode 100644 index 0000000000..4636ffa0db --- /dev/null +++ b/doc/stringio/putc.rdoc @@ -0,0 +1,82 @@ +Replaces one or more bytes at position +pos+ +with bytes of the given argument; +advances the position by the count of bytes written; +returns the argument. + +\StringIO object for 1-byte characters. + + strio = StringIO.new('foo') + strio.pos # => 0 + +With 1-byte argument, replaces one byte: + + strio.putc('b') + strio.string # => "boo" + strio.pos # => 1 + strio.putc('a') # => "a" + strio.string # => "bao" + strio.pos # => 2 + strio.putc('r') # => "r" + strio.string # => "bar" + strio.pos # => 3 + strio.putc('n') # => "n" + strio.string # => "barn" + strio.pos # => 4 + +Fills with null characters if necessary: + + strio.pos = 6 + strio.putc('x') # => "x" + strio.string # => "barn\u0000\u0000x" + strio.pos # => 7 + +With integer argument, replaces one byte with the low-order byte of the integer: + + strio = StringIO.new('foo') + strio.putc(70) + strio.string # => "Foo" + strio.putc(79) + strio.string # => "FOo" + strio.putc(79 + 1024) + strio.string # => "FOO" + +\StringIO object for Multi-byte characters: + + greek = 'αβγδε' # Five 2-byte characters. + strio = StringIO.new(greek) + strio.string# => "αβγδε" + strio.string.b # => "\xCE\xB1\xCE\xB2\xCE\xB3\xCE\xB4\xCE\xB5" + strio.string.bytesize # => 10 + strio.string.chars # => ["α", "β", "γ", "δ", "ε"] + strio.string.size # => 5 + +With 1-byte argument, replaces one byte of the string: + + strio.putc(' ') # 1-byte ascii space. + strio.pos # => 1 + strio.string # => " \xB1βγδε" + strio.string.b # => " \xB1\xCE\xB2\xCE\xB3\xCE\xB4\xCE\xB5" + strio.string.bytesize # => 10 + strio.string.chars # => [" ", "\xB1", "β", "γ", "δ", "ε"] + strio.string.size # => 6 + + strio.putc(' ') + strio.pos # => 2 + strio.string # => " βγδε" + strio.string.b # => " \xCE\xB2\xCE\xB3\xCE\xB4\xCE\xB5" + strio.string.bytesize # => 10 + strio.string.chars # => [" ", " ", "β", "γ", "δ", "ε"] + strio.string.size # => 6 + +With 2-byte argument, replaces two bytes of the string: + + strio.rewind + strio.putc('α') + strio.pos # => 2 + strio.string # => "αβγδε" + strio.string.b # => "\xCE\xB1\xCE\xB2\xCE\xB3\xCE\xB4\xCE\xB5" + strio.string.bytesize # => 10 + strio.string.chars # => ["α", "β", "γ", "δ", "ε"] + strio.string.size # => 5 + +Related: #getc, #ungetc. diff --git a/doc/stringio/read.rdoc b/doc/stringio/read.rdoc new file mode 100644 index 0000000000..46b9fa349f --- /dev/null +++ b/doc/stringio/read.rdoc @@ -0,0 +1,83 @@ +Reads and returns a string containing bytes read from the stream, +beginning at the current position; +advances the position by the count of bytes read. + +With no arguments given, +reads all remaining bytes in the stream; +returns a new string containing bytes read: + + strio = StringIO.new('Hello') # Five 1-byte characters. + strio.read # => "Hello" + strio.pos # => 5 + strio.read # => "" + StringIO.new('').read # => "" + +With non-negative argument +maxlen+ given, +reads +maxlen+ bytes as available; +returns a new string containing the bytes read, or +nil+ if none: + + strio.rewind + strio.read(3) # => "Hel" + strio.read(3) # => "lo" + strio.read(3) # => nil + + russian = 'Привет' # Six 2-byte characters. + russian.b + # => "\xD0\x9F\xD1\x80\xD0\xB8\xD0\xB2\xD0\xB5\xD1\x82" + strio = StringIO.new(russian) + strio.read(6) # => "\xD0\x9F\xD1\x80\xD0\xB8" + strio.read(6) # => "\xD0\xB2\xD0\xB5\xD1\x82" + strio.read(6) # => nil + + japanese = 'こんにちは' + japanese.b + # => "\xE3\x81\x93\xE3\x82\x93\xE3\x81\xAB\xE3\x81\xA1\xE3\x81\xAF" + strio = StringIO.new(japanese) + strio.read(9) # => "\xE3\x81\x93\xE3\x82\x93\xE3\x81\xAB" + strio.read(9) # => "\xE3\x81\xA1\xE3\x81\xAF" + strio.read(9) # => nil + +With argument +max_len+ as +nil+ and string argument +out_string+ given, +reads the remaining bytes in the stream; +clears +out_string+ and writes the bytes into it; +returns +out_string+: + + out_string = 'Will be overwritten' + strio = StringIO.new('Hello') + strio.read(nil, out_string) # => "Hello" + strio.read(nil, out_string) # => "" + +With non-negative argument +maxlen+ and string argument +out_string+ given, +reads the +maxlen bytes from the stream, as availble; +clears +out_string+ and writes the bytes into it; +returns +out_string+ if any bytes were read, or +nil+ if none: + + out_string = 'Will be overwritten' + strio = StringIO.new('Hello') + strio.read(3, out_string) # => "Hel" + strio.read(3, out_string) # => "lo" + strio.read(3, out_string) # => nil + + out_string = 'Will be overwritten' + strio = StringIO.new(russian) + strio.read(6, out_string) # => "При" + strio.read(6, out_string) # => "вет" + strio.read(6, out_string) # => nil + strio.rewind + russian.b + # => "\xD0\x9F\xD1\x80\xD0\xB8\xD0\xB2\xD0\xB5\xD1\x82" + strio.read(3) # => "\xD0\x9F\xD1" + strio.read(3) # => "\x80\xD0\xB8" + + out_string = 'Will be overwritten' + strio = StringIO.new(japanese) + strio.read(9, out_string) # => "こんに" + strio.read(9, out_string) # => "ちは" + strio.read(9, out_string) # => nil + strio.rewind + japanese.b + # => "\xE3\x81\x93\xE3\x82\x93\xE3\x81\xAB\xE3\x81\xA1\xE3\x81\xAF" + strio.read(4) # => "\xE3\x81\x93\xE3" + strio.read(4) # => "\x82\x93\xE3\x81" + +Related: #gets, #readlines. diff --git a/doc/stringio/size.rdoc b/doc/stringio/size.rdoc new file mode 100644 index 0000000000..9323adf8c3 --- /dev/null +++ b/doc/stringio/size.rdoc @@ -0,0 +1,5 @@ +Returns the number of bytes in the string in +self+: + + StringIO.new('hello').size # => 5 # Five 1-byte characters. + StringIO.new('тест').size # => 8 # Four 2-byte characters. + StringIO.new('こんにちは').size # => 15 # Five 3-byte characters. diff --git a/doc/stringio/stringio.md b/doc/stringio/stringio.md new file mode 100644 index 0000000000..8931d1c30c --- /dev/null +++ b/doc/stringio/stringio.md @@ -0,0 +1,700 @@ +\Class \StringIO supports accessing a string as a stream, +similar in some ways to [class IO][io class]. + +You can create a \StringIO instance using: + +- StringIO.new: returns a new \StringIO object containing the given string. +- StringIO.open: passes a new \StringIO object to the given block. + +Like an \IO stream, a \StringIO stream has certain properties: + +- **Read/write mode**: whether the stream may be read, written, appended to, etc.; + see [Read/Write Mode][read/write mode]. +- **Data mode**: text-only or binary; + see [Data Mode][data mode]. +- **Encodings**: internal and external encodings; + see [Encodings][encodings]. +- **Position**: where in the stream the next read or write is to occur; + see [Position][position]. +- **Line number**: a special, line-oriented, "position" (different from the position mentioned above); + see [Line Number][line number]. +- **Open/closed**: whether the stream is open or closed, for reading or writing. + see [Open/Closed Streams][open/closed streams]. +- **BOM**: byte mark order; + see [Byte Order Mark][bom (byte order mark)]. + +## About the Examples + +Examples on this page assume that \StringIO has been required: + +```ruby +require 'stringio' +``` + +And that this constant has been defined: + +```ruby +TEXT = <<EOT +First line +Second line + +Fourth line +Fifth line +EOT +``` + +## Stream Properties + +### Read/Write Mode + +#### Summary + +| Mode | Initial Clear? | Read | Write | +|:--------------------------:|:--------------:|:--------:|:--------:| +| <tt>'r'</tt>: read-only | No | Anywhere | Error | +| <tt>'w'</tt>: write-only | Yes | Error | Anywhere | +| <tt>'a'</tt>: append-only | No | Error | End only | +| <tt>'r+'</tt>: read/write | No | Anywhere | Anywhere | +| <tt>'w+'</tt>: read-write | Yes | Anywhere | Anywhere | +| <tt>'a+'</tt>: read/append | No | Anywhere | End only | + +Each section below describes a read/write mode. + +Any of the modes may be given as a string or as file constants; +example: + +```ruby +strio = StringIO.new('foo', 'a') +strio = StringIO.new('foo', File::WRONLY | File::APPEND) +``` + +#### `'r'`: Read-Only + +Mode specified as one of: + +- String: `'r'`. +- Constant: `File::RDONLY`. + +Initial state: + +```ruby +strio = StringIO.new('foobarbaz', 'r') +strio.pos # => 0 # Beginning-of-stream. +strio.string # => "foobarbaz" # Not cleared. +``` + +May be read anywhere: + +```ruby +strio.gets(3) # => "foo" +strio.gets(3) # => "bar" +strio.pos = 9 +strio.gets(3) # => nil +``` + +May not be written: + +```ruby +strio.write('foo') # Raises IOError: not opened for writing +``` + +#### `'w'`: Write-Only + +Mode specified as one of: + +- String: `'w'`. +- Constant: `File::WRONLY`. + +Initial state: + +```ruby +strio = StringIO.new('foo', 'w') +strio.pos # => 0 # Beginning of stream. +strio.string # => "" # Initially cleared. +``` + +May be written anywhere (even past end-of-stream): + +```ruby +strio.write('foobar') +strio.string # => "foobar" +strio.rewind +strio.write('FOO') +strio.string # => "FOObar" +strio.pos = 3 +strio.write('BAR') +strio.string # => "FOOBAR" +strio.pos = 9 +strio.write('baz') +strio.string # => "FOOBAR\u0000\u0000\u0000baz" # Null-padded. +``` + +May not be read: + +```ruby +strio.read # Raises IOError: not opened for reading +``` + +#### `'a'`: Append-Only + +Mode specified as one of: + +- String: `'a'`. +- Constant: `File::WRONLY | File::APPEND`. + +Initial state: + +```ruby +strio = StringIO.new('foo', 'a') +strio.pos # => 0 # Beginning-of-stream. +strio.string # => "foo" # Not cleared. +``` + +May be written only at the end; position does not affect writing: + +```ruby +strio.write('bar') +strio.string # => "foobar" +strio.write('baz') +strio.string # => "foobarbaz" +strio.pos = 400 +strio.write('bat') +strio.string # => "foobarbazbat" +``` + +May not be read: + +```ruby +strio.gets # Raises IOError: not opened for reading +``` + +#### `'r+'`: Read/Write + +Mode specified as one of: + +- String: `'r+'`. +- Constant: `File::RDRW`. + +Initial state: + +```ruby +strio = StringIO.new('foobar', 'r+') +strio.pos # => 0 # Beginning-of-stream. +strio.string # => "foobar" # Not cleared. +``` + +May be written anywhere (even past end-of-stream): + +```ruby +strio.write('FOO') +strio.string # => "FOObar" +strio.write('BAR') +strio.string # => "FOOBAR" +strio.write('BAZ') +strio.string # => "FOOBARBAZ" +strio.pos = 12 +strio.write('BAT') +strio.string # => "FOOBARBAZ\u0000\u0000\u0000BAT" # Null padded. +``` + +May be read anywhere: + +```ruby +strio.pos = 0 +strio.gets(3) # => "FOO" +strio.pos = 6 +strio.gets(3) # => "BAZ" +strio.pos = 400 +strio.gets(3) # => nil +``` + +#### `'w+'`: Read/Write (Initially Clear) + +Mode specified as one of: + +- String: `'w+'`. +- Constant: `File::RDWR | File::TRUNC`. + +Initial state: + +```ruby +strio = StringIO.new('foo', 'w+') +strio.pos # => 0 # Beginning-of-stream. +strio.string # => "" # Truncated. +``` + +May be written anywhere (even past end-of-stream): + +```ruby +strio.write('foobar') +strio.string # => "foobar" +strio.rewind +strio.write('FOO') +strio.string # => "FOObar" +strio.write('BAR') +strio.string # => "FOOBAR" +strio.write('BAZ') +strio.string # => "FOOBARBAZ" +strio.pos = 12 +strio.write('BAT') +strio.string # => "FOOBARBAZ\u0000\u0000\u0000BAT" # Null-padded. +``` + +May be read anywhere: + +```ruby +strio.rewind +strio.gets(3) # => "FOO" +strio.gets(3) # => "BAR" +strio.pos = 12 +strio.gets(3) # => "BAT" +strio.pos = 400 +strio.gets(3) # => nil +``` + +#### `'a+'`: Read/Append + +Mode specified as one of: + +- String: `'a+'`. +- Constant: `File::RDWR | File::APPEND`. + +Initial state: + +```ruby +strio = StringIO.new('foo', 'a+') +strio.pos # => 0 # Beginning-of-stream. +strio.string # => "foo" # Not cleared. +``` + +May be written only at the end; #rewind; position does not affect writing: + +```ruby +strio.write('bar') +strio.string # => "foobar" +strio.write('baz') +strio.string # => "foobarbaz" +strio.pos = 400 +strio.write('bat') +strio.string # => "foobarbazbat" +``` + +May be read anywhere: + +```ruby +strio.rewind +strio.gets(3) # => "foo" +strio.gets(3) # => "bar" +strio.pos = 9 +strio.gets(3) # => "bat" +strio.pos = 400 +strio.gets(3) # => nil +``` + +### Data Mode + +To specify whether the stream is to be treated as text or as binary data, +either of the following may be suffixed to any of the string read/write modes above: + +- `'t'`: Text; + initializes the encoding as Encoding::UTF_8. +- `'b'`: Binary; + initializes the encoding as Encoding::ASCII_8BIT. + +If neither is given, the stream defaults to text data. + +Examples: + +```ruby +strio = StringIO.new('foo', 'rt') +strio.external_encoding # => #<Encoding:UTF-8> +data = "\u9990\u9991\u9992\u9993\u9994" +strio = StringIO.new(data, 'rb') +strio.external_encoding # => #<Encoding:BINARY (ASCII-8BIT)> +``` + +When the data mode is specified, the read/write mode may not be omitted: + +```ruby +StringIO.new(data, 'b') # Raises ArgumentError: invalid access mode b +``` + +A text stream may be changed to binary by calling instance method #binmode; +a binary stream may not be changed to text. + +### Encodings + +A stream has an encoding; see [Encodings][encodings document]. + +The initial encoding for a new or re-opened stream depends on its [data mode][data mode]: + +- Text: `Encoding::UTF_8`. +- Binary: `Encoding::ASCII_8BIT`. + +These instance methods are relevant: + +- #external_encoding: returns the current encoding of the stream as an `Encoding` object. +- #internal_encoding: returns +nil+; a stream does not have an internal encoding. +- #set_encoding: sets the encoding for the stream. +- #set_encoding_by_bom: sets the encoding for the stream to the stream's BOM (byte order mark). + +Examples: + +```ruby +strio = StringIO.new('foo', 'rt') # Text mode. +strio.external_encoding # => #<Encoding:UTF-8> +data = "\u9990\u9991\u9992\u9993\u9994" +strio = StringIO.new(data, 'rb') # Binary mode. +strio.external_encoding # => #<Encoding:BINARY (ASCII-8BIT)> +strio = StringIO.new('foo') +strio.external_encoding # => #<Encoding:UTF-8> +strio.set_encoding('US-ASCII') +strio.external_encoding # => #<Encoding:US-ASCII> +``` + +### Position + +A stream has a _position_, and integer offset (in bytes) into the stream. +The initial position of a stream is zero. + +#### Getting and Setting the Position + +Each of these methods initializes (to zero) the position of a new or re-opened stream: + +- ::new: returns a new stream. +- ::open: passes a new stream to the block. +- #reopen: re-initializes the stream. + +Each of these methods queries, gets, or sets the position, without otherwise changing the stream: + +- #eof?: returns whether the position is at end-of-stream. +- #pos: returns the position. +- #pos=: sets the position. +- #rewind: sets the position to zero. +- #seek: sets the position. + +Examples: + +```ruby +strio = StringIO.new('foobar') +strio.pos # => 0 +strio.pos = 3 +strio.pos # => 3 +strio.eof? # => false +strio.rewind +strio.pos # => 0 +strio.seek(0, IO::SEEK_END) +strio.pos # => 6 +strio.eof? # => true +``` + +#### Position Before and After Reading + +Except for #pread, a stream reading method (see [Basic Reading][basic reading]) +begins reading at the current position. + +Except for #pread, a read method advances the position past the read substring. + +Examples: + +```ruby +strio = StringIO.new(TEXT) +strio.string # => "First line\nSecond line\n\nFourth line\nFifth line\n" +strio.pos # => 0 +strio.getc # => "F" +strio.pos # => 1 +strio.gets # => "irst line\n" +strio.pos # => 11 +strio.pos = 24 +strio.gets # => "Fourth line\n" +strio.pos # => 36 + +strio = StringIO.new('тест') # Four 2-byte characters. +strio.pos = 0 # At first byte of first character. +strio.read # => "тест" +strio.pos = 1 # At second byte of first character. +strio.read # => "\x82ест" +strio.pos = 2 # At first of second character. +strio.read # => "ест" + +strio = StringIO.new(TEXT) +strio.pos = 15 +a = [] +strio.each_line {|line| a.push(line) } +a # => ["nd line\n", "\n", "Fourth line\n", "Fifth line\n"] +strio.pos # => 47 ## End-of-stream. +``` + +#### Position Before and After Writing + +Each of these methods begins writing at the current position, +and advances the position to the end of the written substring: + +- #putc: writes the given character. +- #write: writes the given objects as strings. +- [Kernel#puts][kernel#puts]: writes given objects as strings, each followed by newline. + +Examples: + +```ruby +strio = StringIO.new('foo') +strio.pos # => 0 +strio.putc('b') +strio.string # => "boo" +strio.pos # => 1 +strio.write('r') +strio.string # => "bro" +strio.pos # => 2 +strio.puts('ew') +strio.string # => "brew\n" +strio.pos # => 5 +strio.pos = 8 +strio.write('foo') +strio.string # => "brew\n\u0000\u0000\u0000foo" +strio.pos # => 11 +``` + +Each of these methods writes _before_ the current position, and decrements the position +so that the written data is next to be read: + +- #ungetbyte: unshifts the given byte. +- #ungetc: unshifts the given character. + +Examples: + +```ruby +strio = StringIO.new('foo') +strio.pos = 2 +strio.ungetc('x') +strio.pos # => 1 +strio.string # => "fxo" +strio.ungetc('x') +strio.pos # => 0 +strio.string # => "xxo" +``` + +This method does not affect the position: + +- #truncate: truncates the stream's string to the given size. + +Examples: + +```ruby +strio = StringIO.new('foobar') +strio.pos # => 0 +strio.truncate(3) +strio.string # => "foo" +strio.pos # => 0 +strio.pos = 500 +strio.truncate(0) +strio.string # => "" +strio.pos # => 500 +``` + +### Line Number + +A stream has a line number, which initially is zero: + +- Method #lineno returns the line number. +- Method #lineno= sets the line number. + +The line number can be affected by reading (but never by writing); +in general, the line number is incremented each time the record separator (default: `"\n"`) is read. + +Examples: + +```ruby +strio = StringIO.new(TEXT) +strio.string # => "First line\nSecond line\n\nFourth line\nFifth line\n" +strio.lineno # => 0 +strio.gets # => "First line\n" +strio.lineno # => 1 +strio.getc # => "S" +strio.lineno # => 1 +strio.gets # => "econd line\n" +strio.lineno # => 2 +strio.gets # => "\n" +strio.lineno # => 3 +strio.gets # => "Fourth line\n" +strio.lineno # => 4 +``` + +Setting the position does not affect the line number: + +```ruby +strio.pos = 0 +strio.lineno # => 4 +strio.gets # => "First line\n" +strio.pos # => 11 +strio.lineno # => 5 +``` + +And setting the line number does not affect the position: + +```ruby +strio.lineno = 10 +strio.pos # => 11 +strio.gets # => "Second line\n" +strio.lineno # => 11 +strio.pos # => 23 +``` + +### Open/Closed Streams + +A new stream is open for either reading or writing, and may be open for both; +see [Read/Write Mode][read/write mode]. + +Each of these methods initializes the read/write mode for a new or re-opened stream: + +- ::new: returns a new stream. +- ::open: passes a new stream to the block. +- #reopen: re-initializes the stream. + +Other relevant methods: + +- #close: closes the stream for both reading and writing. +- #close_read: closes the stream for reading. +- #close_write: closes the stream for writing. +- #closed?: returns whether the stream is closed for both reading and writing. +- #closed_read?: returns whether the stream is closed for reading. +- #closed_write?: returns whether the stream is closed for writing. + +### BOM (Byte Order Mark) + +The string provided for ::new, ::open, or #reopen +may contain an optional [BOM][bom] (byte order mark) at the beginning of the string; +the BOM can affect the stream's encoding. + +The BOM (if provided): + +- Is stored as part of the stream's string. +- Does _not_ immediately affect the encoding. +- Is _initially_ considered part of the stream. + +```ruby +utf8_bom = "\xEF\xBB\xBF" +string = utf8_bom + 'foo' +string.bytes # => [239, 187, 191, 102, 111, 111] +strio.string.bytes.take(3) # => [239, 187, 191] # The BOM. +strio = StringIO.new(string, 'rb') +strio.string.bytes # => [239, 187, 191, 102, 111, 111] # BOM is part of the stored string. +strio.external_encoding # => #<Encoding:BINARY (ASCII-8BIT)> # Default for a binary stream. +strio.gets # => "\xEF\xBB\xBFfoo" # BOM is part of the stream. +``` + +You can call instance method #set_encoding_by_bom to "activate" the stored BOM; +after doing so the BOM: + +- Is _still_ stored as part of the stream's string. +- _Determines_ (and may have changed) the stream's encoding. +- Is _no longer_ considered part of the stream. + +```ruby +strio.set_encoding_by_bom +strio.string.bytes # => [239, 187, 191, 102, 111, 111] # BOM is still part of the stored string. +strio.external_encoding # => #<Encoding:UTF-8> # The new encoding. +strio.rewind # => 0 +strio.gets # => "foo" # BOM is not part of the stream. +``` + +## Basic Stream \IO + +### Basic Reading + +You can read from the stream using these instance methods: + +- #getbyte: reads and returns the next byte. +- #getc: reads and returns the next character. +- #gets: reads and returns all or part of the next line. +- #read: reads and returns all or part of the remaining data in the stream. +- #readlines: reads the remaining data the stream and returns an array of its lines. +- [Kernel#readline][kernel#readline]: like #gets, but raises an exception if at end-of-stream. + +You can iterate over the stream using these instance methods: + +- #each_byte: reads each remaining byte, passing it to the block. +- #each_char: reads each remaining character, passing it to the block. +- #each_codepoint: reads each remaining codepoint, passing it to the block. +- #each_line: reads all or part of each remaining line, passing the read string to the block + +This instance method is useful in a multi-threaded application: + +- #pread: reads and returns all or part of the stream. + +### Basic Writing + +You can write to the stream, advancing the position, using these instance methods: + +- #putc: writes a given character. +- #write: writes the given objects as strings. +- [Kernel#puts][kernel#puts] writes given objects as strings, each followed by newline. + +You can "unshift" to the stream using these instance methods; +each writes _before_ the current position, and decrements the position +so that the written data is next to be read. + +- #ungetbyte: unshifts the given byte. +- #ungetc: unshifts the given character. + +One more writing method: + +- #truncate: truncates the stream's string to the given size. + +## Line \IO + +Reading: + +- #gets: reads and returns the next line. +- [Kernel#readline][kernel#readline]: like #gets, but raises an exception if at end-of-stream. +- #readlines: reads the remaining data the stream and returns an array of its lines. +- #each_line: reads each remaining line, passing it to the block + +Writing: + +- [Kernel#puts][kernel#puts]: writes given objects, each followed by newline. + +## Character \IO + +Reading: + +- #each_char: reads each remaining character, passing it to the block. +- #getc: reads and returns the next character. + +Writing: + +- #putc: writes the given character. +- #ungetc.: unshifts the given character. + +## Byte \IO + +Reading: + +- #each_byte: reads each remaining byte, passing it to the block. +- #getbyte: reads and returns the next byte. + +Writing: + +- #ungetbyte: unshifts the given byte. + +## Codepoint \IO + +Reading: + +- #each_codepoint: reads each remaining codepoint, passing it to the block. + +[bom]: https://en.wikipedia.org/wiki/Byte_order_mark +[encodings document]: https://docs.ruby-lang.org/en/master/language/encodings_rdoc.html +[io class]: https://docs.ruby-lang.org/en/master/IO.html +[kernel#puts]: https://docs.ruby-lang.org/en/master/Kernel.html#method-i-puts +[kernel#readline]: https://docs.ruby-lang.org/en/master/Kernel.html#method-i-readline + +[basic reading]: rdoc-ref:StringIO@Basic+Reading +[basic writing]: rdoc-ref:StringIO@Basic+Writing +[bom (byte order mark)]: rdoc-ref:StringIO@BOM+-28Byte+Order+Mark-29 +[data mode]: rdoc-ref:StringIO@Data+Mode +[encodings]: rdoc-ref:StringIO@Encodings +[end-of-stream]: rdoc-ref:StringIO@End-of-Stream +[line number]: rdoc-ref:StringIO@Line+Number +[open/closed streams]: rdoc-ref:StringIO@Open-2FClosed+Streams +[position]: rdoc-ref:StringIO@Position +[read/write mode]: rdoc-ref:StringIO@Read-2FWrite+Mode diff --git a/doc/strscan/strscan.md b/doc/strscan/strscan.md index 1211a687c2..385e92f84e 100644 --- a/doc/strscan/strscan.md +++ b/doc/strscan/strscan.md @@ -37,7 +37,7 @@ Some examples here assume that certain helper methods are defined: - `match_values_cleared?(scanner)`: Returns whether the scanner's [match values][9] are cleared. -See examples [here][ext/strscan/helper_methods_md.html]. +See examples at [helper methods](helper_methods.md). ## The `StringScanner` \Object @@ -204,7 +204,7 @@ put_situation(scanner) ## Target Substring -The target substring is the the part of the [stored string][1] +The target substring is the part of the [stored string][1] that extends from the current [byte position][2] to the end of the stored string; it is always either: @@ -417,7 +417,7 @@ Each of these methods returns a captured match value: | Method | Return After Match | Return After No Match | |-----------------|-----------------------------------------|-----------------------| | #size | Count of captured substrings. | +nil+. | -| #[](n) | <tt>n</tt>th captured substring. | +nil+. | +| #\[\](n) | <tt>n</tt>th captured substring. | +nil+. | | #captures | Array of all captured substrings. | +nil+. | | #values_at(*n) | Array of specified captured substrings. | +nil+. | | #named_captures | Hash of named captures. | <tt>{}</tt>. | diff --git a/doc/syntax.rdoc b/doc/syntax.rdoc index cb427b6f0f..a48c83ff15 100644 --- a/doc/syntax.rdoc +++ b/doc/syntax.rdoc @@ -2,6 +2,9 @@ The Ruby syntax is large and is split up into the following sections: +{Code Layout}[rdoc-ref:syntax/layout.rdoc] :: + Breaking code in lines + Literals[rdoc-ref:syntax/literals.rdoc] :: Numbers, Strings, Arrays, Hashes, etc. diff --git a/doc/syntax/assignment.rdoc b/doc/syntax/assignment.rdoc index f45f5bc0ea..3988f82e5f 100644 --- a/doc/syntax/assignment.rdoc +++ b/doc/syntax/assignment.rdoc @@ -9,7 +9,7 @@ Assignment creates a local variable if the variable was not previously referenced. An assignment expression result is always the assigned value, including -{assignment methods}[rdoc-ref:syntax/assignment.rdoc@Assignment+Methods]. +{assignment methods}[rdoc-ref:@Assignment+Methods]. == Local Variable Names @@ -279,7 +279,7 @@ An uninitialized global variable has a value of +nil+. Ruby has some special globals that behave differently depending on context such as the regular expression match variables or that have a side-effect when -assigned to. See the {global variables documentation}[rdoc-ref:globals.rdoc] +assigned to. See the {global variables documentation}[rdoc-ref:language/globals.md] for details. == Assignment Methods diff --git a/doc/syntax/calling_methods.rdoc b/doc/syntax/calling_methods.rdoc index bf5916e99a..76babcc3dc 100644 --- a/doc/syntax/calling_methods.rdoc +++ b/doc/syntax/calling_methods.rdoc @@ -355,9 +355,8 @@ as one argument: # Prints the object itself: # #<Name:0x00007f9d07bca650 @name="Jane Doe"> -This allows to handle one or many arguments polymorphically. Note also that +nil+ -has NilClass#to_a defined to return an empty array, so conditional unpacking is -possible: +This allows to handle one or many arguments polymorphically. Note also that <tt>*nil</tt> +is unpacked to an empty list of arguments, so conditional unpacking is possible: my_method(*(some_arguments if some_condition?)) diff --git a/doc/syntax/comments.rdoc b/doc/syntax/comments.rdoc index 00d19d588a..cb6829a984 100644 --- a/doc/syntax/comments.rdoc +++ b/doc/syntax/comments.rdoc @@ -170,7 +170,7 @@ In this mode, all values assigned to constants are made shareable. # shareable_constant_value: experimental_everything FOO = Set[1, 2, {foo: []}] - # same as FOO = Ractor.make_sharable(...) + # same as FOO = Ractor.make_shareable(...) # OR same as `FOO = Set[1, 2, {foo: [].freeze}.freeze].freeze` var = [{foo: []}] diff --git a/doc/syntax/layout.rdoc b/doc/syntax/layout.rdoc new file mode 100644 index 0000000000..f07447587b --- /dev/null +++ b/doc/syntax/layout.rdoc @@ -0,0 +1,118 @@ += Code Layout + +Expressions in Ruby are separated by line breaks: + + x = 1 + y = 2 + z = x + y + +Line breaks also used as logical separators of the headers of some of control structures from their bodies: + + if z > 3 # line break ends the condition and starts the body + puts "more" + end + + while x < 3 # line break ends the condition and starts the body + x += 1 + end + +<tt>;</tt> can be used as an expressions separator instead of a line break: + + x = 1; y = 2; z = x + y + if z > 3; puts "more"; end + +Traditionally, expressions separated by <tt>;</tt> is used only in short scripts and experiments. + +In some control structures, there is an optional keyword that can be used instead of a line break to separate their elements: + + # if, elsif, until and case ... when: 'then' is an optional separator: + + if z > 3 then puts "more" end + + case x + when Numeric then "number" + when String then "string" + else "object" + end + + # while and until: 'do' is an optional separator + while x < 3 do x +=1 end + +Also, line breaks can be skipped in some places where it doesn't create any ambiguity. Note in the example above: no line break needed before +end+, just as no line break needed after +else+. + +== Breaking expressions in lines + +One expression might be split into several lines when each line can be unambiguously identified as "incomplete" without the next one. + +These works: + + x = # incomplete without something after = + 1 + # incomplete without something after + + 2 + + File.read "test.txt", # incomplete without something after , + enconding: "utf-8" + +These would not: + + # unintended interpretation: + x = 1 # already complete expression + + 2 # interpreted as a separate +2 + + # syntax error: + File.read "test.txt" # already complete expression + , encoding: "utf-8" # attempt to parse as a new expression, SyntaxError + +The exceptions to the rule are lines starting with <tt>.</tt> ("leading dot" style of method calls) or logical operators <tt>&&</tt>/<tt>||</tt> and <tt>and</tt>/<tt>or</tt>: + + # OK, interpreted as a chain of calls + File.read('test.txt') + .strip("\n") + .split("\t") + .sort + + # OK, interpreted as a chain of logical operators: + File.empty?('test.txt') + || File.size('test.txt') < 10 + || File.read('test.txt').strip.empty? + +If the expressions is broken into multiple lines in any of the ways described above, comments between separate lines are allowed: + + sum = base_salary + + # see "yearly bonuses section" + yearly_bonus(year) + + # per-employee coefficient is described + # in another module + personal_coeff(employee) + + # We want to short-circuit on empty files + File.empty?('test.txt') + # Or almost empty ones + || File.size('test.txt') < 10 + # Otherwise we check if it is full of spaces + || File.read('test.txt').strip.empty? + +Finally, the code can explicitly tell Ruby that the expression is continued on the next line with <tt>\\</tt>: + + # Unusual, but works + File.read "test.txt" \ + , encoding: "utf-8" + + # More regular usage (joins the strings on parsing instead + # of concatenating them in runtime, as + would do): + TEXT = "One pretty long line" \ + "one more long line" \ + "one other line of the text" + +The <tt>\\</tt> works as a parse time line break escape, so with it, comments can not be inserted between the lines: + + TEXT = "line 1" \ + # here would be line 2: + "line 2" + + # This is interpreted as if there was no line break where \ is, + # i.e. the same as + TEXT = "line 1" # here would be line 2: + "line 2" + + puts TEXT #=> "line 1" diff --git a/doc/syntax/literals.rdoc b/doc/syntax/literals.rdoc index 46bb7673f3..87a891bf2d 100644 --- a/doc/syntax/literals.rdoc +++ b/doc/syntax/literals.rdoc @@ -3,7 +3,7 @@ Literals create objects you can use in your program. Literals include: * {Boolean and Nil Literals}[#label-Boolean+and+Nil+Literals] -* {Number Literals}[#label-Number+Literals] +* {Numeric Literals}[#label-Numeric+Literals] * {Integer Literals}[#label-Integer+Literals] * {Float Literals}[#label-Float+Literals] @@ -36,7 +36,7 @@ Literals create objects you can use in your program. Literals include: +true+ is a true value. All objects except +nil+ and +false+ evaluate to a true value in conditional expressions. -== Number Literals +== \Numeric Literals === \Integer Literals diff --git a/doc/syntax/methods.rdoc b/doc/syntax/methods.rdoc index 8dafa6bb0c..14810a188f 100644 --- a/doc/syntax/methods.rdoc +++ b/doc/syntax/methods.rdoc @@ -100,6 +100,7 @@ operators. <code>/</code> :: divide <code>%</code> :: modulus division, String#% <code>&</code> :: AND +<code>|</code> :: OR <code>^</code> :: XOR (exclusive OR) <code>>></code> :: right-shift <code><<</code> :: left-shift, append diff --git a/doc/syntax/pattern_matching.rdoc b/doc/syntax/pattern_matching.rdoc index c43919ba14..06aae26d49 100644 --- a/doc/syntax/pattern_matching.rdoc +++ b/doc/syntax/pattern_matching.rdoc @@ -253,11 +253,11 @@ The "rest" part of a pattern also can be bound to a variable: case {a: 1, b: 2} in {a: } | Array + # ^ SyntaxError (variable capture in alternative pattern) "matched: #{a}" else "not matched" end - # SyntaxError (illegal variable in alternative pattern (a)) Variables that start with <code>_</code> are the only exclusions from this rule: diff --git a/doc/syntax/refinements.rdoc b/doc/syntax/refinements.rdoc index 17d5e67c21..4095977284 100644 --- a/doc/syntax/refinements.rdoc +++ b/doc/syntax/refinements.rdoc @@ -212,10 +212,7 @@ all refinements from the same module are active when a refined method When looking up a method for an instance of class +C+ Ruby checks: -* If refinements are active for +C+, in the reverse order they were activated: - * The prepended modules from the refinement for +C+ - * The refinement for +C+ - * The included modules from the refinement for +C+ +* The refinements of +C+, in reverse order of activation * The prepended modules of +C+ * +C+ * The included modules of +C+ diff --git a/doc/yarv_frame_layout.md b/doc/yarv_frame_layout.md deleted file mode 100644 index ea8ad013cf..0000000000 --- a/doc/yarv_frame_layout.md +++ /dev/null @@ -1,77 +0,0 @@ -# YARV Frame Layout - -This document is an introduction to what happens on the VM stack as the VM -services calls. The code holds the ultimate truth for this subject, so beware -that this document can become stale. - -We'll walk through the following program, with explanation at selected points -in execution and abridged disassembly listings: - -```ruby -def foo(x, y) - z = x.casecmp(y) -end - -foo(:one, :two) -``` - -First, after arguments are evaluated and right before the `send` to `foo`: - -``` - ┌────────────┐ - putself │ :two │ - putobject :one 0x2 ├────────────┤ - putobject :two │ :one │ -► send <:foo, argc:2> 0x1 ├────────────┤ - leave │ self │ - 0x0 └────────────┘ -``` - -The `put*` instructions have pushed 3 items onto the stack. It's now time to -add a new control frame for `foo`. The following is the shape of the stack -after one instruction in `foo`: - -``` - cfp->sp=0x8 at this point. - 0x8 ┌────────────┐◄──Stack space for temporaries - │ :one │ live above the environment. - 0x7 ├────────────┤ - getlocal x@0 │ < flags > │ foo's rb_control_frame_t -► getlocal y@1 0x6 ├────────────┤◄──has cfp->ep=0x6 - send <:casecmp, argc:1> │ <no block> │ - dup 0x5 ├────────────┤ The flags, block, and CME triple - setlocal z@2 │ <CME: foo> │ (VM_ENV_DATA_SIZE) form an - leave 0x4 ├────────────┤ environment. They can be used to - │ z (nil) │ figure out what local variables - 0x3 ├────────────┤ are below them. - │ :two │ - 0x2 ├────────────┤ Notice how the arguments, now - │ :one │ locals, never moved. This layout - 0x1 ├────────────┤ allows for argument transfer - │ self │ without copying. - 0x0 └────────────┘ -``` - -Given that locals have lower address than `cfp->ep`, it makes sense then that -`getlocal` in `insns.def` has `val = *(vm_get_ep(GET_EP(), level) - idx);`. -When accessing variables in the immediate scope, where `level=0`, it's -essentially `val = cfp->ep[-idx];`. - -Note that this EP-relative index has a different basis the index that comes -after "@" in disassembly listings. The "@" index is relative to the 0th local -(`x` in this case). - -## Q&A - -Q: It seems that the receiver is always at an offset relative to EP, - like locals. Couldn't we use EP to access it instead of using `cfp->self`? - -A: Not all calls put the `self` in the callee on the stack. Two - examples are `Proc#call`, where the receiver is the Proc object, but `self` - inside the callee is `Proc#receiver`, and `yield`, where the receiver isn't - pushed onto the stack before the arguments. - -Q: Why have `cfp->ep` when it seems that everything is below `cfp->sp`? - -A: In the example, `cfp->ep` points to the stack, but it can also point to the - GC heap. Blocks can capture and evacuate their environment to the heap. diff --git a/doc/zjit.md b/doc/zjit.md deleted file mode 100644 index 4cb8056685..0000000000 --- a/doc/zjit.md +++ /dev/null @@ -1,93 +0,0 @@ -# ZJIT: ADVANCED RUBY JIT PROTOTYPE - -## Build Instructions - -To build ZJIT on macOS: -``` -./autogen.sh -./configure --enable-zjit=dev --prefix=$HOME/.rubies/ruby-zjit --disable-install-doc --with-opt-dir="$(brew --prefix openssl):$(brew --prefix readline):$(brew --prefix libyaml)" -make -j miniruby -``` - -## Useful dev commands - -To view YARV output for code snippets: -``` -./miniruby --dump=insns -e0 -``` - -To run code snippets with ZJIT: -``` -./miniruby --zjit -e0 -``` - -You can also try https://www.rubyexplorer.xyz/ to view Ruby YARV disasm output with syntax highlighting -in a way that can be easily shared with other team members. - -## Testing - -Make sure you have a `--enable-zjit=dev` build, and run `brew install cargo-nextest` first. - -### make zjit-test-all - -This command runs all ZJIT tests: `make zjit-test` and `test/ruby/test_zjit.rb`. - -``` -make zjit-test-all -``` - -### make zjit-test - -This command runs Rust unit tests. - -``` -make zjit-test -``` - -You can also run a single test case by specifying the function name: - -``` -make zjit-test ZJIT_TESTS=test_putobject -``` - -If you expect that your changes cause tests to fail and they do, you can have -`expect-test` fix the expected value for you by putting `UPDATE_EXPECT=1` -before your test command, like so: - -``` -UPDATE_EXPECT=1 make zjit-test ZJIT_TESTS=test_putobject -``` - -Test changes will be reviewed alongside code changes. - -<details> - -<summary>Setting up zjit-test</summary> - -ZJIT uses `cargo-nextest` for Rust unit tests instead of `cargo test`. -`cargo-nextest` runs each test in its own process, which is valuable since -CRuby only supports booting once per process, and most APIs are not thread -safe. Use `brew install cargo-nextest` to install it on macOS, otherwise, refer -to <https://nexte.st/docs/installation/pre-built-binaries/> for installation -instructions. - -Since it uses Cargo, you'll also need a `configure --enable-zjit=dev ...` build -for `make zjit-test`. Since the tests need to link against CRuby, directly -calling `cargo test`, or `cargo nextest` likely won't build. Make sure to -use `make`. - -</details> - -### test/ruby/test\_zjit.rb - -This command runs Ruby execution tests. - -``` -make test-all TESTS="test/ruby/test_zjit.rb" -``` - -You can also run a single test case by matching the method name: - -``` -make test-all TESTS="test/ruby/test_zjit.rb -n TestZJIT#test_putobject" -``` |