diff options
Diffstat (limited to 'doc/contributing')
| -rw-r--r-- | doc/contributing/building_ruby.md | 190 | ||||
| -rw-r--r-- | doc/contributing/documentation_guide.md | 133 | ||||
| -rw-r--r-- | doc/contributing/glossary.md | 2 | ||||
| -rw-r--r-- | doc/contributing/testing_ruby.md | 30 |
4 files changed, 268 insertions, 87 deletions
diff --git a/doc/contributing/building_ruby.md b/doc/contributing/building_ruby.md index d4cedbcb69..ce844b5026 100644 --- a/doc/contributing/building_ruby.md +++ b/doc/contributing/building_ruby.md @@ -8,27 +8,30 @@ For RubyGems, you will also need: - * OpenSSL 1.1.x or 3.0.x / LibreSSL - * libyaml 0.1.7 or later - * zlib + * [OpenSSL] 1.1.x or 3.0.x / [LibreSSL] + * [libyaml] 0.1.7 or later + * [zlib] If you want to build from the git repository, you will also need: - * autoconf - 2.67 or later - * gperf - 3.1 or later + * [autoconf] - 2.67 or later + * [gperf] - 3.1 or later * Usually unneeded; only if you edit some source files using gperf - * ruby - 2.5 or later - * We can upgrade this version to system ruby version of the latest Ubuntu LTS. + * ruby - 3.0 or later + * We can upgrade this version to system ruby version of the latest + Ubuntu LTS. 2. Install optional, recommended dependencies: - * libffi (to build fiddle) - * gmp (if you with to accelerate Bignum operations) - * libexecinfo (FreeBSD) - * rustc - 1.58.0 or later (if you wish to build [YJIT](/doc/yjit/yjit.md)) + * [libffi] (to build fiddle) + * [gmp] (if you with to accelerate Bignum operations) + * [rustc] - 1.58.0 or later, if you wish to build + [YJIT](rdoc-ref:RubyVM::YJIT). - If you installed the libraries needed for extensions (openssl, readline, libyaml, zlib) into other than the OS default place, - typically using Homebrew on macOS, add `--with-EXTLIB-dir` options to `CONFIGURE_ARGS` environment variable. + If you installed the libraries needed for extensions (openssl, readline, + libyaml, zlib) into other than the OS default place, typically using + Homebrew on macOS, add `--with-EXTLIB-dir` options to `CONFIGURE_ARGS` + environment variable. ``` shell export CONFIGURE_ARGS="" @@ -37,33 +40,46 @@ done ``` +[OpenSSL]: https://www.openssl.org +[LibreSSL]: https://www.libressl.org +[libyaml]: https://github.com/yaml/libyaml/ +[zlib]: https://www.zlib.net +[autoconf]: https://www.gnu.org/software/autoconf/ +[gperf]: https://www.gnu.org/software/gperf/ +[libffi]: https://sourceware.org/libffi/ +[gmp]: https://gmplib.org +[rustc]: https://www.rust-lang.org + ## Quick start guide 1. Download ruby source code: + Select one of the below. + 1. Build from the tarball: - Download the latest tarball from [ruby-lang.org](https://www.ruby-lang.org/en/downloads/) and - extract it. Example for Ruby 3.0.2: + Download the latest tarball from [Download Ruby] page and extract + it. Example for Ruby 3.0.2: - ``` shell - tar -xzf ruby-3.0.2.tar.gz - cd ruby-3.0.2 - ``` + ``` shell + tar -xzf ruby-3.0.2.tar.gz + cd ruby-3.0.2 + ``` 2. Build from the git repository: - Checkout the CRuby source code: + Checkout the CRuby source code: - ``` shell - git clone https://github.com/ruby/ruby.git - ``` + ``` shell + git clone https://github.com/ruby/ruby.git + cd ruby + ``` - Generate the configure file: + Generate the configure file: - ``` shell - ./autogen.sh - ``` + ``` shell + ./autogen.sh + ``` 2. Create a `build` directory separate from the source directory: @@ -71,7 +87,8 @@ mkdir build && cd build ``` - While it's not necessary to build in a separate directory, it's good practice to do so. + While it's not necessary to build in a separate directory, it's good + practice to do so. 3. We'll install Ruby in `~/.rubies/ruby-master`, so create the directory: @@ -85,19 +102,41 @@ ../configure --prefix="${HOME}/.rubies/ruby-master" ``` - - If you are frequently building Ruby, add the `--disable-install-doc` flag to not build documentation which will speed up the build process. + - Also `-C` (or `--config-cache`) would reduce time to configure from the + next time. 5. Build Ruby: ``` shell - make install + make ``` 6. [Run tests](testing_ruby.md) to confirm your build succeeded. +7. Install Ruby: + + ``` shell + make install + ``` + + - If you need to run `make install` with `sudo` and want to avoid document + generation with different permissions, you can use `make SUDO=sudo + install`. + +[Download Ruby]: https://www.ruby-lang.org/en/downloads/ + ### Unexplainable Build Errors -If you are having unexplainable build errors, after saving all your work, try running `git clean -xfd` in the source root to remove all git ignored local files. If you are working from a source directory that's been updated several times, you may have temporary build artifacts from previous releases which can cause build failures. +If you are having unexplainable build errors, after saving all your work, try +running `git clean -xfd` in the source root to remove all git ignored local +files. If you are working from a source directory that's been updated several +times, you may have temporary build artifacts from previous releases which can +cause build failures. + +## Building on Windows + +The documentation for building on Windows can be found in [the separated +file](../windows.md). ## More details @@ -106,8 +145,9 @@ about Ruby's build to help out. ### Running make scripts in parallel -In GNU make and BSD make implementations, to run a specific make script in parallel, pass the flag `-j<number of processes>`. For instance, -to run tests on 8 processes, use: +In GNU make[^caution-gmake-3] and BSD make implementations, to run a specific make script in +parallel, pass the flag `-j<number of processes>`. For instance, to run tests +on 8 processes, use: ``` shell make test-all -j8 @@ -115,7 +155,9 @@ make test-all -j8 We can also set `MAKEFLAGS` to run _all_ `make` commands in parallel. -Having the right `--jobs` flag will ensure all processors are utilized when building software projects. To do this effectively, you can set `MAKEFLAGS` in your shell configuration/profile: +Having the right `--jobs` flag will ensure all processors are utilized when +building software projects. To do this effectively, you can set `MAKEFLAGS` in +your shell configuration/profile: ``` shell # On macOS with Fish shell: @@ -131,11 +173,15 @@ export MAKEFLAGS="--jobs "(nproc) export MAKEFLAGS="--jobs $(nproc)" ``` +[^caution-gmake-3]: **CAUTION**: GNU make 3 is missing some features for parallel execution, we +recommend to upgrade to GNU make 4 or later. + ### Miniruby vs Ruby -Miniruby is a version of Ruby which has no external dependencies and lacks certain features. -It can be useful in Ruby development because it allows for faster build times. Miniruby is -built before Ruby. A functional Miniruby is required to build Ruby. To build Miniruby: +Miniruby is a version of Ruby which has no external dependencies and lacks +certain features. It can be useful in Ruby development because it allows for +faster build times. Miniruby is built before Ruby. A functional Miniruby is +required to build Ruby. To build Miniruby: ``` shell make miniruby @@ -143,8 +189,9 @@ make miniruby ## Debugging -You can use either lldb or gdb for debugging. Before debugging, you need to create a `test.rb` -with the Ruby script you'd like to run. You can use the following make targets: +You can use either lldb or gdb for debugging. Before debugging, you need to +create a `test.rb` with the Ruby script you'd like to run. You can use the +following make targets: * `make run`: Runs `test.rb` using Miniruby * `make lldb`: Runs `test.rb` using Miniruby in lldb @@ -153,19 +200,65 @@ with the Ruby script you'd like to run. You can use the following make targets: * `make lldb-ruby`: Runs `test.rb` using Ruby in lldb * `make gdb-ruby`: Runs `test.rb` using Ruby in gdb +### Compiling for Debugging + +You should configure Ruby without optimization and other flags that may +interfere with debugging: + +``` shell +./configure --enable-debug-env optflags="-O0 -fno-omit-frame-pointer" +``` + ### Building with Address Sanitizer -Using the address sanitizer is a great way to detect memory issues. +Using the address sanitizer (ASAN) is a great way to detect memory issues. It +can detect memory safety issues in Ruby itself, and also in any C extensions +compiled with and loaded into a Ruby compiled with ASAN. ``` shell ./autogen.sh mkdir build && cd build -export ASAN_OPTIONS="halt_on_error=0:use_sigaltstack=0:detect_leaks=0" -../configure cppflags="-fsanitize=address -fno-omit-frame-pointer" optflags=-O0 LDFLAGS="-fsanitize=address -fno-omit-frame-pointer" +../configure CC=clang-18 cflags="-fsanitize=address -fno-omit-frame-pointer -DUSE_MN_THREADS=0" # and any other options you might like make ``` -On Linux it is important to specify `-O0` when debugging. This is especially true for ASAN which sometimes works incorrectly at higher optimisation levels. +The compiled Ruby will now automatically crash with a report and a backtrace +if ASAN detects a memory safety issue. To run Ruby's test suite under ASAN, +issue the following command. Note that this will take quite a long time (over +two hours on my laptop); the `RUBY_TEST_TIMEOUT_SCALE` and +`SYNTAX_SUGEST_TIMEOUT` variables are required to make sure tests don't +spuriously fail with timeouts when in fact they're just slow. + +``` shell +RUBY_TEST_TIMEOUT_SCALE=5 SYNTAX_SUGGEST_TIMEOUT=600 make check +``` + +Please note, however, the following caveats! + +* ASAN will not work properly on any currently released version of Ruby; the + necessary support is currently only present on Ruby's master branch (and the + whole test suite passes only as of commit [Revision 9d0a5148]). +* Due to [Bug #20243], Clang generates code for threadlocal variables which + doesn't work with M:N threading. Thus, it's necessary to disable M:N + threading support at build time for now (with the `-DUSE_MN_THREADS=0` + configure argument). +* ASAN will only work when using Clang version 18 or later - it requires + [llvm/llvm-project#75290] related to multithreaded `fork`. +* ASAN has only been tested so far with Clang on Linux. It may or may not work + with other compilers or on other platforms - please file an issue on + [Ruby Issue Tracking System] if you run into problems with such configurations + (or, to report that they actually work properly!) +* In particular, although I have not yet tried it, I have reason to believe + ASAN will _not_ work properly on macOS yet - the fix for the multithreaded + fork issue was actually reverted for macOS (see [llvm/llvm-project#75659]). + Please open an issue on [Ruby Issue Tracking System] if this is a problem for + you. + +[Revision 9d0a5148]: https://bugs.ruby-lang.org/projects/ruby-master/repository/git/revisions/9d0a5148ae062a0481a4a18fbeb9cfd01dc10428 +[Bug #20243]: https://bugs.ruby-lang.org/issues/20243 +[llvm/llvm-project#75290]: https://github.com/llvm/llvm-project/pull/75290 +[llvm/llvm-project#75659]: https://github.com/llvm/llvm-project/pull/75659#issuecomment-1861584777 +[Ruby Issue Tracking System]: https://bugs.ruby-lang.org ## How to measure coverage of C and Ruby code @@ -182,11 +275,12 @@ make lcov open lcov-out/index.html ``` -If you need only C code coverage, you can remove `COVERAGE=true` from the above process. -You can also use `gcov` command directly to get per-file coverage. +If you need only C code coverage, you can remove `COVERAGE=true` from the +above process. You can also use `gcov` command directly to get per-file +coverage. -If you need only Ruby code coverage, you can remove `--enable-gcov`. -Note that `test-coverage.dat` accumulates all runs of `make test-all`. -Make sure that you remove the file if you want to measure one test run. +If you need only Ruby code coverage, you can remove `--enable-gcov`. Note +that `test-coverage.dat` accumulates all runs of `make test-all`. Make sure +that you remove the file if you want to measure one test run. You can see the coverage result of CI: https://rubyci.org/coverage diff --git a/doc/contributing/documentation_guide.md b/doc/contributing/documentation_guide.md index 907d935de9..59953d0d52 100644 --- a/doc/contributing/documentation_guide.md +++ b/doc/contributing/documentation_guide.md @@ -20,10 +20,12 @@ build directory: make html ``` +If you don't have a build directory, follow the [quick start +guide](building_ruby.md#label-Quick+start+guide) up to step 4. + Then you can preview your changes by opening `{build folder}/.ext/html/index.html` file in your browser. - ## Goal The goal of Ruby documentation is to impart the most important @@ -41,16 +43,17 @@ Use your judgment about what the user needs to know. - Write short declarative or imperative sentences. - Group sentences into (ideally short) paragraphs, each covering a single topic. -- Organize material with [headers](rdoc-ref:RDoc::Markup@Headers). +- Organize material with + [headings](rdoc-ref:RDoc::MarkupReference@Headings). - Refer to authoritative and relevant sources using - [links](rdoc-ref:RDoc::Markup@Links). + [links](rdoc-ref:RDoc::MarkupReference@Links). - Use simple verb tenses: simple present, simple past, simple future. - Use simple sentence structure, not compound or complex structure. - Avoid: - Excessive comma-separated phrases; - consider a [list](rdoc-ref:RDoc::Markup@Simple+Lists). + consider a [list](rdoc-ref:RDoc::MarkupReference@Lists). - Idioms and culture-specific references. - - Overuse of headers. + - Overuse of headings. - Using US-ASCII-incompatible characters in C source files; see [Characters](#label-Characters) below. @@ -124,16 +127,23 @@ a.shuffle! #=> [2, 3, 1] a #=> [2, 3, 1] ``` -### Headers +### Headings + +Organize a long discussion for a class or module with [headings](rdoc-ref:RDoc::MarkupReference@Headings). + +Do not use formal headings in the documentation for a method or constant. -Organize a long discussion with [headers](rdoc-ref:RDoc::Markup@Headers). +In the rare case where heading-like structures are needed +within the documentation for a method or constant, use +[bold text](rdoc-ref:RDoc::MarkupReference@Bold) +as pseudo-headings. ### Blank Lines A blank line begins a new paragraph. -A [code block](rdoc-ref:RDoc::Markup@Paragraphs+and+Verbatim) -or [list](rdoc-ref:RDoc::Markup@Simple+Lists) +A [code block](rdoc-ref:RDoc::MarkupReference@Code+Blocks) +or [list](rdoc-ref:RDoc::MarkupReference@Lists) should be preceded by and followed by a blank line. This is unnecessary for the HTML output, but helps in the `ri` output. @@ -150,19 +160,76 @@ For a method name in text: or a hash mark for an instance method: <tt>Foo.bar</tt>, <tt>Foo#baz</tt>. +### Embedded Code and Commands + +Code or commands embedded in running text (i.e., not in a code block) +should marked up as +[monofont](rdoc-ref:RDoc::MarkupReference@Monofont). + +Code that is a simple string should include the quote marks. + ### Auto-Linking In general, \RDoc's auto-linking should not be suppressed. For example, we should write `Array`, not `\Array`. -We might consider whether to suppress when: +However, suppress when the word in question: -- The word in question does not refer to a Ruby entity +- Does not refer to a Ruby entity (e.g., some uses of _Class_ or _English_). -- The reference is to the current class document - (e.g., _Array_ in the documentation for class `Array`). -- The same reference is repeated many times - (e.g., _RDoc_ on this page). +- Refers to the current document + (e.g., _Array_ in the documentation for class `Array`); + in that case, the word should be forced to + [monofont](rdoc-ref:RDoc::MarkupReference@Monofont). + +Most often, the name of a class, module, or method +will be auto-linked: + +- Array. +- Enumerable. +- File.new +- File#read. + +If not, or if you suppress autolinking, consider forcing +[monofont](rdoc-ref:RDoc::MarkupReference@Monofont). + +### Explicit Links + +When writing an explicit link, follow these guidelines. + +#### +rdoc-ref+ Scheme + +Use the +rdoc-ref+ scheme for: + +- A link in core documentation to other core documentation. +- A link in core documentation to documentation in a standard library package. +- A link in a standard library package to other documentation in that same + standard library package. + +See section "+rdoc-ref+ Scheme" in {Links}[rdoc-ref:RDoc::MarkupReference@Links]. + +#### URL-Based Link + +Use a full URL-based link for: + +- A link in standard library documentation to documentation in the core. +- A link in standard library documentation to documentation in a different + standard library package. + +Doing so ensures that the link will valid even when the package documentation +is built independently (separately from the core documentation). + +The link should lead to a target in https://docs.ruby-lang.org/en/master/. + +Also use a full URL-based link for a link to an off-site document. + +### Variable Names + +The name of a variable (as specified in its call-seq) should be marked up as +[monofont](rdoc-ref:RDoc::MarkupReference@Monofont). + +Also, use monofont text for the name of a transient variable +(i.e., one defined and used only in the discussion, such as +n+). ### HTML Tags @@ -175,16 +242,21 @@ may not render them properly. In particular, avoid building tables with HTML tags (<tt><table></tt>, etc.). -Alternatives are: - -- The GFM (GitHub Flavored Markdown) table extension, - which is enabled by default. See - {GFM tables extension}[https://github.github.com/gfm/#tables-extension-]. +Alternatives: - A {verbatim text block}[rdoc-ref:RDoc::MarkupReference@Verbatim+Text+Blocks], - using spaces and punctuation to format the text. - Note that {text markup}[rdoc-ref:RDoc::MarkupReference@Text+Markup] - will not be honored. + using spaces and punctuation to format the text; + note that {text markup}[rdoc-ref:RDoc::MarkupReference@Text+Markup] + will not be honored: + + - Example {source}[https://github.com/ruby/ruby/blob/34d802f32f00df1ac0220b62f72605827c16bad8/file.c#L6570-L6596]. + - Corresponding {output}[rdoc-ref:File@Read-2FWrite+Mode]. + +- (Markdown format only): A {Github Flavored Markdown (GFM) table}[https://github.github.com/gfm/#tables-extension-], + using special formatting for the text: + + - Example {source}[https://github.com/ruby/ruby/blob/34d802f32f00df1ac0220b62f72605827c16bad8/doc/contributing/glossary.md?plain=1]. + - Corresponding {output}[https://docs.ruby-lang.org/en/master/contributing/glossary_md.html]. ## Documenting Classes and Modules @@ -214,9 +286,12 @@ Guidelines: - The section title is `What's Here`. - Consider listing the parent class and any included modules; consider - [links](rdoc-ref:RDoc::Markup@Links) + [links](rdoc-ref:RDoc::MarkupReference@Links) to their "What's Here" sections if those exist. -- List methods as a bullet list: +- All methods mentioned in the left-pane table of contents + should be listed (including any methods extended from another class). +- Attributes (which are not included in the TOC) may also be listed. +- Display methods as items in one or more bullet lists: - Begin each item with the method name, followed by a colon and a short description. @@ -224,9 +299,9 @@ Guidelines: (and do not list the aliases separately). - Check the rendered documentation to determine whether \RDoc has recognized the method and linked to it; if not, manually insert a - [link](rdoc-ref:RDoc::Markup@Links). + [link](rdoc-ref:RDoc::MarkupReference@Links). -- If there are numerous entries, consider grouping them into subsections with headers. +- If there are numerous entries, consider grouping them into subsections with headings. - If there are more than a few such subsections, consider adding a table of contents just below the main section title. @@ -249,7 +324,7 @@ For methods written in Ruby, \RDoc documents the calling sequence automatically. For methods written in C, \RDoc cannot determine what arguments the method accepts, so those need to be documented using \RDoc directive -[`call-seq:`](rdoc-ref:RDoc::Markup@Method+arguments). +[`call-seq:`](rdoc-ref:RDoc::MarkupReference@Directives+for+Method+Documentation). For a singleton method, use the form: @@ -388,7 +463,7 @@ argument passed if it is not obvious, not explicitly mentioned in the details, and not implicitly shown in the examples. If there is more than one argument or block argument, use a -[labeled list](rdoc-ref:RDoc::Markup@Labeled+Lists). +[labeled list](rdoc-ref:RDoc::MarkupReference@Labeled+Lists). ### Corner Cases and Exceptions diff --git a/doc/contributing/glossary.md b/doc/contributing/glossary.md index 0d27d28c96..86c6671fbd 100644 --- a/doc/contributing/glossary.md +++ b/doc/contributing/glossary.md @@ -10,7 +10,7 @@ Just a list of acronyms I've run across in the Ruby source code and their meanin | `cd` | Call Data. A data structure that points at the `ci` and the `cc`. `iseq` objects points at the `cd`, and access call information and call caches via this structure | | `cfp`| Control Frame Pointer. Represents a Ruby stack frame. Calling a method pushes a new frame (cfp), returning pops a frame. Points at the `pc`, `sp`, `ep`, and the corresponding `iseq`| | `ci` | Call Information. Refers to an `rb_callinfo` struct. Contains call information about the call site, including number of parameters to be passed, whether it they are keyword arguments or not, etc. Used in conjunction with the `cc` and `cd`. | -| `cref` | Class reference. A structure pointing to the class reference where `klass_or_self`, visibility scope, and refinements are stored. It also stores a pointer to the next class in the hierachy referenced by `rb_cref_struct * next`. The Class reference is lexically scoped. | +| `cref` | Class reference. A structure pointing to the class reference where `klass_or_self`, visibility scope, and refinements are stored. It also stores a pointer to the next class in the hierarchy referenced by `rb_cref_struct * next`. The Class reference is lexically scoped. | | CRuby | Implementation of Ruby written in C | | `cvar` | Class Variable. Refers to a Ruby class variable like `@@foo` | | `dvar` | Dynamic Variable. Used by the parser to refer to local variables that are defined outside of the current lexical scope. For example `def foo; bar = 1; -> { p bar }; end` the "bar" inside the block is a `dvar` | diff --git a/doc/contributing/testing_ruby.md b/doc/contributing/testing_ruby.md index 167ab55c55..dfb7fb3a65 100644 --- a/doc/contributing/testing_ruby.md +++ b/doc/contributing/testing_ruby.md @@ -82,10 +82,10 @@ We can run any of the make scripts [in parallel](building_ruby.md#label-Running+ We can display the help of the `TESTS` option: ``` - $ make test-all TESTS=--help + make test-all TESTS=--help ``` - If we would like to run both the `test/` and `bootstraptest/` test suites, we can run + If we would like to run the `test/`, `bootstraptest/` and `spec/` test suites (the `spec/` is explained in a later section), we can run ``` make check @@ -99,28 +99,28 @@ We can run any of the make scripts [in parallel](building_ruby.md#label-Running+ make test-spec ``` - To run a specific directory, we can use `MSPECOPT` to specify the directory: + To run a specific directory, we can use `SPECOPTS` to specify the directory: ``` - make test-spec MSPECOPT=spec/ruby/core/array + make test-spec SPECOPTS=spec/ruby/core/array ``` - To run a specific file, we can also use `MSPECOPT` to specify the file: + To run a specific file, we can also use `SPECOPTS` to specify the file: ``` - make test-spec MSPECOPT=spec/ruby/core/array/any_spec.rb + make test-spec SPECOPTS=spec/ruby/core/array/any_spec.rb ``` To run a specific test, we can use the `--example` flag to match against the test name: ``` - make test-spec MSPECOPT="../spec/ruby/core/array/any_spec.rb --example='is false if the array is empty'" + make test-spec SPECOPTS="../spec/ruby/core/array/any_spec.rb --example='is false if the array is empty'" ``` To run these specs with logs, we can use: ``` - make test-spec MSPECOPT=-Vfs + make test-spec SPECOPTS=-Vfs ``` To run a ruby-spec file or directory with GNU make, we can use @@ -140,5 +140,17 @@ We can run any of the make scripts [in parallel](building_ruby.md#label-Running+ To run a specific bundler spec file, we can use `BUNDLER_SPECS` as follows: ``` - $ make test-bundler BUNDLER_SPECS=commands/exec_spec.rb + make test-bundler BUNDLER_SPECS=commands/exec_spec.rb ``` + +## Troubleshooting + +### Running test suites on s390x CPU Architecture + +If we see failing tests related to the zlib library on s390x CPU architecture, we can run the test suites with `DFLTCC=0` to pass: + +``` +DFLTCC=0 make check +``` + +The failures can happen with the zlib library applying the patch [madler/zlib#410](https://github.com/madler/zlib/pull/410) to enable the deflate algorithm producing a different compressed byte stream. We manage this issue at [[ruby-core:114942][Bug #19909]](https://bugs.ruby-lang.org/issues/19909). |