diff options
Diffstat (limited to 'doc/contributing/building_ruby.md')
| -rw-r--r-- | doc/contributing/building_ruby.md | 300 |
1 files changed, 243 insertions, 57 deletions
diff --git a/doc/contributing/building_ruby.md b/doc/contributing/building_ruby.md index 1ac7a375f8..286bd1f116 100644 --- a/doc/contributing/building_ruby.md +++ b/doc/contributing/building_ruby.md @@ -1,80 +1,190 @@ # Building Ruby -## Quick start guide +## Dependencies 1. Install the prerequisite dependencies for building the CRuby interpreter: * C compiler - * autoconf - 2.67 or later - * bison - 3.0 or later - * gperf - 3.0.3 or later - * ruby - 2.7 or later + + For RubyGems, you will also need: + + * [OpenSSL] 1.1.x or 3.0.x / [LibreSSL] + * [libyaml] 0.1.7 or later + * [zlib] + + If you want to build from the git repository, you will also need: + + * [autoconf] - 2.67 or later + * [gperf] - 3.1 or later + * Usually unneeded; only if you edit some source files using gperf + * ruby - 3.1 or later + * We can upgrade this version to system ruby version of the latest + Ubuntu LTS. + * git - 2.32 or later + * Anterior versions may work; 2.32 or later will prevent build + errors in case your system `.gitconfig` uses `$HOME` paths. 2. Install optional, recommended dependencies: - * OpenSSL/LibreSSL - * readline/editline (libedit) - * zlib - * libffi - * libyaml - * libexecinfo (FreeBSD) - * rustc - 1.58.0 or later (if you wish to build [YJIT](/doc/yjit/yjit.md)) + * [libffi] (to build fiddle) + * [gmp] (if you wish to accelerate Bignum operations) + * [rustc] - 1.58.0 or later, if you wish to build + [YJIT](rdoc-ref:RubyVM::YJIT). + + If you want to link the libraries (e.g., gmp) installed into other than + the OS default place, typically using Homebrew on macOS, pass the + `--with-opt-dir` (or `--with-gmp-dir` for gmp) option to `configure`. - If you installed the libraries needed for extensions (openssl, readline, libyaml, zlib) into other than the OS default place, - typically using Homebrew on macOS, add `--with-EXTLIB-dir` options to `CONFIGURE_ARGS` environment variable. + ```sh + configure --with-opt-dir=$(brew --prefix gmp):$(brew --prefix jemalloc) + ``` + + As for the libraries needed for particular extensions only and not for + Ruby (openssl, readline, libyaml, zlib), you can add `--with-EXTLIB-dir` + options to the command line or to `CONFIGURE_ARGS` environment variable. + The command line options will be embedded in `rbconfig.rb`, while the + latter environment variable is not embedded and is only used when + building the extension libraries. - ``` shell + ```sh export CONFIGURE_ARGS="" for ext in openssl readline libyaml zlib; do CONFIGURE_ARGS="${CONFIGURE_ARGS} --with-$ext-dir=$(brew --prefix $ext)" done ``` -3. Checkout the CRuby source code: +[OpenSSL]: https://www.openssl.org +[LibreSSL]: https://www.libressl.org +[libyaml]: https://github.com/yaml/libyaml/ +[zlib]: https://www.zlib.net +[autoconf]: https://www.gnu.org/software/autoconf/ +[gperf]: https://www.gnu.org/software/gperf/ +[libffi]: https://sourceware.org/libffi/ +[gmp]: https://gmplib.org +[rustc]: https://www.rust-lang.org - ``` shell - git clone https://github.com/ruby/ruby.git - ``` +## Quick start guide -4. Generate the configure file: +1. Download ruby source code: - ``` shell - ./autogen.sh - ``` + Select one of the below. + + 1. Build from the tarball: -5. Create a `build` directory outside of the source directory: + Download the latest tarball from [Download Ruby] page and extract + it. Example for Ruby 3.0.2: - ``` shell + ```sh + tar -xzf ruby-3.0.2.tar.gz + cd ruby-3.0.2 + ``` + + 2. Build from the git repository: + + Checkout the CRuby source code: + + ```sh + git clone https://github.com/ruby/ruby.git + cd ruby + ``` + + Run the GNU Autoconf script (which generates the `configure` script): + + ```sh + ./autogen.sh + ``` + +2. Create a `build` directory inside the repository directory: + + ```sh mkdir build && cd build ``` - While it's not necessary to build in a separate directory, it's good practice to do so. + While it's not necessary to build in a dedicated directory like this, it's good + practice to do so. -6. We'll install Ruby in `~/.rubies/ruby-master`, so create the directory: +3. We'll eventually install our new Ruby in `~/.rubies/ruby-master`, so we'll create that directory: - ``` shell + ```sh mkdir ~/.rubies ``` -7. Run configure: +4. Run the `configure` script (which generates the `Makefile`): - ``` shell + ```sh ../configure --prefix="${HOME}/.rubies/ruby-master" ``` - - If you are frequently building Ruby, add the `--disable-install-doc` flag to not build documentation which will speed up the build process. + - Also `-C` (or `--config-cache`) would reduce time to configure from the + next time. + +5. Build Ruby: + + ```sh + make + ``` + +6. [Run tests](testing_ruby.md) to confirm your build succeeded. -8. Build Ruby: +7. Install our newly-compiled Ruby into `~/.rubies/ruby-master`: - ``` shell + ```sh make install ``` -9. [Run tests](testing_ruby.md) to confirm your build succeeded. + - If you need to run `make install` with `sudo` and want to avoid document + generation with different permissions, you can use `make SUDO=sudo + install`. + +8. You can then try your new Ruby out, for example: + + ```sh + ~/.rubies/ruby-master/bin/ruby -e "puts 'Hello, World!'" + ``` + +By the end, your repo will look like this: + +```text +ruby +├── autogen.sh # Pre-existing Autoconf script, used in step 1 +├── configure # Generated in step 1, which generates the `Makefile` in step 4 +├── build # Created in step 2 and populated in step 4 +│ ├── GNUmakefile # Generated by `../configure` +│ ├── Makefile # Generated by `../configure` +│ ├── object.o # Compiled object file, built my `make` +│ └── ... other compiled `.o` object files +│ +│ # Other interesting files: +├── include +│ └── ruby.h # The main public header +├── internal +│ ├── object.h +│ └── ... other header files used by the `.c` files in the repo root. +├── lib +│ └── # Default gems, like `bundler`, `erb`, `set`, `yaml`, etc. +├── spec +│ └── # A mirror of the Ruby specification from github.com/ruby/spec +├── test +│ ├── ruby +│ └── ... +├── object.c +└── ... other `.c` files +``` + +[Download Ruby]: https://www.ruby-lang.org/en/downloads/ ### Unexplainable Build Errors -If you are having unexplainable build errors, after saving all your work, try running `git clean -xfd` in the source root to remove all git ignored local files. If you are working from a source directory that's been updated several times, you may have temporary build artifacts from previous releases which can cause build failures. +If you are having unexplainable build errors, after saving all your work, try +running `git clean -xfd` in the source root to remove all git ignored local +files. If you are working from a source directory that's been updated several +times, you may have temporary build artifacts from previous releases which can +cause build failures. + +## Building on Windows + +The documentation for building on Windows can be found in [the separated +file](../distribution/windows.md). ## More details @@ -83,18 +193,21 @@ about Ruby's build to help out. ### Running make scripts in parallel -In GNU make and BSD make implementations, to run a specific make script in parallel, pass the flag `-j<number of processes>`. For instance, -to run tests on 8 processes, use: +In GNU make[^caution-gmake-3] and BSD make implementations, to run a specific make script in +parallel, pass the flag `-j<number of processes>`. For instance, to run tests +on 8 processes, use: -``` shell +```sh make test-all -j8 ``` We can also set `MAKEFLAGS` to run _all_ `make` commands in parallel. -Having the right `--jobs` flag will ensure all processors are utilized when building software projects. To do this effectively, you can set `MAKEFLAGS` in your shell configuration/profile: +Having the right `--jobs` flag will ensure all processors are utilized when +building software projects. To do this effectively, you can set `MAKEFLAGS` in +your shell configuration/profile: -``` shell +```sh # On macOS with Fish shell: export MAKEFLAGS="--jobs "(sysctl -n hw.ncpu) @@ -108,20 +221,25 @@ export MAKEFLAGS="--jobs "(nproc) export MAKEFLAGS="--jobs $(nproc)" ``` +[^caution-gmake-3]: **CAUTION**: GNU make 3 is missing some features for parallel execution, we +recommend to upgrade to GNU make 4 or later. + ### Miniruby vs Ruby -Miniruby is a version of Ruby which has no external dependencies and lacks certain features. -It can be useful in Ruby development because it allows for faster build times. Miniruby is -built before Ruby. A functional Miniruby is required to build Ruby. To build Miniruby: +Miniruby is a version of Ruby which has no external dependencies and lacks +certain features. It can be useful in Ruby development because it allows for +faster build times. Miniruby is built before Ruby. A functional Miniruby is +required to build Ruby. To build Miniruby: -``` shell +```sh make miniruby ``` ## Debugging -You can use either lldb or gdb for debugging. Before debugging, you need to create a `test.rb` -with the Ruby script you'd like to run. You can use the following make targets: +You can use either lldb or gdb for debugging. Before debugging, you need to +create a `test.rb` with the Ruby script you'd like to run. You can use the +following make targets: * `make run`: Runs `test.rb` using Miniruby * `make lldb`: Runs `test.rb` using Miniruby in lldb @@ -130,25 +248,92 @@ with the Ruby script you'd like to run. You can use the following make targets: * `make lldb-ruby`: Runs `test.rb` using Ruby in lldb * `make gdb-ruby`: Runs `test.rb` using Ruby in gdb +For VS Code users, you can set up editor-based debugging experience by running: + +```shell +cp -r misc/.vscode .vscode +``` + +This will add launch configurations for debugging Ruby itself by running `test.rb` with `lldb`. + +**Note**: if you build Ruby under the `./build` folder, you'll need to update `.vscode/launch.json`'s program entry accordingly to: `"${workspaceFolder}/build/ruby"` + +### Compiling for Debugging + +You can compile Ruby with the `RUBY_DEBUG` macro to enable debugging on some +features. One example is debugging object shapes in Ruby with +`RubyVM::Shape.of(object)`. + +Additionally Ruby can be compiled to support the `RUBY_DEBUG` environment +variable to enable debugging on some features. An example is using +`RUBY_DEBUG=gc_stress` to debug GC-related issues. + +There is also support for the `RUBY_DEBUG_LOG` environment variable to log a +lot of information about what the VM is doing, via the `USE_RUBY_DEBUG_LOG` +macro. + +You should also configure Ruby without optimization and other flags that may +interfere with debugging by changing the optimization flags. + +Bringing it all together: + +```sh +./configure cppflags="-DRUBY_DEBUG=1 -DUSE_RUBY_DEBUG_LOG=1" --enable-debug-env optflags="-O0 -fno-omit-frame-pointer" +``` + ### Building with Address Sanitizer -Using the address sanitizer is a great way to detect memory issues. +Using the address sanitizer (ASAN) is a great way to detect memory issues. It +can detect memory safety issues in Ruby itself, and also in any C extensions +compiled with and loaded into a Ruby compiled with ASAN. -``` shell +```sh ./autogen.sh mkdir build && cd build -export ASAN_OPTIONS="halt_on_error=0:use_sigaltstack=0:detect_leaks=0" -../configure cppflags="-fsanitize=address -fno-omit-frame-pointer" optflags=-O0 LDFLAGS="-fsanitize=address -fno-omit-frame-pointer" +../configure CC=clang-18 cflags="-fsanitize=address -fno-omit-frame-pointer -DUSE_MN_THREADS=0" # and any other options you might like make ``` -On Linux it is important to specify `-O0` when debugging. This is especially true for ASAN which sometimes works incorrectly at higher optimisation levels. +The compiled Ruby will now automatically crash with a report and a backtrace +if ASAN detects a memory safety issue. To run Ruby's test suite under ASAN, +issue the following command. Note that this will take quite a long time (over +two hours on my laptop); the `RUBY_TEST_TIMEOUT_SCALE` and +`SYNTAX_SUGEST_TIMEOUT` variables are required to make sure tests don't +spuriously fail with timeouts when in fact they're just slow. + +```sh +RUBY_TEST_TIMEOUT_SCALE=5 SYNTAX_SUGGEST_TIMEOUT=600 make check +``` + +Please note, however, the following caveats! + +* Due to [Bug #20243], Clang generates code for threadlocal variables which + doesn't work with M:N threading. Thus, it's necessary to disable M:N + threading support at build time for now (with the `-DUSE_MN_THREADS=0` + configure argument). +* ASAN will only work when using Clang version 18 or later - it requires + [llvm/llvm-project#75290] related to multithreaded `fork`. +* ASAN has only been tested so far with Clang on Linux. It may or may not work + with other compilers or on other platforms - please file an issue on + [Ruby Issue Tracking System] if you run into problems with such configurations + (or, to report that they actually work properly!) +* In particular, although I have not yet tried it, I have reason to believe + ASAN will _not_ work properly on macOS yet - the fix for the multithreaded + fork issue was actually reverted for macOS (see [llvm/llvm-project#75659]). + Please open an issue on [Ruby Issue Tracking System] if this is a problem for + you. + +[Revision 9d0a5148]: https://bugs.ruby-lang.org/projects/ruby-master/repository/git/revisions/9d0a5148ae062a0481a4a18fbeb9cfd01dc10428 +[Bug #20243]: https://bugs.ruby-lang.org/issues/20243 +[llvm/llvm-project#75290]: https://github.com/llvm/llvm-project/pull/75290 +[llvm/llvm-project#75659]: https://github.com/llvm/llvm-project/pull/75659#issuecomment-1861584777 +[Ruby Issue Tracking System]: https://bugs.ruby-lang.org ## How to measure coverage of C and Ruby code You need to be able to use gcc (gcov) and lcov visualizer. -``` shell +```sh ./autogen.sh ./configure --enable-gcov make @@ -159,11 +344,12 @@ make lcov open lcov-out/index.html ``` -If you need only C code coverage, you can remove `COVERAGE=true` from the above process. -You can also use `gcov` command directly to get per-file coverage. +If you need only C code coverage, you can remove `COVERAGE=true` from the +above process. You can also use `gcov` command directly to get per-file +coverage. -If you need only Ruby code coverage, you can remove `--enable-gcov`. -Note that `test-coverage.dat` accumulates all runs of `make test-all`. -Make sure that you remove the file if you want to measure one test run. +If you need only Ruby code coverage, you can remove `--enable-gcov`. Note +that `test-coverage.dat` accumulates all runs of `make test-all`. Make sure +that you remove the file if you want to measure one test run. You can see the coverage result of CI: https://rubyci.org/coverage |