path: root/doc/contributing/concurrency_guide.md

# Concurrency Guide

This is a guide to thinking about concurrency in the cruby source code, whether that's contributing to Ruby
by writing C or by contributing to one of the JITs. This does not touch on native extensions, only the core
language. It will go over:

* What needs synchronizing?
* How to use the VM lock, and what you can and can't do when you've acquired this lock.
* What you can and can't do when you've acquired other native locks.
* The difference between the VM lock and the GVL.
* What a VM barrier is and when to use it.
* The lock ordering of some important locks.
* How ruby interrupt handling works.
* The timer thread and what it's responsible for.

## What needs synchronizing?

Before ractors, only one ruby thread could run at once. That didn't mean you could forget about concurrency issues, though. The timer thread
is a native thread that interacts with other ruby threads and changes some VM internals, so if these changes can be done in parallel by both the timer
thread and a ruby thread, they need to be synchronized.

When you add ractors to the mix, it gets more complicated. However, ractors allow you to forget about synchronization for non-shareable objects because
they aren't used across ractors. Only one ruby thread can touch the object at once. For shareable objects, they are deeply frozen so there isn't any
mutation on the objects themselves. However, something like reading/writing constants across ractors does need to be synchronized. In this case, ruby threads need to
see a consistent view of the VM. If publishing the update takes 2 steps or even two separate instructions, like in this case, synchronization is required.

Most synchronization is to protect VM internals. These internals include structures for the thread scheduler on each ractor, the global ractor scheduler, the
coordination between ruby threads and ractors, global tables (for `fstrings`, encodings, symbols and global vars), etc. Anything that can be mutated by a ractor
that can also be read or mutated by another ractor at the same time requires proper synchronization.

## The VM Lock

There's only one VM lock and it is for critical sections that can only be entered by one ractor at a time.
Without ractors, the VM lock is useless. It does not stop all ractors from running, as ractors can run
without trying to acquire this lock. If you're updating global (shared) data between ractors and aren't using
atomics, you need to use a lock and this is a convenient one to use. Unlike other locks, you can allocate ruby-managed
memory with it held. When you take the VM lock, there are things you can and can't do during your critical section:

You can (as long as no other locks are also held before the VM lock):

* Create ruby objects, call `ruby_xmalloc`, etc.

You can't:

* Context switch to another ruby thread or ractor. This is important, as many things can cause ruby-level context switches including:

    * Calling any ruby method through, for example, `rb_funcall`. If you execute ruby code, a context switch could happen.
    This also applies to ruby methods defined in C, as they can be redefined in Ruby. Things that call ruby methods such as
    `rb_obj_respond_to` are also disallowed.

    * Calling `rb_raise`. This will call `initialize` on the new exception object. With the VM lock
      held, nothing you call should be able to raise an exception. `NoMemoryError` is allowed, however.

    * Calling `rb_nogvl` or a ruby-level mechanism that can context switch like `rb_mutex_lock`.

    * Enter any blocking operation managed by ruby. This will context switch to another ruby thread using `rb_nogvl` or
    something equivalent. A blocking operation is one that blocks the thread's progress, such as `sleep` or `IO#read`.

Internally, the VM lock is the `vm->ractor.sync.lock`.

You need to be on a ruby thread to take the VM lock. You also can't take it inside any functions that could be called during sweeping, as MMTK sweeps
on another thread and you need a valid `ec` to grab the lock. For this same reason (among others), you can't take it from the timer thread either.

## Other Locks

All native locks that aren't the VM lock share a more strict set of rules for what's allowed during the critical section. By native locks, we mean
anything that uses `rb_native_mutex_lock`. Some important locks include the `interrupt_lock`, the ractor scheduling lock (protects global scheduling data structures),
the thread scheduling lock (local to each ractor, protects per-ractor scheduling data structures) and the ractor lock (local to each ractor, protects ractor data structures).

When you acquire one of these locks,

You can:

* Allocate memory though non-ruby allocation such as raw `malloc` or the standard library. But be careful, some functions like `strdup` use
ruby allocation through the use of macros!

* Use `ccan` lists, as they don't allocate.

* Do the usual things like set variables or struct fields, manipulate linked lists, signal condition variables etc.

You can't:

* Allocate ruby-managed memory. This includes creating ruby objects or using `ruby_xmalloc` or `st_insert`. The reason this
is disallowed is if that allocation causes a GC, then all other ruby threads must join a VM barrier as soon as possible
(when they next check interrupts or acquire the VM lock). This is so that no other ractors are running during GC. If a ruby thread
is waiting (blocked) on this same native lock, it can't join the barrier and a deadlock occurs because the barrier will never finish.

* Raise exceptions. You also can't use `EC_JUMP_TAG` if it jumps out of the critical section.

* Context switch. See the `VM Lock` section for more info.

## Difference Between VM Lock and GVL

The VM Lock is a particular lock in the source code. There is only one VM Lock. The GVL, on the other hand, is more of a combination of locks.
It is "acquired" when a ruby thread is about to run or is running. Since many ruby threads can run at the same time if they're in different ractors,
there are many GVLs (1 per `SNT` + 1 for the main ractor). It can no longer be thought of as a "Global VM Lock" like it once was before ractors.

## VM Barriers

Sometimes, taking the VM Lock isn't enough and you need a guarantee that all ractors have stopped. This happens when running `GC`, for instance.
To get a barrier, you take the VM Lock and call `rb_vm_barrier()`. For the duration that the VM lock is held, no other ractors will be running. It's not used
often as taking a barrier slows ractor performance down considerably, but it's useful to know about and is sometimes the only solution.

## Lock Orderings

It's a good idea to not hold more than 2 locks at once on the same thread. Locking multiple locks can introduce deadlocks, so do it with care. When locking
multiple locks at once, follow an ordering that is consistent across the program, otherwise you can introduce deadlocks. Here are the orderings of some important locks:

* VM lock before ractor_sched_lock
* thread_sched_lock before ractor_sched_lock
* interrupt_lock before timer_th.waiting_lock
* timer_th.waiting_lock before ractor_sched_lock

These orderings are subject to change, so check the source if you're not sure. On top of this:

* During each `ubf` (unblock) function, the VM lock can be taken around it in some circumstances. This happens during VM shutdown, for example.
See the "Interrupt Handling" section for more details.

## Ruby Interrupt Handling

When the VM runs ruby code, ruby's threads intermittently check ruby-level interrupts. These software interrupts
are for various things in ruby and they can be set by other ruby threads or the timer thread.

* Ruby threads check when they should give up their timeslice. The native thread switches to another ruby thread when their time is up.
* The timer thread sends a "trap" interrupt to the main thread if any ruby-level signal handlers are pending.
* Ruby threads can have other ruby threads run tasks for them by sending them an interrupt. For instance, ractors send
the main thread an interrupt when they need to `require` a file so that it's done on the main thread. They wait for the
main thread's result.
* During VM shutdown, a "terminate" interrupt is sent to all ractor main threads top stop them asap.
* When calling `Thread#raise`, the caller sends an interrupt to that thread telling it which exception to raise.
* Unlocking a mutex sends the next waiter (if any) an interrupt telling it to grab the lock.
* Signalling or broadcasting on a condition variable tells the waiter(s) to wake up.

This isn't a complete list.

When sending an interrupt to a ruby thread, the ruby thread can be blocked. For example, it could be in the middle of a `TCPSocket#read` call. If so,
the receiving thread's `ubf` (unblock function) gets called from the thread (ruby thread or timer thread) that sent the interrupt.
Each ruby thread has a `ubf` that is set when it enters a blocking operation and is unset after returning from it. By default, this `ubf` function sends a
`SIGVTALRM` to the receiving thread to try to unblock it from the kernel so it can check its interrupts. There are other `ubfs` that
aren't associated with a syscall, such as when calling `Ractor#join` or `sleep`. All `ubfs` are called with the `interrupt_lock` held,
so take that into account when using locks inside `ubfs`.

Remember, `ubfs` can be called from the timer thread so you cannot assume an `ec` inside them. The `ec` (execution context) is only set on ruby threads.

## The Timer Thread

The timer thread has a few functions. They are:

* Send interrupts to ruby threads that have run for their whole timeslice.
* Wake up M:N ruby threads (threads in non-main ractors) blocked on IO or after a specified timeout. This
uses `kqueue` or `epoll`, depending on the OS, to receive IO events on behalf of the threads.
* Continue calling  the `SIGVTARLM` signal if a thread is still blocked on a syscall after the first `ubf` call.
* Signal native threads (`SNT`) waiting on a ractor if there are ractors waiting in the global run queue.
* Create more `SNT`s if some are blocked, like on IO or on `Ractor#join`.