| Age | Commit message (Collapse) | Author |
|---|
|
|
|
instead of using `rb_native_mutex_lock` directly.
|
|
Fix race between timer thread dequeuing waiting thread and thread
skipping sleeping due to being dequeued. We now use `th->event_serial` which
is protected by `thread_sched_lock`. When a thread is put on timer thread's waiting
list, the event serial is saved on the item. The timer thread checks
that the saved serial is the same as current thread's serial before
calling `thread_sched_to_ready`.
The following script (taken from a test in `test_thread.rb` used to crash on
scheduler debug assertions. It would likely crash in non-debug mode as well.
```ruby
def assert_nil(val)
if val != nil
raise "Expected #{val} to be nil"
end
end
def assert_equal(expected, actual)
if expected != actual
raise "Expected #{expected} to be #{actual}"
end
end
def test_join2
ok = false
t1 = Thread.new { ok = true; sleep }
Thread.pass until ok
Thread.pass until t1.stop?
t2 = Thread.new do
Thread.pass while ok
t1.join(0.01)
end
t3 = Thread.new do
ok = false
t1.join
end
assert_nil(t2.value)
t1.wakeup
assert_equal(t1, t3.value)
ensure
t1&.kill&.join
t2&.kill&.join
t3&.kill&.join
end
rs = 30.times.map do
Ractor.new do
test_join2
end
end
rs.each(&:join)
```
|
|
This race condition was found when calling `Thread#join` with a timeout
inside a ractor. The race is between the polling thread waking up the
thread and the `ubf` getting called (`ubf_event_waiting`). The error was
that the ubf or polling thread would set the thread as ready, but then
the other function would do the same.
Fixes [Bug #21614]
|
|
|
|
[Bug #20146]
Previously we dealt with the main Ractor not being enabled for M:N by
incrementing snt_cnt++. This worked for comparing against ractor count,
but meant that we always had one less SNT than was specified by
RUBY_MAX_CPU.
This was notably a problem for RUBY_MAX_CPU=1, which would cause Ractors
to hang.
This commit instead of adjusting snt, adjusts a
"schedulable_ractor_cnt". This way snt_cnt will actually reach
RUBY_MAX_CPU.
|
|
* Added `Ractor::Port`
* `Ractor::Port#receive` (support multi-threads)
* `Rcator::Port#close`
* `Ractor::Port#closed?`
* Added some methods
* `Ractor#join`
* `Ractor#value`
* `Ractor#monitor`
* `Ractor#unmonitor`
* Removed some methods
* `Ractor#take`
* `Ractor.yield`
* Change the spec
* `Racotr.select`
You can wait for multiple sequences of messages with `Ractor::Port`.
```ruby
ports = 3.times.map{ Ractor::Port.new }
ports.map.with_index do |port, ri|
Ractor.new port,ri do |port, ri|
3.times{|i| port << "r#{ri}-#{i}"}
end
end
p ports.each{|port| pp 3.times.map{port.receive}}
```
In this example, we use 3 ports, and 3 Ractors send messages to them respectively.
We can receive a series of messages from each port.
You can use `Ractor#value` to get the last value of a Ractor's block:
```ruby
result = Ractor.new do
heavy_task()
end.value
```
You can wait for the termination of a Ractor with `Ractor#join` like this:
```ruby
Ractor.new do
some_task()
end.join
```
`#value` and `#join` are similar to `Thread#value` and `Thread#join`.
To implement `#join`, `Ractor#monitor` (and `Ractor#unmonitor`) is introduced.
This commit changes `Ractor.select()` method.
It now only accepts ports or Ractors, and returns when a port receives a message or a Ractor terminates.
We removes `Ractor.yield` and `Ractor#take` because:
* `Ractor::Port` supports most of similar use cases in a simpler manner.
* Removing them significantly simplifies the code.
We also change the internal thread scheduler code (thread_pthread.c):
* During barrier synchronization, we keep the `ractor_sched` lock to avoid deadlocks.
This lock is released by `rb_ractor_sched_barrier_end()`
which is called at the end of operations that require the barrier.
* fix potential deadlock issues by checking interrupts just before setting UBF.
https://bugs.ruby-lang.org/issues/21262
Notes:
Merged: https://github.com/ruby/ruby/pull/13445
|
|
On arm64 macOS, libunwind (both of system library and homebrew
llvm-18) seems not to handle our coroutine switching code.
Notes:
Merged: https://github.com/ruby/ruby/pull/12789
|
|
In this case thread_sched_switch0 never returns, so we would never
end up setting finished to true.
Fixes [Bug #20638]
Notes:
Merged: https://github.com/ruby/ruby/pull/12080
|
|
Use PR_SET_VMA_ANON_NAME to set human-readable names for anonymous
virtual memory areas mapped by `mmap()` when compiled and run on Linux
5.17 or higher. This makes it convenient for developers to debug mmap.
Notes:
Merged: https://github.com/ruby/ruby/pull/12119
|
|
The M:N threading stack cleanup machinery tries to call MADV_FREE on the native
thread's stack, and calls rb_bug if it fails. Unfortunately, there's no way to
distinguish between "You passed bad parameters to madvise" and "MADV_FREE is
not supported on the kernel you are running on"; both cases just return EINVAL.
This means that if you have a Ruby on a system that was built on a system with
MADV_FREE and run it on a system without it, you get a crash in nt_free_stack.
I ran into this because rr actually emulates MADV_FREE by just returning EINVAL
and pretending it's not supported (since it can otherwise introduce
nondeterministic behaviour). So if you run bootstraptest/test_ractor.rb under
rr, you get this crash.
I think we should just get rid of the error handling here; freeing memory like
this is strictly optional anyway.
[Bug #20632]
Notes:
Merged: https://github.com/ruby/ruby/pull/11156
|
|
Introduce `struct rb_thread_sched_waiting` and `timer_th.waiting`
can contain other than `rb_thread_t`.
|
|
Signed-off-by: cui fliter <imcusg@gmail.com>
|
|
Introduce `rb_thread_lock_native_thread()` to allocate dedicated
native thread to the current Ruby thread for M:N threads.
This C API is similar to Go's `runtime.LockOSThread()`.
Accepted at https://github.com/ruby/dev-meeting-log/blob/master/2023/DevMeeting-2023-08-24.md
(and missed to implement on Ruby 3.3.0)
|
|
```
compiling ../thread.c
In file included from ../thread.c:263:
In file included from ../thread_pthread.c:2870:
../thread_pthread_mn.c:777:43: warning: format specifies type 'unsigned long' but the argument has type 'rb_hrtime_t' (aka 'unsigned long long') [-Wformat]
RUBY_DEBUG_LOG("abs:%lu", abs);
~~~ ^~~
%llu
../vm_debug.h:110:74: note: expanded from macro 'RUBY_DEBUG_LOG'
ruby_debug_log(__FILE__, __LINE__, RUBY_FUNCTION_NAME_STRING, "" __VA_ARGS__); \
^~~~~~~~~~~
1 warning generated.
```
|
|
In a similar way to how we do it with fibers in cont.c, we need to call
__sanitize_start_switch_fiber and __sanitize_finish_switch_fiber around
the call to coroutine_transfer to let ASAN save & restore the fake stack
pointer.
When a M:N thread is exiting, we pass `to_dead` to the new
coroutine_transfer0 function, so that we can pass NULL for saving the
stack pointer. This signals to ASAN that the fake stack can be freed
(otherwise it would be leaked)
[Bug #20220]
|
|
|
|
`timer_th.waiting` should be protected by `timer_th.waiting_lock`
|
|
* In the epoll implementation, you get an EEXIST if you try to register the same event for the same fd more than once for a particular epoll instance
* Otherwise kevent will just override the previous event registration, and if multiple threads listen on the same fd only the last one to register will ever finish, the others are stuck
* This approach will lead to native threads getting created, similar to the epoll implementation. This is not ideal, but it fixes certain test cases for now, like test/socket/test_tcp.rb#test_accept_multithread
|
|
|
|
* When we have the thread already, it saves a lookup
* `event_wait`, not `kq`
Clean up the `thread_sched_wait_events_timeval` calls
* By handling the PTHREAD check inside the function, all the other code can become much simpler and just call the function directly without additional checks
|
|
* Allows macOS users to use M:N threads (and technically FreeBSD, though it has not been verified on FreeBSD)
* Include sys/event.h header check for macros, and include sys/event.h when present
* Rename epoll_fd to more generic kq_fd (Kernel event Queue) for use by both epoll and kqueue
* MAP_STACK is not available on macOS so conditionall apply it to mmap flags
* Set fd to close on exec
* Log debug messages specific to kqueue and epoll on creation
* close_invalidate raises an error for the kqueue fd on child process fork. It's unclear rn if that's a bug, or if it's kqueue specific behavior
Use kq with rb_thread_wait_for_single_fd
* Only platforms with `USE_POLL` (linux) had changes applied to take advantage of kernel event queues. It needed to be applied to the `select` so that kqueue could be properly applied
* Clean up kqueue specific code and make sure only flags that were actually set are removed (or an error is raised)
* Also handle kevent specific errnos, since most don't apply from epoll to kqueue
* Use the more platform standard close-on-exec approach of `fcntl` and `FD_CLOEXEC`. The io-event gem uses `ioctl`, but fcntl seems to be the recommended choice. It is also what Go, Bun, and Libuv use
* We're making changes in this file anyways - may as well fix a couple spelling mistakes while here
Make sure FD_CLOEXEC carries over in dup
* Otherwise the kqueue descriptor should have FD_CLOEXEC, but doesn't and fails in assert_close_on_exec
|
|
|
|
This reverts commit ad54fbf281ca1935e79f4df1460b0106ba76761e.
|
|
[Bug #20019]
This fixes GVL instrumentation in three locations it was missing:
- Suspending when blocking on a Ractor
- Suspending when doing a coroutine transfer from an M:N thread
- Resuming after an M:N thread starts
Co-authored-by: Matthew Draper <matthew@trebex.net>
|
|
|
|
|
|
Some systems use not 4096 page size (64KB for example).
|
|
This patch introduce M:N thread scheduler for Ractor system.
In general, M:N thread scheduler employs N native threads (OS threads)
to manage M user-level threads (Ruby threads in this case).
On the Ruby interpreter, 1 native thread is provided for 1 Ractor
and all Ruby threads are managed by the native thread.
From Ruby 1.9, the interpreter uses 1:1 thread scheduler which means
1 Ruby thread has 1 native thread. M:N scheduler change this strategy.
Because of compatibility issue (and stableness issue of the implementation)
main Ractor doesn't use M:N scheduler on default. On the other words,
threads on the main Ractor will be managed with 1:1 thread scheduler.
There are additional settings by environment variables:
`RUBY_MN_THREADS=1` enables M:N thread scheduler on the main ractor.
Note that non-main ractors use the M:N scheduler without this
configuration. With this configuration, single ractor applications
run threads on M:1 thread scheduler (green threads, user-level threads).
`RUBY_MAX_CPU=n` specifies maximum number of native threads for
M:N scheduler (default: 8).
This patch will be reverted soon if non-easy issues are found.
[Bug #19842]
|
