| Age | Commit message (Collapse) | Author |
|---|
|
[Backport #21342]"
This reverts commit c414b9871f263331cde0af1c08cf5c1a47e1aedf.
|
|
[Backport #21342]"
This reverts commit 4306c9048fb674d24b92dc46b6746a4749564147.
|
|
Properly unlock locked mutexes on thread cleanup.
Mutexes were being improperly unlocked on thread cleanup. This bug was
introduced in 050a8954395.
We must keep a reference from the mutex to the thread, because if the fiber
is collected before the mutex is, then we cannot unlink it from the thread in
`mutex_free`. If it's not unlinked from the the thread when it's freed, it
causes bugs in `rb_thread_unlock_all_locking_mutexes`.
We now mark the fiber when a mutex is locked, and the thread is marked
as well. However, a fiber can still be freed in the same GC cycle as the
mutex, so the reference to the thread is still needed.
The reason we need to mark the fiber is that `mutex_owned_p()` has an ABA
issue where if the fiber is collected while it's locked, a new fiber could be
allocated at the same memory address and we could get false positives.
Fixes [Bug #21342]
Co-authored-by: John Hawthorn <john@hawthorn.email>
|
|
Fix coverage measurement for negative line numbers
Fixes [Bug #21220]
Co-Authored-By: Mike Bourgeous <mike@mikebourgeous.com>
Co-Authored-By: Jean Boussier <jean.boussier@gmail.com>
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/12328
|
|
This commit adds an environment variable `RUBY_THREAD_TIMESLICE` for
specifying the default thread quantum in milliseconds. You can adjust
this variable to tune throughput, which is especially useful on
multithreaded systems that are mixing CPU bound work and IO bound work.
The default quantum remains 100ms.
[Feature #20861]
Co-Authored-By: John Hawthorn <john@hawthorn.email>
Notes:
Merged: https://github.com/ruby/ruby/pull/11981
|
|
Redirect `rb_nogvl` blocking operations to the fiber scheduler if possible
to prevent stalling the event loop.
[Feature #20876]
Notes:
Merged-By: ioquatix <samuel@codeotaku.com>
|
|
to avoid TLS issue with N:M threads.
Notes:
Merged: https://github.com/ruby/ruby/pull/11142
|
|
introduce
- rb_threadptr_interrupt_exec
- rb_ractor_interrupt_exec
to intercept the thread/ractor execution.
Notes:
Merged: https://github.com/ruby/ruby/pull/11142
|
|
Notes:
Merged-By: ioquatix <samuel@codeotaku.com>
|
|
This reverts some of commit 87fb44dff6409a19d12052cf0fc07ba80a4c45ac.
We will rename and propose a slightly different interface.
Notes:
Merged-By: ioquatix <samuel@codeotaku.com>
|
|
`RUBY_VM_CRITICAL_SECTION` is not used anywhere.
|
|
Notes:
Merged-By: ioquatix <samuel@codeotaku.com>
|
|
If one thread is reading and another closes that socket, the close
blocks waiting for the read to abort cleanly. This ensures that Ruby is
totally done with the file descriptor _BEFORE_ we tell the OS to close
and potentially re-use it.
When the read is correctly terminated, the close should be unblocked.
That currently works if closing is happening on a thread, but if it's
happening on a fiber with a fiber scheduler, it does NOT work.
This patch ensures that if the close happened in a fiber scheduled
thread, that the scheduler is notified that the fiber is unblocked.
[Bug #20723]
Notes:
Merged: https://github.com/ruby/ruby/pull/11614
|
|
Fixes this compiler warning:
thread.c:4530:18: warning: storing the address of local variable ‘stack_end’ in ‘*stack_end_p’ [-Wdangling-pointer=]
4530 | *stack_end_p = &stack_end;
| ~~~~~~~~~~~~~^~~~~~~~~~~~
|
|
* This PR from the timeout gem (https://github.com/ruby/timeout/pull/30) made it so you have to handle_interrupt on Timeout::ExitException instead of Timeout::Error
* Efficiency changes to the gem (one shared thread) mean you can't consistently handle timeout errors using handle_timeout: https://github.com/ruby/timeout/issues/41
Notes:
Merged: https://github.com/ruby/ruby/pull/11474
|
|
Previously, a TypeError was not raised if there were no thread
variables, because the conversion to symbol was done after that
check. Convert to symbol before checking for whether thread
variables are set to make the behavior consistent.
Fixes [Bug #20606]
|
|
* Speed up chunkypng benchmark
Since d037c5196a14c03e72746ccdf0437b5dd4f80a69 we're seeing a slowdown
in ChunkyPNG benchmarks in YJIT bench. This patch addresses the
slowdown. Making the thread volatile speeds up the benchmark by 2 or 3%
on my machine.
```
before: ruby 3.4.0dev (2024-07-02T18:48:43Z master b2b8306b46) [x86_64-linux]
after: ruby 3.4.0dev (2024-07-02T20:07:44Z speed-chunkypng 418334dba9) [x86_64-linux]
---------- ----------- ---------- ---------- ---------- ------------- ------------
bench before (ms) stddev (%) after (ms) stddev (%) after 1st itr before/after
chunky-png 1000.2 0.1 980.6 0.1 1.02 1.02
---------- ----------- ---------- ---------- ---------- ------------- ------------
Legend:
- after 1st itr: ratio of before/after time for the first benchmarking iteration.
- before/after: ratio of before/after time. Higher is better for after. Above 1 represents a speedup.
Output:
./data/output_015.csv
```
* Update thread.c
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
---------
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
|
|
|
|
|
|
At 7afc16aa48beb093b06eb978bc430f90dd771690, now `BLOCKING_REGION`
contains `setjmp` call in `RB_VM_SAVE_MACHINE_CONTEXT`. By this
change, variables in blocks for this macro may be clobbered.
|
|
There's an exhaustive explanation of this in the linked redmine bug, but
the short version is as follows:
blocking_region_begin can spill callee-saved registers to the stack for
its own use. That means they're not saved to ec->machine by the call to
setjmp, since by that point they're already on the stack and new,
different values are in the real registers. ec->machine's end-of-stack
pointer is also bumped to accomodate this, BUT, after
blocking_region_begin returns, that points past the end of the stack!
As far as C is concerned, that's fine; the callee-saved registers are
restored when blocking_region_begin returns. But, if another thread
triggers GC, it is relying on finding references to Ruby objects by
walking the stack region pointed to by ec->machine.
If the C code in exec; subsequently does things that use that stack
memory, then the value will be overwritten and the GC might prematurely
collect something it shouldn't.
[Bug #20493]
|
|
Previously it would bypass the `FL_ABLE` check, but
since shapes introduction, it started having a different
behavior than `OBJ_FREEZE`, as it would onyl set the `FL_FREEZE`
flag, but not update the shape.
I have no indication of this causing a bug yet, but it seems
like a trap waiting to happen.
|
|
|
|
The return value is used as a boolean value in C. Since it's not used as
a Ruby object, it just seems confusing that it returns a VALUE.
|
|
similarly to 332f4938cf3adbff8f15b647767dc660583a5bef
|
|
It gives true/TRUE (int) instead of Qtrue (VALUE).
|
|
It's really a property of the EC; each fiber (which has its own EC) also
has its own asan_fake_stack_handle.
[Bug #20310]
|
|
|
|
|
|
|
|
|
|
|
|
[Feature #16495]
|
|
|
|
ASAN leaves a pointer to the fake frame on the stack; we can use the
__asan_addr_is_in_fake_stack API to work out the extent of the fake
stack and thus mark any VALUEs contained therein.
[Bug #20001]
|
|
This commit changes how stack extents are calculated for both the main
thread and other threads. Ruby uses the address of a local variable as
part of the calculation for machine stack extents:
* pthreads uses it as a lower-bound on the start of the stack, because
glibc (and maybe other libcs) can store its own data on the stack
before calling into user code on thread creation.
* win32 uses it as an argument to VirtualQuery, which gets the extent of
the memory mapping which contains the variable
However, the local being used for this is actually too low (too close to
the leaf function call) in both the main thread case and the new thread
case.
In the main thread case, we have the `INIT_STACK` macro, which is used
for pthreads to set the `native_main_thread->stack_start` value. This
value is correctly captured at the very top level of the program (in
main.c). However, this is _not_ what's used to set the execution context
machine stack (`th->ec->machine_stack.stack_start`); that gets set as
part of a call to `ruby_thread_init_stack` in `Init_BareVM`, using the
address of a local variable allocated _inside_ `Init_BareVM`. This is
too low; we need to use a local allocated closer to the top of the
program.
In the new thread case, the lolcal is allocated inside
`native_thread_init_stack`, which is, again, too low.
In both cases, this means that we might have VALUEs lying outside the
bounds of `th->ec->machine.stack_{start,end}`, which won't be marked
correctly by the GC machinery.
To fix this,
* In the main thread case: We already have `INIT_STACK` at the right
level, so just pass that local var to `ruby_thread_init_stack`.
* In the new thread case: Allocate the local one level above the call to
`native_thread_init_stack` in `call_thread_start_func2`.
[Bug #20001]
fix
|
|
This reverts commit 4ba8f0dc993953d3ddda6328e3ef17a2fc2cbde5.
|
|
This reverts commit d10bc3a2b8300cffc383e10c3730871e851be24c.
|
|
ASAN leaves a pointer to the fake frame on the stack; we can use the
__asan_addr_is_in_fake_stack API to work out the extent of the fake
stack and thus mark any VALUEs contained therein.
[Bug #20001]
|
|
The implementation of `native_thread_init_stack` for the various
threading models can use the address of a local variable as part of the
calculation of the machine stack extents:
* pthreads uses it as a lower-bound on the start of the stack, because
glibc (and maybe other libcs) can store its own data on the stack
before calling into user code on thread creation.
* win32 uses it as an argument to VirtualQuery, which gets the extent of
the memory mapping which contains the variable
However, the local being used for this is actually allocated _inside_
the `native_thread_init_stack` frame; that means the caller might
allocate a VALUE on the stack that actually lies outside the bounds
stored in machine.stack_{start,end}.
A local variable from one level above the topmost frame that stores
VALUEs on the stack must be drilled down into the call to
`native_thread_init_stack` to be used in the calculation. This probably
doesn't _really_ matter for the win32 case (they'll be in the same
memory mapping so VirtualQuery should return the same thing), but
definitely could matter for the pthreads case.
[Bug #20001]
|
|
`rb_thread_wait_for_single_fd(fd)` waits until `fd` is ready.
Without MN it shouldn't use `thread_io_wait_events()` for the
retry checking (alwasy false if MN is not active).
|
|
|
|
Before this patch, the MN scheduler waits for the IO with the
following steps:
1. `poll(fd, timeout=0)` to check fd is ready or not.
2. if fd is not ready, waits with MN thread scheduler
3. call `func` to issue the blocking I/O call
The advantage of advanced `poll()` is we can wait for the
IO ready for any fds. However `poll()` becomes overhead
for already ready fds.
This patch changes the steps like:
1. call `func` to issue the blocking I/O call
2. if the `func` returns `EWOULDBLOCK` the fd is `O_NONBLOCK`
and we need to wait for fd is ready so that waits with MN
thread scheduler.
In this case, we can wait only for `O_NONBLOCK` fds. Otherwise
it waits with blocking operations such as `read()` system call.
However we don't need to call `poll()` to check fd is ready
in advance.
With this patch we can observe performance improvement
on microbenchmark which repeats blocking I/O (not
`O_NONBLOCK` fd) with and without MN thread scheduler.
```ruby
require 'benchmark'
f = open('/dev/null', 'w')
f.sync = true
TN = 1
N = 1_000_000 / TN
Benchmark.bm{|x|
x.report{
TN.times.map{
Thread.new{
N.times{f.print '.'}
}
}.each(&:join)
}
}
__END__
TN = 1
user system total real
ruby32 0.393966 0.101122 0.495088 ( 0.495235)
ruby33 0.493963 0.089521 0.583484 ( 0.584091)
ruby33+MN 0.639333 0.200843 0.840176 ( 0.840291) <- Slow
this+MN 0.512231 0.099091 0.611322 ( 0.611074) <- Good
```
|
|
Assrsion was `events == RB_WAITFD_IN || events == RB_WAITFD_OUT`
but it should accept `RB_WAITFD_IN | RB_WAITFD_OUT`.
|
|
If the Fiber is nonblocking mode, fiber scheduler needs to handle
IO events.
|
|
Introduce `thread_io_wait_events()` to make 1 function to call
`thread_sched_wait_events()`.
In ``thread_io_wait_events()`, manipulate `waiting_fd` to raise
an exception when closing the IO correctly.
|
|
* When we have the thread already, it saves a lookup
* `event_wait`, not `kq`
Clean up the `thread_sched_wait_events_timeval` calls
* By handling the PTHREAD check inside the function, all the other code can become much simpler and just call the function directly without additional checks
|
|
* Allows macOS users to use M:N threads (and technically FreeBSD, though it has not been verified on FreeBSD)
* Include sys/event.h header check for macros, and include sys/event.h when present
* Rename epoll_fd to more generic kq_fd (Kernel event Queue) for use by both epoll and kqueue
* MAP_STACK is not available on macOS so conditionall apply it to mmap flags
* Set fd to close on exec
* Log debug messages specific to kqueue and epoll on creation
* close_invalidate raises an error for the kqueue fd on child process fork. It's unclear rn if that's a bug, or if it's kqueue specific behavior
Use kq with rb_thread_wait_for_single_fd
* Only platforms with `USE_POLL` (linux) had changes applied to take advantage of kernel event queues. It needed to be applied to the `select` so that kqueue could be properly applied
* Clean up kqueue specific code and make sure only flags that were actually set are removed (or an error is raised)
* Also handle kevent specific errnos, since most don't apply from epoll to kqueue
* Use the more platform standard close-on-exec approach of `fcntl` and `FD_CLOEXEC`. The io-event gem uses `ioctl`, but fcntl seems to be the recommended choice. It is also what Go, Bun, and Libuv use
* We're making changes in this file anyways - may as well fix a couple spelling mistakes while here
Make sure FD_CLOEXEC carries over in dup
* Otherwise the kqueue descriptor should have FD_CLOEXEC, but doesn't and fails in assert_close_on_exec
|
|
`thread_sched_wait_events()` suspend the thread until the target
fd is ready. Howver, other threads can close the target fd and
suspended thread should be awake. To support it, setup `waiting_fd`
before `thread_sched_wait_events()`.
`rb_thread_io_wake_pending_closer()` should be called before
`RUBY_VM_CHECK_INTS_BLOCKING()` because it can return this function.
This patch introduces additional overhead (setup/cleanup `waiting_fd`)
and maybe we can reduce the overhead.
|
