summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2018-07-30thread_pthread.c (unregister_ubf_list): assert unblock.func is unsetnormal
We must not allow reentry into ubf_list_head once we delete ourselves, otherwise we could hang in there forever. [ruby-core:88218] [Bug #14945] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64134 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-30thread_pthread.c (gvl_yield): do ubf wakeups when uncontendednormal
Not having contention for GVL could mean everybody else is stuck in blocking region without GVL, so we kick the ubf list in that case. I expect this to fix test_thread_fd_close timeout: http://ci.rvm.jp/results/trunk-test@ruby-sky3/1173398 git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64133 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-30* 2018-07-31svn
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64132 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-30thread_pthread.c: revert r64123nobu
* thread_pthread.c (USE_NATIVE_SLEEP_COND): revised wrongly removed line with the ifndef guard. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64131 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-30thread_pthread.h (native_thread_data): split condvars on some platformsnormal
Maybe some platforms have strange condition variable implementations which have a "memory" of which mutexes they're associated with. In any case, it makes documentation easier even on GNU/Linux and FreeBSD. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64124 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-30thread_pthread.c: remove duplicate definenobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64123 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-30fix typo(#2), sorry!shyouhei
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64122 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-30fix typo; sorry!shyouhei
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64121 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-30bootstraptest/runner.rb: speed up assert_finish with IO.select (take #2)normal
Resurrect r63754 in a 1.8-compatible way. While we're at it, add a note to maintain 1.8 compatibility (cf. r63757). git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64120 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-30move #pragma out of functionsshyouhei
gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) reportedly fails to compile cf: https://rubyci.org/logs/rubyci.s3.amazonaws.com/centos6/ruby-trunk/log/20180726T093003Z.fail.html.gz git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64119 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-30reduce copy & pasteshyouhei
We see several occurrence of "diagnostic push/pop" so why not make them macros. Tested on GCC8 / Clang 6. Note that ruby.h is intentionally left untouched because we don't want to introduce new public macros. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64118 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-30process.c (waitpid_nogvl): prevent conflicting use of sleep_condnormal
We reuse sleep_cond for waitpid notifications as well as GVL waiting. So we must take care to not hold onto sleep_cond when we try to reacquire GVL. [ruby-core:88183] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64117 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-30thread_pthread.c (rb_sigwait_sleep): th may be 0 from MJITnormal
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64116 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-30thread*.c: waiting on sigwait_fd performs periodic ubf wakeupsnormal
We need to be able to perform periodic ubf_list wakeups when a thread is sleeping and waiting on signals. [ruby-core:88088] [Misc #14937] [Bug #5343] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64115 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-30thread_pthread.c (gvl_destroy): make no-op on GVL bitsnormal
It's possible for another thread to take vm->gvl.lock during gvl_release at the end of thread_start_func_2 during VM shutdown, at least. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64114 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-30thread_pthread.c (rb_sigwait_sleep): fix uninitialized poll set in UBF casenormal
[ruby-core:88088] [Misc #14937] [Bug #5343] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64113 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-30non-constant aggregate initializer is a C99ismshyouhei
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64112 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-30thread_pthread.c (rb_sigwait_sleep): re-fix [Bug #5343] hardernormal
We can't always designate a timer thread, so any sleepers must also perform ubf wakeups. Note: a similar change needs to be made for rb_thread_fd_select and rb_wait_for_single_fd. [ruby-core:88088] [Misc #14937] [Bug #5343] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64111 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-30thread.c: move ppoll wrapper into thread_pthread.cnormal
thread_pthread.c relies on ppoll for rb_sigwait_sleep, so ensure the compatibility wrapper is available for it. [Bug #14950] Reported-by: SHIBATA Hiroshi <hsbt@ruby-lang.org> Reported-by: Greg L <Greg.mpls@gmail.com> git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64110 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-29thread.c: move ppoll wrapper before thread_pthread.cnormal
thread_pthread.c relies on ppoll for rb_sigwait_sleep, so ensure the compatibility wrapper is available for it. Reported-by: SHIBATA Hiroshi <hsbt@ruby-lang.org> git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64109 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-29thread_pthread.c (ubf_select): refix [Bug #5343]normal
We still need to to designate a timer thread after registering target thread for the ubf list. Oops :x Note: I was never able to reproduce test/thread/test_queue.rb::test_thr_kill failures on my on Debian machines. [ruby-core:88088] [Misc #14937] [Bug #5343] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64108 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-29thread_pthread: remove timer-thread by restructuring GVLnormal
To reduce resource use and reduce CI failure; remove timer-thread. Single-threaded Ruby processes (including forked children) will never see extra thread overhead. This prevents glibc and jemalloc from going into multi-threaded mode and initializing locks or causing fragmentation via arena explosion. The GVL is implements its own wait-queue as a ccan/list to permit controlling wakeup order. Timeslice under contention is handled by a designated timer thread (similar to choosing a "patrol_thread" for current deadlock checking). There is only one self-pipe, now, as wakeups for timeslice are done independently using condition variables. This reduces FD pressure slightly. Signal handling is handled directly by a Ruby Thread (instead of timer-thread) by exposing signal self-pipe to callers of rb_thread_fd_select, native_sleep, rb_wait_for_single_fd, etc... Acquiring, using, and releasing the self-pipe is exposed via 4 new internal functions: 1) rb_sigwait_fd_get - exclusively acquire timer_thread_pipe.normal[0] 2) rb_sigwait_fd_sleep - sleep and wait for signal (and no other FDs) 3) rb_sigwait_fd_put - release acquired result from rb_sigwait_fd_get 4) rb_sigwait_fd_migrate - migrate signal handling to another thread after calling rb_sigwait_fd_put. rb_sigwait_fd_migrate is necessary for waitpid callers because only one thread can wait on self-pipe at a time, otherwise a deadlock will occur if threads fight over the self-pipe. TRAP_INTERRUPT_MASK is now set for the main thread directly in signal handler via rb_thread_wakeup_timer_thread. Originally, I wanted to use POSIX timers (timer_create/timer_settime) for this. Unfortunately, this proved unfeasible as Mutex#sleep resumes on spurious wakeups and test/thread/test_cv.rb::test_condvar_timed_wait failed. Using pthread_sigmask to mask out SIGVTALRM fixed that test, but test/fiddle/test_function.rb::test_nogvl_poll proved there'd be some unavoidable (and frequent) incompatibilities from that approach. Finally, this allows us to drop thread_destruct_lock and interrupt current ec directly. We don't need to rely on vm->thread_destruct_lock or a coherent vm->running_thread on any platform. Separate timer-thread for time slice and signal handling is relegated to thread_win32.c, now. [ruby-core:88088] [Misc #14937] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64107 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-29* 2018-07-30svn
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64106 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-29test/ruby/test_process.rb (test_wait_and_sigchld): improve reliabilitynormal
This should fix CI failure under MJIT, 100ms wait may not be enough for signal handler to fire: http://ci.rvm.jp/results/trunk-mjit@silicon-docker/1169472 git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64105 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-29common.mk: install for test-all iff load-relative is disablednobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64104 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-29common.mk: test-all requires install for nowk0kubun
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64103 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-29thread_pthread.c: clear altstacks in thread cache at GVL destructionnormal
Otherwise, an altstack may live past ObjectSpace destruction and xfree-ing the altstack will segfault. [ruby-core:85621] [Feature #14487] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64102 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-29reduce LibreSSL warningsnobu
* ext/openssl/extconf.rb: LibreSSL headers emit "overriding WinCrypt defines" warnings if wincrypt.h has been included (except for x509.h) on Windows. get rid of including the header by defining NOCRYPT macro. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64101 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-29mjit.c: keep unit->o_file on --jit-save-tempsk0kubun
to use compaction with --jit-save-temps. Prior to this commit, JIT compaction didn't work with --jit-save-temps but it wasn't intentional. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64100 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-29test_jit.rb: skip known test failure on MinGWk0kubun
for the ease of MSP-Greg's CI. I'll track this failure as Bug#14948 and fix it later. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64099 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-29test_jit.rb: use the appropiate namek0kubun
for better C-level backtrace git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64098 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-29mjit.c: disable compaction on empty queue w/ --jit-waitk0kubun
When --jit-wait is specified, `unit_queue.length` is always 0 and it's not a good metric. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64097 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-29repack structsnobu
Repack rb_thread_struct, rb_execution_context_struct, args_info and iseq_compile_data to save 1 word per struct. re_pattern_buffer remains unpacked due to the possible binary compatibility. [Fix GH-1907] Based on the patch From: Lourens Naudé <lourens@bearmetal.eu> git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64096 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-29mkexports.rb: flip-flopnobu
* win32/mkexports.rb (Exports::Mswin#each_export): get rid of flip-flop warning. [ruby-core:88147] [Bug #14946] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64095 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-28mjit.c: introduce JIT compaction [experimental]k0kubun
When all compilation finishes or the number of JIT-ed code reaches --jit-max-cache, this compacts all generated code to a single .so file and re-loads all methods from it. In the future, it may trigger compaction more frequently and/or limit the maximum times of compaction to prevent unlimited memory usage. So the current behavior is experimental, but at least the performance improvement in this commit won't be removed. === Benchmark === In this benchmark, I'll compare following four conditions: * trunk: r64082 * trunk JIT: r64082 w/ --jit * single-so JIT: This commit w/ --jit * objfcn JIT: This branch https://github.com/k0kubun/ruby/tree/objfcn w/ --jit, which is shinh's objfcn https://github.com/shinh/ruby/tree/objfcn rebased from this commit ``` $ uname -a Linux bionic 4.15.0-29-generic #31-Ubuntu SMP Tue Jul 17 15:39:52 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux ``` * Micro benchmark Using this script https://gist.github.com/k0kubun/10e6d3387c9ab1b134622b2c9d76ef51, calls some amount of different methods that just return `nil`. The following tables are its average duration seconds of 3 measurements. Smaller is better. ** 1 method (seconds) | | trunk | trunk JIT | single-so JIT | objfcn JIT | |:------|:------------------|:------------------|:------------------|:------------------| | Time | 5.576067774333296 | 5.915551971666446 | 5.833641665666619 | 5.845915191666639 | | Ratio | 1.00x | 1.06x | 1.05x | 1.05x | ** 50 methods (seconds) | | trunk | trunk JIT | single-so JIT | objfcn JIT | |:------|:------------------|:------------------|:------------------|:------------------| | Time | 3.1661167996666677| 6.125825928333342 | 4.135432743666665 | 3.750358728333348 | | Ratio | 1.00x | 1.93x | 1.31x | 1.18x | ** 1500 methods (seconds) | | trunk | trunk JIT | single-so JIT | objfcn JIT | |:------|:------------------|:------------------|:------------------|:------------------| | Time | 5.971650823666664 | 19.579182102999994| 10.511108153999961| 10.854653588999932| | Ratio | 1.00x | 3.28x | 1.76x | 1.82x | * Discourse Using the same benchmark strategy as https://bugs.ruby-lang.org/issues/14490 with this branch https://github.com/k0kubun/discourse/commits/benchmark2 forked from discourse v1.8.11 to support running trunk. 1. Run ruby script/bench.rb to warm up profiling database 2. Run RUBYOPT='--jit-verbose=1 --jit-max-cache=10000' RAILS_ENV=profile bin/puma -e production 3. WAIT 5-15 or so minutes for all jitting to stop so we have no cross talk 4. Run ab -n 100 http://localhost:9292/ 5. Wait for all new jitting to finish 6. Run ab -n 100 http://localhost:9292/ ** Response time (ms) Here is the response time milliseconds for each percentile. Skipping 99%ile because it's the same as 100%ile in 100 calls. | | trunk| trunk|single|objfcn| | | | JIT|so JIT| JIT| |:----|:-----|:-----|:-----|:-----| | 50% | 38 | 45 | 41 | 43 | | 66% | 39 | 50 | 44 | 44 | | 75% | 47 | 51 | 46 | 45 | | 80% | 49 | 52 | 47 | 47 | | 90% | 50 | 63 | 50 | 52 | | 95% | 60 | 79 | 52 | 55 | | 98% | 91 | 114 | 91 | 91 | |100% | 97 | 133 | 96 | 99 | ** Ratio (smaller is better) Here is the response time increase ratio against no-JIT trunk's one. | | trunk| trunk|single|objfcn| | | | JIT|so JIT| JIT| |:----|:-----|:-----|:-----|:-----| | 50% | 1.00x| 1.18x| 1.08x| 1.13x| | 66% | 1.00x| 1.28x| 1.13x| 1.13x| | 75% | 1.00x| 1.09x| 0.98x| 0.96x| | 80% | 1.00x| 1.06x| 0.96x| 0.96x| | 90% | 1.00x| 1.26x| 1.00x| 1.04x| | 95% | 1.00x| 1.32x| 0.87x| 0.92x| | 98% | 1.00x| 1.25x| 1.00x| 1.00x| |100% | 1.00x| 1.37x| 0.99x| 1.02x| While 50 and 60 %ile are still worse than no-JIT trunk, 75, 80, 90, 95, 98 and 100% are not slower than that. So now it's a little harder to say "MJIT slows down Rails applications". Probably I can close [Bug #14490] now. Let's start improving it. Close https://github.com/ruby/ruby/pull/1921 git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64094 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-28* 2018-07-29svn
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64093 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-28win32.c: limit write size on consolenobu
* win32/win32.c (constat_parse): split long buffer and limit write size on a console, as well as rb_w32_write. [ruby-dev:50597] [Bug #14942] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64092 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-28test_function.rb: fix messagesnobu
* test/fiddle/test_function.rb (test_nogvl_poll): fix messages as failed conditions, with errno description. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64091 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-28make-snapshot: fetch from the official git sitenobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64090 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-28Fix filename in comment [ci skip]kazu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64089 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-28use https:// instead of http://kazu
[ci skip] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64088 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-28fix range check for Hangul jamo trailers in Unicode normalizationduerst
* lib/unicode_normalize/normalize.rb: Fix the range check for trailing Hangul jamo characters in Unicode normalization. Different from leading or vowel jamos, where LBASE and VBASE are actual characters, a value equal to TBASE expresses the absence of a trailing jamo. This fix is technically correct, but there was no bug because the regular expressions in lib/unicode_normalize/tables.rb eliminate jamos equal to TBASE from normalization processing. * test/test_unicode_normalize.rb: Add preventive test test_no_trailing_jamo based on https://github.com/python/cpython/commit/d134809cd3764c6a634eab7bb8995e3e2eff14d5 just for the case we ever get a regression. This closes issue #14934, thanks to MaLin (Lin Ma) for reporting. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64087 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-28always run unicode normalization that do not depend on data fileduerst
* test/test_unicode_normalize.rb: extract tests that do not depend on NormalizationTest.txt data file from conditionally constructed part of TestUnicodeNormalize class, to always run them even if the data file isn't found. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64086 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-28update unicode data files directory for normalization testsduerst
* test/test_unicode_normalize.rb: search ucd directory first if it exists. This change follows r61415. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64085 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-28use https:// instead of git:// when possiblenormal
Avoid MitM when downloading from insecure networks. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64084 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-28thread.c (blocking_region_end): clear ubf before unregister_ubf_listnormal
If we keep ubf set after unregistering, there is a window for other threads (including timer thread) to put this thread back on the ubf_list right away. Entering ubf_list unexpectedly after GVL acquisition may cause spurious wakeup and trigger unexpected behavior. Finally, clear ubf before acquiring GVL, to since ubf is useless during GVL acquisition anyways and we don't want to waste cycles in other threads calling ubf for useless work. [ruby-core:88141] [Bug #14945] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64083 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-27mjit.c: clean up unit link from iseqk0kubun
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64082 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-27Include Hash#size in the examplestenderlove
Both methods Hash#length and Hash#size share the same source code in Ruby, but they also share the same documentation. Now when you look at the documentation of Hash#size you only see examples for Hash#length, which is confusing. This commit includes Hash#size in the examples and also remarks that both methods are equivalent to each other. Co-authored-by: Alberto Almagro <alberto.almagro@rakuten.com> git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64081 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-27* 2018-07-28svn
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64080 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-07-27Escape debug output in InvalidURIError exceptions.tenderlove
Co-authored-by: Brad Landers <brad@bradlanders.com> git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64079 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-02 02:26:02 +0000