summaryrefslogtreecommitdiff
path: root/mjit.c
AgeCommit message (Collapse)Author
2020-06-09mjit_mark_cc_entries: should consider VM_CALLCACHE_UNMARKABLE卜部昌平
Now that vm_empty_cc is VM_CALLCACHE_UNMARKABLE, it has to be properly ruled out from being GCed.
2020-05-26Eliminate a call instruction on JIT cancel pathTakashi Kokubun
by calling combined functions specialized for each cancel type. I'm hoping to improve locality of hot code, but this patch's impact should be insignificant.
2020-05-11sed -i 's|ruby/impl|ruby/internal|'卜部昌平
To fix build failures. Notes: Merged: https://github.com/ruby/ruby/pull/3079
2020-05-11sed -i s|ruby/3|ruby/impl|g卜部昌平
This shall fix compile errors. Notes: Merged: https://github.com/ruby/ruby/pull/3079
2020-05-01Deduplicate functions in compacted JIT codeTakashi Kokubun
to improve code locality. Using benchmark-driver/sinatra with 100 methods JIT-ed, [Before] 12149.97 rps 1.3M /tmp/_ruby_mjit_p31171u145.so [After] 12818.83 rps 260K /tmp/_ruby_mjit_p32155u145.so (VM is 13714.89 rps)
2020-04-30Do not stop the world during JIT compactionTakashi Kokubun
Running C compiler for JIT compaction inside a critical section may lock main thread for a long time when it triggers GC. As I'm planning to increase this duration a bit, I'd like to make sure this doesn't stop the world. For now, I chose to give up unloading units when it's during JIT compaction, assuming other calls may unload them later.
2020-04-12Add MJIT_COUNTER macro to dump total_callsTakashi Kokubun
2020-04-08Merge pull request #2991 from shyouhei/ruby.h卜部昌平
Split ruby.h Notes: Merged-By: shyouhei <shyouhei@ruby-lang.org>
2020-03-15Add debug counter for unload_unitsTakashi Kokubun
changing add_iseq_to_process's debug counter name as well for comparison
2020-03-12Mark all cc_entries associated to compiled_iseqTakashi Kokubun
2020-03-12Move code to mark jit_unit's cc_entries to mjit.cTakashi Kokubun
2020-03-10Capture inlined iseq's cc entries in root iseq'sTakashi Kokubun
jit_unit to avoid marking wrong cc entries when inlined iseq is compiled multiple times, resolving the TODO added by daf7c48d88. This obviates pseudo jit_unit in inlined iseq introduced by 7ec2359374 and fixes memory leak of the adhoc unit.
2020-02-28Prevent unloading methods used in root_fiber while calling another Fiber (#2939)Takashi Kokubun
Fixing SEGVs like: http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/2744905 http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/2744420 http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/2741400 Notes: Merged-By: k0kubun <takashikkbn@gmail.com>
2020-02-28Avoid infinite loop on --jit-waitTakashi Kokubun
2020-02-26Internalize rb_mjit_unit definition againTakashi Kokubun
Fixed a TODO in b9007b6c548f91e88fd3f2ffa23de740431fa969
2020-02-25Remove an unused declarationTakashi Kokubun
It was unnecessary in b9007b6c548f91e88fd3f2ffa23de740431fa969
2020-02-25should be initialize jit_unit->cc_entries.Koichi Sasada
GC can invoke just after allocation of jit_unit->cc_entries so it should be zero-cleared.
2020-02-25prevent GC from mjit worker.Koichi Sasada
ALLOC_N() can causes GC. Sometimes `mjit_copy_job_handler()` can be called by mjit_worker thread which is not a Ruby thread, so we need to prevent GC in this function. This patch has some issues, but I introduce it to pass the tests.
2020-02-22Introduce disposable call-cache.Koichi Sasada
This patch contains several ideas: (1) Disposable inline method cache (IMC) for race-free inline method cache * Making call-cache (CC) as a RVALUE (GC target object) and allocate new CC on cache miss. * This technique allows race-free access from parallel processing elements like RCU. (2) Introduce per-Class method cache (pCMC) * Instead of fixed-size global method cache (GMC), pCMC allows flexible cache size. * Caching CCs reduces CC allocation and allow sharing CC's fast-path between same call-info (CI) call-sites. (3) Invalidate an inline method cache by invalidating corresponding method entries (MEs) * Instead of using class serials, we set "invalidated" flag for method entry itself to represent cache invalidation. * Compare with using class serials, the impact of method modification (add/overwrite/delete) is small. * Updating class serials invalidate all method caches of the class and sub-classes. * Proposed approach only invalidate the method cache of only one ME. See [Feature #16614] for more details. Notes: Merged: https://github.com/ruby/ruby/pull/2888
2020-02-22VALUE size packed callinfo (ci).Koichi Sasada
Now, rb_call_info contains how to call the method with tuple of (mid, orig_argc, flags, kwarg). Most of cases, kwarg == NULL and mid+argc+flags only requires 64bits. So this patch packed rb_call_info to VALUE (1 word) on such cases. If we can not represent it in VALUE, then use imemo_callinfo which contains conventional callinfo (rb_callinfo, renamed from rb_call_info). iseq->body->ci_kw_size is removed because all of callinfo is VALUE size (packed ci or a pointer to imemo_callinfo). To access ci information, we need to use these functions: vm_ci_mid(ci), _flag(ci), _argc(ci), _kwarg(ci). struct rb_call_info_kw_arg is renamed to rb_callinfo_kwarg. rb_funcallv_with_cc() and rb_method_basic_definition_p_with_cc() is temporary removed because cd->ci should be marked. Notes: Merged: https://github.com/ruby/ruby/pull/2888
2020-02-06function pointers are not void*卜部昌平
The same as 8427fca49bd85205f5a8766292dd893f003c0e48. Notes: Merged: https://github.com/ruby/ruby/pull/2885
2019-12-26decouple internal.h headers卜部昌平
Saves comitters' daily life by avoid #include-ing everything from internal.h to make each file do so instead. This would significantly speed up incremental builds. We take the following inclusion order in this changeset: 1. "ruby/config.h", where _GNU_SOURCE is defined (must be the very first thing among everything). 2. RUBY_EXTCONF_H if any. 3. Standard C headers, sorted alphabetically. 4. Other system headers, maybe guarded by #ifdef 5. Everything else, sorted alphabetically. Exceptions are those win32-related headers, which tend not be self- containing (headers have inclusion order dependencies). Notes: Merged: https://github.com/ruby/ruby/pull/2711
2019-12-03mjit.c: fix a mismatch of malloc'ed typeYusuke Endoh
Coverity Scan found this issue.
2019-12-01Simplify variable declaration by C99Takashi Kokubun
2019-12-01Constified mjit_initNobuyoshi Nakada
2019-12-01ConstifiedNobuyoshi Nakada
2019-12-01Fix a mistake excluding NULL in the endTakashi Kokubun
2019-12-01Workaround missing strndup on WindowsTakashi Kokubun
https://ci.appveyor.com/project/ruby/ruby/builds/29230976/job/c910t37313edb97k
2019-12-01Allow specifying arbitrary MJIT flags by --jit-debugTakashi Kokubun
This is a secret feature for me. It's only for testing and any behavior with this flag override is unsupported. I needed this because I sometimes want to add debug options but do not want to disable optimizations, for using Linux perf.
2019-10-24Combine call info and cache to speed up method invocationAlan Wu
To perform a regular method call, the VM needs two structs, `rb_call_info` and `rb_call_cache`. At the moment, we allocate these two structures in separate buffers. In the worst case, the CPU needs to read 4 cache lines to complete a method call. Putting the two structures together reduces the maximum number of cache line reads to 2. Combining the structures also saves 8 bytes per call site as the current layout uses separate two pointers for the call info and the call cache. This saves about 2 MiB on Discourse. This change improves the Optcarrot benchmark at least 3%. For more details, see attached bugs.ruby-lang.org ticket. Complications: - A new instruction attribute `comptime_sp_inc` is introduced to calculate SP increase at compile time without using call caches. At compile time, a `TS_CALLDATA` operand points to a call info struct, but at runtime, the same operand points to a call data struct. Instruction that explicitly define `sp_inc` also need to define `comptime_sp_inc`. - MJIT code for copying call cache becomes slightly more complicated. - This changes the bytecode format, which might break existing tools. [Misc #16258] Notes: Merged: https://github.com/ruby/ruby/pull/2564
2019-10-13Remove the quick stop path after convert_unit_to_funcTakashi Kokubun
Now I'm not exactly sure why I needed to check `stop_worker_p` after `mjit_copy_cache_from_main_thread` of `convert_unit_to_func` in 4161674b2fbea6bdd01783ac5d3b39d88db22972. If it's for avoiding deadlock under `in_gc` condition, we should keep it. However, if it's not the case and it's just for retrying accidental compilation failure or just to avoid `MJIT_ATOMIC_SET` and `compact_all_jit_code`, I think this quick stop path is not mandatory. Because this path is somewhat problematic in my upcoming fix in mjit_worker, let me try to remove this first and see how CI goes.
2019-09-26RubyVM::MJIT.pause(wait: true) should waitTakashi Kokubun
for all compilations and compaction. Prior to this commit, the last-compiled code has not been used because MJIT worker is stopped before setting the code, and compaction has also been skipped. But it was not intentional and `wait: true` pause should wait until those two things by its feature.
2019-09-20Fix wrong allocation failure handlingTakashi Kokubun
`iseq->body->jit_unit->compile_info` should not be referenced before the null check of `iseq->body->jit_unit`.
2019-08-22* expand tabs. [ci skip]git
Tabs are expanded because previously the file did not have any tab indentation. Please update your editor config, and use misc/expand_tabs.rb in the pre-commit hook.
2019-08-22Improve the description of MJIT cancel-all [ci skip]Takashi Kokubun
2019-05-18Rename rb_gc_new_location to rb_gc_locationAaron Patterson
The function will return new or existing locations depending on whether or not the object actually moved, so give it a more appropriate name.
2019-05-15Rename mjit_gc_finish_hook to mjit_gc_exit_hookTakashi Kokubun
because @ko1 said "gc_finish" is confusing like a finish of entire GC process
2019-04-29glibc says memcpy cannot take NULLUrabe, Shyouhei
At least since 2004, glibc's <string.h> annotates memcpy as __attribute__((__nonnull__)). This basedir is passed to it. When LOAD_RELATIVE is not defined and MJIT_SEARCH_BUILD_DIR is not set, this variable is never updated. Should initialize with meaningful default value.
2019-04-20Fix wrong critical section labelk0kubun
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67644 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-20Prefix rb_ to MJIT wait call since it's exportedk0kubun
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67643 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-20Revert "Revert "Skip ISeq reference for stale_units for debugging""k0kubun
This reverts commit b79899b56a9ebadf5c493c4df2419ebf63934381. I wanted to test r67638 first. Now let me try this as well. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67639 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-20Invalidate JIT-ed code if ISeq is moved by GC.compactk0kubun
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67638 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-20Revert "Skip ISeq reference for stale_units for debugging"k0kubun
This reverts commit 4bd4d4e4cfd164f767e88f844cf10b06faee14b9. Sorry, let me test another one first git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67637 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-20Skip ISeq reference for stale_units for debuggingk0kubun
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67636 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-20Check ISeq references in stale_units took0kubun
This is a possible bug from recent "JIT recompile" introduction. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67631 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-20Update ISeq references in stale_units as wellk0kubun
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67628 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-20fix strict compile errorstenderlove
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67625 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-20Update MJIT referencestenderlove
ISeq can move, so we need to tell MJIT where the new location is. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67624 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-14Do not execute MJIT copy job when ISeq is GC-edk0kubun
I assumed that ISeq is never GC-ed by `in_jit` + `mjit_mark` on copy job ISeq, but unfortunately I found SEGV on `mjit_copy_job_handler` in which iseq->body was somehow Qnil. And it seems to be fixed by disabling the job when `mjit_free_iseq` is called for the ISeq. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67551 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-14Prefer uintptr_t over ptrdiff_t for i686k0kubun
as debugged in PR [close https://github.com/ruby/ruby/pull/2130] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67549 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.25.1) at 2024-04-27 16:06:00 +0000