summaryrefslogtreecommitdiff
path: root/vm_insnhelper.c
AgeCommit message (Collapse)Author
2020-06-04Properly resolve refinements in defined? on method call [Bug #16932]Nobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/3180
2020-06-03vm_invoke_proc_block: reduce recursion卜部昌平
According to nobu recursion can be longer than my expectation. Limit them here. Notes: Merged: https://github.com/ruby/ruby/pull/3152
2020-06-03vm_call_symbol: check stack overflow卜部昌平
VM stack could overflow here. The condition is when a symbol is passed to a block-taking method via &variable, and that symbol has never been used for actual method names (thus yielding that results in calling method_missing), and the VM stack is full (no single word left). This is a once-in-a-blue-moon event. Yet there is a very tiny room of stack overflow. We need to check that. Notes: Merged: https://github.com/ruby/ruby/pull/3152
2020-06-03vm_invoke_block: remove auto qualifier卜部昌平
Was (harmless but) redundant. Notes: Merged: https://github.com/ruby/ruby/pull/3152
2020-06-03vm_insnhelper.c: add space [ci skip]卜部昌平
Just cosmetic change to improve readability. Notes: Merged: https://github.com/ruby/ruby/pull/3152
2020-06-03vm_invoke_symbol_block: reduce MEMCPY卜部昌平
This commit changes the number of calls of MEMCPY from... | send | &:sym -------------------------|-------|------- Symbol already interned | once | twice Symbol not pinned yet | none | once to: | send | &:sym -------------------------|-------|------- Symbol already interned | once | none Symbol not pinned yet | twice | once So it sacrifices exceptional situation for normal path. Notes: Merged: https://github.com/ruby/ruby/pull/3152
2020-06-03vm_invoke_symbol_block: call vm_call_opt_send卜部昌平
Symbol#to_proc and Object#send are closely related each other. Why not share their implementations. By doing so we can skip recursive call of vm_exec(), which could benefit for speed. Notes: Merged: https://github.com/ruby/ruby/pull/3152
2020-06-03vm_invoke_block: force indirect jump卜部昌平
This changeset slightly speeds up on my machine. Calculating ------------------------------------- before after Optcarrot Lan_Master.nes 38.33488426546287 40.89825082589147 fps 40.91288557922081 41.48687465359386 40.96591995270991 41.98499064664184 41.20461943032173 43.67314690779162 42.38344888176518 44.02777536251875 43.43563728880915 44.88695892714136 43.88082889062643 45.11226186242523 Notes: Merged: https://github.com/ruby/ruby/pull/3152
2020-06-03vm_invoke_block: insertion of unused args卜部昌平
This makes it possible for vm_invoke_block to pass its passed arguments verbatimly to calling functions. Because they are tail-called the function calls can be strength-recuced into indirect jumps, which is a huge win. Notes: Merged: https://github.com/ruby/ruby/pull/3152
2020-06-03vm_invoke_block: eliminate goto卜部昌平
Use recursion for better readability. Notes: Merged: https://github.com/ruby/ruby/pull/3152
2020-06-03vm_invoke_block: move logics around卜部昌平
Moved block handler -> captured block conversion from vm_invokeblock to each vm_invoke_*_block functions. Notes: Merged: https://github.com/ruby/ruby/pull/3152
2020-06-02Fixed `defined?` against protected method callNobuyoshi Nakada
Protected methods are restricted to be called according to the class/module in where it is defined, not the actual receiver's class. [Bug #16931]
2020-06-02vm_insnhelper.c: merge opt_eq_func / opt_eql_func卜部昌平
These two function were almost identical, except in case of T_STRING/T_FLOAT. Why not merge them into one, and let the difference be handled in normal method calls (slowpath). This does not improve runtime performance for me, but at least reduces for instance rb_eql_opt from 653 bytes to 86 bytes on my machine, according to nm(1). Notes: Merged: https://github.com/ruby/ruby/pull/3169
2020-05-26Mark vm_stackoverflow as NOINLINE COLDFUNC on JITTakashi Kokubun
to reduce code size and improve locality of hot code.
2020-05-22Fix origin iclass pointer for modulesJeremy Evans
If a module has an origin, and that module is included in another module or class, previously the iclass created for the module had an origin pointer to the module's origin instead of the iclass's origin. Setting the origin pointer correctly requires using a stack, since the origin iclass is not created until after the iclass itself. Use a hidden ruby array to implement that stack. Correctly assigning the origin pointers in the iclass caused a use-after-free in GC. If a module with an origin is included in a class, the iclass shares a method table with the module and the iclass origin shares a method table with module origin. Mark iclass origin with a flag that notes that even though the iclass is an origin, it shares a method table, so the method table should not be garbage collected. The shared method table will be garbage collected when the module origin is garbage collected. I've tested that this does not introduce a memory leak. This change caused a VM assertion failure, which was traced to callable method entries using the incorrect defined_class. Update rb_vm_check_redefinition_opt_method and find_defined_class_by_owner to treat iclass origins different than class origins to avoid this issue. This also includes a fix for Module#included_modules to skip iclasses with origins. Fixes [Bug #16736] Notes: Merged: https://github.com/ruby/ruby/pull/3136
2020-05-22fix memory leak of ccsKoichi Sasada
rb_callable_method_entry() creates ccs entry in cc_tbl, but this code overwrite by insert newly created ccs and overwrote ccs never freed. [Bug #16900] Notes: Merged: https://github.com/ruby/ruby/pull/3128
2020-05-11more on NULL versus functions卜部昌平
Function pointers are not void*. See also 115fec062ccf7c6d72c8d5f64b7a5d84c9fb2dd8 ce4ea956d24eab5089a143bba38126f2b11b55b6 8427fca49bd85205f5a8766292dd893f003c0e48
2020-05-11sed -i 's|ruby/impl|ruby/internal|'卜部昌平
To fix build failures. Notes: Merged: https://github.com/ruby/ruby/pull/3079
2020-05-11sed -i s|ruby/3|ruby/impl|g卜部昌平
This shall fix compile errors. Notes: Merged: https://github.com/ruby/ruby/pull/3079
2020-05-03Disable -Wswitch warning when VM_CHECK_MODENobuyoshi Nakada
2020-04-14Invalidate fastpath when calling attr_reader by superTakashi Kokubun
The same bug as 8355a99883 existed in attr_reader too.
2020-04-14Invalidate fastpath when calling attr_writer by superTakashi Kokubun
We started to use fastpath on invokesuper when a method is not refinements since 5c27681813, but we shouldn't have used fastpath for attr_writer either. `cc->aux_.attr_index` is for an actual receiver class, while we store its superclass in `cc->klass` and therefore there's no way to properly invalidate attr_writer's inline cache when it's called by super. [Bug #16785] I suspect the same bug also exists in attr_reader. I'll address that in another commit.
2020-04-13Make vm_call_cfunc_with_frame a fastpath (#3027)Takashi Kokubun
when there's no need to call CALLER_SETUP_ARG and CALLER_REMOVE_EMPTY_KW_SPLAT (i.e. !rb_splat_or_kwargs_p(ci) && !calling->kw_splat). Micro benchmark: ``` $ benchmark-driver -v --rbenv 'before;after' benchmark/vm_send_cfunc.yml --repeat-count=4 before: ruby 2.8.0dev (2020-04-13T23:45:05Z master b9d3ceee8f) [x86_64-linux] after: ruby 2.8.0dev (2020-04-14T00:48:52Z no-splat-fastpath 418d363722) [x86_64-linux] Calculating ------------------------------------- before after vm_send_cfunc 69.585M 88.724M i/s - 100.000M times in 1.437097s 1.127096s Comparison: vm_send_cfunc after: 88723605.2 i/s before: 69584737.1 i/s - 1.28x slower ``` Optcarrot: ``` $ benchmark-driver -v --rbenv 'before;after' benchmark.yml --repeat-count=12 --output=all before: ruby 2.8.0dev (2020-04-13T23:45:05Z master b9d3ceee8f) [x86_64-linux] after: ruby 2.8.0dev (2020-04-14T00:48:52Z no-splat-fastpath 418d363722) [x86_64-linux] Calculating ------------------------------------- before after Optcarrot Lan_Master.nes 50.76119601545175 42.73858236484051 fps 50.76388649761503 51.04211379912850 50.80930672252514 51.39455790755538 50.90236000778749 51.75656936556145 51.01744746340430 51.86875277356489 51.06495279015112 51.88692482485558 51.07785337168974 51.93429603190578 51.20163525187862 51.95768145071314 51.34671771913112 52.45577266040274 51.35918340835583 52.53163888762858 51.46641337418146 52.62172484121034 51.50835463462257 52.85064021113239 ``` Notes: Merged-By: k0kubun <takashikkbn@gmail.com>
2020-04-11Enable fastpath on invokesuper (#3021)Takashi Kokubun
Fastpath has not been used for invokesuper since it has set vm_call_super_method on every invocation. Because it seems to be blocked only by refinements, try enabling fastpath on invokesuper when cme is not for refinements. While this patch itself should be helpful for VM performance, a part of this patch's motivation is to unblock inlining invokesuper on JIT. $ benchmark-driver -v --rbenv 'before;after' benchmark/vm2_super.yml --repeat-count=4 before: ruby 2.8.0dev (2020-04-11T15:19:58Z master a01bda5949) [x86_64-linux] after: ruby 2.8.0dev (2020-04-12T02:00:08Z invokesuper-fastpath c171984ee3) [x86_64-linux] Calculating ------------------------------------- before after vm2_super 20.031M 32.860M i/s - 6.000M times in 0.299534s 0.182593s Comparison: vm2_super after: 32859885.2 i/s before: 20031097.3 i/s - 1.64x slower Notes: Merged-By: k0kubun <takashikkbn@gmail.com>
2020-04-10Turn class variable warnings into exceptionsJeremy Evans
This changes the following warnings: * warning: class variable access from toplevel * warning: class variable @foo of D is overtaken by C into RuntimeErrors. Handle defined?(@@foo) at toplevel by returning nil instead of raising an exception (the previous behavior warned before returning nil when defined? was used). Refactor the specs to avoid the warnings even in older versions. The specs were checking for the warnings, but the purpose of the related specs as evidenced from their description is to test for behavior, not for warnings. Fixes [Bug #14541] Notes: Merged: https://github.com/ruby/ruby/pull/2987
2020-04-08Merge pull request #2991 from shyouhei/ruby.h卜部昌平
Split ruby.h Notes: Merged-By: shyouhei <shyouhei@ruby-lang.org>
2020-03-17Reduce allocations for keyword argument hashesJeremy Evans
Previously, passing a keyword splat to a method always allocated a hash on the caller side, and accepting arbitrary keywords in a method allocated a separate hash on the callee side. Passing explicit keywords to a method that accepted a keyword splat did not allocate a hash on the caller side, but resulted in two hashes allocated on the callee side. This commit makes passing a single keyword splat to a method not allocate a hash on the caller side. Passing multiple keyword splats or a mix of explicit keywords and a keyword splat still generates a hash on the caller side. On the callee side, if arbitrary keywords are not accepted, it does not allocate a hash. If arbitrary keywords are accepted, it will allocate a hash, but this commit uses a callinfo flag to indicate whether the caller already allocated a hash, and if so, the callee can use the passed hash without duplicating it. So this commit should make it so that a maximum of a single hash is allocated during method calls. To set the callinfo flag appropriately, method call argument compilation checks if only a single keyword splat is given. If only one keyword splat is given, the VM_CALL_KW_SPLAT_MUT callinfo flag is not set, since in that case the keyword splat is passed directly and not mutable. If more than one splat is used, a new hash needs to be generated on the caller side, and in that case the callinfo flag is set, indicating the keyword splat is mutable by the callee. In compile_hash, used for both hash and keyword argument compilation, if compiling keyword arguments and only a single keyword splat is used, pass the argument directly. On the caller side, in vm_args.c, the callinfo flag needs to be recognized and handled. Because the keyword splat argument may not be a hash, it needs to be converted to a hash first if not. Then, unless the callinfo flag is set, the hash needs to be duplicated. The temporary copy of the callinfo flag, kw_flag, is updated if a hash was duplicated, to prevent the need to duplicate it again. If we are converting to a hash or duplicating a hash, we need to update the argument array, which can including duplicating the positional splat array if one was passed. CALLER_SETUP_ARG and a couple other places needs to be modified to handle similar issues for other types of calls. This includes fairly comprehensive tests for different ways keywords are handled internally, checking that you get equal results but that keyword splats on the caller side result in distinct objects for keyword rest parameters. Included are benchmarks for keyword argument calls. Brief results when compiled without optimization: def kw(a: 1) a end def kws(**kw) kw end h = {a: 1} kw(a: 1) # about same kw(**h) # 2.37x faster kws(a: 1) # 1.30x faster kws(**h) # 2.19x faster kw(a: 1, **h) # 1.03x slower kw(**h, **h) # about same kws(a: 1, **h) # 1.16x faster kws(**h, **h) # 1.14x faster Notes: Merged: https://github.com/ruby/ruby/pull/2945
2020-03-04%p is for void *卜部昌平
See also 35eb12c06397e770392a41343cbffc4b204e15c9 6f5eb285077d9abf8f97056531996c58674b570c 687308cf0dab0af675e40da2b6ab8ccd5f77c072 b6a2d63eb3dbc31e110e8cb95e054dd71d49a611
2020-03-03method_missing_reason should be set.Koichi Sasada
send() has special method launcher in VM and it has special method_missing caller. This path doesn't set ec->method_missing_reason which is used at exception creation, so setup this information. Without this setting, NoMethodError exception becomes NameError. This patch will fix: http://ci.rvm.jp/results/trunk-random1@phosphorus-docker/2761643
2020-02-27check imemo_typeKoichi Sasada
check imemo_type to debug http://ci.rvm.jp/results/trunk-vm-asserts@silicon-docker/2744755
2020-02-22Introduce disposable call-cache.Koichi Sasada
This patch contains several ideas: (1) Disposable inline method cache (IMC) for race-free inline method cache * Making call-cache (CC) as a RVALUE (GC target object) and allocate new CC on cache miss. * This technique allows race-free access from parallel processing elements like RCU. (2) Introduce per-Class method cache (pCMC) * Instead of fixed-size global method cache (GMC), pCMC allows flexible cache size. * Caching CCs reduces CC allocation and allow sharing CC's fast-path between same call-info (CI) call-sites. (3) Invalidate an inline method cache by invalidating corresponding method entries (MEs) * Instead of using class serials, we set "invalidated" flag for method entry itself to represent cache invalidation. * Compare with using class serials, the impact of method modification (add/overwrite/delete) is small. * Updating class serials invalidate all method caches of the class and sub-classes. * Proposed approach only invalidate the method cache of only one ME. See [Feature #16614] for more details. Notes: Merged: https://github.com/ruby/ruby/pull/2888
2020-02-22VALUE size packed callinfo (ci).Koichi Sasada
Now, rb_call_info contains how to call the method with tuple of (mid, orig_argc, flags, kwarg). Most of cases, kwarg == NULL and mid+argc+flags only requires 64bits. So this patch packed rb_call_info to VALUE (1 word) on such cases. If we can not represent it in VALUE, then use imemo_callinfo which contains conventional callinfo (rb_callinfo, renamed from rb_call_info). iseq->body->ci_kw_size is removed because all of callinfo is VALUE size (packed ci or a pointer to imemo_callinfo). To access ci information, we need to use these functions: vm_ci_mid(ci), _flag(ci), _argc(ci), _kwarg(ci). struct rb_call_info_kw_arg is renamed to rb_callinfo_kwarg. rb_funcallv_with_cc() and rb_method_basic_definition_p_with_cc() is temporary removed because cd->ci should be marked. Notes: Merged: https://github.com/ruby/ruby/pull/2888
2020-02-22Adjusted indent [ci skip]Nobuyoshi Nakada
2020-02-13should be compared with called_idKoichi Sasada
me->called_id and me->def->original_id can be different sometimes so we should compare with called_id, which is mtbl's key. (fix GH-PR #2869)
2020-02-13Use inline cache for super callsJohn Hawthorn
Notes: Merged: https://github.com/ruby/ruby/pull/2869
2020-02-03support MJIT with debug option.Koichi Sasada
VM_CHECK_MODE > 0 with optflags=-O0 can not run JIT tests because of link problems. This patch fix them.
2020-01-02Fully separate positional arguments and keyword argumentsJeremy Evans
This removes the warnings added in 2.7, and changes the behavior so that a final positional hash is not treated as keywords or vice-versa. To handle the arg_setup_block splat case correctly with keyword arguments, we need to check if we are taking a keyword hash. That case didn't have a test, but it affects real-world code, so add a test for it. This removes rb_empty_keyword_given_p() and related code, as that is not needed in Ruby 3. The empty keyword case is the same as the no keyword case in Ruby 3. This changes rb_scan_args to implement keyword argument separation for C functions when the : character is used. For backwards compatibility, it returns a duped hash. This is a bad idea for performance, but not duping the hash breaks at least Enumerator::ArithmeticSequence#inspect. Instead of having RB_PASS_CALLED_KEYWORDS be a number, simplify the code by just making it be rb_keyword_given_p(). Notes: Merged: https://github.com/ruby/ruby/pull/2794
2019-12-26decouple internal.h headers卜部昌平
Saves comitters' daily life by avoid #include-ing everything from internal.h to make each file do so instead. This would significantly speed up incremental builds. We take the following inclusion order in this changeset: 1. "ruby/config.h", where _GNU_SOURCE is defined (must be the very first thing among everything). 2. RUBY_EXTCONF_H if any. 3. Standard C headers, sorted alphabetically. 4. Other system headers, maybe guarded by #ifdef 5. Everything else, sorted alphabetically. Exceptions are those win32-related headers, which tend not be self- containing (headers have inclusion order dependencies). Notes: Merged: https://github.com/ruby/ruby/pull/2711
2019-12-26add several __has_something macro卜部昌平
With these macros implemented we can write codes just like we can assume the compiler being clang. MSC_VERSION_SINCE is defined to implement those macros, but turned out to be handy for other places. The -fdeclspec compiler flag is necessary for clang to properly handle __has_declspec(). Notes: Merged: https://github.com/ruby/ruby/pull/2711
2019-12-20Fixed misspellingsNobuyoshi Nakada
Fixed misspellings reported at [Bug #16437], only in ruby and rubyspec.
2019-12-18per-method serial number卜部昌平
Methods and their definitions can be allocated/deallocated on-the-fly. One pathological situation is when a method is deallocated then another one is allocated immediately after that. Address of those old/new method entries/definitions can be the same then, depending on underlying malloc/free implementation. So pointer comparison is insufficient. We have to check the contents. To do so we introduce def->method_serial, which is an integer unique to that specific method definition. PS: Note that method_serial being uintptr_t rather than rb_serial_t is intentional. This is because rb_serial_t can be bigger than a pointer on a 32bit system (rb_serial_t is at least 64bit). In order to preserve old packing of struct rb_call_cache, rb_serial_t is inappropriate. Notes: Merged: https://github.com/ruby/ruby/pull/2759
2019-12-17add debug counter to count `call` reusing cases.Koichi Sasada
2019-12-16ensure cc->def == cc->me->def卜部昌平
The equation shall hold for every call cache. However prior to this changeset cc->me could be updated without also updating cc->def. Let's make it sure by introducing new macro named CC_SET_ME which sets cc->me and cc->def at once.
2019-12-12Make super in instance_eval in method in module raise TypeErrorJeremy Evans
This makes behavior the same as super in instance_eval in method in class. The reason this wasn't implemented before is that there is a check to determine if the self in the current context is of the expected class, and a module itself can be included in multiple classes, so it doesn't have an expected class. Implementing this requires giving iclasses knowledge of which class created them, so that super call in the module method knows the expected class for super calls. This reference is called includer, and should only be set for iclasses. Note that the approach Ruby uses in this check is not robust. If you instance_eval another object of the same class and call super, instead of an TypeError, you get super called with the instance_eval receiver instead of the method receiver. Truly fixing super would require keeping a reference to the super object (method receiver) in each frame where scope has changed, and using that instead of current self when calling super. Fixes [Bug #11636] Notes: Merged: https://github.com/ruby/ruby/pull/2717
2019-12-05Introduce an "Inline IVAR cache" structAaron Patterson
This commit introduces an "inline ivar cache" struct. The reason we need this is so compaction can differentiate from an ivar cache and a regular inline cache. Regular inline caches contain references to `VALUE` and ivar caches just contain references to the ivar index. With this new struct we can easily update references for inline caches (but not inline var caches as they just contain an int)
2019-11-29check interrupts at each frame pop timing.Koichi Sasada
Asynchronous events such as signal trap, finalization timing, thread switching and so on are managed by "interrupt_flag". Ruby's threads check this flag periodically and if a thread does not check this flag, above events doesn't happen. This checking is CHECK_INTS() (related) macro and it is placed at some places (laeve instruction and so on). However, at the end of C methods, C blocks (IMEMO_IFUNC) etc there are no checking and it can introduce uninterruptible thread. To modify this situation, we decide to place CHECK_INTS() at vm_pop_frame(). It increases interrupt checking points. [Bug #16366] This patch can introduce unexpected events...
2019-11-29Reduce duplicated warnings for the change of Ruby 3 keyword argumentsYusuke Endoh
By this change, the following code prints only one warning. ``` def foo(**opt); end 100.times { foo({kw:1}) } ``` A global variable `st_table *caller_to_callees` is a map from caller to a set of callee methods. It remembers that a warning is already printed for each pair of caller and callee. [Feature #16289] Notes: Merged: https://github.com/ruby/ruby/pull/2458
2019-11-29Revert "export for MJIT"Koichi Sasada
This reverts commit 2e6f1cf8b264f4c8499c4e5f18bf662fdade04ff.
2019-11-29Revert "* remove trailing spaces. [ci skip]"Koichi Sasada
This reverts commit 27d0d7c0d39076d4bbacd3c3f3864322699db7b4.
2019-11-29* remove trailing spaces. [ci skip]git