summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authork0kubun <k0kubun@b2dd03c8-39d4-4d8f-98ff-823fe69b080e>2018-02-04 11:22:28 (GMT)
committerk0kubun <k0kubun@b2dd03c8-39d4-4d8f-98ff-823fe69b080e>2018-02-04 11:22:28 (GMT)
commited935aa5be0e5e6b8d53c3e7d76a9ce395dfa18b (patch)
tree04c71c64100ed07478bbaed89228c1323a34f2d0
parent56530b541b2601d2d8bfc4e53796faaba414ae2f (diff)
mjit_compile.c: merge initial JIT compiler
which has been developed by Takashi Kokubun <takashikkbn@gmail> as YARV-MJIT. Many of its bugs are fixed by wanabe <s.wanabe@gmail.com>. This JIT compiler is designed to be a safe migration path to introduce JIT compiler to MRI. So this commit does not include any bytecode changes or dynamic instruction modifications, which are done in original MJIT. This commit even strips off some aggressive optimizations from YARV-MJIT, and thus it's slower than YARV-MJIT too. But it's still fairly faster than Ruby 2.5 in some benchmarks (attached below). Note that this JIT compiler passes `make test`, `make test-all`, `make test-spec` without JIT, and even with JIT. Not only it's perfectly safe with JIT disabled because it does not replace VM instructions unlike MJIT, but also with JIT enabled it stably runs Ruby applications including Rails applications. I'm expecting this version as just "initial" JIT compiler. I have many optimization ideas which are skipped for initial merging, and you may easily replace this JIT compiler with a faster one by just replacing mjit_compile.c. `mjit_compile` interface is designed for the purpose. common.mk: update dependencies for mjit_compile.c. internal.h: declare `rb_vm_insn_addr2insn` for MJIT. vm.c: exclude some definitions if `-DMJIT_HEADER` is provided to compiler. This avoids to include some functions which take a long time to compile, e.g. vm_exec_core. Some of the purpose is achieved in transform_mjit_header.rb (see `IGNORED_FUNCTIONS`) but others are manually resolved for now. Load mjit_helper.h for MJIT header. mjit_helper.h: New. This is a file used only by JIT-ed code. I'll refactor `mjit_call_cfunc` later. vm_eval.c: add some #ifdef switches to skip compiling some functions like Init_vm_eval. win32/mkexports.rb: export thread/ec functions, which are used by MJIT. include/ruby/defines.h: add MJIT_FUNC_EXPORTED macro alis to clarify that a function is exported only for MJIT. array.c: export a function used by MJIT. bignum.c: ditto. class.c: ditto. compile.c: ditto. error.c: ditto. gc.c: ditto. hash.c: ditto. iseq.c: ditto. numeric.c: ditto. object.c: ditto. proc.c: ditto. re.c: ditto. st.c: ditto. string.c: ditto. thread.c: ditto. variable.c: ditto. vm_backtrace.c: ditto. vm_insnhelper.c: ditto. vm_method.c: ditto. I would like to improve maintainability of function exports, but I believe this way is acceptable as initial merging if we clarify the new exports are for MJIT (so that we can use them as TODO list to fix) and add unit tests to detect unresolved symbols. I'll add unit tests of JIT compilations in succeeding commits. Author: Takashi Kokubun <takashikkbn@gmail.com> Contributor: wanabe <s.wanabe@gmail.com> Part of [Feature #14235] --- * Known issues * Code generated by gcc is faster than clang. The benchmark may be worse in macOS. Following benchmark result is provided by gcc w/ Linux. * Performance is decreased when Google Chrome is running * JIT can work on MinGW, but it doesn't improve performance at least in short running benchmark. * Currently it doesn't perform well with Rails. We'll try to fix this before release. --- * Benchmark reslts Benchmarked with: Intel 4.0GHz i7-4790K with 16GB memory under x86-64 Ubuntu 8 Cores - 2.0.0-p0: Ruby 2.0.0-p0 - r62186: Ruby trunk (early 2.6.0), before MJIT changes - JIT off: On this commit, but without `--jit` option - JIT on: On this commit, and with `--jit` option ** Optcarrot fps Benchmark: https://github.com/mame/optcarrot | |2.0.0-p0 |r62186 |JIT off |JIT on | |:--------|:--------|:--------|:--------|:--------| |fps |37.32 |51.46 |51.31 |58.88 | |vs 2.0.0 |1.00x |1.38x |1.37x |1.58x | ** MJIT benchmarks Benchmark: https://github.com/benchmark-driver/mjit-benchmarks (Original: https://github.com/vnmakarov/ruby/tree/rtl_mjit_branch/MJIT-benchmarks) | |2.0.0-p0 |r62186 |JIT off |JIT on | |:----------|:--------|:--------|:--------|:--------| |aread |1.00 |1.09 |1.07 |2.19 | |aref |1.00 |1.13 |1.11 |2.22 | |aset |1.00 |1.50 |1.45 |2.64 | |awrite |1.00 |1.17 |1.13 |2.20 | |call |1.00 |1.29 |1.26 |2.02 | |const2 |1.00 |1.10 |1.10 |2.19 | |const |1.00 |1.11 |1.10 |2.19 | |fannk |1.00 |1.04 |1.02 |1.00 | |fib |1.00 |1.32 |1.31 |1.84 | |ivread |1.00 |1.13 |1.12 |2.43 | |ivwrite |1.00 |1.23 |1.21 |2.40 | |mandelbrot |1.00 |1.13 |1.16 |1.28 | |meteor |1.00 |2.97 |2.92 |3.17 | |nbody |1.00 |1.17 |1.15 |1.49 | |nest-ntimes|1.00 |1.22 |1.20 |1.39 | |nest-while |1.00 |1.10 |1.10 |1.37 | |norm |1.00 |1.18 |1.16 |1.24 | |nsvb |1.00 |1.16 |1.16 |1.17 | |red-black |1.00 |1.02 |0.99 |1.12 | |sieve |1.00 |1.30 |1.28 |1.62 | |trees |1.00 |1.14 |1.13 |1.19 | |while |1.00 |1.12 |1.11 |2.41 | ** Discourse's script/bench.rb Benchmark: https://github.com/discourse/discourse/blob/v1.8.7/script/bench.rb NOTE: Rails performance was somehow a little degraded with JIT for now. We should fix this. (At least I know opt_aref is performing badly in JIT and I have an idea to fix it. Please wait for the fix.) *** JIT off Your Results: (note for timings- percentile is first, duration is second in millisecs) categories_admin: 50: 17 75: 18 90: 22 99: 29 home_admin: 50: 21 75: 21 90: 27 99: 40 topic_admin: 50: 17 75: 18 90: 22 99: 32 categories: 50: 35 75: 41 90: 43 99: 77 home: 50: 39 75: 46 90: 49 99: 95 topic: 50: 46 75: 52 90: 56 99: 101 *** JIT on Your Results: (note for timings- percentile is first, duration is second in millisecs) categories_admin: 50: 19 75: 21 90: 25 99: 33 home_admin: 50: 24 75: 26 90: 30 99: 35 topic_admin: 50: 19 75: 20 90: 25 99: 30 categories: 50: 40 75: 44 90: 48 99: 76 home: 50: 42 75: 48 90: 51 99: 89 topic: 50: 49 75: 55 90: 58 99: 99 git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62197 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
-rw-r--r--Makefile.in2
-rw-r--r--array.c6
-rw-r--r--bignum.c2
-rw-r--r--class.c4
-rw-r--r--common.mk5
-rw-r--r--compile.c2
-rw-r--r--error.c2
-rw-r--r--gc.c5
-rw-r--r--hash.c10
-rw-r--r--include/ruby/defines.h4
-rw-r--r--insns.def15
-rw-r--r--internal.h1
-rw-r--r--iseq.c2
-rw-r--r--mjit.c62
-rw-r--r--mjit.h3
-rw-r--r--mjit_compile.c158
-rw-r--r--numeric.c6
-rw-r--r--object.c10
-rw-r--r--proc.c4
-rw-r--r--re.c2
-rw-r--r--st.c2
-rw-r--r--string.c10
-rw-r--r--test/-ext-/thread_fd_close/test_thread_fd_close.rb2
-rw-r--r--test/lib/minitest/unit.rb4
-rw-r--r--test/ruby/test_io.rb8
-rw-r--r--test/ruby/test_settracefunc.rb3
-rw-r--r--test/ruby/test_thread.rb1
-rw-r--r--test/rubygems/test_gem_util.rb1
-rw-r--r--test/thread/test_cv.rb2
-rw-r--r--test/thread/test_sync.rb1
-rw-r--r--test/webrick/test_httpserver.rb1
-rw-r--r--test/webrick/test_server.rb1
-rw-r--r--thread.c2
-rw-r--r--tool/ruby_vm/views/_mjit_compile_insn.erb133
-rw-r--r--tool/ruby_vm/views/_mjit_compile_send.erb74
-rw-r--r--tool/ruby_vm/views/mjit_compile.inc.erb66
-rw-r--r--tool/transform_mjit_header.rb1
-rwxr-xr-xtool/update-deps1
-rw-r--r--variable.c20
-rw-r--r--vm.c36
-rw-r--r--vm_backtrace.c4
-rw-r--r--vm_core.h3
-rw-r--r--vm_eval.c18
-rw-r--r--vm_exec.h19
-rw-r--r--vm_insnhelper.c33
-rw-r--r--vm_insnhelper.h5
-rw-r--r--vm_method.c19
-rw-r--r--vm_trace.c2
-rw-r--r--win32/Makefile.sub2
-rwxr-xr-xwin32/mkexports.rb2
50 files changed, 680 insertions, 101 deletions
diff --git a/Makefile.in b/Makefile.in
index 128dffa..e75cfe5 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -516,7 +516,7 @@ update-simplecov:
update-coverage: update-simplecov update-simplecov-html update-doclie
INSNS = opt_sc.inc optinsn.inc optunifs.inc insns.inc insns_info.inc \
- vmtc.inc vm.inc
+ vmtc.inc vm.inc mjit_compile.inc
$(INSNS): $(srcdir)/insns.def vm_opts.h \
$(srcdir)/defs/opt_operand.def $(srcdir)/defs/opt_insn_unif.def \
diff --git a/array.c b/array.c
index 619a435..7ce72f9 100644
--- a/array.c
+++ b/array.c
@@ -506,7 +506,7 @@ VALUE
return ary;
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_ary_tmp_new_from_values(VALUE klass, long n, const VALUE *elts)
{
VALUE ary;
@@ -640,7 +640,7 @@ rb_check_array_type(VALUE ary)
return rb_check_convert_type_with_id(ary, T_ARRAY, "Array", idTo_ary);
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_check_to_array(VALUE ary)
{
return rb_check_convert_type_with_id(ary, T_ARRAY, "Array", idTo_a);
@@ -1297,7 +1297,7 @@ rb_ary_aref2(VALUE ary, VALUE b, VALUE e)
return rb_ary_subseq(ary, beg, len);
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_ary_aref1(VALUE ary, VALUE arg)
{
long beg, len;
diff --git a/bignum.c b/bignum.c
index 94aabf6..9e73927 100644
--- a/bignum.c
+++ b/bignum.c
@@ -4490,7 +4490,7 @@ rb_uint128t2big(uint128_t n)
return big;
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_int128t2big(int128_t n)
{
int neg = 0;
diff --git a/class.c b/class.c
index 364f258..df7904f 100644
--- a/class.c
+++ b/class.c
@@ -616,7 +616,7 @@ rb_define_class_id(ID id, VALUE super)
* \return the value \c Class#inherited's returns
* \pre Each of \a super and \a klass must be a \c Class object.
*/
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_class_inherited(VALUE super, VALUE klass)
{
ID inherited;
@@ -1773,7 +1773,7 @@ rb_define_attr(VALUE klass, const char *name, int read, int write)
rb_attr(klass, rb_intern(name), read, write, FALSE);
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_keyword_error_new(const char *error, VALUE keys)
{
const VALUE *ptr = RARRAY_CONST_PTR(keys);
diff --git a/common.mk b/common.mk
index 3eb576e..1511614 100644
--- a/common.mk
+++ b/common.mk
@@ -890,6 +890,7 @@ $(srcs_vpath)insns.inc: $(srcdir)/tool/ruby_vm/views/insns.inc.erb
$(srcs_vpath)insns_info.inc: $(srcdir)/tool/ruby_vm/views/insns_info.inc.erb
$(srcs_vpath)vmtc.inc: $(srcdir)/tool/ruby_vm/views/vmtc.inc.erb
$(srcs_vpath)vm.inc: $(srcdir)/tool/ruby_vm/views/vm.inc.erb
+$(srcs_vpath)mjit_compile.inc: $(srcdir)/tool/ruby_vm/views/mjit_compile.inc.erb $(srcdir)/tool/ruby_vm/views/_mjit_compile_insn.erb $(srcdir)/tool/ruby_vm/views/_mjit_compile_send.erb
common-srcs: $(srcs_vpath)parse.c $(srcs_vpath)lex.c $(srcs_vpath)enc/trans/newline.c $(srcs_vpath)id.c \
srcs-lib srcs-ext incs
@@ -2003,8 +2004,12 @@ mjit.$(OBJEXT): {$(VPATH)}mjit.h
mjit.$(OBJEXT): {$(VPATH)}ruby_assert.h
mjit.$(OBJEXT): {$(VPATH)}version.h
mjit.$(OBJEXT): {$(VPATH)}vm_core.h
+mjit_compile.$(OBJEXT): {$(VPATH)}insns.inc
+mjit_compile.$(OBJEXT): {$(VPATH)}insns_info.inc
mjit_compile.$(OBJEXT): {$(VPATH)}internal.h
+mjit_compile.$(OBJEXT): {$(VPATH)}mjit.h
mjit_compile.$(OBJEXT): {$(VPATH)}mjit_compile.c
+mjit_compile.$(OBJEXT): {$(VPATH)}mjit_compile.inc
mjit_compile.$(OBJEXT): {$(VPATH)}vm_core.h
load.$(OBJEXT): $(CCAN_DIR)/check_type/check_type.h
load.$(OBJEXT): $(CCAN_DIR)/container_of/container_of.h
diff --git a/compile.c b/compile.c
index c7d6a41..b559703 100644
--- a/compile.c
+++ b/compile.c
@@ -754,7 +754,7 @@ rb_iseq_translate_threaded_code(rb_iseq_t *iseq)
}
#if OPT_DIRECT_THREADED_CODE || OPT_CALL_THREADED_CODE
-static int
+int
rb_vm_insn_addr2insn(const void *addr) /* cold path */
{
int insn;
diff --git a/error.c b/error.c
index eebfa1d..b0a2a12 100644
--- a/error.c
+++ b/error.c
@@ -1161,7 +1161,7 @@ exc_set_backtrace(VALUE exc, VALUE bt)
return rb_ivar_set(exc, id_bt, rb_check_backtrace(bt));
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_exc_set_backtrace(VALUE exc, VALUE bt)
{
return exc_set_backtrace(exc, bt);
diff --git a/gc.c b/gc.c
index cfa345e..be0dd70 100644
--- a/gc.c
+++ b/gc.c
@@ -2216,6 +2216,7 @@ obj_free(rb_objspace_t *objspace, VALUE obj)
break;
case T_MODULE:
case T_CLASS:
+ mjit_remove_class_serial(RCLASS_SERIAL(obj));
rb_id_table_free(RCLASS_M_TBL(obj));
if (RCLASS_IV_TBL(obj)) {
st_free_table(RCLASS_IV_TBL(obj));
@@ -4054,7 +4055,7 @@ stack_check(rb_execution_context_t *ec, int water_mark)
#define STACKFRAME_FOR_CALL_CFUNC 838
-int
+MJIT_FUNC_EXPORTED int
rb_ec_stack_check(rb_execution_context_t *ec)
{
return stack_check(ec, STACKFRAME_FOR_CALL_CFUNC);
@@ -6053,7 +6054,7 @@ rb_gc_writebarrier_unprotect(VALUE obj)
/*
* remember `obj' if needed.
*/
-void
+MJIT_FUNC_EXPORTED void
rb_gc_writebarrier_remember(VALUE obj)
{
rb_objspace_t *objspace = &rb_objspace;
diff --git a/hash.c b/hash.c
index 43db795..85c6e85 100644
--- a/hash.c
+++ b/hash.c
@@ -443,7 +443,7 @@ rb_hash_new_compare_by_id(void)
return hash;
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_hash_new_with_size(st_index_t size)
{
VALUE ret = rb_hash_new();
@@ -495,7 +495,7 @@ rb_hash_tbl(VALUE hash)
return hash_tbl(hash);
}
-struct st_table *
+MJIT_FUNC_EXPORTED struct st_table *
rb_hash_tbl_raw(VALUE hash)
{
return hash_tbl(hash);
@@ -2155,7 +2155,7 @@ keys_i(VALUE key, VALUE value, VALUE ary)
*
*/
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_hash_keys(VALUE hash)
{
VALUE keys;
@@ -2243,7 +2243,7 @@ rb_hash_values(VALUE hash)
* See also Enumerable#include?
*/
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_hash_has_key(VALUE hash, VALUE key)
{
if (!RHASH(hash)->ntbl)
@@ -2949,7 +2949,7 @@ rb_hash_compare_by_id(VALUE hash)
*
*/
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_hash_compare_by_id_p(VALUE hash)
{
if (!RHASH(hash)->ntbl)
diff --git a/include/ruby/defines.h b/include/ruby/defines.h
index 5006305..b49fabe 100644
--- a/include/ruby/defines.h
+++ b/include/ruby/defines.h
@@ -270,6 +270,10 @@ void xfree(void*);
#define RUBY_FUNC_EXPORTED
#endif
+/* MJIT_FUNC_EXPORTED is used for functions which are exported only for MJIT
+ and NOT ensured to be exported in future versions. */
+#define MJIT_FUNC_EXPORTED RUBY_FUNC_EXPORTED
+
#ifndef RUBY_EXTERN
#define RUBY_EXTERN extern
#endif
diff --git a/insns.def b/insns.def
index eabace4..fb5e83c 100644
--- a/insns.def
+++ b/insns.def
@@ -695,8 +695,7 @@ defineclass
class_iseq->body->iseq_encoded, GET_SP(),
class_iseq->body->local_table_size,
class_iseq->body->stack_max);
- RESTORE_REGS();
- NEXT_INSN();
+ EXEC_EC_CFP();
}
/**********************************************************/
@@ -823,8 +822,7 @@ invokeblock
val = vm_invoke_block(ec, GET_CFP(), &calling, ci, block_handler);
if (val == Qundef) {
- RESTORE_REGS();
- NEXT_INSN();
+ EXEC_EC_CFP();
}
}
@@ -1090,7 +1088,9 @@ opt_neq
val = vm_opt_neq(ci, cc, ci_eq, cc_eq, recv, obj);
if (val == Qundef) {
+#ifndef MJIT_HEADER
ADD_PC(2); /* !!! */
+#endif
DISPATCH_ORIGINAL_INSN(opt_send_without_block);
}
}
@@ -1206,9 +1206,11 @@ opt_aset_with
val = tmp;
}
else {
+#ifndef MJIT_HEADER
TOPN(0) = rb_str_resurrect(key);
PUSH(val);
ADD_PC(1); /* !!! */
+#endif
DISPATCH_ORIGINAL_INSN(opt_send_without_block);
}
}
@@ -1223,8 +1225,10 @@ opt_aref_with
val = vm_opt_aref_with(recv, key);
if (val == Qundef) {
+#ifndef MJIT_HEADER
PUSH(rb_str_resurrect(key));
ADD_PC(1); /* !!! */
+#endif
DISPATCH_ORIGINAL_INSN(opt_send_without_block);
}
}
@@ -1339,8 +1343,7 @@ opt_call_c_function
THROW_EXCEPTION(err);
}
- RESTORE_REGS();
- NEXT_INSN();
+ EXEC_EC_CFP();
}
/* BLT */
diff --git a/internal.h b/internal.h
index 8b1ee22..2c65ebc 100644
--- a/internal.h
+++ b/internal.h
@@ -1124,6 +1124,7 @@ int rb_dvar_defined(ID, const struct rb_block *);
int rb_local_defined(ID, const struct rb_block *);
const char * rb_insns_name(int i);
VALUE rb_insns_name_array(void);
+int rb_vm_insn_addr2insn(const void *);
/* complex.c */
VALUE rb_complex_plus(VALUE, VALUE);
diff --git a/iseq.c b/iseq.c
index 724f81b..b703baa 100644
--- a/iseq.c
+++ b/iseq.c
@@ -1480,7 +1480,7 @@ rb_iseq_line_no(const rb_iseq_t *iseq, size_t pos)
}
}
-rb_event_flag_t
+MJIT_FUNC_EXPORTED rb_event_flag_t
rb_iseq_event_flags(const rb_iseq_t *iseq, size_t pos)
{
const struct iseq_insn_info_entry *entry = get_insn_info(iseq, pos);
diff --git a/mjit.c b/mjit.c
index f90e8a2..826c5b3 100644
--- a/mjit.c
+++ b/mjit.c
@@ -83,6 +83,8 @@
#include "mjit.h"
#include "version.h"
#include "gc.h"
+#include "constant.h"
+#include "id_table.h"
#include "ruby_assert.h"
extern void rb_native_mutex_lock(rb_nativethread_lock_t *lock);
@@ -194,6 +196,9 @@ static char *header_file;
static char *pch_file;
/* Path of "/tmp", which can be changed to $TMP in MinGW. */
static char *tmp_dir;
+/* Hash like { 1 => true, 2 => true, ... } whose keys are valid `class_serial`s.
+ This is used to invalidate obsoleted CALL_CACHE. */
+static VALUE valid_class_serials;
/* Ruby level interface module. */
VALUE rb_mMJIT;
@@ -1081,6 +1086,19 @@ child_after_fork(void)
/* TODO: Should we initiate MJIT in the forked Ruby. */
}
+static enum rb_id_table_iterator_result
+valid_class_serials_add_i(ID key, VALUE v, void *unused)
+{
+ rb_const_entry_t *ce = (rb_const_entry_t *)v;
+ VALUE value = ce->value;
+
+ if (!rb_is_const_id(key)) return ID_TABLE_CONTINUE;
+ if (RB_TYPE_P(value, T_MODULE) || RB_TYPE_P(value, T_CLASS)) {
+ mjit_add_class_serial(RCLASS_SERIAL(value));
+ }
+ return ID_TABLE_CONTINUE;
+}
+
/* Default permitted number of units with a JIT code kept in
memory. */
#define DEFAULT_CACHE_SIZE 1000
@@ -1149,6 +1167,14 @@ mjit_init(struct mjit_options *opts)
rb_native_cond_initialize(&mjit_worker_wakeup, RB_CONDATTR_CLOCK_MONOTONIC);
rb_native_cond_initialize(&mjit_gc_wakeup, RB_CONDATTR_CLOCK_MONOTONIC);
+ /* Initialize class_serials cache for compilation */
+ valid_class_serials = rb_hash_new();
+ rb_obj_hide(valid_class_serials);
+ rb_gc_register_mark_object(valid_class_serials);
+ if (RCLASS_CONST_TBL(rb_cObject)) {
+ rb_id_table_foreach(RCLASS_CONST_TBL(rb_cObject), valid_class_serials_add_i, NULL);
+ }
+
/* Initialize worker thread */
finish_worker_p = FALSE;
worker_finished = FALSE;
@@ -1233,3 +1259,39 @@ mjit_mark(void)
CRITICAL_SECTION_FINISH(4, "mjit_mark");
RUBY_MARK_LEAVE("mjit");
}
+
+/* A hook to update valid_class_serials. This should NOT be used in MJIT worker. */
+void
+mjit_add_class_serial(rb_serial_t class_serial)
+{
+ if (!mjit_init_p)
+ return;
+
+ CRITICAL_SECTION_START(3, "in mjit_add_class_serial");
+ rb_hash_aset(valid_class_serials, LONG2FIX(class_serial), Qtrue);
+ CRITICAL_SECTION_FINISH(3, "in mjit_add_class_serial");
+}
+
+/* A hook to update valid_class_serials. This should NOT be used in MJIT worker. */
+void
+mjit_remove_class_serial(rb_serial_t class_serial)
+{
+ if (!mjit_init_p)
+ return;
+
+ CRITICAL_SECTION_START(3, "in mjit_remove_class_serial");
+ rb_hash_delete_entry(valid_class_serials, LONG2FIX(class_serial));
+ CRITICAL_SECTION_FINISH(3, "in mjit_remove_class_serial");
+}
+
+/* Return TRUE if class_serial is not obsoleted. This can be used in MJIT worker. */
+int
+mjit_valid_class_serial_p(rb_serial_t class_serial)
+{
+ int found_p;
+
+ CRITICAL_SECTION_START(3, "in valid_class_serial_p");
+ found_p = st_lookup(RHASH_TBL_RAW(valid_class_serials), LONG2FIX(class_serial), NULL);
+ CRITICAL_SECTION_FINISH(3, "in valid_class_serial_p");
+ return found_p;
+}
diff --git a/mjit.h b/mjit.h
index 452110e..90e0f6b 100644
--- a/mjit.h
+++ b/mjit.h
@@ -80,6 +80,9 @@ extern void mjit_free_iseq(const rb_iseq_t *iseq);
extern void mjit_mark(void);
extern struct mjit_cont *mjit_cont_new(rb_execution_context_t *ec);
extern void mjit_cont_free(struct mjit_cont *cont);
+extern void mjit_add_class_serial(rb_serial_t class_serial);
+extern void mjit_remove_class_serial(rb_serial_t class_serial);
+extern int mjit_valid_class_serial_p(rb_serial_t class_serial);
/* A threshold used to reject long iseqs from JITting as such iseqs
takes too much time to be compiled. */
diff --git a/mjit_compile.c b/mjit_compile.c
index a48ae84..b323192 100644
--- a/mjit_compile.c
+++ b/mjit_compile.c
@@ -8,11 +8,163 @@
#include "internal.h"
#include "vm_core.h"
+#include "vm_exec.h"
+#include "mjit.h"
+#include "insns.inc"
+#include "insns_info.inc"
+#include "vm_insnhelper.h"
-/* Compile ISeq to C code in F. Return TRUE if it succeeds to compile. */
+/* Storage to keep compiler's status. This should have information
+ which is global during one `mjit_compile` call. Ones conditional
+ in each branch should be stored in `compile_branch`. */
+struct compile_status {
+ int success; /* has TRUE if compilation has had no issue */
+ int *compiled_for_pos; /* compiled_for_pos[pos] has TRUE if the pos is compiled */
+};
+
+/* Storage to keep data which is consistent in each conditional branch.
+ This is created and used for one `compile_insns` call and its values
+ should be copied for extra `compile_insns` call. */
+struct compile_branch {
+ unsigned int stack_size; /* this simulates sp (stack pointer) of YARV */
+ int finish_p; /* if TRUE, compilation in this branch should stop and let another branch to be compiled */
+};
+
+struct case_dispatch_var {
+ FILE *f;
+ unsigned int base_pos;
+ VALUE last_value;
+};
+
+/* Returns iseq from cc if it's available and still not obsoleted. */
+static const rb_iseq_t *
+get_iseq_if_available(CALL_CACHE cc)
+{
+ if (GET_GLOBAL_METHOD_STATE() == cc->method_state
+ && mjit_valid_class_serial_p(cc->class_serial)
+ && cc->me && cc->me->def->type == VM_METHOD_TYPE_ISEQ) {
+ return rb_iseq_check(cc->me->def->body.iseq.iseqptr);
+ }
+ return NULL;
+}
+
+/* TODO: move to somewhere shared with vm_args.c */
+#define IS_ARGS_SPLAT(ci) ((ci)->flag & VM_CALL_ARGS_SPLAT)
+#define IS_ARGS_KEYWORD(ci) ((ci)->flag & VM_CALL_KWARG)
+
+/* Returns TRUE if iseq is inlinable, otherwise NULL. This becomes TRUE in the same condition
+ as CI_SET_FASTPATH (in vm_callee_setup_arg) is called from vm_call_iseq_setup. */
+static int
+inlinable_iseq_p(CALL_INFO ci, CALL_CACHE cc, const rb_iseq_t *iseq)
+{
+ extern int simple_iseq_p(const rb_iseq_t *iseq);
+ return iseq != NULL
+ && simple_iseq_p(iseq) && !(ci->flag & VM_CALL_KW_SPLAT) /* top of vm_callee_setup_arg */
+ && (!IS_ARGS_SPLAT(ci) && !IS_ARGS_KEYWORD(ci) && !(METHOD_ENTRY_VISI(cc->me) == METHOD_VISI_PROTECTED)); /* CI_SET_FASTPATH */
+}
+
+static int
+compile_case_dispatch_each(VALUE key, VALUE value, VALUE arg)
+{
+ struct case_dispatch_var *var = (struct case_dispatch_var *)arg;
+ unsigned int offset;
+
+ if (var->last_value != value) {
+ offset = FIX2INT(value);
+ var->last_value = value;
+ fprintf(var->f, " case %d:\n", offset);
+ fprintf(var->f, " goto label_%d;\n", var->base_pos + offset);
+ fprintf(var->f, " break;\n");
+ }
+ return ST_CONTINUE;
+}
+
+static void compile_insns(FILE *f, const struct rb_iseq_constant_body *body, unsigned int stack_size,
+ unsigned int pos, struct compile_status *status);
+
+/* Main function of JIT compilation, vm_exec_core counterpart for JIT. Compile one insn to `f`, may modify
+ b->stack_size and return next position.
+
+ When you add a new instruction to insns.def, it would be nice to have JIT compilation support here but
+ it's optional. This JIT compiler just ignores ISeq which includes unknown instruction, and ISeq which
+ does not have it can be compiled as usual. */
+static unsigned int
+compile_insn(FILE *f, const struct rb_iseq_constant_body *body, const int insn, const VALUE *operands,
+ const unsigned int pos, struct compile_status *status, struct compile_branch *b)
+{
+ unsigned int next_pos = pos + insn_len(insn);
+
+/*****************/
+ #include "mjit_compile.inc"
+/*****************/
+
+ return next_pos;
+}
+
+/* Compile one conditional branch. If it has branchXXX insn, this should be
+ called multiple times for each branch. */
+static void
+compile_insns(FILE *f, const struct rb_iseq_constant_body *body, unsigned int stack_size,
+ unsigned int pos, struct compile_status *status)
+{
+ int insn;
+ struct compile_branch branch;
+
+ branch.stack_size = stack_size;
+ branch.finish_p = FALSE;
+
+ while (pos < body->iseq_size && !status->compiled_for_pos[pos] && !branch.finish_p) {
+#if OPT_DIRECT_THREADED_CODE || OPT_CALL_THREADED_CODE
+ insn = rb_vm_insn_addr2insn((void *)body->iseq_encoded[pos]);
+#else
+ insn = (int)body->iseq_encoded[pos];
+#endif
+ status->compiled_for_pos[pos] = TRUE;
+
+ fprintf(f, "\nlabel_%d: /* %s */\n", pos, insn_name(insn));
+ pos = compile_insn(f, body, insn, body->iseq_encoded + (pos+1), pos, status, &branch);
+ if (status->success && branch.stack_size > body->stack_max) {
+ if (mjit_opts.warnings || mjit_opts.verbose)
+ fprintf(stderr, "MJIT warning: JIT stack exceeded its max\n");
+ status->success = FALSE;
+ }
+ if (!status->success)
+ break;
+ }
+}
+
+/* Compile ISeq to C code in F. It returns 1 if it succeeds to compile. */
int
mjit_compile(FILE *f, const struct rb_iseq_constant_body *body, const char *funcname)
{
- /* TODO: Write your own JIT compiler here. */
- return FALSE;
+ struct compile_status status;
+ status.success = TRUE;
+ status.compiled_for_pos = ZALLOC_N(int, body->iseq_size);
+
+ fprintf(f, "VALUE %s(rb_execution_context_t *ec, rb_control_frame_t *reg_cfp) {\n", funcname);
+ fprintf(f, " VALUE *stack = reg_cfp->sp;\n");
+
+ /* Simulate `opt_pc` in setup_parameters_complex */
+ if (body->param.flags.has_opt) {
+ int i;
+ fprintf(f, "\n");
+ fprintf(f, " switch (reg_cfp->pc - reg_cfp->iseq->body->iseq_encoded) {\n");
+ for (i = 0; i <= body->param.opt_num; i++) {
+ VALUE pc_offset = body->param.opt_table[i];
+ fprintf(f, " case %"PRIdVALUE":\n", pc_offset);
+ fprintf(f, " goto label_%"PRIdVALUE";\n", pc_offset);
+ }
+ fprintf(f, " }\n");
+ }
+
+ /* ISeq might be used for catch table too. For that usage, this code cancels JIT execution. */
+ fprintf(f, " if (reg_cfp->pc != 0x%"PRIxVALUE") {\n", (VALUE)body->iseq_encoded);
+ fprintf(f, " return Qundef;\n");
+ fprintf(f, " }\n");
+
+ compile_insns(f, body, 0, 0, &status);
+ fprintf(f, "}\n");
+
+ xfree(status.compiled_for_pos);
+ return status.success;
}
diff --git a/numeric.c b/numeric.c
index 9cc3f4d..0e28043 100644
--- a/numeric.c
+++ b/numeric.c
@@ -1157,7 +1157,7 @@ flodivmod(double x, double y, double *divp, double *modp)
* An error will be raised if y == 0.
*/
-double
+MJIT_FUNC_EXPORTED double
ruby_float_mod(double x, double y)
{
double mod;
@@ -1333,7 +1333,7 @@ num_equal(VALUE x, VALUE y)
* so an implementation-dependent value is returned.
*/
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_float_equal(VALUE x, VALUE y)
{
volatile double a, b;
@@ -1436,7 +1436,7 @@ flo_cmp(VALUE x, VALUE y)
return rb_dbl_cmp(a, b);
}
-int
+MJIT_FUNC_EXPORTED int
rb_float_cmp(VALUE x, VALUE y)
{
return NUM2INT(flo_cmp(x, y));
diff --git a/object.c b/object.c
index 0550283..44d0d54 100644
--- a/object.c
+++ b/object.c
@@ -198,7 +198,7 @@ rb_eql(VALUE obj1, VALUE obj2)
* \private
*++
*/
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_obj_equal(VALUE obj1, VALUE obj2)
{
if (obj1 == obj2) return Qtrue;
@@ -217,7 +217,7 @@ VALUE rb_obj_hash(VALUE obj);
*++
*/
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_obj_not(VALUE obj)
{
return RTEST(obj) ? Qfalse : Qtrue;
@@ -233,7 +233,7 @@ rb_obj_not(VALUE obj)
*++
*/
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_obj_not_equal(VALUE obj1, VALUE obj2)
{
VALUE result = rb_funcall(obj1, id_eq, 1, obj2);
@@ -304,7 +304,7 @@ rb_obj_singleton_class(VALUE obj)
}
/*! \private */
-void
+MJIT_FUNC_EXPORTED void
rb_obj_copy_ivar(VALUE dest, VALUE obj)
{
if (!(RBASIC(dest)->flags & ROBJECT_EMBED) && ROBJECT_IVPTR(dest)) {
@@ -3019,7 +3019,7 @@ rb_check_convert_type(VALUE val, int type, const char *tname, const char *method
}
/*! \private */
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_check_convert_type_with_id(VALUE val, int type, const char *tname, ID method)
{
VALUE v;
diff --git a/proc.c b/proc.c
index 86610dd..3213ced 100644
--- a/proc.c
+++ b/proc.c
@@ -677,7 +677,7 @@ rb_vm_ifunc_new(VALUE (*func)(ANYARGS), const void *data, int min_argc, int max_
return IFUNC_NEW(func, data, arity.packed);
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_func_proc_new(rb_block_call_func_t func, VALUE val)
{
struct vm_ifunc *ifunc = rb_vm_ifunc_proc_new(func, (void *)val);
@@ -1206,7 +1206,7 @@ rb_hash_proc(st_index_t hash, VALUE prc)
return rb_hash_uint(hash, (st_index_t)proc->block.as.captured.ep >> 16);
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_sym_to_proc(VALUE sym)
{
static VALUE sym_proc_cache = Qfalse;
diff --git a/re.c b/re.c
index df7a561..0508e5f 100644
--- a/re.c
+++ b/re.c
@@ -2891,7 +2891,7 @@ rb_reg_init_str_enc(VALUE re, VALUE s, rb_encoding *enc, int options)
return re;
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_reg_new_ary(VALUE ary, int opt)
{
return rb_reg_new_str(rb_reg_preprocess_dregexp(ary, opt), opt);
diff --git a/st.c b/st.c
index c5a2069..56ae30c 100644
--- a/st.c
+++ b/st.c
@@ -2171,7 +2171,7 @@ st_insert_generic(st_table *tab, long argc, const VALUE *argv, VALUE hash)
/* Mimics ruby's { foo => bar } syntax. This function is placed here
because it touches table internals and write barriers at once. */
-void
+MJIT_FUNC_EXPORTED void
rb_hash_bulk_insert(long argc, const VALUE *argv, VALUE hash)
{
st_index_t n;
diff --git a/string.c b/string.c
index 9034724..28b8735 100644
--- a/string.c
+++ b/string.c
@@ -373,7 +373,7 @@ rb_setup_fake_str(struct RString *fake_str, const char *name, long len, rb_encod
return setup_fake_str(fake_str, name, len, rb_enc_to_index(enc));
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_fstring_new(const char *ptr, long len)
{
struct RString fake_str;
@@ -1445,7 +1445,7 @@ rb_obj_as_string(VALUE obj)
return rb_obj_as_string_result(str, obj);
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_obj_as_string_result(VALUE str, VALUE obj)
{
if (!RB_TYPE_P(str, T_STRING))
@@ -2933,7 +2933,7 @@ rb_str_append(VALUE str, VALUE str2)
#define MIN_PRE_ALLOC_SIZE 48
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_str_concat_literals(size_t num, const VALUE *strary)
{
VALUE str;
@@ -10391,7 +10391,7 @@ rb_str_quote_unprintable(VALUE str)
return str;
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_id_quote_unprintable(ID id)
{
return rb_str_quote_unprintable(rb_id2str(id));
@@ -10468,7 +10468,7 @@ sym_to_sym(VALUE sym)
return sym;
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_sym_proc_call(ID mid, int argc, const VALUE *argv, VALUE passed_proc)
{
VALUE obj;
diff --git a/test/-ext-/thread_fd_close/test_thread_fd_close.rb b/test/-ext-/thread_fd_close/test_thread_fd_close.rb
index e6c3895..90993cf 100644
--- a/test/-ext-/thread_fd_close/test_thread_fd_close.rb
+++ b/test/-ext-/thread_fd_close/test_thread_fd_close.rb
@@ -6,6 +6,8 @@ require 'io/wait'
class TestThreadFdClose < Test::Unit::TestCase
def test_thread_fd_close
+ skip "MJIT thread is unexpected for this" if RubyVM::MJIT.enabled?
+
IO.pipe do |r, w|
th = Thread.new do
begin
diff --git a/test/lib/minitest/unit.rb b/test/lib/minitest/unit.rb
index 89b6a8e..aa9a09f 100644
--- a/test/lib/minitest/unit.rb
+++ b/test/lib/minitest/unit.rb
@@ -956,7 +956,9 @@ module MiniTest
puts if @verbose
$stdout.flush
- leakchecker.check("#{inst.class}\##{inst.__name__}")
+ unless RubyVM::MJIT.enabled? # compiler process is wrongly considered as leaked
+ leakchecker.check("#{inst.class}\##{inst.__name__}")
+ end
inst._assertions
}
diff --git a/test/ruby/test_io.rb b/test/ruby/test_io.rb
index 2f0a8a7..f1a9453 100644
--- a/test/ruby/test_io.rb
+++ b/test/ruby/test_io.rb
@@ -2135,6 +2135,12 @@ class TestIO < Test::Unit::TestCase
end
def test_autoclose_true_closed_by_finalizer
+ if RubyVM::MJIT.enabled?
+ # This is skipped but this test passes with AOT mode.
+ # At least it should not be a JIT compiler's bug.
+ skip "MJIT worker does IO which is unexpected for this test"
+ end
+
feature2250 = '[ruby-core:26222]'
pre = 'ft2250'
t = Tempfile.new(pre)
@@ -2150,7 +2156,7 @@ class TestIO < Test::Unit::TestCase
assert_raise(Errno::EBADF, feature2250) {t.close}
end
ensure
- t.close!
+ t&.close!
end
def test_autoclose_false_closed_by_finalizer
diff --git a/test/ruby/test_settracefunc.rb b/test/ruby/test_settracefunc.rb
index 69ddc05..bff3b8c 100644
--- a/test/ruby/test_settracefunc.rb
+++ b/test/ruby/test_settracefunc.rb
@@ -1819,6 +1819,9 @@ class TestSetTraceFunc < Test::Unit::TestCase
}
# it is dirty hack. usually we shouldn't use such technique
Thread.pass until t.status == 'sleep'
+ # When MJIT thread exists, t.status becomes 'sleep' even if it does not reach m2t_q.pop.
+ # This sleep forces it to reach m2t_q.pop for --jit-wait.
+ sleep 1 if RubyVM::MJIT.enabled?
t.add_trace_func proc{|ev, file, line, *args|
if file == __FILE__
diff --git a/test/ruby/test_thread.rb b/test/ruby/test_thread.rb
index 6171c3a..2d84ee3 100644
--- a/test/ruby/test_thread.rb
+++ b/test/ruby/test_thread.rb
@@ -280,6 +280,7 @@ class TestThread < Test::Unit::TestCase
s += 1
end
Thread.pass until t.stop?
+ sleep 1 if RubyVM::MJIT.enabled? # t.stop? behaves unexpectedly with --jit-wait
assert_equal(1, s)
t.wakeup
Thread.pass while t.alive?
diff --git a/test/rubygems/test_gem_util.rb b/test/rubygems/test_gem_util.rb
index b85db44..dc85d25 100644
--- a/test/rubygems/test_gem_util.rb
+++ b/test/rubygems/test_gem_util.rb
@@ -5,6 +5,7 @@ require 'rubygems/util'
class TestGemUtil < Gem::TestCase
def test_class_popen
+ skip "MJIT executes process and it's caught by Process.wait(-1)" if RubyVM::MJIT.enabled?
assert_equal "0\n", Gem::Util.popen(Gem.ruby, '-e', 'p 0')
assert_raises Errno::ECHILD do
diff --git a/test/thread/test_cv.rb b/test/thread/test_cv.rb
index 1e15d2e..70cf448 100644
--- a/test/thread/test_cv.rb
+++ b/test/thread/test_cv.rb
@@ -33,6 +33,8 @@ class TestConditionVariable < Test::Unit::TestCase
end
def test_condvar_wait_exception_handling
+ skip "MJIT thread is unexpected for this test, especially with --jit-wait" if RubyVM::MJIT.enabled?
+
# Calling wait in the only thread running should raise a ThreadError of
# 'stopping only thread'
mutex = Mutex.new
diff --git a/test/thread/test_sync.rb b/test/thread/test_sync.rb
index 8241445..e3294ff 100644
--- a/test/thread/test_sync.rb
+++ b/test/thread/test_sync.rb
@@ -59,6 +59,7 @@ class SyncTest < Test::Unit::TestCase
}
sleep 0.1 until t.stop?
+ sleep 1 if RubyVM::MJIT.enabled? # t.stop? behaves unexpectedly with --jit-wait
t.raise
t.join
diff --git a/test/webrick/test_httpserver.rb b/test/webrick/test_httpserver.rb
index daeb7b9..5ae105d 100644
--- a/test/webrick/test_httpserver.rb
+++ b/test/webrick/test_httpserver.rb
@@ -253,6 +253,7 @@ class TestWEBrickHTTPServer < Test::Unit::TestCase
server.virtual_host(WEBrick::HTTPServer.new(vhost_config))
Thread.pass while server.status != :Running
+ sleep 1 if RubyVM::MJIT.enabled? # server.status behaves unexpectedly with --jit-wait
assert_equal(1, started, log.call)
assert_equal(0, stopped, log.call)
assert_equal(0, accepted, log.call)
diff --git a/test/webrick/test_server.rb b/test/webrick/test_server.rb
index 4d539d0..5f7f3a0 100644
--- a/test/webrick/test_server.rb
+++ b/test/webrick/test_server.rb
@@ -65,6 +65,7 @@ class TestWEBrickServer < Test::Unit::TestCase
}
TestWEBrick.start_server(Echo, config){|server, addr, port, log|
true while server.status != :Running
+ sleep 1 if RubyVM::MJIT.enabled? # server.status behaves unexpectedly with --jit-wait
assert_equal(1, started, log.call)
assert_equal(0, stopped, log.call)
assert_equal(0, accepted, log.call)
diff --git a/thread.c b/thread.c
index f58dccb..96cadd6 100644
--- a/thread.c
+++ b/thread.c
@@ -2089,7 +2089,7 @@ threadptr_get_interrupts(rb_thread_t *th)
return interrupt & (rb_atomic_t)~ec->interrupt_mask;
}
-void
+MJIT_FUNC_EXPORTED void
rb_threadptr_execute_interrupts(rb_thread_t *th, int blocking_timing)
{
rb_atomic_t interrupt;
diff --git a/tool/ruby_vm/views/_mjit_compile_insn.erb b/tool/ruby_vm/views/_mjit_compile_insn.erb
new file mode 100644
index 0000000..60ba250
--- /dev/null
+++ b/tool/ruby_vm/views/_mjit_compile_insn.erb
@@ -0,0 +1,133 @@
+% # -*- mode:c; style:ruby; coding: utf-8; indent-tabs-mode: nil -*-
+% # Copyright (c) 2018 Takashi Kokubun. All rights reserved.
+% #
+% # This file is a part of the programming language Ruby. Permission is hereby
+% # granted, to either redistribute and/or modify this file, provided that the
+% # conditions mentioned in the file COPYING are met. Consult the file for
+% # details.
+%
+% trace_enablable_insns = [
+% 'opt_send_without_block',
+% 'send',
+% 'invokeblock',
+% 'invokesuper',
+% ]
+%
+% to_cstr = lambda do |line|
+% normalized = line.gsub(/\t/, ' ' * 8)
+% indented = normalized.sub(/\A(?!#)/, ' ') # avoid indenting preprocessor
+% rstring2cstr(indented.rstrip).sub(/"\z/, '\\n"')
+% end
+%
+ fprintf(f, "{\n");
+ {
+% # compiler: Prepare operands which may be used by `insn.call_attribute`
+% insn.opes.each_with_index do |ope, i|
+ MAYBE_UNUSED(<%= ope.fetch(:decl) %>) = (<%= ope.fetch(:type) %>)operands[<%= i %>];
+% end
+%
+% # JIT: Declare variables for operands, popped values and return values
+% ret_decls = insn.rets.map { |r| "MAYBE_UNUSED(#{r.fetch(:type)}) #{r.fetch(:name)}"} # TODO: fix #declarations to return Hash...
+% insn.declarations.each do |decl|
+% next if dispatched && ret_decls.include?(decl) # return value should be propagated to dispatcher. TODO: assert it's the same as dispatcher
+ fprintf(f, " <%= decl %>;\n");
+% end
+
+% # JIT: Set const expressions for `RubyVM::OperandsUnifications` insn
+% insn.preamble.each do |amble|
+ fprintf(f, "<%= amble.expr %>\n");
+% end
+%
+% # JIT: Initialize operands
+% insn.opes.each_with_index do |ope, i|
+ fprintf(f, " <%= ope.fetch(:name) %> = (<%= ope.fetch(:type) %>)0x%"PRIxVALUE";\n", operands[<%= i %>]);
+% end
+%
+% # JIT: Initialize popped values
+% insn.pops.reverse_each.with_index.reverse_each do |pop, i|
+ fprintf(f, " <%= pop.fetch(:name) %> = stack[%d];\n", b->stack_size - <%= i + 1 %>);
+% end
+%
+% # JIT: move sp and pc if necessary
+% if insn.handles_frame?
+ fprintf(f, " reg_cfp->pc = (VALUE *)0x%"PRIxVALUE";\n", (VALUE)(body->iseq_encoded + next_pos)); /* ADD_PC(INSN_ATTR(width)); */
+ fprintf(f, " reg_cfp->sp = reg_cfp->bp + %d;\n", b->stack_size + 1 - <%= insn.pops.size %>); /* POPN(INSN_ATTR(popn)); */
+% else
+ fprintf(f, " reg_cfp->pc = (VALUE *)0x%"PRIxVALUE";\n", (VALUE)(body->iseq_encoded + pos));
+ fprintf(f, " reg_cfp->sp = reg_cfp->bp + %d;\n", b->stack_size + 1);
+% end
+%
+% # JIT: Print insn body in insns.def
+% insn.expr.expr.each_line do |line|
+% # Special macro expansion for ones that can't be resolved by macro redefinition.
+% if line =~ /\A\s+DISPATCH_ORIGINAL_INSN\((?<insn_name>[^)]+)\);\s+\z/
+ fprintf(f, " return Qundef; /* cancel JIT */\n");
+% elsif line =~ /\A\s+JUMP\((?<dest>[^)]+)\);\s+\z/
+% # Before we `goto` next insn, we need to set return values, especially for getinlinecache
+% insn.rets.reverse_each.with_index do |ret, i|
+% # TOPN(n) = ...
+ fprintf(f, " stack[%d] = <%= ret.fetch(:name) %>;\n", b->stack_size + (int)<%= insn.call_attribute('sp_inc') %> - <%= i + 1 %>);
+% end
+%
+% dest = Regexp.last_match[:dest]
+% if insn.name == 'opt_case_dispatch' # special case... TODO: use another macro to avoid checking name
+ {
+ struct case_dispatch_var arg;
+ arg.f = f;
+ arg.base_pos = pos + insn_len(insn);
+ arg.last_value = Qundef;
+
+ fprintf(f, " switch (<%= dest %>) {\n");
+ st_foreach(RHASH_TBL_RAW(hash), compile_case_dispatch_each, (VALUE)&arg);
+ fprintf(f, " case %lu:\n", else_offset);
+ fprintf(f, " goto label_%lu;\n", arg.base_pos + else_offset);
+ fprintf(f, " }\n");
+ }
+% else
+ next_pos = pos + insn_len(insn) + (unsigned int)<%= dest %>;
+ fprintf(f, " goto label_%d;\n", next_pos);
+% end
+% elsif line =~ /\A\s+RESTORE_REGS\(\);\s+\z/ # for `leave` only
+#if OPT_CALL_THREADED_CODE
+ fprintf(f, " rb_ec_thread_ptr(ec)->retval = val;\n");
+ fprintf(f, " return 0;\n");
+#else
+ fprintf(f, " return val;\n");
+#endif
+% else
+ fprintf(f, <%= to_cstr.call(line) %>);
+% end
+% end
+%
+% # JIT: Set return values
+% unless dispatched
+% insn.rets.reverse_each.with_index do |ret, i|
+% # TOPN(n) = ...
+ fprintf(f, " stack[%d] = <%= ret.fetch(:name) %>;\n", b->stack_size + (int)<%= insn.call_attribute('sp_inc') %> - <%= i + 1 %>);
+% end
+% end
+%
+% # JIT: We should evaluate ISeq modified for TracePoint if it's enabled. Note: This is slow.
+% if trace_enablable_insns.include?(insn.name)
+ fprintf(f, " if (UNLIKELY(ruby_vm_event_enabled_flags & ISEQ_TRACE_EVENTS)) {\n");
+ fprintf(f, " reg_cfp->sp = reg_cfp->bp + %d;\n", b->stack_size + (int)<%= insn.call_attribute('sp_inc') %> + 1);
+ fprintf(f, " return Qundef; /* cancel JIT */\n");
+ fprintf(f, " }\n");
+% end
+%
+% # compiler: Move JIT compiler's internal stack pointer
+% unless dispatched
+ b->stack_size += <%= insn.call_attribute('sp_inc') %>;
+% end
+ }
+ fprintf(f, "}\n");
+%
+% # compiler: If insn has conditional JUMP, the branch which is not targeted by JUMP should be compiled too.
+% if insn.expr.expr =~ /if\s+\([^{}]+\)\s+\{[^{}]+JUMP\([^)]+\);[^{}]+\}/
+ compile_insns(f, body, b->stack_size, pos + insn_len(insn), status);
+% end
+%
+% # compiler: If insn returns (leave) or does longjmp (throw), the branch should no longer be compieled. TODO: create attr for it?
+% if insn.expr.expr =~ /\sTHROW_EXCEPTION\([^)]+\);/ || insn.expr.expr =~ /\sRESTORE_REGS\(\);/
+ b->finish_p = TRUE;
+% end
diff --git a/tool/ruby_vm/views/_mjit_compile_send.erb b/tool/ruby_vm/views/_mjit_compile_send.erb
new file mode 100644
index 0000000..6624cfe
--- /dev/null
+++ b/tool/ruby_vm/views/_mjit_compile_send.erb
@@ -0,0 +1,74 @@
+% # -*- mode:c; style:ruby; coding: utf-8; indent-tabs-mode: nil -*-
+% # Copyright (c) 2018 Takashi Kokubun. All rights reserved.
+% #
+% # This file is a part of the programming language Ruby. Permission is hereby
+% # granted, to either redistribute and/or modify this file, provided that the
+% # conditions mentioned in the file COPYING are met. Consult the file for
+% # details.
+%
+% # Optimized case of send / opt_send_without_block instructions.
+ {
+% # compiler: Prepare operands which may be used by `insn.call_attribute`
+% insn.opes.each_with_index do |ope, i|
+ MAYBE_UNUSED(<%= ope.fetch(:decl) %>) = (<%= ope.fetch(:type) %>)operands[<%= i %>];
+% end
+%
+ const rb_iseq_t *iseq;
+ unsigned int argc = ci->orig_argc; /* unlike `ci->orig_argc`, `argc` may include blockarg */
+% if insn.name == 'send'
+ argc += ((ci->flag & VM_CALL_ARGS_BLOCKARG) ? 1 : 0);
+% end
+
+ if (inlinable_iseq_p(ci, cc, iseq = get_iseq_if_available(cc))) {
+ int param_size = iseq->body->param.size; /* TODO: check calling->argc for argument_arity_error */
+
+% # JIT: move sp and pc if necessary
+ fprintf(f, " reg_cfp->pc = (VALUE *)0x%"PRIxVALUE";\n", (VALUE)(body->iseq_encoded + next_pos)); /* ADD_PC(INSN_ATTR(width)); */
+ fprintf(f, " reg_cfp->sp = reg_cfp->bp + %d;\n", b->stack_size + 1 - <%= insn.pops.size %>); /* POPN(INSN_ATTR(popn)); */
+
+% # JIT: Invalidate call cache if it requires vm_search_method. This allows to inline some of following things.
+ fprintf(f, " if (UNLIKELY(GET_GLOBAL_METHOD_STATE() != %llu || RCLASS_SERIAL(CLASS_OF(stack[%d])) != %llu)) {\n", cc->method_state, b->stack_size - 1 - argc, cc->class_serial);
+ fprintf(f, " reg_cfp->pc = (VALUE *)0x%"PRIxVALUE";\n", (VALUE)(body->iseq_encoded + pos));
+ fprintf(f, " return Qundef; /* cancel JIT */\n");
+ fprintf(f, " }\n");
+
+% # JIT: Print insn body in insns.def
+ fprintf(f, " {\n");
+ fprintf(f, " struct rb_calling_info calling;\n");
+% if insn.name == 'send'
+ fprintf(f, " vm_caller_setup_arg_block(ec, reg_cfp, &calling, 0x%"PRIxVALUE", 0x%"PRIxVALUE", FALSE);\n", operands[0], operands[2]);
+% else
+ fprintf(f, " calling.block_handler = VM_BLOCK_HANDLER_NONE;\n");
+% end
+ fprintf(f, " calling.argc = %d;\n", ci->orig_argc);
+ fprintf(f, " calling.recv = stack[%d];\n", b->stack_size - 1 - argc);
+
+% # JIT: Special CALL_METHOD. Inline vm_call_iseq_setup_normal for vm_call_iseq_setup_func FASTPATH. TODO: modify VM to share code with here
+ fprintf(f, " {\n");
+ fprintf(f, " VALUE v;\n");
+ fprintf(f, " VALUE *argv = reg_cfp->sp - calling.argc;\n");
+ fprintf(f, " reg_cfp->sp = argv - 1;\n"); /* recv */
+ fprintf(f, " vm_push_frame(ec, 0x%"PRIxVALUE", VM_FRAME_MAGIC_METHOD | VM_ENV_FLAG_LOCAL, calling.recv, "
+ "calling.block_handler, 0x%"PRIxVALUE", 0x%"PRIxVALUE", argv + %d, %d, %d);\n",
+ (VALUE)iseq, (VALUE)cc->me, (VALUE)iseq->body->iseq_encoded, param_size, iseq->body->local_table_size - param_size, iseq->body->stack_max);
+ fprintf(f, " if ((v = mjit_exec(ec)) == Qundef) {\n");
+ fprintf(f, " VM_ENV_FLAGS_SET(ec->cfp->ep, VM_FRAME_FLAG_FINISH);\n"); /* This is vm_call0_body's code after vm_call_iseq_setup */
+ fprintf(f, " v = vm_exec(ec);\n");
+ fprintf(f, " }\n");
+ fprintf(f, " stack[%d] = v;\n", b->stack_size - argc - 1);
+ fprintf(f, " }\n");
+
+ fprintf(f, " }\n");
+
+% # JIT: We should evaluate ISeq modified for TracePoint if it's enabled. Note: This is slow.
+ fprintf(f, " if (UNLIKELY(ruby_vm_event_enabled_flags & ISEQ_TRACE_EVENTS)) {\n");
+ fprintf(f, " reg_cfp->sp = reg_cfp->bp + %d;\n", b->stack_size + (int)<%= insn.call_attribute('sp_inc') %> + 1);
+ fprintf(f, " return Qundef; /* cancel JIT */\n");
+ fprintf(f, " }\n");
+
+% # compiler: Move JIT compiler's internal stack pointer
+ b->stack_size += <%= insn.call_attribute('sp_inc') %>;
+
+ break;
+ }
+ }
diff --git a/tool/ruby_vm/views/mjit_compile.inc.erb b/tool/ruby_vm/views/mjit_compile.inc.erb
new file mode 100644
index 0000000..7437a44
--- /dev/null
+++ b/tool/ruby_vm/views/mjit_compile.inc.erb
@@ -0,0 +1,66 @@
+/* -*- mode:c; style:ruby; coding: utf-8; indent-tabs-mode: nil -*- */
+
+% # Copyright (c) 2018 Takashi Kokubun. All rights reserved.
+% #
+% # This file is a part of the programming language Ruby. Permission is hereby
+% # granted, to either redistribute and/or modify this file, provided that the
+% # conditions mentioned in the file COPYING are met. Consult the file for
+% # details.
+<%= render 'copyright' %>
+%
+% # This is an ERB template that generates Ruby code that generates C code that
+% # generates JIT-ed C code.
+<%= render 'notice', locals: {
+ this_file: 'is the main part of compile_insn() in mjit_compile.c',
+ edit: __FILE__,
+} -%>
+%
+% unsupported_insns = [
+% 'getblockparamproxy', # TODO: support this
+% 'defineclass', # low priority
+% 'opt_call_c_function', # low priority
+% ]
+%
+% # Available variables and macros in JIT-ed function:
+% # ec: the first argument of _mjitXXX
+% # reg_cfp: the second argument of _mjitXXX
+% # GET_CFP(): refers to `reg_cfp`
+% # GET_EP(): refers to `reg_cfp->ep`
+% # GET_SP(): refers to `reg_cfp->sp`
+% # INC_SP(): refers to `reg_cfp->sp`
+% # SET_SV(): refers to `reg_cfp->sp`
+% # PUSH(): refers to `SET_SV()`, `INC_SP()`
+% # GET_SELF(): refers to `reg_cfp->self`
+% # GET_LEP(): refers to `VM_EP_LEP(reg_cfp->ep)`
+% # EXEC_EC_CFP(): refers to `val = vm_exec(ec)` with frame setup
+% # CALL_METHOD(): using `GET_CFP()` and `EXEC_EC_CFP()`
+% # TOPN(): refers to `reg_cfp->sp`, which needs to have correct sp (of course)
+% # STACK_ADDR_FROM_TOP(): refers to `reg_cfp->sp`, same problem here
+% # DISPATCH_ORIGINAL_INSN(): expanded in _mjit_compile_insn.erb
+% # THROW_EXCEPTION(): specially defined for JIT
+% # RESTORE_REGS(): specially defined for `leave`
+
+switch (insn) {
+% (RubyVM::BareInstructions.to_a + RubyVM::OperandsUnifications.to_a).each do |insn|
+% next if unsupported_insns.include?(insn.name)
+ case BIN(<%= insn.name %>):
+% if %w[opt_send_without_block send].include?(insn.name)
+<%= render 'mjit_compile_send', locals: { insn: insn } -%>
+% end
+<%= render 'mjit_compile_insn', locals: { insn: insn, dispatched: false } -%>
+ break;
+% end
+% # We don't support InstructionsUnifications yet because it's not used for now.
+% # We don't support TraceInstructions yet. There is no blocker for it but it's just not implemented.
+ default:
+ if (mjit_opts.warnings || mjit_opts.verbose >= 3)
+ /* passing excessive arguments to suppress warning in insns_info.inc as workaround... */
+ fprintf(stderr, "MJIT warning: Failed to compile instruction: %s (%s: %d...)\n",
+ insn_name(insn), insn_op_types(insn), insn_len(insn) > 0 ? insn_op_type(insn, 0) : 0);
+ status->success = FALSE;
+ break;
+}
+
+/* if next_pos is already compiled, next instruction won't be compiled in C code and needs `goto`. */
+if ((next_pos < body->iseq_size && status->compiled_for_pos[next_pos]))
+ fprintf(f, " goto label_%d;\n", next_pos);
diff --git a/tool/transform_mjit_header.rb b/tool/transform_mjit_header.rb
index 3083791..4c3377b 100644
--- a/tool/transform_mjit_header.rb
+++ b/tool/transform_mjit_header.rb
@@ -17,6 +17,7 @@ module MJITHeader
]
IGNORED_FUNCTIONS = [
+ 'vm_search_method_slowpath', # This increases the time to compile when inlined. So we use it as external function.
'rb_equal_opt', # Not used from VM and not compilable
]
diff --git a/tool/update-deps b/tool/update-deps
index ccfc160..200afe7 100755
--- a/tool/update-deps
+++ b/tool/update-deps
@@ -120,6 +120,7 @@ FILES_NEED_VPATH = %w[
known_errors.inc
lex.c
miniprelude.c
+ mjit_compile.inc
newline.c
node_name.inc
opt_sc.inc
diff --git a/variable.c b/variable.c
index ca7e498..433f97e 100644
--- a/variable.c
+++ b/variable.c
@@ -480,7 +480,7 @@ struct rb_global_variable {
struct trace_var *trace;
};
-struct rb_global_entry*
+MJIT_FUNC_EXPORTED struct rb_global_entry*
rb_global_entry(ID id)
{
struct rb_global_entry *entry;
@@ -790,7 +790,7 @@ rb_f_untrace_var(int argc, const VALUE *argv)
return Qnil;
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_gvar_get(struct rb_global_entry *entry)
{
struct rb_global_variable *var = entry->var;
@@ -823,7 +823,7 @@ trace_en(struct rb_global_variable *var)
return Qnil; /* not reached */
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_gvar_set(struct rb_global_entry *entry, VALUE val)
{
struct trace_data trace;
@@ -858,7 +858,7 @@ rb_gv_get(const char *name)
return rb_gvar_get(entry);
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_gvar_defined(struct rb_global_entry *entry)
{
if (entry->var->getter == rb_gvar_undef_getter) return Qfalse;
@@ -2010,7 +2010,7 @@ check_autoload_required(VALUE mod, ID id, const char **loadingpath)
return 0;
}
-int
+MJIT_FUNC_EXPORTED int
rb_autoloading_value(VALUE mod, ID id, VALUE* value)
{
VALUE load;
@@ -2211,7 +2211,7 @@ rb_autoload_p(VALUE mod, ID id)
return (ele = check_autoload_data(load)) ? ele->feature : Qnil;
}
-void
+MJIT_FUNC_EXPORTED void
rb_const_warn_if_deprecated(const rb_const_entry_t *ce, VALUE klass, ID id)
{
if (RB_CONST_DEPRECATED_P(ce)) {
@@ -2294,7 +2294,7 @@ rb_const_get_at(VALUE klass, ID id)
return rb_const_get_0(klass, id, TRUE, FALSE, FALSE);
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_public_const_get_from(VALUE klass, ID id)
{
return rb_const_get_0(klass, id, TRUE, TRUE, TRUE);
@@ -2306,7 +2306,7 @@ rb_public_const_get(VALUE klass, ID id)
return rb_const_get_0(klass, id, FALSE, TRUE, TRUE);
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_public_const_get_at(VALUE klass, ID id)
{
return rb_const_get_0(klass, id, TRUE, FALSE, TRUE);
@@ -2544,7 +2544,7 @@ rb_const_defined_at(VALUE klass, ID id)
return rb_const_defined_0(klass, id, TRUE, FALSE, FALSE);
}
-int
+MJIT_FUNC_EXPORTED int
rb_public_const_defined_from(VALUE klass, ID id)
{
return rb_const_defined_0(klass, id, TRUE, TRUE, TRUE);
@@ -3123,7 +3123,7 @@ rb_st_copy(VALUE obj, struct st_table *orig_tbl)
return new_tbl;
}
-rb_const_entry_t *
+MJIT_FUNC_EXPORTED rb_const_entry_t *
rb_const_lookup(VALUE klass, ID id)
{
struct rb_id_table *tbl = RCLASS_CONST_TBL(klass);
diff --git a/vm.c b/vm.c
index 0ff180a..7fc7aee 100644
--- a/vm.c
+++ b/vm.c
@@ -293,7 +293,7 @@ static void vm_collect_usage_register(int reg, int isset);
#endif
static VALUE vm_make_env_object(const rb_execution_context_t *ec, rb_control_frame_t *cfp);
-static VALUE vm_invoke_bmethod(rb_execution_context_t *ec, rb_proc_t *proc, VALUE self, int argc, const VALUE *argv, VALUE block_handler);
+extern VALUE vm_invoke_bmethod(rb_execution_context_t *ec, rb_proc_t *proc, VALUE self, int argc, const VALUE *argv, VALUE block_handler);
static VALUE vm_invoke_proc(rb_execution_context_t *ec, rb_proc_t *proc, VALUE self, int argc, const VALUE *argv, VALUE block_handler);
static VALUE rb_block_param_proxy;
@@ -302,17 +302,24 @@ static VALUE rb_block_param_proxy;
#include "vm_insnhelper.h"
#include "vm_exec.h"
#include "vm_insnhelper.c"
+
+#ifndef MJIT_HEADER
+
#include "vm_exec.c"
#include "vm_method.c"
+#endif /* #ifndef MJIT_HEADER */
#include "vm_eval.c"
+#ifndef MJIT_HEADER
#define PROCDEBUG 0
rb_serial_t
rb_next_class_serial(void)
{
- return NEXT_CLASS_SERIAL();
+ rb_serial_t class_serial = NEXT_CLASS_SERIAL();
+ mjit_add_class_serial(class_serial);
+ return class_serial;
}
VALUE rb_cRubyVM;
@@ -339,7 +346,7 @@ rb_vm_inc_const_missing_count(void)
VALUE rb_class_path_no_cache(VALUE _klass);
-int
+MJIT_FUNC_EXPORTED int
rb_dtrace_setup(rb_execution_context_t *ec, VALUE klass, ID id,
struct ruby_dtrace_method_hook_args *args)
{
@@ -499,7 +506,7 @@ rb_vm_get_binding_creatable_next_cfp(const rb_execution_context_t *ec, const rb_
return 0;
}
-rb_control_frame_t *
+MJIT_FUNC_EXPORTED rb_control_frame_t *
rb_vm_get_ruby_level_next_cfp(const rb_execution_context_t *ec, const rb_control_frame_t *cfp)
{
if (RUBY_VM_CONTROL_FRAME_STACK_OVERFLOW_P(ec, cfp)) bp();
@@ -512,6 +519,8 @@ rb_vm_get_ruby_level_next_cfp(const rb_execution_context_t *ec, const rb_control
return 0;
}
+#endif /* #ifndef MJIT_HEADER */
+
static rb_control_frame_t *
vm_get_ruby_level_caller_cfp(const rb_execution_context_t *ec, const rb_control_frame_t *cfp)
{
@@ -546,6 +555,8 @@ rb_vm_pop_cfunc_frame(void)
vm_pop_frame(ec, cfp, cfp->ep);
}
+#ifndef MJIT_HEADER
+
void
rb_vm_rewind_cfp(rb_execution_context_t *ec, rb_control_frame_t *cfp)
{
@@ -880,7 +891,7 @@ rb_proc_dup(VALUE self)
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_vm_make_proc_lambda(const rb_execution_context_t *ec, const struct rb_captured_block *captured, VALUE klass, int8_t is_lambda)
{
VALUE procval;
@@ -1157,14 +1168,14 @@ vm_invoke_proc(rb_execution_context_t *ec, rb_proc_t *proc, VALUE self,
return invoke_block_from_c_proc(ec, proc, self, argc, argv, passed_block_handler, proc->is_lambda);
}
-static VALUE
+MJIT_FUNC_EXPORTED VALUE
vm_invoke_bmethod(rb_execution_context_t *ec, rb_proc_t *proc, VALUE self,
int argc, const VALUE *argv, VALUE block_handler)
{
return invoke_block_from_c_proc(ec, proc, self, argc, argv, block_handler, TRUE);
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_vm_invoke_proc(rb_execution_context_t *ec, rb_proc_t *proc,
int argc, const VALUE *argv, VALUE passed_block_handler)
{
@@ -1391,7 +1402,7 @@ make_localjump_error(const char *mesg, VALUE value, int reason)
return exc;
}
-void
+MJIT_FUNC_EXPORTED void
rb_vm_localjump_error(const char *mesg, VALUE value, int reason)
{
VALUE exc = make_localjump_error(mesg, value, reason);
@@ -1775,7 +1786,7 @@ hook_before_rewind(rb_execution_context_t *ec, const rb_control_frame_t *cfp, in
};
*/
-static VALUE
+MJIT_FUNC_EXPORTED VALUE
vm_exec(rb_execution_context_t *ec)
{
enum ruby_tag_type state;
@@ -1789,8 +1800,8 @@ vm_exec(rb_execution_context_t *ec)
if ((state = EC_EXEC_TAG()) == TAG_NONE) {
result = mjit_exec(ec);
vm_loop_start:
- if (result == Qundef)
- result = vm_exec_core(ec, initial);
+ if (result == Qundef)
+ result = vm_exec_core(ec, initial);
VM_ASSERT(ec->tag == &_tag);
if ((state = _tag.state) != TAG_NONE) {
err = (struct vm_throw_data *)result;
@@ -2000,6 +2011,7 @@ vm_exec(rb_execution_context_t *ec)
state = 0;
ec->tag->state = TAG_NONE;
ec->errinfo = Qnil;
+
result = Qundef;
goto vm_loop_start;
}
@@ -3424,4 +3436,6 @@ vm_collect_usage_register(int reg, int isset)
}
#endif
+#endif /* #ifndef MJIT_HEADER */
+
#include "vm_call_iseq_optimized.inc" /* required from vm_insnhelper.c */
diff --git a/vm_backtrace.c b/vm_backtrace.c
index d6a8a13..5934029 100644
--- a/vm_backtrace.c
+++ b/vm_backtrace.c
@@ -520,7 +520,7 @@ bt_iter_cfunc(void *ptr, const rb_control_frame_t *cfp, ID mid)
loc->body.cfunc.prev_loc = arg->prev_loc;
}
-VALUE
+MJIT_FUNC_EXPORTED VALUE
rb_ec_backtrace_object(const rb_execution_context_t *ec)
{
struct bt_iter_arg arg;
@@ -595,7 +595,7 @@ rb_backtrace_to_str_ary(VALUE self)
return bt->strary;
}
-void
+MJIT_FUNC_EXPORTED void
rb_backtrace_use_iseq_first_lineno_for_last_location(VALUE self)
{
const rb_backtrace_t *bt;
diff --git a/vm_core.h b/vm_core.h
index def9051..c39bcad 100644
--- a/vm_core.h
+++ b/vm_core.h
@@ -686,9 +686,10 @@ typedef struct rb_control_frame_struct {
VALUE self; /* cfp[3] / block[0] */
const VALUE *ep; /* cfp[4] / block[1] */
const void *block_code; /* cfp[5] / block[2] */ /* iseq or ifunc */
+ const VALUE *bp; /* cfp[6] */
#if VM_DEBUG_BP_CHECK
- VALUE *bp_check; /* cfp[6] */
+ VALUE *bp_check; /* cfp[7] */
#endif
} rb_control_frame_t;
diff --git a/vm_eval.c b/vm_eval.c
index 3a82a08..61c112a 100644
--- a/vm_eval.c
+++ b/vm_eval.c
@@ -20,7 +20,7 @@ static inline VALUE vm_yield_with_cref(rb_execution_context_t *ec, int argc, con
static inline VALUE vm_yield(rb_execution_context_t *ec, int argc, const VALUE *argv);
static inline VALUE vm_yield_with_block(rb_execution_context_t *ec, int argc, const VALUE *argv, VALUE block_handler);
static inline VALUE vm_yield_force_blockarg(rb_execution_context_t *ec, VALUE args);
-static VALUE vm_exec(rb_execution_context_t *ec);
+VALUE vm_exec(rb_execution_context_t *ec);
static void vm_set_eval_stack(rb_execution_context_t * th, const rb_iseq_t *iseq, const rb_cref_t *cref, const struct rb_block *base_block);
static int vm_collect_local_variables_in_heap(const VALUE *dfp, const struct local_var_list *vars);
@@ -38,7 +38,9 @@ typedef enum call_type {
static VALUE send_internal(int argc, const VALUE *argv, VALUE recv, call_type scope);
static VALUE vm_call0_body(rb_execution_context_t* ec, struct rb_calling_info *calling, const struct rb_call_info *ci, struct rb_call_cache *cc, const VALUE *argv);
-static VALUE
+#ifndef MJIT_HEADER
+
+MJIT_FUNC_EXPORTED VALUE
vm_call0(rb_execution_context_t *ec, VALUE recv, ID id, int argc, const VALUE *argv, const rb_callable_method_entry_t *me)
{
struct rb_calling_info calling_entry, *calling;
@@ -252,6 +254,8 @@ rb_current_receiver(void)
return cfp->self;
}
+#endif /* #ifndef MJIT_HEADER */
+
static inline void
stack_check(rb_execution_context_t *ec)
{
@@ -262,6 +266,8 @@ stack_check(rb_execution_context_t *ec)
}
}
+#ifndef MJIT_HEADER
+
static inline const rb_callable_method_entry_t *rb_search_method_entry(VALUE recv, ID mid);
static inline enum method_missing_reason rb_method_call_status(rb_execution_context_t *ec, const rb_callable_method_entry_t *me, call_type scope, VALUE self);
@@ -633,7 +639,7 @@ rb_method_missing(int argc, const VALUE *argv, VALUE obj)
UNREACHABLE;
}
-static VALUE
+MJIT_FUNC_EXPORTED VALUE
make_no_method_exception(VALUE exc, VALUE format, VALUE obj,
int argc, const VALUE *argv, int priv)
{
@@ -659,6 +665,8 @@ make_no_method_exception(VALUE exc, VALUE format, VALUE obj,
return rb_class_new_instance(n, args, exc);
}
+#endif /* #ifndef MJIT_HEADER */
+
static void
raise_method_missing(rb_execution_context_t *ec, int argc, const VALUE *argv, VALUE obj,
enum method_missing_reason last_call_status)
@@ -740,6 +748,8 @@ method_missing(VALUE obj, ID id, int argc, const VALUE *argv, enum method_missin
return result;
}
+#ifndef MJIT_HEADER
+
/*!
* Calls a method
* \param recv receiver of the method
@@ -2175,3 +2185,5 @@ Init_vm_eval(void)
id_tag = rb_intern_const("tag");
id_value = rb_intern_const("value");
}
+
+#endif /* #ifndef MJIT_HEADER */
diff --git a/vm_exec.h b/vm_exec.h
index e0ceb64..76bdea1 100644
--- a/vm_exec.h
+++ b/vm_exec.h
@@ -167,8 +167,26 @@ default: \
#endif
+#ifdef MJIT_HEADER
+#define EXEC_EC_CFP() do { \
+ VM_ENV_FLAGS_SET(ec->cfp->ep, VM_FRAME_FLAG_FINISH); \
+ val = vm_exec(ec); \
+} while (0)
+#else
+#define EXEC_EC_CFP() do { \
+ RESTORE_REGS(); \
+ NEXT_INSN(); \
+} while (0)
+#endif
+
#define VM_SP_CNT(ec, sp) ((sp) - (ec)->vm_stack)
+#ifdef MJIT_HEADER
+#define THROW_EXCEPTION(exc) do { \
+ ec->errinfo = (VALUE)(exc); \
+ EC_JUMP_TAG(ec, ec->tag->state); \
+} while (0)
+#else
#if OPT_CALL_THREADED_CODE
#define THROW_EXCEPTION(exc) do { \
ec->errinfo = (VALUE)(exc); \
@@ -177,6 +195,7 @@ default: \
#else
#define THROW_EXCEPTION(exc) return (VALUE)(exc)
#endif
+#endif
#define SCREG(r) (reg_##r)
diff --git a/vm_insnhelper.c b/vm_insnhelper.c
index acb7d79..bc54acb 100644
--- a/vm_insnhelper.c
+++ b/vm_insnhelper.c
@@ -243,7 +243,8 @@ vm_push_frame(rb_execution_context_t *ec,
*sp++ = specval /* ep[-1] / block handler or prev env ptr */;
*sp = type; /* ep[-0] / ENV_FLAGS */
- cfp->ep = sp;
+ /* Store initial value of ep as bp to skip calculation cost of bp on JIT cancellation. */
+ cfp->ep = cfp->bp = sp;
cfp->sp = sp + 1;
#if VM_DEBUG_BP_CHECK
@@ -1295,6 +1296,18 @@ vm_expandarray(rb_control_frame_t *cfp, VALUE ary, rb_num_t num, int flag)
static VALUE vm_call_general(rb_execution_context_t *ec, rb_control_frame_t *reg_cfp, struct rb_calling_info *calling, const struct rb_call_info *ci, struct rb_call_cache *cc);
+MJIT_FUNC_EXPORTED void
+vm_search_method_slowpath(const struct rb_call_info *ci, struct rb_call_cache *cc, VALUE klass)
+{
+ cc->me = rb_callable_method_entry(klass, ci->mid);
+ VM_ASSERT(callable_method_entry_p(cc->me));
+ cc->call = vm_call_general;
+#if OPT_INLINE_METHOD_CACHE
+ cc->method_state = GET_GLOBAL_METHOD_STATE();
+ cc->class_serial = RCLASS_SERIAL(klass);
+#endif
+}
+
static void
vm_search_method(const struct rb_call_info *ci, struct rb_call_cache *cc, VALUE recv)
{
@@ -1312,13 +1325,7 @@ vm_search_method(const struct rb_call_info *ci, struct rb_call_cache *cc, VALUE
}
RB_DEBUG_COUNTER_INC(mc_inline_miss);
#endif
- cc->me = rb_callable_method_entry(klass, ci->mid);
- VM_ASSERT(callable_method_entry_p(cc->me));
- cc->call = vm_call_general;
-#if OPT_INLINE_METHOD_CACHE
- cc->method_state = GET_GLOBAL_METHOD_STATE();
- cc->class_serial = RCLASS_SERIAL(klass);
-#endif
+ vm_search_method_slowpath(ci, cc, klass);
}
static inline int
@@ -1458,7 +1465,7 @@ rb_eql_opt(VALUE obj1, VALUE obj2)
return opt_eql_func(obj1, obj2, &ci, &cc);
}
-static VALUE vm_call0(rb_execution_context_t *ec, VALUE, ID, int, const VALUE*, const rb_callable_method_entry_t *);
+extern VALUE vm_call0(rb_execution_context_t *ec, VALUE, ID, int, const VALUE*, const rb_callable_method_entry_t *);
static VALUE
check_match(rb_execution_context_t *ec, VALUE pattern, VALUE target, enum vm_check_match_type type)
@@ -1562,9 +1569,9 @@ static inline VALUE vm_call_method(rb_execution_context_t *ec, rb_control_frame_
static vm_call_handler vm_call_iseq_setup_func(const struct rb_call_info *ci, const int param_size, const int local_size);
-static rb_method_definition_t *method_definition_create(rb_method_type_t type, ID mid);
-static void method_definition_set(const rb_method_entry_t *me, rb_method_definition_t *def, void *opts);
-static int rb_method_definition_eq(const rb_method_definition_t *d1, const rb_method_definition_t *d2);
+extern rb_method_definition_t *method_definition_create(rb_method_type_t type, ID mid);
+extern void method_definition_set(const rb_method_entry_t *me, rb_method_definition_t *def, void *opts);
+extern int rb_method_definition_eq(const rb_method_definition_t *d1, const rb_method_definition_t *d2);
static const rb_iseq_t *
def_iseq_ptr(rb_method_definition_t *def)
@@ -1590,7 +1597,7 @@ vm_call_iseq_setup_normal_0start(rb_execution_context_t *ec, rb_control_frame_t
return vm_call_iseq_setup_normal(ec, cfp, calling, ci, cc, 0, param, local);
}
-static inline int
+int
simple_iseq_p(const rb_iseq_t *iseq)
{
return iseq->body->param.flags.has_opt == FALSE &&
diff --git a/vm_insnhelper.h b/vm_insnhelper.h
index 31dbdac..1dd491c 100644
--- a/vm_insnhelper.h
+++ b/vm_insnhelper.h
@@ -130,8 +130,7 @@ enum vm_regan_acttype {
#define CALL_METHOD(calling, ci, cc) do { \
VALUE v = (*(cc)->call)(ec, GET_CFP(), (calling), (ci), (cc)); \
if (v == Qundef && (v = mjit_exec(ec)) == Qundef) { \
- RESTORE_REGS(); \
- NEXT_INSN(); \
+ EXEC_EC_CFP(); \
} \
else { \
val = v; \
@@ -185,7 +184,7 @@ enum vm_regan_acttype {
#define GET_GLOBAL_CONSTANT_STATE() (ruby_vm_global_constant_state)
#define INC_GLOBAL_CONSTANT_STATE() (++ruby_vm_global_constant_state)
-static VALUE make_no_method_exception(VALUE exc, VALUE format, VALUE obj,
+extern VALUE make_no_method_exception(VALUE exc, VALUE format, VALUE obj,
int argc, const VALUE *argv, int priv);
static inline struct vm_throw_data *
diff --git a/vm_method.c b/vm_method.c
index 695a985..1062b70 100644
--- a/vm_method.c
+++ b/vm_method.c
@@ -62,6 +62,7 @@ static struct {
static void
rb_class_clear_method_cache(VALUE klass, VALUE arg)
{
+ mjit_remove_class_serial(RCLASS_SERIAL(klass));
RCLASS_SERIAL(klass) = rb_next_class_serial();
if (RB_TYPE_P(klass, T_ICLASS)) {
@@ -171,7 +172,7 @@ rb_free_method_entry(const rb_method_entry_t *me)
}
static inline rb_method_entry_t *search_method(VALUE klass, ID id, VALUE *defined_class_ptr);
-static int rb_method_definition_eq(const rb_method_definition_t *d1, const rb_method_definition_t *d2);
+extern int rb_method_definition_eq(const rb_method_definition_t *d1, const rb_method_definition_t *d2);
static inline rb_method_entry_t *
lookup_method_table(VALUE klass, ID id)
@@ -222,7 +223,7 @@ setup_method_cfunc_struct(rb_method_cfunc_t *cfunc, VALUE (*func)(), int argc)
cfunc->invoker = call_cfunc_invoker_func(argc);
}
-static void
+MJIT_FUNC_EXPORTED void
method_definition_set(const rb_method_entry_t *me, rb_method_definition_t *def, void *opts)
{
*(rb_method_definition_t **)&me->def = def;
@@ -336,7 +337,7 @@ method_definition_reset(const rb_method_entry_t *me)
}
}
-static rb_method_definition_t *
+MJIT_FUNC_EXPORTED rb_method_definition_t *
method_definition_create(rb_method_type_t type, ID mid)
{
rb_method_definition_t *def;
@@ -401,7 +402,7 @@ rb_method_entry_clone(const rb_method_entry_t *src_me)
return me;
}
-const rb_callable_method_entry_t *
+MJIT_FUNC_EXPORTED const rb_callable_method_entry_t *
rb_method_entry_complement_defined_class(const rb_method_entry_t *src_me, ID called_id, VALUE defined_class)
{
rb_method_definition_t *def = src_me->def;
@@ -812,7 +813,7 @@ method_entry_get(VALUE klass, ID id, VALUE *defined_class_ptr)
return method_entry_get_without_cache(klass, id, defined_class_ptr);
}
-const rb_method_entry_t *
+MJIT_FUNC_EXPORTED const rb_method_entry_t *
rb_method_entry(VALUE klass, ID id)
{
return method_entry_get(klass, id, NULL);
@@ -853,7 +854,7 @@ prepare_callable_method_entry(VALUE defined_class, ID id, const rb_method_entry_
return cme;
}
-const rb_callable_method_entry_t *
+MJIT_FUNC_EXPORTED const rb_callable_method_entry_t *
rb_callable_method_entry(VALUE klass, ID id)
{
VALUE defined_class;
@@ -886,7 +887,7 @@ method_entry_resolve_refinement(VALUE klass, ID id, int with_refinement, VALUE *
return me;
}
-const rb_callable_method_entry_t *
+MJIT_FUNC_EXPORTED const rb_callable_method_entry_t *
rb_callable_method_entry_with_refinements(VALUE klass, ID id, VALUE *defined_class_ptr)
{
VALUE defined_class, *dcp = defined_class_ptr ? defined_class_ptr : &defined_class;
@@ -900,7 +901,7 @@ rb_method_entry_without_refinements(VALUE klass, ID id, VALUE *defined_class_ptr
return method_entry_resolve_refinement(klass, id, FALSE, defined_class_ptr);
}
-const rb_callable_method_entry_t *
+MJIT_FUNC_EXPORTED const rb_callable_method_entry_t *
rb_callable_method_entry_without_refinements(VALUE klass, ID id, VALUE *defined_class_ptr)
{
VALUE defined_class, *dcp = defined_class_ptr ? defined_class_ptr : &defined_class;
@@ -1462,7 +1463,7 @@ original_method_definition(const rb_method_definition_t *def)
return def;
}
-static int
+MJIT_FUNC_EXPORTED int
rb_method_definition_eq(const rb_method_definition_t *d1, const rb_method_definition_t *d2)
{
d1 = original_method_definition(d1);
diff --git a/vm_trace.c b/vm_trace.c
index 5734dba..3b4f223 100644
--- a/vm_trace.c
+++ b/vm_trace.c
@@ -325,7 +325,7 @@ exec_hooks_protected(rb_execution_context_t *ec, rb_vm_t *vm, rb_hook_list_t *li
return state;
}
-void
+MJIT_FUNC_EXPORTED void
rb_exec_event_hooks(rb_trace_arg_t *trace_arg, int pop_p)
{
rb_execution_context_t *ec = trace_arg->ec;
diff --git a/win32/Makefile.sub b/win32/Makefile.sub
index d42625c..3013fbe 100644
--- a/win32/Makefile.sub
+++ b/win32/Makefile.sub
@@ -1192,7 +1192,7 @@ rb_mjit_header.h: PHONY probes.h
$(Q) $(IFCHANGE) $@ vm.i
INSNS = opt_sc.inc optinsn.inc optunifs.inc insns.inc insns_info.inc \
- vmtc.inc vm.inc
+ vmtc.inc vm.inc mjit_compile.inc
!if [exit > insns_rules.mk]
!else if [for %I in ($(INSNS)) do \
diff --git a/win32/mkexports.rb b/win32/mkexports.rb
index 7672ac3..887b5d9 100755
--- a/win32/mkexports.rb
+++ b/win32/mkexports.rb
@@ -7,7 +7,7 @@ module RbConfig
end
class Exports
- PrivateNames = /(?:Init_|ruby_static_id_|.*_threadptr_|rb_ec_|DllMain\b)/
+ PrivateNames = /(?:Init_|ruby_static_id_|DllMain\b)/
@@subclass = []
def self.inherited(klass)