summaryrefslogtreecommitdiff
path: root/tool/transform_mjit_header.rb
AgeCommit message (Collapse)Author
2018-05-11transform_mjit_header.rb: workaround for Solaris 10 with old GCCngoto
* tool/transform_mjit_header.rb (MJITHeader.conflicting_types?): Add workaround for Solaris 10 with old GCC (4.6.2), that is essentially the same as for AIX (commit r62326), but probably due to different GCC versions, different error message is shown. [Bug #14751] [ruby-dev:50541] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63400 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-17transform_mjit_header.rb: read output and errorsnobu
* tool/transform_mjit_header.rb (MJITHeader.check_code): read output and errors than discarding errors. also cl.exe prints the source file name which is a garbage at this time. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62788 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-07transform_mjit_header.rb: inline ivar operationsk0kubun
* Optcarrot before,--jit: ruby 2.6.0dev (2018-03-07 trunk 62689) +JIT [x86_64-linux] after,--jit: ruby 2.6.0dev (2018-03-07 trunk 62689) +JIT [x86_64-linux] last_commit=transform_mjit_header.rb: inline ivar operations Calculating ------------------------------------- before,--jit after,--jit optcarrot 67.852 69.359 fps Comparison: optcarrot after,--jit: 69.4 fps before,--jit: 67.9 fps - 1.02x slower git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62693 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-06transform_mjit_header.rb: inline VM instructionsk0kubun
* Optcarrot benchmark (--jit) Before: 62.42 fps After: 65.31 fps git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62677 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-06transform_mjit_header.rb: drop obsoleted debug codek0kubun
Now RubyCI is stable for this part. We no longer use this. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62676 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-11common.mk: add comments about r62355 [ci skip]k0kubun
transform_mjit_header.rb: ditto git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62357 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-09transform_mjit_header.rb: fix typo in r62326k0kubun
Today's AIX CI failed due to this... :innocent: git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62332 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-09transform_mjit_header.rb: pass -std=c99 for AIX gcck0kubun
I heard `error: conflicting types for 'restrict'` can be solved by adding `-std=c99`. Ideally we should use the same cflags which are used to compile vm.c, but let me try this and see what happens on AIX. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62326 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-07transform_mjit_header.rb: very verbose error logk0kubun
for debugging AIX..... I have no idea why AIX's gcc is failing to transform MJIT header.... Today's CI output: https://rubyci.org/logs/rubyci.s3.amazonaws.com/aix71_ppc/ruby-trunk/log/20180207T113303Z.log.html.gz#make git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62290 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-07transform_mjit_header.rb: apply Regexp.escapek0kubun
to prevent future breakage Following up r62285 git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62287 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-07transform_mjit_header.rb: ignore unsupported cck0kubun
to generate MJIT header. Even if we can't build MJIT header, Ruby's build should success. And compilers which are not explicitly supported are likely to fail to transform MJIT header. Also you can pass only gcc or clang to --jit-cc=xxx for now. Thus generating header does never make sense. So I decided to conservatively give up MJIT header generation. But please feel free to add your favorite compiler's macro if you think it's working. (Another workaround is passing -D__GNUC__ :p) [Bug #14447] [Bug #14446] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62285 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-06transform_mjit_header.rb: resurrect `exit false`k0kubun
which was just unintentionally dropped at r62252. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62257 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-06transform_mjit_header.rb: add output for debugk0kubun
AIX's header build is failing like: ./miniruby -I./lib -I. -I.ext/common ./tool/transform_mjit_header.rb "/usr/bin/gcc " rb_mjit_header.h .ext/include/powerpc-aix7.1.3.0/rb_mjit_min_header-2.6.0.h error in initial header file: /home/odaira/chkbuild/tmp/build/20180206T113302Z/tmp/20180206-15335556-aaiego.c:19:59: error: conflicting types for 'restrict' extern size_t fread(void *restrict, size_t, size_t, FILE *restrict); ^ https://rubyci.org/logs/rubyci.s3.amazonaws.com/aix71_ppc/ruby-trunk/log/20180206T113302Z.log.html.gz#make It's so hard to know the cause from current output. Let me add debug output and see tomorrow's CI result. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62252 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-06Remove unused capturekazu
Because `Regexp#to_s` add `(?-mix:)`, it is not unnecessary. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62250 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-05transform_mjit_header.rb: restrict transformationk0kubun
to whitelisted patterns. This fix is for NetBSD. [Bug #14439] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62235 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-05transform_mjit_header.rb: same command after errornobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62232 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-05transform_mjit_header.rb: check initial codek0kubun
separating the macros. Applying the kept macros to code which is already affected by the macros may cause errors in initial code. This is hopefully the final fix for icc build failure. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62226 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-05transform_mjit_header.rb: remove predefined macrosk0kubun
semi-automatically. This is basically for icc's __DATE__, __TIME__, and many other families. It causes an error by predefined macro redefinition. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62225 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-05transform_mjit_header.rb: no extra CRsnobu
* tool/transform_mjit_header.rb (MJITHeader.check_code!): open temporary file in binary mode too, not to include extra CRs. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62222 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-05transform_mjit_header.rb: take program name from $0nobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62216 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-05transform_mjit_header.rb: read/write in binmodenobu
* tool/transform_mjit_header.rb: read and write as ASCII-8BIT to make single-byte-optimizable always. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62215 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-05transform_mjit_header.rb: separate macro and codenobu
* tool/transform_mjit_header.rb (separate_macro_and_code): return macro and code separately as the name, and concat before output. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62214 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-05transform_mjit_header.rb: scan by regexpnobu
* tool/transform_mjit_header.rb (find_decl): scan by regexp instead of char-by-char loop. return nil when finished. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62213 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-05Use system option instead of shell redirectkazu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62212 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04transform_mjit_header.rb: refactor messages and exitnobu
* tool/transform_mjit_header.rb: print non-error messages to STDOUT instead of STDERR. exit with false or abort instead of exit 1. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62205 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04match whole wordnobu
* tool/transform_mjit_header.rb: add word boundary anchors and match whole word to get rid of false `static` declarations, e.g., rb_str_new_static. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62199 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04Fix typos and remove redundant `/#{}/`kazu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62198 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04mjit_compile.c: merge initial JIT compilerk0kubun
which has been developed by Takashi Kokubun <takashikkbn@gmail> as YARV-MJIT. Many of its bugs are fixed by wanabe <s.wanabe@gmail.com>. This JIT compiler is designed to be a safe migration path to introduce JIT compiler to MRI. So this commit does not include any bytecode changes or dynamic instruction modifications, which are done in original MJIT. This commit even strips off some aggressive optimizations from YARV-MJIT, and thus it's slower than YARV-MJIT too. But it's still fairly faster than Ruby 2.5 in some benchmarks (attached below). Note that this JIT compiler passes `make test`, `make test-all`, `make test-spec` without JIT, and even with JIT. Not only it's perfectly safe with JIT disabled because it does not replace VM instructions unlike MJIT, but also with JIT enabled it stably runs Ruby applications including Rails applications. I'm expecting this version as just "initial" JIT compiler. I have many optimization ideas which are skipped for initial merging, and you may easily replace this JIT compiler with a faster one by just replacing mjit_compile.c. `mjit_compile` interface is designed for the purpose. common.mk: update dependencies for mjit_compile.c. internal.h: declare `rb_vm_insn_addr2insn` for MJIT. vm.c: exclude some definitions if `-DMJIT_HEADER` is provided to compiler. This avoids to include some functions which take a long time to compile, e.g. vm_exec_core. Some of the purpose is achieved in transform_mjit_header.rb (see `IGNORED_FUNCTIONS`) but others are manually resolved for now. Load mjit_helper.h for MJIT header. mjit_helper.h: New. This is a file used only by JIT-ed code. I'll refactor `mjit_call_cfunc` later. vm_eval.c: add some #ifdef switches to skip compiling some functions like Init_vm_eval. win32/mkexports.rb: export thread/ec functions, which are used by MJIT. include/ruby/defines.h: add MJIT_FUNC_EXPORTED macro alis to clarify that a function is exported only for MJIT. array.c: export a function used by MJIT. bignum.c: ditto. class.c: ditto. compile.c: ditto. error.c: ditto. gc.c: ditto. hash.c: ditto. iseq.c: ditto. numeric.c: ditto. object.c: ditto. proc.c: ditto. re.c: ditto. st.c: ditto. string.c: ditto. thread.c: ditto. variable.c: ditto. vm_backtrace.c: ditto. vm_insnhelper.c: ditto. vm_method.c: ditto. I would like to improve maintainability of function exports, but I believe this way is acceptable as initial merging if we clarify the new exports are for MJIT (so that we can use them as TODO list to fix) and add unit tests to detect unresolved symbols. I'll add unit tests of JIT compilations in succeeding commits. Author: Takashi Kokubun <takashikkbn@gmail.com> Contributor: wanabe <s.wanabe@gmail.com> Part of [Feature #14235] --- * Known issues * Code generated by gcc is faster than clang. The benchmark may be worse in macOS. Following benchmark result is provided by gcc w/ Linux. * Performance is decreased when Google Chrome is running * JIT can work on MinGW, but it doesn't improve performance at least in short running benchmark. * Currently it doesn't perform well with Rails. We'll try to fix this before release. --- * Benchmark reslts Benchmarked with: Intel 4.0GHz i7-4790K with 16GB memory under x86-64 Ubuntu 8 Cores - 2.0.0-p0: Ruby 2.0.0-p0 - r62186: Ruby trunk (early 2.6.0), before MJIT changes - JIT off: On this commit, but without `--jit` option - JIT on: On this commit, and with `--jit` option ** Optcarrot fps Benchmark: https://github.com/mame/optcarrot | |2.0.0-p0 |r62186 |JIT off |JIT on | |:--------|:--------|:--------|:--------|:--------| |fps |37.32 |51.46 |51.31 |58.88 | |vs 2.0.0 |1.00x |1.38x |1.37x |1.58x | ** MJIT benchmarks Benchmark: https://github.com/benchmark-driver/mjit-benchmarks (Original: https://github.com/vnmakarov/ruby/tree/rtl_mjit_branch/MJIT-benchmarks) | |2.0.0-p0 |r62186 |JIT off |JIT on | |:----------|:--------|:--------|:--------|:--------| |aread |1.00 |1.09 |1.07 |2.19 | |aref |1.00 |1.13 |1.11 |2.22 | |aset |1.00 |1.50 |1.45 |2.64 | |awrite |1.00 |1.17 |1.13 |2.20 | |call |1.00 |1.29 |1.26 |2.02 | |const2 |1.00 |1.10 |1.10 |2.19 | |const |1.00 |1.11 |1.10 |2.19 | |fannk |1.00 |1.04 |1.02 |1.00 | |fib |1.00 |1.32 |1.31 |1.84 | |ivread |1.00 |1.13 |1.12 |2.43 | |ivwrite |1.00 |1.23 |1.21 |2.40 | |mandelbrot |1.00 |1.13 |1.16 |1.28 | |meteor |1.00 |2.97 |2.92 |3.17 | |nbody |1.00 |1.17 |1.15 |1.49 | |nest-ntimes|1.00 |1.22 |1.20 |1.39 | |nest-while |1.00 |1.10 |1.10 |1.37 | |norm |1.00 |1.18 |1.16 |1.24 | |nsvb |1.00 |1.16 |1.16 |1.17 | |red-black |1.00 |1.02 |0.99 |1.12 | |sieve |1.00 |1.30 |1.28 |1.62 | |trees |1.00 |1.14 |1.13 |1.19 | |while |1.00 |1.12 |1.11 |2.41 | ** Discourse's script/bench.rb Benchmark: https://github.com/discourse/discourse/blob/v1.8.7/script/bench.rb NOTE: Rails performance was somehow a little degraded with JIT for now. We should fix this. (At least I know opt_aref is performing badly in JIT and I have an idea to fix it. Please wait for the fix.) *** JIT off Your Results: (note for timings- percentile is first, duration is second in millisecs) categories_admin: 50: 17 75: 18 90: 22 99: 29 home_admin: 50: 21 75: 21 90: 27 99: 40 topic_admin: 50: 17 75: 18 90: 22 99: 32 categories: 50: 35 75: 41 90: 43 99: 77 home: 50: 39 75: 46 90: 49 99: 95 topic: 50: 46 75: 52 90: 56 99: 101 *** JIT on Your Results: (note for timings- percentile is first, duration is second in millisecs) categories_admin: 50: 19 75: 21 90: 25 99: 33 home_admin: 50: 24 75: 26 90: 30 99: 35 topic_admin: 50: 19 75: 20 90: 25 99: 30 categories: 50: 40 75: 44 90: 48 99: 76 home: 50: 42 75: 48 90: 51 99: 89 topic: 50: 49 75: 55 90: 58 99: 99 git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62197 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04common.mk: install a single header file for JITk0kubun
compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-11 17:27:25 +0000