diff options
| author | Nobuyoshi Nakada <nobu@ruby-lang.org> | 2023-11-29 20:16:36 +0900 |
|---|---|---|
| committer | Nobuyoshi Nakada <nobu@ruby-lang.org> | 2023-11-29 20:16:36 +0900 |
| commit | 8e93bf8e1fbac73b677c333b19a8b55ae9daddc3 (patch) | |
| tree | 92a47d3fe8bb981af5a476cd4a30e4f7e2a0997b /test/ruby/test_integer.rb | |
| parent | 79eb75a8dd64848f23e9efc465f06326b5d4b680 (diff) | |
[Bug #17037] Improve accuracy of division near precision limits
When dividing near the precision limit of `double`, use Bignum
division to get rid of rounding errors.
Diffstat (limited to 'test/ruby/test_integer.rb')
| -rw-r--r-- | test/ruby/test_integer.rb | 8 |
1 files changed, 8 insertions, 0 deletions
diff --git a/test/ruby/test_integer.rb b/test/ruby/test_integer.rb index 31cb8d6cf5..3349a1c493 100644 --- a/test/ruby/test_integer.rb +++ b/test/ruby/test_integer.rb @@ -704,6 +704,14 @@ class TestInteger < Test::Unit::TestCase def test_fdiv assert_equal(1.0, 1.fdiv(1)) assert_equal(0.5, 1.fdiv(2)) + + m = 50 << Float::MANT_DIG + prev = 1.0 + (1..100).each do |i| + val = (m + i).fdiv(m) + assert_operator val, :>=, prev, "1+epsilon*(#{i}/100)" + prev = val + end end def test_obj_fdiv |