summaryrefslogtreecommitdiff
path: root/test/json
AgeCommit message (Collapse)Author
2025-12-11[ruby/json] Revert "Skip test failing with JRuby in CI"Étienne Barrié
This reverts commit https://github.com/ruby/json/commit/b7e1734d9ca0. https://github.com/ruby/json/commit/5793694ee6
2025-12-11[ruby/json] Add `allow_control_characters` parsing optionJean Boussier
While it's not allowed by the spec, some parsers like Oj do accept it, and it can be blocking a transition. Having this feature can help people migrate. https://github.com/ruby/json/commit/3459499cb3
2025-12-10[ruby/json] Add a specific error for unescaped newlinesJean Boussier
It's the most likely control character so it's worth giving a better error message for it. https://github.com/ruby/json/commit/1da3fd9233
2025-12-04[ruby/json] Fix a regression in parsing of unicode surogate pairsJean Boussier
Fix: https://github.com/ruby/json/issues/912 In the case of surogate pairs we consume two backslashes, so `json_next_backslash` need to ensure it's not sending us back in the stream. https://github.com/ruby/json/commit/0fce370c41
2025-12-03[ruby/json] Fix handling of depthÉtienne Barrié
https://github.com/ruby/json/commit/ccca602274
2025-12-03[ruby/json] Fix duplicated test_unsafe_load_with_options test caseJean Boussier
https://github.com/ruby/json/commit/7b62fac525
2025-12-03[ruby/json] Reproduce C ext behavior of ignoring mutated depth in arraysÉtienne Barrié
https://github.com/ruby/json/commit/e0257b9f82
2025-12-03[ruby/json] Test and restore behavior around to_json changing depthÉtienne Barrié
When serializing an Array, and one of the elements of the Array requires calling `to_json`, if the depth is changed, it will be used for the next entries, which wasn't the case before https://github.com/ruby/json/commit/5abd43490714, and is not the case with TruffleRuby and JRuby. Additionally, with TruffleRuby and JRuby the state's depth after the `to_json` call is used to close the Array, which isn't the case with CRuby. https://github.com/ruby/json/commit/386b36fde5
2025-12-03[ruby/json] Improve `JSON.load` and `JSON.unsafe_load` to allow passing ↵Jean Boussier
options as second argument Otherwise it's very error prone. https://github.com/ruby/json/commit/c54de70f90
2025-12-02[ruby/json] Don't call to_json on the return value of as_json for Float::NANÉtienne Barrié
https://github.com/ruby/json/commit/28c57df8f7
2025-11-27[ruby/json] Respect Coder depth when generatingÉtienne Barrié
https://github.com/ruby/json/commit/9c36681b17
2025-11-27[ruby/json] Test current behavior regarding depth for CoderÉtienne Barrié
Coder currently ignores its depth and always resets it to 0 when generating a new JSON document. https://github.com/ruby/json/commit/cca1f38316
2025-11-26[ruby/json] Test to_json using State#depthÉtienne Barrié
https://github.com/ruby/json/commit/ac0a980668
2025-11-26[ruby/json] Test depthÉtienne Barrié
https://github.com/ruby/json/commit/d02e40324a
2025-11-26[ruby/json] Test that depth of unfrozen State does not changeÉtienne Barrié
https://github.com/ruby/json/commit/9d32cf4618
2025-11-22[ruby/json] Fix the parser to not accept invalid escapesJean Boussier
Only `"\/bfnrtu` are valid after a backslash. https://github.com/ruby/json/commit/f7f8f552ed
2025-11-21[ruby/json] Skip test failing with JRuby in CIÉtienne Barrié
https://github.com/ruby/json/commit/305d3832db
2025-11-21[ruby/json] Ractor-shareable JSON::CoderÉtienne Barrié
https://github.com/ruby/json/commit/58d60d6b76
2025-11-07[ruby/json] Deprecate `JSON::State#[]` and `JSON::State#[]=`Jean Boussier
This prevent from freezing and sharing state instances. If you needs some sort of arguments or extra state to the generator methods, consider using `JSON::Coder` instead. https://github.com/ruby/json/commit/e9fbc8937f
2025-11-07[ruby/json] Get rid of JSON.deep_const_get (private API)Jean Boussier
https://github.com/ruby/json/commit/826cb2a4f4
2025-11-03[ruby/json] parser.c: Skip checking for escape sequences in ↵Jean Boussier
`rstring_cache_fetch` The caller already know if the string contains escape sequences so this check is redundant. Also stop calling `rstring_cache_fetch` from `json_string_unescape` as we know it won't match anyways. ``` == Parsing twitter.json (567916 bytes) ruby 3.4.6 (2025-09-16 revision https://github.com/ruby/json/commit/dbd83256b1) +YJIT +PRISM [arm64-darwin24] Warming up -------------------------------------- after 122.000 i/100ms Calculating ------------------------------------- after 1.226k (± 0.3%) i/s (815.85 μs/i) - 6.222k in 5.076282s Comparison: before: 1206.2 i/s after: 1225.7 i/s - 1.02x faster ``` https://github.com/ruby/json/commit/b8cdf3282d Co-Authored-By: Scott Myron <samyron@gmail.com>
2025-11-02[ruby/json] Invoke `as_json` callback for strings with invalid encodingJean Boussier
Fix: https://github.com/ruby/json/issues/873 This allow users to encode binary strings if they so wish. e.g. they can use Base64 or similar, or chose to replace invalid characters with something else. https://github.com/ruby/json/commit/b1b16c416f
2025-11-01[ruby/json] Enable JSON_DEBUG for parser/extconf.rbJean Boussier
https://github.com/ruby/json/commit/82b030f294
2025-11-01[ruby/json] Add test coverage for T_BIGNUM parsingJean Boussier
https://github.com/ruby/json/commit/f0150e2944
2025-10-30[ruby/json] Add ryu float parser.Josef Šimánek
https://github.com/ruby/json/commit/9c4db31908 Co-Authored-By: Jean Boussier <jean.boussier@gmail.com>
2025-10-28[ruby/json] Use Vector API in the Java ExtensionScott Myron
Overview This PR uses the [jdk.incubator.vector module](https://docs.oracle.com/en/java/javase/20/docs/api/jdk.incubator.vector/jdk/incubator/vector/package-summary.html) as mentioned in [issue #739](https://github.com/ruby/json/issues/739) to accelerate generating JSON with the same algorithm as the C extension. The PR as it exists right now, it will attempt to build the `json.ext.VectorizedEscapeScanner` class with a target release of `16`. This is the first version of Java with support for the `jdk.incubator.vector` module. The remaining code is built for Java 1.8. The code will attempt to load the `json.ext.VectorizedEscapeScanner` only if the `json.enableVectorizedEscapeScanner` system property is set to `true` (or `1`). I'm not entirely sure how this is packaged / included with JRuby so I'd love @byroot and @headius's (and others?) thought about how to potential package and/or structure the JARs. I did consider adding the `json.ext.VectorizedEscapeScanner` to a separate `generator-vectorized.jar` but I thought I'd solicit feedback before spending any more time on the build / package process. Benchmarks Machine M1 Macbook Air Note: I've had trouble modifying the `compare.rb` I was using for the C extension to work reliability with the Java extension. I'll probably spend more time trying to get it to work, but as of right now these are pretty raw benchmarks. Below are two sample runs of the real-world benchmarks. The benchmarks are much more variable then the C extension for some reason. I'm not sure if HotSpot is doing something slightly different per execution. Vector API Enabled ``` scott@Scotts-MacBook-Air json % ONLY=json JAVA_OPTS='--add-modules jdk.incubator.vector -Djson.enableVectorizedEscapeScanner=true' ruby -I"lib" benchmark/encoder-realworld.rb WARNING: Using incubator modules: jdk.incubator.vector == Encoding activitypub.json (52595 bytes) jruby 9.4.12.0 (3.1.4) 2025-02-11 https://github.com/ruby/json/commit/f4ab75096a Java HotSpot(TM) 64-Bit Server VM 21.0.7+8-LTS-245 on 21.0.7+8-LTS-245 +jit [arm64-darwin] Warming up -------------------------------------- json 1.384k i/100ms Calculating ------------------------------------- json 15.289k (± 0.8%) i/s (65.41 μs/i) - 153.624k in 10.048481s == Encoding citm_catalog.json (500298 bytes) jruby 9.4.12.0 (3.1.4) 2025-02-11 https://github.com/ruby/json/commit/f4ab75096a Java HotSpot(TM) 64-Bit Server VM 21.0.7+8-LTS-245 on 21.0.7+8-LTS-245 +jit [arm64-darwin] Warming up -------------------------------------- json 76.000 i/100ms Calculating ------------------------------------- json 753.787 (± 3.6%) i/s (1.33 ms/i) - 7.524k in 9.997059s == Encoding twitter.json (466906 bytes) jruby 9.4.12.0 (3.1.4) 2025-02-11 https://github.com/ruby/json/commit/f4ab75096a Java HotSpot(TM) 64-Bit Server VM 21.0.7+8-LTS-245 on 21.0.7+8-LTS-245 +jit [arm64-darwin] Warming up -------------------------------------- json 173.000 i/100ms Calculating ------------------------------------- json 1.751k (± 1.1%) i/s (571.24 μs/i) - 17.646k in 10.081260s == Encoding ohai.json (20147 bytes) jruby 9.4.12.0 (3.1.4) 2025-02-11 https://github.com/ruby/json/commit/f4ab75096a Java HotSpot(TM) 64-Bit Server VM 21.0.7+8-LTS-245 on 21.0.7+8-LTS-245 +jit [arm64-darwin] Warming up -------------------------------------- json 2.390k i/100ms Calculating ------------------------------------- json 23.829k (± 0.8%) i/s (41.97 μs/i) - 239.000k in 10.030503s ``` Vector API Disabled ``` scott@Scotts-MacBook-Air json % ONLY=json JAVA_OPTS='--add-modules jdk.incubator.vector -Djson.enableVectorizedEscapeScanner=false' ruby -I"lib" benchmark/encoder-realworld.rb WARNING: Using incubator modules: jdk.incubator.vector VectorizedEscapeScanner disabled. == Encoding activitypub.json (52595 bytes) jruby 9.4.12.0 (3.1.4) 2025-02-11 https://github.com/ruby/json/commit/f4ab75096a Java HotSpot(TM) 64-Bit Server VM 21.0.7+8-LTS-245 on 21.0.7+8-LTS-245 +jit [arm64-darwin] Warming up -------------------------------------- json 1.204k i/100ms Calculating ------------------------------------- json 12.937k (± 1.1%) i/s (77.30 μs/i) - 130.032k in 10.052234s == Encoding citm_catalog.json (500298 bytes) jruby 9.4.12.0 (3.1.4) 2025-02-11 https://github.com/ruby/json/commit/f4ab75096a Java HotSpot(TM) 64-Bit Server VM 21.0.7+8-LTS-245 on 21.0.7+8-LTS-245 +jit [arm64-darwin] Warming up -------------------------------------- json 80.000 i/100ms Calculating ------------------------------------- json 817.378 (± 1.0%) i/s (1.22 ms/i) - 8.240k in 10.082058s == Encoding twitter.json (466906 bytes) jruby 9.4.12.0 (3.1.4) 2025-02-11 https://github.com/ruby/json/commit/f4ab75096a Java HotSpot(TM) 64-Bit Server VM 21.0.7+8-LTS-245 on 21.0.7+8-LTS-245 +jit [arm64-darwin] Warming up -------------------------------------- json 147.000 i/100ms Calculating ------------------------------------- json 1.499k (± 1.3%) i/s (667.08 μs/i) - 14.994k in 10.004181s == Encoding ohai.json (20147 bytes) jruby 9.4.12.0 (3.1.4) 2025-02-11 https://github.com/ruby/json/commit/f4ab75096a Java HotSpot(TM) 64-Bit Server VM 21.0.7+8-LTS-245 on 21.0.7+8-LTS-245 +jit [arm64-darwin] Warming up -------------------------------------- json 2.269k i/100ms Calculating ------------------------------------- json 22.366k (± 5.7%) i/s (44.71 μs/i) - 224.631k in 10.097069s ``` `master` as of commit `https://github.com/ruby/json/commit/c5af1b68c582` ``` scott@Scotts-MacBook-Air json % ONLY=json ruby -I"lib" benchmark/encoder-realworld.rb == Encoding activitypub.json (52595 bytes) jruby 9.4.12.0 (3.1.4) 2025-02-11 https://github.com/ruby/json/commit/f4ab75096a Java HotSpot(TM) 64-Bit Server VM 21.0.7+8-LTS-245 on 21.0.7+8-LTS-245 +jit [arm64-darwin] Warming up -------------------------------------- json 886.000 i/100ms Calculating ------------------------------------- json^C% scott@Scotts-MacBook-Air json % ONLY=json ruby -I"lib" benchmark/encoder-realworld.rb == Encoding activitypub.json (52595 bytes) jruby 9.4.12.0 (3.1.4) 2025-02-11 https://github.com/ruby/json/commit/f4ab75096a Java HotSpot(TM) 64-Bit Server VM 21.0.7+8-LTS-245 on 21.0.7+8-LTS-245 +jit [arm64-darwin] Warming up -------------------------------------- json 1.031k i/100ms Calculating ------------------------------------- json 10.812k (± 1.3%) i/s (92.49 μs/i) - 108.255k in 10.014260s == Encoding citm_catalog.json (500298 bytes) jruby 9.4.12.0 (3.1.4) 2025-02-11 https://github.com/ruby/json/commit/f4ab75096a Java HotSpot(TM) 64-Bit Server VM 21.0.7+8-LTS-245 on 21.0.7+8-LTS-245 +jit [arm64-darwin] Warming up -------------------------------------- json 82.000 i/100ms Calculating ------------------------------------- json 824.921 (± 1.0%) i/s (1.21 ms/i) - 8.282k in 10.040787s == Encoding twitter.json (466906 bytes) jruby 9.4.12.0 (3.1.4) 2025-02-11 https://github.com/ruby/json/commit/f4ab75096a Java HotSpot(TM) 64-Bit Server VM 21.0.7+8-LTS-245 on 21.0.7+8-LTS-245 +jit [arm64-darwin] Warming up -------------------------------------- json 141.000 i/100ms Calculating ------------------------------------- json 1.421k (± 0.7%) i/s (703.85 μs/i) - 14.241k in 10.023979s == Encoding ohai.json (20147 bytes) jruby 9.4.12.0 (3.1.4) 2025-02-11 https://github.com/ruby/json/commit/f4ab75096a Java HotSpot(TM) 64-Bit Server VM 21.0.7+8-LTS-245 on 21.0.7+8-LTS-245 +jit [arm64-darwin] Warming up -------------------------------------- json 2.274k i/100ms Calculating ------------------------------------- json 22.612k (± 0.9%) i/s (44.22 μs/i) - 227.400k in 10.057516s ``` Observations `activitypub.json` and `twitter.json` seem to be consistently faster with the Vector API enabled. `citm_catalog.json` seems consistently a bit slower and `ohai.json` is fairly close to even. https://github.com/ruby/json/commit/d40b2703a8
2025-10-25[ruby/json] Fix concurrent usage of JSON::Coder#dumpJean Boussier
Fix: https://github.com/rails/rails/commit/90616277e3d8fc46c9cf35d6a7470ff1ea0092f7#r168784389 Because the `depth` counter is inside `JSON::State` it can't be used concurrently, and in case of a circular reference the counter may be left at the max value. The depth counter should be moved outside `JSON_Generator_State` and into `struct generate_json_data`, but it's a larger refactor. In the meantime, `JSON::Coder` calls `State#generate_new` so I changed that method so that it first copy the state on the stack. https://github.com/ruby/json/commit/aefa671eca
2025-10-06[ruby/json] Fix sliced string escapingtompng
https://github.com/ruby/json/commit/d7baf015d9
2025-09-19`JSON::Coder` callback now recieve a second argument to mark object keysJean Boussier
e.g. ```ruby { 1 => 2 } ``` The callback will be invoked for `1` as while it has a native JSON equivalent, it's not legal as an object name.
2025-09-19[ruby/json] Avoid scientific notation before exponent 15Jean Boussier
Fix: https://github.com/ruby/json/issues/861 It's not incorrect to use scientific notation, but it tend to throw people off a bit, so it's best to keep it for very large numbers. https://github.com/ruby/json/commit/1566cd01a6
2025-09-19[ruby/json] fix issue reading off the end of the ByteBuffer if ptr > 0Scott Myron
Fix: https://github.com/ruby/json/issues/859 https://github.com/ruby/json/commit/67ebabec75 Co-Authored-By: Jean Boussier <jean.boussier@gmail.com>
2025-09-19[ruby/json] Only enable test coverage when running the test suite standaloneJean Boussier
https://github.com/ruby/json/commit/08b9eb0ee6
2025-09-19[ruby/json] Add branch test coverage when available. Force track all files ↵Robin Miller
to prevent implicit file discovery missing files. https://github.com/ruby/json/commit/6bded942c4
2025-09-19[ruby/json] parser: Reject invalid surogate pairs more consistently.Jean Boussier
https://github.com/ruby/json/commit/5855f4f603
2025-09-17Revert "Handle `uninitialized constant JSON::GenericObject` at ruby/ruby."Hiroshi SHIBATA
This reverts commit 0dc1cd407e7775610f2bcaef6c1282369867f91c. 1213adfe5526d65cce81a9fb127074130c8faea7 is fixed this issue.
2025-09-16[ruby/json] Better handle missing ostructÉtienne Barrié
In the Ruby test suite, this test class is causing trouble because ostruct is not available. Having an autoload for JSON::GenericObject but causing it not to define the constant causes a warning. See https://github.com/ruby/json/commit/0dc1cd407e77 and https://github.com/ruby/json/commit/caa5d8cdd748 in ruby. We can skip defining the test class entirely instead when ostruct is not available. https://github.com/ruby/json/commit/6f6a4cdfd7
2025-09-12[ruby/json] Enable coverage before loading simplecovJean Boussier
Fix: https://github.com/ruby/json/pull/853 Simplecov end up requiring json so we need to start collecting coverage before. https://github.com/ruby/json/commit/ca72019fd3
2025-09-11Handle `uninitialized constant JSON::GenericObject` at ruby/ruby.Hiroshi SHIBATA
``` 1) Error: JSONGenericObjectTest#test_from_hash: Test::Unit::ProxyError: uninitialized constant JSON::GenericObject /Users/hsbt/Documents/github.com/ruby/ruby/.ext/common/json/common.rb:1006:in 'JSON.const_missing' /Users/hsbt/Documents/github.com/ruby/ruby/test/json/json_generic_object_test.rb:8:in 'JSONGenericObjectTest#setup' ```
2025-09-11[ruby/json] Improve test coverageÉtienne Barrié
Test SubArrayWrapper#[] and SubOpenStruct#[] https://github.com/ruby/json/commit/bae760aa19 Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2025-09-11[ruby/json] Stop checking for String#@-Étienne Barrié
https://github.com/ruby/json/commit/a9e05d6ff3 Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2025-09-11[ruby/json] Change invalid encoding test to use assert_raiseÉtienne Barrié
https://github.com/ruby/json/commit/efc61682ee Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2025-09-11[ruby/json] Fix deprecation warning backtraceÉtienne Barrié
The gem root was including the test/ directory, so lines from tests were skipped, and the deprecation warnings were shown as coming from the test framework. https://github.com/ruby/json/commit/2ec31a7363 Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2025-09-11[ruby/json] Fix JSON::GenericObject testsÉtienne Barrié
https://github.com/ruby/json/commit/7a2a24d663 Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2025-09-11[ruby/json] Enable test coverage when simplecov is availableÉtienne Barrié
https://github.com/ruby/json/commit/f37e73cf44 Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2025-09-03Use Time.new because JSON.unsafe_load losts fractions of a second andHiroshi SHIBATA
JSON.unsafe_load with Time object returns String ``` 1) Failure: JSONCommonInterfaceTest#test_unsafe_load_default_options [/path/to/ruby/test/json/json_common_interface_test.rb:230]: <2025-09-03 14:28:31.293635 +0200> expected but was <"2025-09-03 14:28:31 +0200">. ```
2025-09-03[ruby/json] Added testing for JSON.unsafe_load. Fixes NoMethodErrorRobin Miller
when passing proc to JSON.unsafe_load, matching the changes made in https://github.com/ruby/json/commit/73d2137fd3ad. https://github.com/ruby/json/commit/77292cbc9b
2025-09-03[ruby/json] Ensure the SWAR encoder in the java extension checks every byte.Scott Myron
https://github.com/ruby/json/commit/9ebe105144
2025-08-27JSON.generate: warn or raise on duplicated keyJean Boussier
Because both strings and symbols keys are serialized the same, it always has been possible to generate documents with duplicated keys: ```ruby >> puts JSON.generate({ foo: 1, "foo" => 2 }) {"foo":1,"foo":2} ``` This is pretty much always a mistake and can cause various issues because it's not guaranteed how various JSON parsers will handle this. Until now I didn't think it was possible to catch such case without tanking performance, hence why I only made the parser more strict. But I finally found a way to check for duplicated keys cheaply enough.
2025-08-27Fix `JSON::Coder` to cast non-string keys.Jean Boussier
2025-08-27Fix `JSON.generate` `strict: true` mode to also restrict hash keysJean Boussier
generated by cgit v1.2.3 (git 2.25.1) at 2026-01-15 23:08:30 +0000