summaryrefslogtreecommitdiff
path: root/test/json/json_generator_test.rb
AgeCommit message (Collapse)Author
7 days[ruby/json] Use __builtin_memcpy, if available, to copy overlapping byte ↵Scott Myron
ranges in copy_remaining_bytes to avoid a branch to MEMCPY. Additionally use a space as padding byte instead of an 'X' so it can be represented diretly on AArch64 with a single instruction. https://github.com/ruby/json/commit/643ee11fed
2025-12-03[ruby/json] Fix handling of depthÉtienne Barrié
https://github.com/ruby/json/commit/ccca602274
2025-12-03[ruby/json] Reproduce C ext behavior of ignoring mutated depth in arraysÉtienne Barrié
https://github.com/ruby/json/commit/e0257b9f82
2025-12-03[ruby/json] Test and restore behavior around to_json changing depthÉtienne Barrié
When serializing an Array, and one of the elements of the Array requires calling `to_json`, if the depth is changed, it will be used for the next entries, which wasn't the case before https://github.com/ruby/json/commit/5abd43490714, and is not the case with TruffleRuby and JRuby. Additionally, with TruffleRuby and JRuby the state's depth after the `to_json` call is used to close the Array, which isn't the case with CRuby. https://github.com/ruby/json/commit/386b36fde5
2025-12-02[ruby/json] Don't call to_json on the return value of as_json for Float::NANÉtienne Barrié
https://github.com/ruby/json/commit/28c57df8f7
2025-11-26[ruby/json] Test to_json using State#depthÉtienne Barrié
https://github.com/ruby/json/commit/ac0a980668
2025-11-26[ruby/json] Test depthÉtienne Barrié
https://github.com/ruby/json/commit/d02e40324a
2025-11-26[ruby/json] Test that depth of unfrozen State does not changeÉtienne Barrié
https://github.com/ruby/json/commit/9d32cf4618
2025-11-21[ruby/json] Ractor-shareable JSON::CoderÉtienne Barrié
https://github.com/ruby/json/commit/58d60d6b76
2025-11-07[ruby/json] Deprecate `JSON::State#[]` and `JSON::State#[]=`Jean Boussier
This prevent from freezing and sharing state instances. If you needs some sort of arguments or extra state to the generator methods, consider using `JSON::Coder` instead. https://github.com/ruby/json/commit/e9fbc8937f
2025-09-19`JSON::Coder` callback now recieve a second argument to mark object keysJean Boussier
e.g. ```ruby { 1 => 2 } ``` The callback will be invoked for `1` as while it has a native JSON equivalent, it's not legal as an object name.
2025-09-19[ruby/json] Avoid scientific notation before exponent 15Jean Boussier
Fix: https://github.com/ruby/json/issues/861 It's not incorrect to use scientific notation, but it tend to throw people off a bit, so it's best to keep it for very large numbers. https://github.com/ruby/json/commit/1566cd01a6
2025-09-03[ruby/json] Ensure the SWAR encoder in the java extension checks every byte.Scott Myron
https://github.com/ruby/json/commit/9ebe105144
2025-08-27JSON.generate: warn or raise on duplicated keyJean Boussier
Because both strings and symbols keys are serialized the same, it always has been possible to generate documents with duplicated keys: ```ruby >> puts JSON.generate({ foo: 1, "foo" => 2 }) {"foo":1,"foo":2} ``` This is pretty much always a mistake and can cause various issues because it's not guaranteed how various JSON parsers will handle this. Until now I didn't think it was possible to catch such case without tanking performance, hence why I only made the parser more strict. But I finally found a way to check for duplicated keys cheaply enough.
2025-08-27Fix `JSON.generate` `strict: true` mode to also restrict hash keysJean Boussier
2025-05-26[ruby/json] Fix: generate_json_float to reserve enough memory for large ↵Jean Boussier
negative floats. Fix: https://github.com/ruby/json/issues/807 Since https://github.com/ruby/json/pull/800, `fpconv_dtoa` can actually generate up to 28 chars. https://github.com/ruby/json/commit/d73ae93d3c
2025-05-19[ruby/json] fix for pretty_generate throwing wrong number of arguments errorCody Horton
https://github.com/ruby/json/commit/8433571dcf
2025-05-12[ruby/json] Favor decimal notation over scientific notation for floatsJean Boussier
e.g. ``` JSON.dump(1746861937.7842371) ``` master: ``` "1.https://github.com/ruby/json/commit/746861937784+9" ``` This branch and older json versions: ``` https://github.com/ruby/json/commit/1746861937.7842371 ``` In the end it's shorter, and according to `canada.json` benchmark performance is the same. https://github.com/ruby/json/commit/866f72a437
2025-04-30[ruby/json] Introduce ARM Neon and SSE2 SIMD.Scott Myron
(https://github.com/ruby/json/pull/743) See the pull request for the long development history: https://github.com/ruby/json/pull/743 ``` == Encoding activitypub.json (52595 bytes) ruby 3.4.2 (2025-02-15 revision https://github.com/ruby/json/commit/d2930f8e7a) +YJIT +PRISM [arm64-darwin24] Warming up -------------------------------------- after 2.913k i/100ms Calculating ------------------------------------- after 29.377k (± 2.0%) i/s (34.04 μs/i) - 148.563k in 5.059169s Comparison: before: 23314.1 i/s after: 29377.3 i/s - 1.26x faster == Encoding citm_catalog.json (500298 bytes) ruby 3.4.2 (2025-02-15 revision https://github.com/ruby/json/commit/d2930f8e7a) +YJIT +PRISM [arm64-darwin24] Warming up -------------------------------------- after 152.000 i/100ms Calculating ------------------------------------- after 1.569k (± 0.8%) i/s (637.49 μs/i) - 7.904k in 5.039001s Comparison: before: 1485.6 i/s after: 1568.7 i/s - 1.06x faster == Encoding twitter.json (466906 bytes) ruby 3.4.2 (2025-02-15 revision https://github.com/ruby/json/commit/d2930f8e7a) +YJIT +PRISM [arm64-darwin24] Warming up -------------------------------------- after 309.000 i/100ms Calculating ------------------------------------- after 3.115k (± 3.1%) i/s (321.01 μs/i) - 15.759k in 5.063776s Comparison: before: 2508.3 i/s after: 3115.2 i/s - 1.24x faster ``` https://github.com/ruby/json/commit/49003523da
2025-04-30[ruby/json] Handle non-string keys returning immediate values via `to_s`Jean Boussier
We can't directly call `RBASIC_CLASS` as the return value of `to_s` may be an immediate. https://github.com/ruby/json/commit/12dc394d11
2025-03-28[ruby/json] Add missing assert_deprecated_warningJean Boussier
https://github.com/ruby/json/commit/1b16a82980 Notes: Merged: https://github.com/ruby/ruby/pull/13004
2025-03-28[ruby/json] Cleanup JSON.pretty_generateJean Boussier
https://github.com/ruby/json/commit/01c47a0555 Notes: Merged: https://github.com/ruby/ruby/pull/13004
2025-03-28[ruby/json] Deprecate `JSON.fast_generate`Jean Boussier
https://github.com/ruby/json/commit/6508455d82 Notes: Merged: https://github.com/ruby/ruby/pull/13004
2025-03-27Removed trailing spacesHiroshi SHIBATA
2025-03-27Faster integer formattingeno
This commit provides an alternative implementation for a long → decimal conversion. The main difference is that it uses an algorithm pulled from https://github.com/jeaiii/itoa. The source there is C++, it was converted by hand to C for inclusion with this gem. jeaiii's algorithm is covered by the MIT License, see source code. On addition this version now also generates the string directly into the fbuffer, foregoing the need to run a separate memory copy. As a result, I see a speedup of 32% on Apple Silicon M1 for an integer set of benchmarks.
2025-03-24[ruby/json] Adjust fpconv to add ".0" to integerseno
Adds a test case fix https://github.com/ruby/json/commit/fa5bdf87cb
2025-03-06[ruby/json] Fix JSON::GeneratorError#detailed_message with Ruby < 3.2Rahim Packir Saibo
https://github.com/ruby/json/commit/2e015ff839
2025-02-12Fix a compatibility issue with `MultiJson.dump(obj, pretty: true)`Jean Boussier
Fix: https://github.com/ruby/json/issues/748 `MultiJson` pass `State#to_h` as options, and the `as_json` property defaults to `false` but `false` wasn't accepted by the constructor.
2025-02-06Optimize Symbol generation in strict modeÉtienne Barrié
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2025-02-03[ruby/json] Refactor convert_UTF8_to_JSON to split searching and escaping codeJean Boussier
The goal is to be able to dispatch to more optimized search implementations without having to duplicate the escaping code. Somehow, this is a few % faster already: ``` == Encoding activitypub.json (52595 bytes) ruby 3.4.1 (2024-12-25 revision https://github.com/ruby/json/commit/48d4efcb85) +YJIT +PRISM [arm64-darwin23] Warming up -------------------------------------- after 2.257k i/100ms Calculating ------------------------------------- after 22.930k (± 1.3%) i/s (43.61 μs/i) - 115.107k in 5.020814s Comparison: before: 21604.0 i/s after: 22930.1 i/s - 1.06x faster == Encoding citm_catalog.json (500298 bytes) ruby 3.4.1 (2024-12-25 revision https://github.com/ruby/json/commit/48d4efcb85) +YJIT +PRISM [arm64-darwin23] Warming up -------------------------------------- after 137.000 i/100ms Calculating ------------------------------------- after 1.397k (± 1.1%) i/s (715.57 μs/i) - 6.987k in 5.000408s Comparison: before: 1344.4 i/s after: 1397.5 i/s - 1.04x faster == Encoding twitter.json (466906 bytes) ruby 3.4.1 (2024-12-25 revision https://github.com/ruby/json/commit/48d4efcb85) +YJIT +PRISM [arm64-darwin23] Warming up -------------------------------------- after 249.000 i/100ms Calculating ------------------------------------- after 2.464k (± 1.8%) i/s (405.81 μs/i) - 12.450k in 5.054131s Comparison: before: 2326.5 i/s after: 2464.2 i/s - 1.06x faster ``` https://github.com/ruby/json/commit/8fb5ae807f
2025-01-28Allow JSON::Fragment to be used even in strict modeJean Boussier
2025-01-28Introduce JSON::CoderÉtienne Barrié
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2025-01-20[ruby/json] Introduce JSON::FragmentÉtienne Barrié
https://github.com/ruby/json/commit/9e3500f345 Co-authored-by: Jean Boussier <jean.boussier@gmail.com> Notes: Merged: https://github.com/ruby/ruby/pull/12602
2024-12-05[ruby/json] Fix generate(script_safe: true) to not confuse unrelated charactersJean Boussier
Fix: https://github.com/ruby/json/issues/715 The first byte check was missing. https://github.com/ruby/json/commit/93a7f8717d Notes: Merged: https://github.com/ruby/ruby/pull/12267
2024-11-26JSON::GeneratorError expose invalid objectJean Boussier
Fix: https://github.com/ruby/json/issues/710 Makes it easier to debug why a given tree of objects can't be dumped as JSON. Co-Authored-By: Étienne Barrié <etienne.barrie@gmail.com>
2024-11-05[ruby/json] ResyncJean Boussier
Notes: Merged: https://github.com/ruby/ruby/pull/12003
2024-11-05[ruby/json] Add tests for the behavior of JSON.generate with base types ↵Jean Boussier
subclasses Ref: https://github.com/ruby/json/pull/674 Ref: https://github.com/ruby/json/pull/668 The behavior on such case it quite unclear, the goal here is to figure out whatever was the behavior on Cext version of `json 2.7.0` and get all implementations to converge. We can then decide to make them all behave differently if we so wish. https://github.com/ruby/json/commit/614921dcef Notes: Merged: https://github.com/ruby/ruby/pull/12003
2024-11-01JSON.generate: call to_json on String subclassesJean Boussier
Fix: https://github.com/ruby/json/issues/667 This is yet another behavior on which the various implementations differed, but the C implementation used to call `to_json` on String subclasses used as keys. This was optimized out in e125072130229e54a651f7b11d7d5a782ae7fb65 but there is an Active Support test case for it, so it's best to make all 3 implementation respect this behavior.
2024-11-01[ruby/json] Emit warnings when dumping binary stringsJean Boussier
Because of it's Ruby 1.8 heritage, the C extension doesn't care much about strings encoding. We should get stricter over time. https://github.com/ruby/json/commit/42402fc13f
2024-11-01Elide Generator::State allocation until a `to_json` method has to be calledJean Boussier
Fix: https://github.com/ruby/json/issues/655 For very small documents, the biggest performance gap with alternatives is that the API impose that we allocate the `State` object. In a real world app this doesn't make much of a difference, but when running in a micro-benchmark this doubles the allocations, causing twice the amount of GC runs, making us look bad. However, unless we have to call a `to_json` method, the `State` object isn't visible, so with some refactoring, we can elude that allocation entirely. Instead we allocate the State internal struct on the stack, and if we need to call a `to_json` method, we allocate the `State` and spill the struct on the heap. As a result, `JSON.generate` is now as fast as re-using a `State` instance, as long as only primitives are generated. Before: ``` == Encoding small mixed (34 bytes) ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23] Warming up -------------------------------------- json (reuse) 598.654k i/100ms json 400.542k i/100ms oj 533.353k i/100ms Calculating ------------------------------------- json (reuse) 6.371M (± 8.6%) i/s (156.96 ns/i) - 31.729M in 5.059195s json 4.120M (± 6.6%) i/s (242.72 ns/i) - 20.828M in 5.090549s oj 5.622M (± 6.4%) i/s (177.86 ns/i) - 28.268M in 5.061473s Comparison: json (reuse): 6371126.6 i/s oj: 5622452.0 i/s - same-ish: difference falls within error json: 4119991.1 i/s - 1.55x slower == Encoding small nested array (121 bytes) ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23] Warming up -------------------------------------- json (reuse) 248.125k i/100ms json 215.255k i/100ms oj 217.531k i/100ms Calculating ------------------------------------- json (reuse) 2.628M (± 6.1%) i/s (380.55 ns/i) - 13.151M in 5.030281s json 2.185M (± 6.7%) i/s (457.74 ns/i) - 10.978M in 5.057655s oj 2.217M (± 6.7%) i/s (451.10 ns/i) - 11.094M in 5.044844s Comparison: json (reuse): 2627799.4 i/s oj: 2216824.8 i/s - 1.19x slower json: 2184669.5 i/s - 1.20x slower == Encoding small hash (65 bytes) ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23] Warming up -------------------------------------- json (reuse) 641.334k i/100ms json 322.745k i/100ms oj 642.450k i/100ms Calculating ------------------------------------- json (reuse) 7.133M (± 6.5%) i/s (140.19 ns/i) - 35.915M in 5.068201s json 4.615M (± 7.0%) i/s (216.70 ns/i) - 22.915M in 5.003718s oj 6.912M (± 6.4%) i/s (144.68 ns/i) - 34.692M in 5.047690s Comparison: json (reuse): 7133123.3 i/s oj: 6911977.1 i/s - same-ish: difference falls within error json: 4614696.6 i/s - 1.55x slower ``` After: ``` == Encoding small mixed (34 bytes) ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23] Warming up -------------------------------------- json (reuse) 572.751k i/100ms json 457.741k i/100ms oj 512.247k i/100ms Calculating ------------------------------------- json (reuse) 6.324M (± 6.9%) i/s (158.12 ns/i) - 31.501M in 5.023093s json 6.263M (± 6.9%) i/s (159.66 ns/i) - 31.126M in 5.017086s oj 5.569M (± 6.6%) i/s (179.56 ns/i) - 27.661M in 5.003739s Comparison: json (reuse): 6324183.5 i/s json: 6263204.9 i/s - same-ish: difference falls within error oj: 5569049.2 i/s - same-ish: difference falls within error == Encoding small nested array (121 bytes) ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23] Warming up -------------------------------------- json (reuse) 258.505k i/100ms json 242.335k i/100ms oj 220.678k i/100ms Calculating ------------------------------------- json (reuse) 2.589M (± 9.6%) i/s (386.17 ns/i) - 12.925M in 5.071853s json 2.594M (± 6.6%) i/s (385.46 ns/i) - 13.086M in 5.083035s oj 2.250M (± 2.3%) i/s (444.43 ns/i) - 11.255M in 5.004707s Comparison: json (reuse): 2589499.6 i/s json: 2594321.0 i/s - same-ish: difference falls within error oj: 2250064.0 i/s - 1.15x slower == Encoding small hash (65 bytes) ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23] Warming up -------------------------------------- json (reuse) 656.373k i/100ms json 644.135k i/100ms oj 650.283k i/100ms Calculating ------------------------------------- json (reuse) 7.202M (± 7.1%) i/s (138.84 ns/i) - 36.101M in 5.051438s json 7.278M (± 1.7%) i/s (137.40 ns/i) - 36.716M in 5.046300s oj 7.036M (± 1.7%) i/s (142.12 ns/i) - 35.766M in 5.084729s Comparison: json (reuse): 7202447.9 i/s json: 7277883.0 i/s - same-ish: difference falls within error oj: 7036115.2 i/s - same-ish: difference falls within error ```
2024-10-29Handle all formatting configs potentially being `nil`.Jean Boussier
Fix: https://github.com/ruby/json/issues/653 I don't think this was really fully supported in the past, but it kinda worked with some of the implementations.
2024-10-26[ruby/json] Workaround rubygems $LOAD_PATH bugJean Boussier
Ref: https://github.com/ruby/json/issues/647 Ref: https://github.com/rubygems/rubygems/pull/6490 Older rubygems are executing `extconf.rb` with a broken `$LOAD_PATH` causing the `json` gem native extension to be loaded with the stdlib version of the `.rb` files. This fails with ``` json/common.rb:82:in `initialize': wrong number of arguments (given 1, expected 0) (ArgumentError) ``` Since this is just for `extconf.rb` we can probably just accept that extra argument and ignore it. The bug was fixed in rubygems 3.4.9 / 2023-03-20 https://github.com/ruby/json/commit/1f5e849fe0
2024-10-26[ruby/json] Modernize heredocsJean Boussier
https://github.com/ruby/json/commit/fb25e94aea
2024-10-26pretty_generate: don't apply object_nl / array_nl for empty containersJean Boussier
Fix: https://github.com/ruby/json/issues/437 Before: ```json { "foo": { }, "bar": [ ] } ``` After: ```json { "foo": {}, "bar": [] } ```
2024-10-26[ruby/json] JSON.dump / String#to_json: raise on invalid encodingJean Boussier
This regressed since 2.7.2. https://github.com/ruby/json/commit/35407d6635
2024-10-26Raise the correct exception in fast_serialize_stringBenoit Daloze
* Related to https://github.com/ruby/json/issues/344
2024-10-26Use frozen string literalsÉtienne Barrié
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2024-10-26Use Encoding constants, String#bÉtienne Barrié
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2024-10-18[ruby/json] Assume Encoding is definedJean Boussier
https://github.com/ruby/json/commit/8713aa4812
2024-10-16Restore missing test-case from ↵Hiroshi SHIBATA
https://github.com/ruby/json/commit/c5a6d8042752dca312cfb407a6d3e594458d1cd6 Co-authored-by: "Jean Boussier" <byroot@ruby-lang.org>