summaryrefslogtreecommitdiff
path: root/ext/json/generator/generator.c
AgeCommit message (Collapse)Author
2025-12-03[ruby/json] Fix handling of depthÉtienne Barrié
https://github.com/ruby/json/commit/ccca602274
2025-12-03[ruby/json] Test and restore behavior around to_json changing depthÉtienne Barrié
When serializing an Array, and one of the elements of the Array requires calling `to_json`, if the depth is changed, it will be used for the next entries, which wasn't the case before https://github.com/ruby/json/commit/5abd43490714, and is not the case with TruffleRuby and JRuby. Additionally, with TruffleRuby and JRuby the state's depth after the `to_json` call is used to close the Array, which isn't the case with CRuby. https://github.com/ruby/json/commit/386b36fde5
2025-12-03[ruby/json] Fix macro argumentsNobuyoshi Nakada
`ALWAYS_INLINE()` and `NOINLINE()` are defined with one argument. https://github.com/ruby/json/commit/8fb727901e
2025-11-27[ruby/json] Respect Coder depth when generatingÉtienne Barrié
https://github.com/ruby/json/commit/9c36681b17
2025-11-26[ruby/json] Don't copy JSON_Generator_State in generate_newÉtienne Barrié
Now that the state isn't mutated in generate_new, we no longer need to copy the struct, we can just use it. https://github.com/ruby/json/commit/d7964f8892
2025-11-26[ruby/json] Don't write depth to JSON_Generator_State in some casesÉtienne Barrié
For `JSON.generate` and `JSON::State#generate_new`, don't copy generate_json_data::depth to JSON_Generator_State::depth. In `JSON.generate`, the JSON_Generator_State is on the stack and discarded anyway. In `JSON::State#generate_new`, we copy the struct to avoid mutating the original one. https://github.com/ruby/json/commit/873b29ea34
2025-11-26[ruby/json] Add depth to struct generate_json_dataÉtienne Barrié
Instead of incrementing JSON_Generator_State::depth, we now increment generate_json_data::depth, and only copied at the end. https://github.com/ruby/json/commit/5abd434907
2025-11-21[ruby/json] Move RUBY_TYPED_FROZEN_SHAREABLE macro to json.hÉtienne Barrié
https://github.com/ruby/json/commit/2a4ebe8250
2025-11-21[ruby/json] Ractor-shareable JSON::CoderÉtienne Barrié
https://github.com/ruby/json/commit/58d60d6b76
2025-11-20[ruby/json] Remove unused symbolsÉtienne Barrié
https://github.com/ruby/json/commit/9364d0c761
2025-11-03[ruby/json] Fix duplicate 'inline' declaration specifierJean Boussier
Followup: https://github.com/ruby/json/pull/889 https://github.com/ruby/json/commit/591510392a
2025-11-03[ruby/json] Centralize macro definitionsJean Boussier
https://github.com/ruby/json/commit/1576ea7d47
2025-11-02[ruby/json] Invoke `as_json` callback for strings with invalid encodingJean Boussier
Fix: https://github.com/ruby/json/issues/873 This allow users to encode binary strings if they so wish. e.g. they can use Base64 or similar, or chose to replace invalid characters with something else. https://github.com/ruby/json/commit/b1b16c416f
2025-10-31[ruby/json] Fix memory leak when exception is raised during JSON generation ↵Kazuki Yamaguchi
part 2 Commit https://github.com/ruby/json/commit/44df509dc2de fixed it for StandardError, but other exceptions and jumps are also possible. Use rb_ensure() to release FBuffer instead of rb_rescue(). A reproducer: o = Object.new def o.to_json(a) = throw :a a = ["make heap allocation"*100, o] 10.times do 100_000.times do catch(:a) { JSON(a) } end puts `ps -o rss= -p #{$$}` end https://github.com/ruby/json/commit/9b7b648ecd
2025-10-30[ruby/json] Fix GeneratorError messages to be UTF-8 encodedJean Boussier
https://github.com/ruby/json/commit/965ba6c5d4
2025-10-25[ruby/json] Fix concurrent usage of JSON::Coder#dumpJean Boussier
Fix: https://github.com/rails/rails/commit/90616277e3d8fc46c9cf35d6a7470ff1ea0092f7#r168784389 Because the `depth` counter is inside `JSON::State` it can't be used concurrently, and in case of a circular reference the counter may be left at the max value. The depth counter should be moved outside `JSON_Generator_State` and into `struct generate_json_data`, but it's a larger refactor. In the meantime, `JSON::Coder` calls `State#generate_new` so I changed that method so that it first copy the state on the stack. https://github.com/ruby/json/commit/aefa671eca
2025-09-19`JSON::Coder` callback now recieve a second argument to mark object keysJean Boussier
e.g. ```ruby { 1 => 2 } ``` The callback will be invoked for `1` as while it has a native JSON equivalent, it's not legal as an object name.
2025-09-19[ruby/json] Avoid scientific notation before exponent 15Jean Boussier
Fix: https://github.com/ruby/json/issues/861 It's not incorrect to use scientific notation, but it tend to throw people off a bit, so it's best to keep it for very large numbers. https://github.com/ruby/json/commit/1566cd01a6
2025-09-03[ruby/json] Fix a -Wreturn-type warningJean Boussier
Fix: https://github.com/ruby/json/pull/843 https://github.com/ruby/json/commit/d3f7f0452b Co-Authored-By: Takashi Kokubun <takashikkbn@gmail.com>
2025-08-27JSON.generate: warn or raise on duplicated keyJean Boussier
Because both strings and symbols keys are serialized the same, it always has been possible to generate documents with duplicated keys: ```ruby >> puts JSON.generate({ foo: 1, "foo" => 2 }) {"foo":1,"foo":2} ``` This is pretty much always a mistake and can cause various issues because it's not guaranteed how various JSON parsers will handle this. Until now I didn't think it was possible to catch such case without tanking performance, hence why I only made the parser more strict. But I finally found a way to check for duplicated keys cheaply enough.
2025-08-27Fix `JSON::Coder` to cast non-string keys.Jean Boussier
2025-08-27Fix `JSON.generate` `strict: true` mode to also restrict hash keysJean Boussier
2025-08-27[ruby/json] Extract `fbuffer_append_str_repeat` functionJean Boussier
https://github.com/ruby/json/commit/12656777dc
2025-08-18Fix typosDouglas Eichelberger
2025-08-04Only define `String.json_create` & al when `json/add` is requiredJean Boussier
All the `json/add` related methods for string were always defined unconditionally from the generators. It's preferable to only define them if `json/add` is actually used.
2025-07-25[ruby/json] Fix missing write barrier on Generator StateJohn Hawthorn
Found by wbcheck WBCHECK ERROR: Missed write barrier detected! Parent object: 0x7b7b8487c450 (wb_protected: true) rb_obj_info_dump: 0x00007b7b8487c450 JSON/Generator/State/JSON::Ext::Generator::State JSON/Generator/State Reference counts - snapshot: 1, writebarrier: 0, current: 6, missed: 5 Missing reference to: 0x7b7b82f35a10 rb_obj_info_dump: 0x00007b7b82f35a10 T_STRING/String len: 1, capa: 15 "1" Missing reference to: 0x7b7b82f35e90 rb_obj_info_dump: 0x00007b7b82f35e90 T_STRING/String len: 1, capa: 15 "2" Missing reference to: 0x7b7b83629e50 rb_obj_info_dump: 0x00007b7b83629e50 T_STRING/String len: 1, capa: 15 "3" Missing reference to: 0x7b7b83b62190 rb_obj_info_dump: 0x00007b7b83b62190 T_STRING/String len: 1, capa: 15 "4" Missing reference to: 0x7b7b83629490 rb_obj_info_dump: 0x00007b7b83629490 T_STRING/String len: 1, capa: 15 "5" https://github.com/ruby/json/commit/c24342d801
2025-07-07[ruby/json] Improve consistency of code styleJean Boussier
https://github.com/ruby/json/commit/a497c71960
2025-06-30Optimize 'json_parse_string' using SIMD.Scott Myron
2025-06-24[ruby/json] Remove trailing spaces [ci skip]Nobuyoshi Nakada
https://github.com/ruby/json/commit/6c41162522
2025-05-26[ruby/json] fbuffer.c: add debug mode with bound checks.Jean Boussier
This would have caught https://github.com/ruby/json/pull/808 on CI. https://github.com/ruby/json/commit/8109421fb4
2025-05-26[ruby/json] Fix: generate_json_float to reserve enough memory for large ↵Jean Boussier
negative floats. Fix: https://github.com/ruby/json/issues/807 Since https://github.com/ruby/json/pull/800, `fpconv_dtoa` can actually generate up to 28 chars. https://github.com/ruby/json/commit/d73ae93d3c
2025-05-01Sync ruby/jsonJean Boussier
Fix: https://github.com/ruby/json/issues/796 Notes: Merged: https://github.com/ruby/ruby/pull/13227
2025-04-30[ruby/json] Fix i686 buildsJean Boussier
We should test compilation with `-msse2` because we need to test with whatever arguments Ruby will be compiled with. https://github.com/ruby/json/commit/0a871365db
2025-04-30[ruby/json] SIMD: Match control char and double quote in one passJean Boussier
`c < 32 || c == 34` is equivalent to `c ^ 2 < 33`. Found in: https://lemire.me/blog/2025/04/13/detect-control-characters-quotes-and-backslashes-efficiently-using-swar/ The gain seem mostly present on micro-benchmark, and even there aren't very consistent, but it's never slower. ``` == Encoding long string (124001 bytes) ruby 3.4.2 (2025-02-15 revision https://github.com/ruby/json/commit/d2930f8e7a) +YJIT +PRISM [arm64-darwin24] Warming up -------------------------------------- after 5.295k i/100ms Calculating ------------------------------------- after 55.796k (± 3.4%) i/s (17.92 μs/i) - 280.635k in 5.035690s Comparison: before: 49840.7 i/s after: 55795.8 i/s - 1.12x faster ``` https://github.com/ruby/json/commit/034c5debd8
2025-04-30[ruby/json] Introduce ARM Neon and SSE2 SIMD.Scott Myron
(https://github.com/ruby/json/pull/743) See the pull request for the long development history: https://github.com/ruby/json/pull/743 ``` == Encoding activitypub.json (52595 bytes) ruby 3.4.2 (2025-02-15 revision https://github.com/ruby/json/commit/d2930f8e7a) +YJIT +PRISM [arm64-darwin24] Warming up -------------------------------------- after 2.913k i/100ms Calculating ------------------------------------- after 29.377k (± 2.0%) i/s (34.04 μs/i) - 148.563k in 5.059169s Comparison: before: 23314.1 i/s after: 29377.3 i/s - 1.26x faster == Encoding citm_catalog.json (500298 bytes) ruby 3.4.2 (2025-02-15 revision https://github.com/ruby/json/commit/d2930f8e7a) +YJIT +PRISM [arm64-darwin24] Warming up -------------------------------------- after 152.000 i/100ms Calculating ------------------------------------- after 1.569k (± 0.8%) i/s (637.49 μs/i) - 7.904k in 5.039001s Comparison: before: 1485.6 i/s after: 1568.7 i/s - 1.06x faster == Encoding twitter.json (466906 bytes) ruby 3.4.2 (2025-02-15 revision https://github.com/ruby/json/commit/d2930f8e7a) +YJIT +PRISM [arm64-darwin24] Warming up -------------------------------------- after 309.000 i/100ms Calculating ------------------------------------- after 3.115k (± 3.1%) i/s (321.01 μs/i) - 15.759k in 5.063776s Comparison: before: 2508.3 i/s after: 3115.2 i/s - 1.24x faster ``` https://github.com/ruby/json/commit/49003523da
2025-04-30[ruby/json] Use RB_TYPE_PJean Boussier
https://github.com/ruby/json/commit/b14250f1da
2025-04-30[ruby/json] Handle non-string keys returning immediate values via `to_s`Jean Boussier
We can't directly call `RBASIC_CLASS` as the return value of `to_s` may be an immediate. https://github.com/ruby/json/commit/12dc394d11
2025-04-30[ruby/json] Stop caching the generator state pointerJean Boussier
Fix: https://github.com/ruby/json/issues/790 If we end up calling something that spills the state on the heap, the pointer we received is outdated and may be out of sync. https://github.com/ruby/json/commit/2ffa4ea46b
2025-03-27Refactor jeaiii-ltoa.hJean Boussier
Some relatively minor change to make the library more in line with the gem. Some renaming, etc. Notes: Merged: https://github.com/ruby/ruby/pull/12994
2025-03-27Faster integer formattingeno
This commit provides an alternative implementation for a long → decimal conversion. The main difference is that it uses an algorithm pulled from https://github.com/jeaiii/itoa. The source there is C++, it was converted by hand to C for inclusion with this gem. jeaiii's algorithm is covered by the MIT License, see source code. On addition this version now also generates the string directly into the fbuffer, foregoing the need to run a separate memory copy. As a result, I see a speedup of 32% on Apple Silicon M1 for an integer set of benchmarks.
2025-03-24Removed trailing spaceHiroshi SHIBATA
2025-03-24Reorganize `fpconv` vendoringJean Boussier
Make it a single file and declare the dependency.
2025-03-24[ruby/json] Adjust fpconv to add ".0" to integerseno
Adds a test case fix https://github.com/ruby/json/commit/fa5bdf87cb
2025-03-24[ruby/json] Faster float formattingeno
This commit provides an alternative implementation for a float → decimal conversion. It integrates a C implementation of Fabian Loitsch's Grisu-algorithm [[pdf]](http://florian.loitsch.com/publications/dtoa-pldi2010.pdf), extracted from https://github.com/night-shift/fpconv. The relevant files are added in this PR, they are, as is all of https://github.com/night-shift/fpconv, available under a MIT License. As a result, I see a speedup of 900% on Apple Silicon M1 for a float set of benchmarks. floats don't have a single correct string representation: a float like `1000.0` can be represented as "1000", "1e3", "1000.0" (and more). The Grisu algorithm converts floating point numbers to an optimal decimal string representation without loss of precision. As a result, a float that is exactly an integer (like `Float(10)`) will be converted by that algorithm into `"10"`. While technically correct – the JSON format treats floats and integers identically –, this differs from the current behaviour of the `"json"` gem. To address this, the integration checks for that case, and explicitely adds a ".0" suffix in those cases. This is sufficient to meet all existing tests; there is, however, a chance that the current implementation and this implementation occasionally encode floats differently. ``` == Encoding floats (4179311 bytes) ruby 3.4.1 (2024-12-25 revision https://github.com/ruby/json/commit/48d4efcb85) +YJIT +PRISM [arm64-darwin24] Warming up -------------------------------------- json (local) 4.000 i/100ms Calculating ------------------------------------- json (local) 46.046 (± 2.2%) i/s (21.72 ms/i) - 232.000 in 5.039611s Normalize to 2090234 byte == Encoding floats (4179242 bytes) ruby 3.4.1 (2024-12-25 revision https://github.com/ruby/json/commit/48d4efcb85) +YJIT +PRISM [arm64-darwin24] Warming up -------------------------------------- json (2.10.2) 1.000 i/100ms Calculating ------------------------------------- json (2.10.2) 4.614 (± 0.0%) i/s (216.74 ms/i) - 24.000 in 5.201871s ``` These benchmarks are run via a script ([link](https://gist.github.com/radiospiel/04019402726a28b31616df3d0c17bd1c)) which is based on the gem's `benchmark/encoder.rb` file. There are probably better ways to run benchmarks :) My version allows to combine multiple test cases into a single one. The `dumps` benchmark, which covers the JSON files in `benchmark/data/*.json` – with the exception of `canada.json` – , reported a minor speedup within statistical uncertainty. https://github.com/ruby/json/commit/7d77415108
2025-02-12Fix a compatibility issue with `MultiJson.dump(obj, pretty: true)`Jean Boussier
Fix: https://github.com/ruby/json/issues/748 `MultiJson` pass `State#to_h` as options, and the `as_json` property defaults to `false` but `false` wasn't accepted by the constructor.
2025-02-06Optimize Symbol generation in strict modeÉtienne Barrié
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2025-02-06Fix JSON::Coder to call as_json proc for NaN and InfinityÉtienne Barrié
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2025-02-03[ruby/json] Refactor further to expose the simpler escape search possibleJean Boussier
https://github.com/ruby/json/commit/e03515ac8b
2025-02-03[ruby/json] Refactor convert_UTF8_to_JSON to split searching and escaping codeJean Boussier
The goal is to be able to dispatch to more optimized search implementations without having to duplicate the escaping code. Somehow, this is a few % faster already: ``` == Encoding activitypub.json (52595 bytes) ruby 3.4.1 (2024-12-25 revision https://github.com/ruby/json/commit/48d4efcb85) +YJIT +PRISM [arm64-darwin23] Warming up -------------------------------------- after 2.257k i/100ms Calculating ------------------------------------- after 22.930k (± 1.3%) i/s (43.61 μs/i) - 115.107k in 5.020814s Comparison: before: 21604.0 i/s after: 22930.1 i/s - 1.06x faster == Encoding citm_catalog.json (500298 bytes) ruby 3.4.1 (2024-12-25 revision https://github.com/ruby/json/commit/48d4efcb85) +YJIT +PRISM [arm64-darwin23] Warming up -------------------------------------- after 137.000 i/100ms Calculating ------------------------------------- after 1.397k (± 1.1%) i/s (715.57 μs/i) - 6.987k in 5.000408s Comparison: before: 1344.4 i/s after: 1397.5 i/s - 1.04x faster == Encoding twitter.json (466906 bytes) ruby 3.4.1 (2024-12-25 revision https://github.com/ruby/json/commit/48d4efcb85) +YJIT +PRISM [arm64-darwin23] Warming up -------------------------------------- after 249.000 i/100ms Calculating ------------------------------------- after 2.464k (± 1.8%) i/s (405.81 μs/i) - 12.450k in 5.054131s Comparison: before: 2326.5 i/s after: 2464.2 i/s - 1.06x faster ``` https://github.com/ruby/json/commit/8fb5ae807f
2025-01-28Introduce JSON::CoderÉtienne Barrié
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
generated by cgit v1.2.3 (git 2.25.1) at 2026-05-13 22:36:23 +0000