Age | Commit message (Collapse) | Author |
|
multithreaded use
I'm not sure whether this handles all multithreaded use cases,
but this handles the example that crashes almost immediately
and does 10,000,000 total deflates using 100 separate threads.
To prevent the tests from taking forever, the committed test
for this uses only 10,000 deflates across 10 separate threads,
which still causes a segfault in the previous implementation
almost immediately.
Fixes [Bug #17803]
https://github.com/ruby/zlib/commit/4b1023b3f2
|
|
Seems like the test `TestZlibGzipFile#test_gzip_reader_zcat` fails when
the timestamp has `\n\n`.
https://ci.appveyor.com/project/ruby/ruby/builds/38597932
```
1) Error:
TestZlibGzipFile#test_gzip_reader_zcat:
Zlib::DataError: invalid distance too far back
C:/projects/ruby/test/zlib/test_zlib.rb:522:in `initialize'
C:/projects/ruby/test/zlib/test_zlib.rb:522:in `new'
C:/projects/ruby/test/zlib/test_zlib.rb:522:in `zcat'
C:/projects/ruby/test/zlib/test_zlib.rb:522:in `block (2 levels) in test_gzip_reader_zcat'
C:/projects/ruby/test/zlib/test_zlib.rb:521:in `open'
C:/projects/ruby/test/zlib/test_zlib.rb:521:in `block in test_gzip_reader_zcat'
C:/projects/ruby/lib/tempfile.rb:358:in `create'
C:/projects/ruby/test/zlib/test_zlib.rb:510:in `test_gzip_reader_zcat'
```
The test time is around 2021-04-08 04:40 +0900. Maybe the following time
should trigger the bug.
```
irb(main):001:0> Time.at(1617824266)
=> 2021-04-08 04:37:46 +0900
irb(main):002:0> [1617824266].pack("V")
=> "\n\nn`"
```
|
|
The platform defines a constant File::TMPFILE, but it seems unavailable
(maybe depending on the file system?).
http://rubyci.s3.amazonaws.com/scw-9d6766/ruby-master/log/20210322T171707Z.fail.html.gz
This change adds some rescue cluases to the test, copied from
test/ruby/test_file.rb.
|
|
follow-up: commit cf831f49189c4a890da6845e39199a5dfaf4fb48
|
|
|
|
https://github.com/ruby/zlib/commit/7d3fc1f325
|
|
|
|
|
|
If a buffer keyword argument is given, it is used as the buffer,
instead of creating new strings. This can result in significantly
lower memory usage during inflation.
Implements #19
https://github.com/ruby/zlib/commit/dac9a9b57d
|
|
This reads from the IO in 8192 byte chunks, so you don't need to
have the entire string in memory.
Fixes #16
https://github.com/ruby/zlib/commit/ba9793c550
|
|
gz file
Most gzip tools support concatenated gz streams in a gz file. This
offers a way to handle such gz files in Ruby.
Fixes [Bug #9790]
Fixes [Bug #11180]
Fixes [Bug #14804]
https://github.com/ruby/zlib/commit/e2ce56de7d
|
|
Before this change, it was not possible to write out zero for the
timestamp part of a Gzip file's header, as calling GzipWriter#mtime with
zero was ignored.
Judging from the docs for `GzipWriter#mtime=`, it should be possible to
indicate that no timestamp is available by calling the method with zero.
https://github.com/ruby/zlib/commit/310be39cac
|
|
Ruby 2.7 deprecates taint and it no longer has an effect.
The lack of taint support should not cause a problem in
previous Ruby versions.
https://github.com/ruby/zlib/commit/21711ed0ce
|
|
|
|
Apparently, a component of Rails implements a buffering .write
method which keeps the String buffer around and makes it unsafe
for us to clear it after calling .write.
This caused Rack::Deflater to give empty results when enabled.
Fortunately, per r61631 / a55abcc0ca6f628fc05304f81e5a044d65ab4a68,
this misguided optimization was only worth a small (0.5MB) savings
and we still benefit from the majority of the memory savings in
that change.
Thanks to zunda for the bug report.
[ruby-core:90133] [Bug #15356]
Fixes: r61631 (commit a55abcc0ca6f628fc05304f81e5a044d65ab4a68)
("zlib: reduce garbage on gzip writes (deflate)")
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66268 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63171 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
* ext/zlib/zlib.c (rb_gzwriter_write): accepts multiple arguments.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60379 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
* ext/zlib/zlib.c (zlib_gunzip): clear zstream to fix memory leak.
[ruby-core:83162] [Bug #13982]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60130 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
of file [Bug #13616]
patched by Andrew Haines <andrew@haines.org.nz> [ruby-core:81488]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59333 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
* ext/zlib/zlib.c (zstream): manage capacity and size of `buf`
instead of size and separated member `buf_filled`. reported by
Christian Jalio (jalio) at https://hackerone.com/reports/211958
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58526 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57564 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
StringIO.new makes the buffer IO.default_external, while
StringIO.new("".dup) makes source encoding which is defaulted to
UTF-8.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57328 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57323 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57127 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57126 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
* ext/zlib/zlib.c (zstream_discard_input): replace with unread
portion substring, not modifying the input buffer directly.
[ruby-core:78567] [Bug #13021]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57042 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
Encode and Decode gzip data without creating files.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57035 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
Skip two tests on AIX because zconf.h in zlib does not correctly
recognize _LARGE_FILES in AIX. The problem was already reported
to zlib, and skip these tests until it is fixed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54002 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
When you change this to true, you may need to add more tests.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53141 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50727 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
define GzipReader#external_encoding.
[Bug #10900]
* test/zlib/test_zlib.rb: test for above.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50460 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
close for consistent to IO#close.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49893 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
[ruby-core:67444] [Feature #10718]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49260 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
* ext/zlib/zlib.c (gzfile_reset): preserve ZSTREAM_FLAG_GZFILE
[Bug #10101]
* test/zlib/test_zlib.rb (test_rewind): test each_byte
We must preserve the ZSTREAM_FLAG_GZFILE flag to prevent
zstream_detach_buffer from:
a) returning Qnil and breaking out of the `each_byte' loop
b) yielding a large string to each_byte
Note: the test case in bug report takes a long time. I found this
bug because I noticed the massive time descrepancy between
`each_byte' and `readbyte' loop before this patch. With this patch,
`each_byte' and `readbyte' both take very long.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47327 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45214 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
* test/socket/test_nonblock.rb: ditto.
* test/socket/test_socket.rb: ditto.
* test/socket/test_unix.rb: ditto.
* test/testunit/test_parallel.rb: ditto.
* test/webrick/test_filehandler.rb: ditto.
* test/xmlrpc/test_features.rb: ditto.
* test/zlib/test_zlib.rb: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45034 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
need a dictionary but are being decompressed by Zlib::Inflate.inflate
(which has no option to set a dictionary). Now Zlib::NeedDict is
raised instead of crashing. [ruby-trunk - Bug #8829]
* test/zlib/test_zlib.rb (TestZlibInflate): Test for the above.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@42720 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
* ext/zlib/zlib.c (gzfile_read, gzfile_read_all, gzfile_getc),
(gzreader_gets): check EOF. [ruby-core:55220] [Bug #8467]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@41014 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
test/mkmf/test_have_macro.rb, test/net/http/test_http.rb,
test/openssl/test_config.rb, test/psych/test_encoding.rb,
test/psych/test_exception.rb, test/psych/test_psych.rb,
test/psych/test_tainted.rb, test/readline/test_readline.rb,
test/rexml/test_contrib.rb, test/ruby/test_autoload.rb,
test/ruby/test_beginendblock.rb, test/ruby/test_exception.rb,
test/ruby/test_file.rb, test/ruby/test_io.rb,
test/ruby/test_marshal.rb, test/ruby/test_process.rb,
test/ruby/test_require.rb, test/ruby/test_rubyoptions.rb,
test/syslog/test_syslog_logger.rb, test/webrick/test_httpauth.rb,
test/zlib/test_zlib.rb: Use Tempfile.create.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40400 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
* test/ruby/test_rubyoptions.rb: Ditto.
* test/logger/test_logger.rb: Ditto.
* test/psych/test_psych.rb: Ditto.
* test/readline/test_readline.rb: Ditto.
* test/syslog/test_syslog_logger.rb: Ditto.
* test/webrick/test_httpauth.rb: Ditto.
* test/zlib/test_zlib.rb: Ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@39899 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
This allows zlib streams to be processed without huge memory growth.
[Feature #6612]
* NEWS: ditto
* ext/zlib/zlib.c (zstream_expand_buffer): Uses rb_yield when a block
is given for streaming support. Refactored to use
zstream_expand_buffer_into to remove duplicate code.
* ext/zlib/zlib.c (zstream_expand_buffer_protect): Added wrapper
function to pass jump state back through GVL-free section to allow
zstream clean-up before terminating the ruby call.
* ext/zlib/zlib.c (zstream_expand_buffer_without_gvl): Acquire GVL to
yield processed chunk of output stream.
* ext/zlib/zlib.c (zstream_detach_buffer): When a block is given,
returns Qnil mid-stream and yields the output buffer at the end of
the stream.
* test/zlib/test_zlib.rb: Updated tests
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36356 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
processing. rb_block_given_p() is not callable without the GVL.
* ext/zlib/extconf.rb: ditto
* NEWS: ditto
* test/zlib/test_zlib.rb: ditto
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36354 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
This allows zlib streams to be processed without huge memory growth.
[Feature #6612]
* NEWS: ditto
* ext/zlib/zlib.c (zstream_expand_buffer): Uses rb_yield when a block
is given for streaming support. Refactored to use
zstream_expand_buffer_into to remove duplicate code.
* ext/zlib/zlib.c (zstream_expand_buffer_protect): Added wrapper
function to pass jump state back through GVL-free section to allow
zstream clean-up before terminating the ruby call.
* ext/zlib/zlib.c (zstream_expand_buffer_without_gvl): Acquire GVL to
yield processed chunk of output stream.
* ext/zlib/zlib.c (zstream_detach_buffer): When a block is given,
returns Qnil mid-stream and yields the output buffer at the end of
the stream.
* ext/zlib/extconf.rb: Update INCFLAGS to find internal.h for
rb_thread_call_with_gvl
* test/zlib/test_zlib.rb: Updated tests
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36349 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
inflating incomplete zlib streams.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36295 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
patched by headius (Charles Nutter). [ruby-core:44859] [Bug #6398]
* test/zlib/test_zlib.rb (test_deflate): add a test for Zlib.deflate.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@35540 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
encoding testcases for GzipReader#read. read() emits
Encoding.default_external in contrast to read(size) emits BINARY.
See also: http://bugs.jruby.org/6208
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@34790 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
Zlib::Inflate#add_dictionary to allow users to pre-specify
for using during #inflate. [ruby-trunk - Feature #5937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@34553 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
exists. Allows Zlib::Inflate#set_dictionary to work.
[ruby-trunk - Bug #5929]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@34552 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
binmode explicitly for fixing test error on Windows. This is consistent
with r34243.
[ruby-dev:45149] [Bug #5812]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@34387 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
binmode explicitly.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@34243 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|