Apparently, a component of Rails implements a buffering .write
method which keeps the String buffer around and makes it unsafe
for us to clear it after calling .write.

This caused Rack::Deflater to give empty results when enabled.

Fortunately, per r61631 / a55abcc0ca6f628fc05304f81e5a044d65ab4a68,
this misguided optimization was only worth a small (0.5MB) savings
and we still benefit from the majority of the memory savings in
that change.

Thanks to zunda for the bug report.

[ruby-core:90133] [Bug #15356]

Fixes: r61631 (commit a55abcc0ca6f628fc05304f81e5a044d65ab4a68)
       ("zlib: reduce garbage on gzip writes (deflate)")

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66268 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65141 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

  They are assigned automatically when pushing gem file to rubygems.org.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64213 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

No point in having a long-lived cbuf in "struct gzfile"
since GZFILE_CBUF_CAPA is smaller than RSTRING_EMBED_LEN_MAX
(even on 32-bit).  We can also have rb_econv_convert write
directly to the return value instead of an intermediate buffer.

This brings "struct gzfile" from 264 to 256 bytes on 64-bit
systems to avoid taking an additional cache line.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63993 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Init functions are called only once, cache is useless.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62429 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62046 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

For garbage-concious users who use the `outbuf' argument of
`readpartial' to supply a destination buffer, this provides
a drastic reduction in garbage when inflating large inputs
in a streaming fashion.

This results in a anonymous RSS reduction in the reader
similar to the reduction in the writer from r61631.

Results using the test script from r61631
<https://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision&revision=61631>

Before:

        writer   7.359999   0.000000   7.359999 (  7.360639)
        writer RssAnon:     4040 kB
        reader   6.346667   0.070000   6.416667 (  7.387654)
        reader RssAnon:    98272 kB

After:

        writer   7.309999   0.000000   7.309999 (  7.310651)
        writer RssAnon:	    4048 kB
        reader   6.146666   0.003333   6.149999 (  7.334868)
        reader RssAnon:	    4300 kB

* ext/zlib/zlib.c (struct read_raw_arg): new struct
  (gzfile_read_raw_partial): use read_raw_arg
  (gzfile_read_raw_rescue): ditto
  (gzfile_read_raw): accept outbuf, use read_raw_arg
  (gzfile_read_raw_ensure): accept outbuf
  (gzfile_read_header): ditto
  (gzfile_check_footer): ditto
  (gzfile_read_more): ditto
  (gzfile_read_raw_until_zero): adjust for changes
  (gzfile_fill): ditto
  (gzfile_readpartial): ditto
  (gzfile_read_all): ditto
  (gzfile_getc): ditto
  (gzfile_reader_end_run): ditto
  (gzfile_reader_get_unused): ditto
  (rb_gzreader_initialize): ditto
  (gzreader_skip_linebreaks): ditto
  (gzreader_gets): ditto
  (zlib_gunzip_run): ditto
  [ruby-core:84660] [Feature #14319]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61665 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Zlib::GzipWriter generated large amounts of garbage from
(struct zstream).input.  Reuse the .input field when it is
hidden, and recycle it when its lifetime is over.  This change
alone reduced memory usage of the writer from 90MB to 4.5MB.

For the detached buffer of compressed data used by
gzfile_write_raw, we can only clear the string (not recycle it)
since user code may hold references to it (but the data would be
clobbered, anyways).  This reduced memory usage slightly by
around 0.5MB (because it's smaller compressed data).

Combined, these changes reduce the anonymous RSS memory of a
dedicated writer process from over 90MB to under 4MB.

before:

    #      user     system      total        real

    writer   7.823332   0.053333   7.876665 (  7.881464)
    writer RssAnon:    92944 kB
    reader   6.969999   0.076666   7.046665 (  7.906377)
    reader RssAnon:   109820 kB

after:

    writer   7.359999   0.000000   7.359999 (  7.360639)
    writer RssAnon:     4040 kB
    reader   6.346667   0.070000   6.416667 (  7.387654)
    reader RssAnon:    98272 kB

Script used:
-------
require 'zlib'
require 'benchmark'
nr = 16384 * 2

def stats(pfx, bm)
  str = "#{bm}#{File.readlines("/proc/#$$/status").grep(/^RssAnon:/)[0]}"
  puts str.gsub!(/^/m, pfx)
end

rd, wr = IO.pipe
pid = fork do
  buf = ((0..255).map(&:chr).join * 128).freeze
  rd.close
  gzip = Zlib::GzipWriter.new(wr)
  bm = Benchmark.measure do
    nr.times { gzip.write(buf) }
    gzip.close
    wr.close
  end
  stats('writer ', bm)
end

wr.close
buf = ''
gunzip = Zlib::GzipReader.new(rd)
n = 0
bm = Benchmark.measure do
  begin
    gunzip.readpartial(16384, buf)
    n += buf.size
  rescue EOFError
    break
  end while true
end
stats('reader ', bm)
Process.waitall
-------
* ext/zlib/zlib.c (zstream_discard_input): reuse or recycle hidden input
  (zstream_reset_input): clear hidden input
  (zstream_run): detach input and recycle after use
  (gzfile_write_raw): clear buffer after write
  [ruby-core:84638] [Feature #14315]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61631 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

No need to reveal strings freshly created with rb_str_new.

* ext/zlib/zlib.c (zstream_detach_input): remove redundant call

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61612 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61115 b2dd03c8-39d4-4d8f-98ff-823fe69b080e