<feed xmlns='http://www.w3.org/2005/Atom'>
<title>ruby.git/lib/open-uri.rb, branch v3_0_4</title>
<subtitle>The Ruby Programming Language</subtitle>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/'/>
<entry>
<title>Remove the deprecated override of Kernel#open in open-uri</title>
<updated>2020-08-16T14:41:40+00:00</updated>
<author>
<name>Jeremy Evans</name>
<email>code@jeremyevans.net</email>
</author>
<published>2020-07-30T19:48:19+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=a73b5cc556bd131fe924ed6bb02b3c5bdf1593e8'/>
<id>a73b5cc556bd131fe924ed6bb02b3c5bdf1593e8</id>
<content type='text'>
This was deprecated in 2.7 to resolve [Misc #15893].
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This was deprecated in 2.7 to resolve [Misc #15893].
</pre>
</div>
</content>
</entry>
<entry>
<title>[DOC] get rid of parsing as TIDYLINK unintentionally</title>
<updated>2020-04-07T04:59:38+00:00</updated>
<author>
<name>Nobuyoshi Nakada</name>
<email>nobu@ruby-lang.org</email>
</author>
<published>2020-04-07T04:59:38+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=bc646e6715bc0113e800f4d7f4dec7bb34123cc1'/>
<id>bc646e6715bc0113e800f4d7f4dec7bb34123cc1</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Add `URI#open` to warning message</title>
<updated>2019-12-19T16:40:00+00:00</updated>
<author>
<name>Kazuhiro NISHIYAMA</name>
<email>zn@mbf.nifty.com</email>
</author>
<published>2019-12-19T16:40:00+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=3816cd945d68eac7ca8fecbc9d71f878ff3e7b3d'/>
<id>3816cd945d68eac7ca8fecbc9d71f878ff3e7b3d</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>open-uri: change global docs to reflect that URI.open syntax is preferred</title>
<updated>2019-10-26T17:24:20+00:00</updated>
<author>
<name>zverok</name>
<email>zverok.offline@gmail.com</email>
</author>
<published>2019-10-26T08:29:43+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=a24bff461d67e8d5e1f076a9ce773bc876a8b3d4'/>
<id>a24bff461d67e8d5e1f076a9ce773bc876a8b3d4</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Make rb_scan_args handle keywords more similar to Ruby methods (#2460)</title>
<updated>2019-09-25T18:18:49+00:00</updated>
<author>
<name>Jeremy Evans</name>
<email>code@jeremyevans.net</email>
</author>
<published>2019-09-25T18:18:49+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=80b5a0ff2a7709367178f29d4ebe1c54122b1c27'/>
<id>80b5a0ff2a7709367178f29d4ebe1c54122b1c27</id>
<content type='text'>
Cfuncs that use rb_scan_args with the : entry suffer similar keyword
argument separation issues that Ruby methods suffer if the cfuncs
accept optional or variable arguments.

This makes the following changes to : handling.

* Treats as **kw, prompting keyword argument separation warnings
  if called with a positional hash.

* Do not look for an option hash if empty keywords are provided.
  For backwards compatibility, treat an empty keyword splat as a empty
  mandatory positional hash argument, but emit a a warning, as this
  behavior will be removed in Ruby 3.  The argument number check
  needs to be moved lower so it can correctly handle an empty
  positional argument being added.

* If the last argument is nil and it is necessary to treat it as an option
  hash in order to make sure all arguments are processed, continue to
  treat the last argument as the option hash. Emit a warning in this case,
  as this behavior will be removed in Ruby 3.

* If splitting the keyword hash into two hashes, issue a warning, as we
  will not be splitting hashes in Ruby 3.

* If the keyword argument is required to fill a mandatory positional
  argument, continue to do so, but emit a warning as this behavior will
  be going away in Ruby 3.

* If keyword arguments are provided and the last argument is not a hash,
  that indicates something wrong. This can happen if a cfunc is calling
  rb_scan_args multiple times, and providing arguments that were not
  passed to it from Ruby.  Callers need to switch to the new
  rb_scan_args_kw function, which allows passing of whether keywords
  were provided.

This commit fixes all warnings caused by the changes above.

It switches some function calls to *_kw versions with appropriate
kw_splat flags. If delegating arguments, RB_PASS_CALLED_KEYWORDS
is used.  If creating new arguments, RB_PASS_KEYWORDS is used if
the last argument is a hash to be treated as keywords.

In open_key_args in io.c, use rb_scan_args_kw.
In this case, the arguments provided come from another C
function, not Ruby.  The last argument may or may not be a hash,
so we can't set keyword argument mode.  However, if it is a
hash, we don't want to warn when treating it as keywords.

In Ruby files, make sure to appropriately use keyword splats
or literal keywords when calling Cfuncs that now issue keyword
argument separation warnings through rb_scan_args.  Also, make
sure not to pass nil in place of an option hash.

Work around Kernel#warn warnings due to problems in the Rubygems
override of the method.  There is an open pull request to fix
these issues in Rubygems, but part of the Rubygems tests for
their override fail on ruby-head due to rb_scan_args not
recognizing empty keyword splats, which this commit fixes.

Implementation wise, adding rb_scan_args_kw is kind of a pain,
because rb_scan_args takes a variable number of arguments.
In order to not duplicate all the code, the function internals need
to be split into two functions taking a va_list, and to avoid passing
in a ton of arguments, a single struct argument is used to handle
the variables previously local to the function.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Cfuncs that use rb_scan_args with the : entry suffer similar keyword
argument separation issues that Ruby methods suffer if the cfuncs
accept optional or variable arguments.

This makes the following changes to : handling.

* Treats as **kw, prompting keyword argument separation warnings
  if called with a positional hash.

* Do not look for an option hash if empty keywords are provided.
  For backwards compatibility, treat an empty keyword splat as a empty
  mandatory positional hash argument, but emit a a warning, as this
  behavior will be removed in Ruby 3.  The argument number check
  needs to be moved lower so it can correctly handle an empty
  positional argument being added.

* If the last argument is nil and it is necessary to treat it as an option
  hash in order to make sure all arguments are processed, continue to
  treat the last argument as the option hash. Emit a warning in this case,
  as this behavior will be removed in Ruby 3.

* If splitting the keyword hash into two hashes, issue a warning, as we
  will not be splitting hashes in Ruby 3.

* If the keyword argument is required to fill a mandatory positional
  argument, continue to do so, but emit a warning as this behavior will
  be going away in Ruby 3.

* If keyword arguments are provided and the last argument is not a hash,
  that indicates something wrong. This can happen if a cfunc is calling
  rb_scan_args multiple times, and providing arguments that were not
  passed to it from Ruby.  Callers need to switch to the new
  rb_scan_args_kw function, which allows passing of whether keywords
  were provided.

This commit fixes all warnings caused by the changes above.

It switches some function calls to *_kw versions with appropriate
kw_splat flags. If delegating arguments, RB_PASS_CALLED_KEYWORDS
is used.  If creating new arguments, RB_PASS_KEYWORDS is used if
the last argument is a hash to be treated as keywords.

In open_key_args in io.c, use rb_scan_args_kw.
In this case, the arguments provided come from another C
function, not Ruby.  The last argument may or may not be a hash,
so we can't set keyword argument mode.  However, if it is a
hash, we don't want to warn when treating it as keywords.

In Ruby files, make sure to appropriately use keyword splats
or literal keywords when calling Cfuncs that now issue keyword
argument separation warnings through rb_scan_args.  Also, make
sure not to pass nil in place of an option hash.

Work around Kernel#warn warnings due to problems in the Rubygems
override of the method.  There is an open pull request to fix
these issues in Rubygems, but part of the Rubygems tests for
their override fail on ruby-head due to rb_scan_args not
recognizing empty keyword splats, which this commit fixes.

Implementation wise, adding rb_scan_args_kw is kind of a pain,
because rb_scan_args takes a variable number of arguments.
In order to not duplicate all the code, the function internals need
to be split into two functions taking a va_list, and to avoid passing
in a ton of arguments, a single struct argument is used to handle
the variables previously local to the function.</pre>
</div>
</content>
</entry>
<entry>
<title>The default charset of text/* media type is UTF-8.</title>
<updated>2019-07-15T00:36:52+00:00</updated>
<author>
<name>Tanaka Akira</name>
<email>akr@fsij.org</email>
</author>
<published>2019-07-15T00:36:52+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=8f7884761e30c453287d73de6ea733d565635ebc'/>
<id>8f7884761e30c453287d73de6ea733d565635ebc</id>
<content type='text'>
Thanks for the patch  gareth (Gareth Adams).  [Bug #15933]

-------

Combines two small, but very related changes

1: Treat HTTPS the same as HTTP

Previously, OpenURI followed guidance in RFC2616/3.7.1:

&gt; When no explicit charset parameter is provided by the sender, media
&gt; subtypes of the "text" type are defined to have a default charset
&gt; value of "ISO-8859-1" when received via HTTP.

However this RFC was written before TLS was established and OpenURI was
never updated to treat HTTPS traffic the same way. So, HTTPS documents
received a different default to HTTP documents.

This commit removes the scheme check so that all text/* documents
processed by OpenURI are treated the same way.

In theory this processing gets applied to FTP URIs too, but there's no
mechanism in OpenURI for FTP documents to have Content-Type metadata
appended to them, so this ends up being a no-op.

2: Change default charset for text/* to UTF-8

Replaces the default ISO-8859-1 charset previously defined in RFC2616 (now
obsoleted) with a UTF-8 charset as defined in RFC6838.

Fixes: https://bugs.ruby-lang.org/issues/15933
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Thanks for the patch  gareth (Gareth Adams).  [Bug #15933]

-------

Combines two small, but very related changes

1: Treat HTTPS the same as HTTP

Previously, OpenURI followed guidance in RFC2616/3.7.1:

&gt; When no explicit charset parameter is provided by the sender, media
&gt; subtypes of the "text" type are defined to have a default charset
&gt; value of "ISO-8859-1" when received via HTTP.

However this RFC was written before TLS was established and OpenURI was
never updated to treat HTTPS traffic the same way. So, HTTPS documents
received a different default to HTTP documents.

This commit removes the scheme check so that all text/* documents
processed by OpenURI are treated the same way.

In theory this processing gets applied to FTP URIs too, but there's no
mechanism in OpenURI for FTP documents to have Content-Type metadata
appended to them, so this ends up being a no-op.

2: Change default charset for text/* to UTF-8

Replaces the default ISO-8859-1 charset previously defined in RFC2616 (now
obsoleted) with a UTF-8 charset as defined in RFC6838.

Fixes: https://bugs.ruby-lang.org/issues/15933
</pre>
</div>
</content>
</entry>
<entry>
<title>Warn open-uri's "open" method at Kernel.</title>
<updated>2019-07-14T08:18:17+00:00</updated>
<author>
<name>Tanaka Akira</name>
<email>akr@fsij.org</email>
</author>
<published>2019-07-14T08:18:17+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=05aac90a1bcfeb180f5e78ea8b00a4d1b04d5eed'/>
<id>05aac90a1bcfeb180f5e78ea8b00a4d1b04d5eed</id>
<content type='text'>
Use URI.open instead.

Thanks for the patch by jeremyevans0 (Jeremy Evans) [Misc #15893].
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Use URI.open instead.

Thanks for the patch by jeremyevans0 (Jeremy Evans) [Misc #15893].
</pre>
</div>
</content>
</entry>
<entry>
<title>open-uri: clear string after buffering</title>
<updated>2018-01-08T01:11:33+00:00</updated>
<author>
<name>normal</name>
<email>normal@b2dd03c8-39d4-4d8f-98ff-823fe69b080e</email>
</author>
<published>2018-01-08T01:11:33+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=e1dd1fc35cac1984bed4209a3fe2d47256708cf5'/>
<id>e1dd1fc35cac1984bed4209a3fe2d47256708cf5</id>
<content type='text'>
Since r58846 (in Ruby 2.5), it is safe to clear the string
yielded to Net::HTTPResponse#read_body methods.  This
reduces malloc garbage (anonymous RSS) using the Linux-only
script below:

before:  user     system      total        real
      0.030000   0.250000   0.280000 (  0.280511)
    RssAnon:	   60240 kB

 after:  user     system      total        real
      0.050000   0.223333   0.273333 (  0.273118)
    RssAnon:	    6676 kB

------
  # warning this script requires 1G free space for buffering
require 'open-uri'
require 'socket'
require 'benchmark'

s = TCPServer.new('127.0.0.1', 0)
len = 1024 * 1024 * 1024
buf = ((0..255).map(&amp;:chr).join * 128)
nr = len / buf.size
pid = fork do
  c = s.accept
  c.readpartial(16384).clear
  c.write("HTTP/1.1 200 OK\r\n" \
	  "Content-Length: #{len}\r\n" \
          "Content-Type: application/octet-stream\r\n" \
          "\r\n")
  buf.freeze # speeds up IO#write slightly
  nr.times { c.write(buf) }
  c.close
end

addr = s.addr
open("http://#{addr[3]}:#{addr[1]}/", "rb") do |fp|
  bm = Benchmark.measure do
    while fp.read(16384, buf)
    end
  end
  puts bm
end
puts File.readlines("/proc/#$$/status").grep(/RssAnon/)[0]
Process.waitpid2(pid)
------

* lib/open-uri.rb: clear string yielded by Net::HTTPResponse#read_body
  [ruby-core:84662] [Feature #14320]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61664 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Since r58846 (in Ruby 2.5), it is safe to clear the string
yielded to Net::HTTPResponse#read_body methods.  This
reduces malloc garbage (anonymous RSS) using the Linux-only
script below:

before:  user     system      total        real
      0.030000   0.250000   0.280000 (  0.280511)
    RssAnon:	   60240 kB

 after:  user     system      total        real
      0.050000   0.223333   0.273333 (  0.273118)
    RssAnon:	    6676 kB

------
  # warning this script requires 1G free space for buffering
require 'open-uri'
require 'socket'
require 'benchmark'

s = TCPServer.new('127.0.0.1', 0)
len = 1024 * 1024 * 1024
buf = ((0..255).map(&amp;:chr).join * 128)
nr = len / buf.size
pid = fork do
  c = s.accept
  c.readpartial(16384).clear
  c.write("HTTP/1.1 200 OK\r\n" \
	  "Content-Length: #{len}\r\n" \
          "Content-Type: application/octet-stream\r\n" \
          "\r\n")
  buf.freeze # speeds up IO#write slightly
  nr.times { c.write(buf) }
  c.close
end

addr = s.addr
open("http://#{addr[3]}:#{addr[1]}/", "rb") do |fp|
  bm = Benchmark.measure do
    while fp.read(16384, buf)
    end
  end
  puts bm
end
puts File.readlines("/proc/#$$/status").grep(/RssAnon/)[0]
Process.waitpid2(pid)
------

* lib/open-uri.rb: clear string yielded by Net::HTTPResponse#read_body
  [ruby-core:84662] [Feature #14320]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61664 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
</pre>
</div>
</content>
</entry>
<entry>
<title>open-uri defines URI.open defined as an alias.</title>
<updated>2017-12-21T14:15:04+00:00</updated>
<author>
<name>akr</name>
<email>akr@b2dd03c8-39d4-4d8f-98ff-823fe69b080e</email>
</author>
<published>2017-12-21T14:15:04+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=bf287424fd00c0304c836525bb52d89fc1f4a84a'/>
<id>bf287424fd00c0304c836525bb52d89fc1f4a84a</id>
<content type='text'>
open-uri's Kernel.open will be deprecated in future.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61392 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
open-uri's Kernel.open will be deprecated in future.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61392 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
</pre>
</div>
</content>
</entry>
<entry>
<title>lib/open-uri.rb: accept :encoding option as well as encoding in mode string.</title>
<updated>2017-10-21T06:22:53+00:00</updated>
<author>
<name>akr</name>
<email>akr@b2dd03c8-39d4-4d8f-98ff-823fe69b080e</email>
</author>
<published>2017-10-21T06:22:53+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=a2831955b26a07241c8c45ab2affdbafd4a36db2'/>
<id>a2831955b26a07241c8c45ab2affdbafd4a36db2</id>
<content type='text'>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60232 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60232 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
</pre>
</div>
</content>
</entry>
</feed>
