summaryrefslogtreecommitdiff
path: root/lib/open-uri.rb
AgeCommit message (Collapse)Author
2019-07-15The default charset of text/* media type is UTF-8.Tanaka Akira
Thanks for the patch gareth (Gareth Adams). [Bug #15933] ------- Combines two small, but very related changes 1: Treat HTTPS the same as HTTP Previously, OpenURI followed guidance in RFC2616/3.7.1: > When no explicit charset parameter is provided by the sender, media > subtypes of the "text" type are defined to have a default charset > value of "ISO-8859-1" when received via HTTP. However this RFC was written before TLS was established and OpenURI was never updated to treat HTTPS traffic the same way. So, HTTPS documents received a different default to HTTP documents. This commit removes the scheme check so that all text/* documents processed by OpenURI are treated the same way. In theory this processing gets applied to FTP URIs too, but there's no mechanism in OpenURI for FTP documents to have Content-Type metadata appended to them, so this ends up being a no-op. 2: Change default charset for text/* to UTF-8 Replaces the default ISO-8859-1 charset previously defined in RFC2616 (now obsoleted) with a UTF-8 charset as defined in RFC6838. Fixes: https://bugs.ruby-lang.org/issues/15933
2019-07-14Warn open-uri's "open" method at Kernel.Tanaka Akira
Use URI.open instead. Thanks for the patch by jeremyevans0 (Jeremy Evans) [Misc #15893].
2018-01-08open-uri: clear string after bufferingnormal
Since r58846 (in Ruby 2.5), it is safe to clear the string yielded to Net::HTTPResponse#read_body methods. This reduces malloc garbage (anonymous RSS) using the Linux-only script below: before: user system total real 0.030000 0.250000 0.280000 ( 0.280511) RssAnon: 60240 kB after: user system total real 0.050000 0.223333 0.273333 ( 0.273118) RssAnon: 6676 kB ------ # warning this script requires 1G free space for buffering require 'open-uri' require 'socket' require 'benchmark' s = TCPServer.new('127.0.0.1', 0) len = 1024 * 1024 * 1024 buf = ((0..255).map(&:chr).join * 128) nr = len / buf.size pid = fork do c = s.accept c.readpartial(16384).clear c.write("HTTP/1.1 200 OK\r\n" \ "Content-Length: #{len}\r\n" \ "Content-Type: application/octet-stream\r\n" \ "\r\n") buf.freeze # speeds up IO#write slightly nr.times { c.write(buf) } c.close end addr = s.addr open("http://#{addr[3]}:#{addr[1]}/", "rb") do |fp| bm = Benchmark.measure do while fp.read(16384, buf) end end puts bm end puts File.readlines("/proc/#$$/status").grep(/RssAnon/)[0] Process.waitpid2(pid) ------ * lib/open-uri.rb: clear string yielded by Net::HTTPResponse#read_body [ruby-core:84662] [Feature #14320] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61664 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-21open-uri defines URI.open defined as an alias.akr
open-uri's Kernel.open will be deprecated in future. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61392 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21lib/open-uri.rb: accept :encoding option as well as encoding in mode string.akr
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60232 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-10-20* lib/open-uri.rb: Improved documentation grammar forhsbt
open-uri#open option. [Misc #11329][ruby-core:69868][ci skip] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56454 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-09-07lib/open-uri.rb: Allow http to https redirection.akr
* lib/open-uri.rb: Allow http to https redirection. Note that https to http is still forbidden. [ruby-core:20485] [Feature #859] by Roman Shterenzon. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56085 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-03-30* lib/open-uri.rb: Use `userinfo` for authenticated proxy.hsbt
[fix GH-1148] Patch by @SokichiFujita * test/open-uri/test_open-uri.rb: ditto. [fix GH-1309] Patch by @jdamick git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54432 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-01-10stdlib: avoid extra calls to eliminate "\n" from Base64normal
We may use the '0' (zero) to avoid adding the line feed. Furthermore, the '*' (asterisk) modifier is not needed for a single-element arrays. * ext/psych/lib/psych/visitors/yaml_tree.rb (visit_String): eliminate chomp * lib/net/http.rb (connect): eliminate delete * lib/net/http/header.rb (basic_encode): ditto * lib/net/imap.rb (authenticate): eliminate gsub (self.encode_utf7): shorten delete arg * lib/net/smtp.rb (base64_encode): eliminate gsub * lib/open-uri.rb (OpenURI.open_http): eliminate delete * lib/rss/rss.rb: ditto * lib/securerandom.rb (base64): ditto (urlsafe_base64): eliminate delete! * lib/webrick/httpauth/digestauth.rb (split_param_value): eliminate chop * lib/webrick/httpproxy.rb (do_CONNECT): eliminate delete (setup_upstream_proxy_authentication): ditto [ruby-core:72666] [Feature #11938] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53488 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-11-11* lib/net/ftp.rb (initialize): Connections are in passive mode pershugo
default now. The default mode can be changed by Net::FTP.default_passive=. * lib/net/ftp.rb (default_passive=, default_passive): new methods. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52532 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-11-10* lib/open-uri.rb: Remove indicator for "frozen_string_literal: true".akr
* lib/pp.rb: Ditto. * lib/prettyprint.rb: Ditto. * lib/resolv.rb: Ditto. * lib/securerandom.rb: Ditto. * lib/tmpdir.rb: Ditto. * lib/unicode_normalize/tables.rb: Ditto. * test/net/ftp/test_buffered_socket.rb: Ditto. * test/net/ftp/test_mlsx_entry.rb: Ditto. * test/open-uri/test_open-uri.rb: Ditto. * test/open-uri/test_ssl.rb: Ditto. * test/pathname/test_pathname.rb: Ditto. * test/test_pp.rb: Ditto. * test/test_prettyprint.rb: Ditto. * tool/transcode-tblgen.rb: Ditto. * ext/pathname/lib/pathname.rb: Ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52526 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-10-18* lib/open-uri.rb: Specify frozen_string_literal: true.akr
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52186 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-12-24* lib/open-uri.rb (OpenURI.open_http): refactoring of r48941.usa
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48969 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-12-23* lib/open-uri.rb (OpenURI.open_http): accept multiple certs path inusa
ssl_ca_certs. * tool/downloader.rb: use certs of rubygems for downloading gems. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48941 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-10open-uri: accept :open_timeout optionnormal
* lib/open-uri.rb (OpenURI::Options): add :open_timeout default * (def OpenURI.open_http): check :open_timeout option * (module OpenURI): rdoc for :open_timeout * test/open-uri/test_open-uri.rb (test_open_timeout): new test [Feature #10361] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47869 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-08-08* lib/open-uri.rb: remove needless condition for old ruby version.hsbt
* test/open-uri/test_open-uri.rb: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47103 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-05* lib/open-uri.rb (OpenURI.open_uri): Call StringIO#close only ifakr
the StringIO object is not closed yet. Reported by Jordi Massaguer Pla. [ruby-core:42538] [Bug #6010] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45835 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-02-07* lib/open-uri.rb: [DOC] use lower case version of core classes, samezzak
as commit r44878, based on patch by Jonathan Jackson [Bug #9483] [ci skip] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44879 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-01-18* lib/open-uri.rb: Make proxy disabling working again.akr
Fixed by Christophe Philemotte. [ruby-core:59650] [Bug #9385] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44637 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-01-13* lib/net/smtp.rb: [DOC] Remove dead link to RAA by Giorgos Tsiftsiszzak
Fixes the following bugs: [Bug #9152] [Bug #9268] [Bug #9394] * lib/open-uri.rb: ditto git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44585 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-10-26* lib/open-uri.rb (meta_add_field): : Re-implemented.akr
[ruby-core:58017] [Bug #9051] patch by Eamonn Webster. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@43430 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-04-09* lib/open-uri.rb: Support multiple fields with same fieldakr
name (like Set-Cookie). (OpenURI::Meta#metas): New accessor to obtain fields as a Hash from field name (string) to field values (array of strings). [ruby-core:37734] [Bug #4964] reported by ren li. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40207 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-10-20* lib/open-uri.rb: Documentation for OpenURIzzak
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@37273 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-07-23* lib/open-uri.rb: use respond_to? to test Tempfile.akr
[ruby-dev:45995] [Bug #6781] reported by hsbt (Hiroshi SHIBATA). git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36500 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-07-20* lib/net/http.rb: Net::HTTP now automatically detects and usesdrbrain
proxies from the environment. A proxy may also be specified as before. Net::HTTP::Proxy still creates anonymous classes, but these classes are only used to store configuration information. When an HTTP instance is created the configuration is now copied. Additionally, Net::HTTP::ProxyDelta is no longer used by Net::HTTP [Feature #6546] * lib/open-uri.rb: Moved URI::Generic#find_proxy to uri/generic. * lib/uri/generic.rb: Imported find_proxy from open-uri. * test/open-uri/test_open-uri.rb: Moved proxy-discovery tests to URI. * test/uri/test_generic.rb: Imported proxy-discovery tests from open-uri. * test/net/http/test_http.rb: Added tests for proxy behavior. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36476 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-07-17* lib/open-uri.rb: delete space.ayumin
* test/psych/test_alias_and_anchor.rb: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36418 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-07-17* lib/open-uri.rb: call io.close! for Tempfile.akr
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36417 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-08-26* lib/open-uri.rb: Fix indentation of OpenURI::OpenRead#open. Use ++drbrain
instead of `' for method arguments in open-uri.rb git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@33087 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-08-26* ext/pathname/lib/pathname.rb: Fix typos and grammar mistakes. Patchdrbrain
by Luke Gruber. [#5203] * ext/pty/lib/expect.rb: ditto * lib/mathn.rb: ditto * lib/net/http.rb: ditto * lib/open-uri.rb: ditto * lib/ostruct.rb: ditto * lib/tempfile.rb: ditto * lib/thread.rb: ditto * lib/weakref.rb: ditto * sample/webrick/httpproxy.rb: ditto git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@33086 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-05-27update comment.akr
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@31759 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-10-29* lib/resolv-replace.rb: suppress warning.akr
* lib/open-uri.rb: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@29633 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-10-06* lib/uri/generic.rb (URI::Generic#hostname): new method.akr
(URI::Generic#hostname=): ditto. * lib/open-uri.rb: use URI#hostname * lib/net/http.rb: ditto. [ruby-core:32056] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@29416 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-11-05fix a typo.akr
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25662 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-09-13* lib/open-uri.rb (OpenURI::Meta#content_type_parse): strip quotes.akr
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24885 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-09-12* lib/open-uri.rb (URI::FTP#buffer_open): fix the %2F handling.akr
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24875 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-09-12* lib/open-uri.rb (URI::FTP#buffer_open): use the port specified inakr
the URI. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24865 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2009-01-07* lib/open-uri.rb (OpenURI.redirectable?): reverted https redirection.akr
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21381 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-12-27* lib/open-uri.rb (OpenURI.redirectable?): permit https redirection.akr
patch from Roman Shterenzon. [ruby-core:20485] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21087 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-10-05* lib/open-uri.rb: remove debug code introduced by previous change.akr
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19688 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-10-05* lib/open-uri.rb (OpenURI.open_uri): set encoding to strio.naruse
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19687 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-14trailing spaces removed.akr
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19345 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-06-04* lib/rdoc.rb: massive spelling correction patch from Evan Farrarmatz
<evanfarrar at gmail.com> in [ruby-doc:1382] applied. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@16810 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-02-28* lib/open-uri.rb (OpenURI::Meta#meta_setup_encoding): use ASCII-8BITakr
for charset unspecified non-text data. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15633 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-02-28* lib/open-uri.rb (OpenURI::Meta#meta_setup_encoding): setup encodingakr
by charset. (OpenURI::Meta#meta_add_field): call meta_setup_encoding when content-type. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15630 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-02-08* lib/open-uri.rb (OpenURI.open_http): rescue URI::InvalidURIError byakr
URI.parse for location URI. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15406 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-01-23* lib/rexml/text.rb, lib/rubygems/open-uri.rb, lib/open-uri.rb,naruse
test/logger/test_logger.rb, test/ruby/test_regexp.rb: fix tests. [ruby-dev:33336] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15200 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-24* lib/open-uri.rb (OpenURI::Buffer): use Meta ===. [ruby-core:14295]akr
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14609 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-23Fix method redefined warning in open-uri.rb. [ruby-core:14304].drbrain
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14521 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-10-28* lib/open-uri.rb: :redirect option implemented to disable redirects.akr
(OpenURI::HTTPRedirect): new exception class for redirection. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13788 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-10-13* lib/open-uri.rb (OpenURI.open_http): fix :ssl_ca_cert option.akr
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@13691 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-22 00:22:14 +0000