summaryrefslogtreecommitdiff
path: root/ext/socket
AgeCommit message (Collapse)Author
2024-12-18Wrap `do_fast_fallback_getaddrinfo` with `rb_thread_prevent_fork` (#12366)Misaki Shioi
Wrap `do_fast_fallback_getaddrinfo` with `rb_thread_prevent_fork` Referencing PR #10864, wrap `do_fast_fallback_getaddrinfo` with `rb_thread_prevent_fork` to avoid fork safety issues. `do_fast_fallback_getaddrinfo` internally uses getaddrinfo(3), leading to fork safety issues, as described in PR #10864. This change ensures that `do_fast_fallback_getaddrinfo` is guarded by `rb_thread_prevent_fork`, preventing fork during its execution and avoiding related issues. Notes: Merged-By: shioimm <shioi.mm@gmail.com>
2024-12-16[DOC] Fix broken parameter display and see links for Socketnvh0412
Notes: Merged: https://github.com/ruby/ruby/pull/12350
2024-12-15Fix broken links to `Socket.tcp_fast_fallback=`Stan Lo
Since it's a singleton method, it should be referenced as `Socket.tcp_fast_fallback=`, not `Socket#tcp_fast_fallback=`. Notes: Merged: https://github.com/ruby/ruby/pull/12348
2024-12-14Improve APIs for Globally Enabling/Disabling fast_fallback in Socket (#12257)Misaki Shioi
This change includes the following updates: - Added an environment variable `RUBY_TCP_NO_FAST_FALLBACK` to control enabling/disabling fast_fallback - Updated documentation and man pages - Revised the implementation of Socket.tcp_fast_fallback= and Socket.tcp_fast_fallback, which previously performed dynamic name resolution of constants and variables. As a result, the following performance improvements were achieved: (Case of 1000 executions of `TCPSocket.new` to the local host) Rehearsal ----------------------------------------- before 0.031462 0.147946 0.179408 ( 0.249279) after 0.031164 0.146839 0.178003 ( 0.346935) -------------------------------- total: 0.178003sec user system total real before 0.027584 0.138712 0.166296 ( 0.233356) after 0.025953 0.127608 0.153561 ( 0.237971) Notes: Merged-By: shioimm <shioi.mm@gmail.com>
2024-12-11Use ruby_strdup/xfree in fast_fallbackJohn Hawthorn
Any memory allocated with xmalloc needs to be matched with xfree rather than plain free. Ruby unfortunately redefines strdup to be ruby_strdup, which uses xmalloc so needs to be xfreed. Previously these were mismatched. This commit changes the copy to be an explicit ruby_strdup (to avoid confusion) and the free to be xfree. Notes: Merged: https://github.com/ruby/ruby/pull/12311
2024-12-11Use `rb_thread_fd_select` instead of select(2) (#12292)Misaki Shioi
* Use `rb_thread_fd_select` instead of select(2) For fixing https://bugs.ruby-lang.org/issues/20932 . `TCPSocket.new`, which internally uses select(2) for HEv2, can cause SEGV if the number of file descriptors exceeds `FD_SETSIZE`. This change avoids that issue by replacing select(2) with `rb_thread_fd_select`, which is provided as part of Ruby's internal API. --- This includes the following changes. * rb_thread_fd_select does not need common pipe Notes: Merged-By: shioimm <shioi.mm@gmail.com>
2024-12-10Fix Connection Attempt Delay of `Socket.tcp` (#12291)Misaki Shioi
The following two commits fix the proper clearing of the Connection Attempt Delay in `TCPSocket.new`. - https://github.com/ruby/ruby/pull/12087/commits/b2f610b0edf8f84d9d0fce038f3ebfa9f2cb0223 - https://github.com/ruby/ruby/pull/12223/commits/6f4efaec5352cdeb6cab86036a83f53c9fa5c3d2 The same fix will be applied to `Socket.tcp`. Notes: Merged-By: shioimm <shioi.mm@gmail.com>
2024-12-03Fix use of getaddrinfo_shared->lockJohn Hawthorn
In some locations we were using shared->lock and in others &shared->lock, and we were leaking the allocated memory. Notes: Merged: https://github.com/ruby/ruby/pull/12239
2024-11-30Improve the conditions for clearing the Connection Attempt Delay upon ↵Misaki Shioi
connection failure (#12223) * Improve the conditions for clearing the Connection Attempt Delay upon connection failure This change addresses a case that was overlooked in ruby/ruby#12087. In the previous change, the Connection Attempt Delay was cleared at the point of a connection failure only if both of the following conditions were met: - No other sockets were attempting a connection - There were addresses still available to start a new connection In this update, the second condition has been removed. As a result, if name resolution succeeds after a connection failure and new addresses are obtained, it will be able to immediately attempt a connection to one of them. If there are no sockets attempting a connection, no addresses available for connection, and name resolution has completed, an exception will still be raised as before. --- Additionally, the following minor fixes have been made: * Refactor: Remove unnecessary members Notes: Merged-By: shioimm <shioi.mm@gmail.com>
2024-11-29Ensure to close pipes when `TCPSocket.new` finishes processing (#12181)Misaki Shioi
`TCPSocket.new` with HEv2 uses three threads. The last of these threads to exit closed pipes. However, if pipes were open at the end of the main thread, they would leak. This change avoids this by closing pipes at the end of the main thread. Notes: Merged-By: shioimm <shioi.mm@gmail.com>
2024-11-29Allow disable to `fast_fallback` of `TCPSocket.new` (#12210)Misaki Shioi
with `Socket.tcp_fast_fallback=` The functions that `Socket.tcp` had are now also available in `TCPSocket.new`. Notes: Merged-By: shioimm <shioi.mm@gmail.com>
2024-11-28Ensure to free fast_fallback_getaddrinfo_shared with single family (#12199)Misaki Shioi
With https://github.com/ruby/ruby/pull/12156, the memory of the `struct fast_fallback_getaddrinfo_shared` is now allocated even if there is only one address family. This change will always free it when `TCPSocket.new` finishes. Notes: Merged-By: shioimm <shioi.mm@gmail.com>
2024-11-25Prevent memory leakYusuke Endoh
``` for (int i = 0; i < arg->family_size; i++) { arg->getaddrinfo_entries[i] = allocate_fast_fallback_getaddrinfo_entry(); if (!(arg->getaddrinfo_entries[i])) rb_syserr_fail(errno, "calloc(3)"); ``` If the allocation fails in the second interation, the memory allocated in the first iteration would be leaked. This change prevents the memory leak by allocating the memory in advance. (The struct name `fast_fallback_getaddrinfo_shared` might no longer be good.) Notes: Merged: https://github.com/ruby/ruby/pull/12163
2024-11-25Fix initialization of `struct wait_fast_fallback_arg::cancelled`Nobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/12161
2024-11-25Do not save the last error without sockets in the connection attempt (#12153)Misaki Shioi
* Do not save the last_error if there are no sockets waiting to be connected In this implementation, the results of both name resolution and connection attempts are awaited using select(2). When it returned, the implementation attempted to check for connections even if there were no sockets currently attempting to connect, treating the absence of connected sockets as a connection failure. With this fix, it will no longer check for connections when there are no sockets waiting to be connected. Additionally, the following minor fixes have been made: * Handle failure of getsockopt(2) and removed unnecessary continue in the loop * Tweak: Use common API to check in_progress_fds * Safely call TCPServer.new in test * Set empty writefds when there is no socket waiting to be connected * Enable fast_fallback option Notes: Merged-By: shioimm <shioi.mm@gmail.com>
2024-11-24UBF is also required for synchronous name resolution (#12156)Misaki Shioi
`rb_thread_call_without_gvl2` is used to wait for the results of name resolution and connection attempts. When there is only one address family to resolve, the necessary resources were not being passed to the UBF. With this change, the handling of resources has been revised and organized to work consistently, whether there are two address families to resolve or only one. Notes: Merged-By: shioimm <shioi.mm@gmail.com>
2024-11-23Save the error that occurred during name resolution (#12155)Misaki Shioi
even if a system call error happens after the name resolution failure in the child thread. pipe and write(2) are used to notify the main thread of the name resolution results from the child thread. After name resolution is completed in the child thread, if the call to write(2) fails, the main thread retrieves the resolved addresses. However, when name resolution failed, the corresponding error was not being saved in `last_error`. With this change, name resolution failures will now be saved in last_error even if the write(2) call in the child thread fails. Notes: Merged-By: shioimm <shioi.mm@gmail.com>
2024-11-21Fix the usage of reallocYusuke Endoh
http://ci.rvm.jp/results/trunk-repeat50@ruby-sp2-noble-docker/5420911 ``` /tmp/ruby/src/trunk-repeat50/ext/socket/ipsocket.c: In function ‘reallocate_connection_attempt_fds’: /tmp/ruby/src/trunk-repeat50/ext/socket/ipsocket.c:292:62: warning: pointer ‘fds’ may be used after ‘realloc’ [-Wuse-after-free] 292 | for (int i = current_capacity; i < new_capacity; i++) fds[i] = -1; | ^ /tmp/ruby/src/trunk-repeat50/ext/socket/ipsocket.c:288:9: note: call to ‘realloc’ here 288 | if (realloc(fds, new_capacity * sizeof(int)) == NULL) { | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ```
2024-11-17Fix stack-use-after-return (#12105)Misaki Shioi
http://ci.rvm.jp/results/trunk_asan@ruby-sp1/5409001 ``` ================================================================= ==3263562==ERROR: AddressSanitizer: stack-use-after-return on address 0x735a8f190da8 at pc 0x735a6f58dabc bp 0x735a639ffd10 sp 0x735a639ffd08 READ of size 4 at 0x735a8f190da8 thread T211 ================================================================= ``` Notes: Merged-By: shioimm <shioi.mm@gmail.com>
2024-11-15Make fast_fallback option false by default temporarily (#12070)Misaki Shioi
to suppress failing output in CI. Notes: Merged-By: shioimm <shioi.mm@gmail.com>
2024-11-14Fix a stack-buffer-overflow bugYusuke Endoh
http://ci.rvm.jp/results/trunk_asan@ruby-sp1/5408428 ``` ==3159643==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x796cf8f09041 at pc 0x6539bbf68ded bp 0x796cfadffcf0 sp 0x796cfadff4b8 READ of size 2 at 0x796cf8f09041 thread T13 #0 0x6539bbf68dec in strlen (/tmp/ruby/build/trunk_asan/ruby+0x18edec) (BuildId: cca267c7ae091060e1b82a6b4ed1aeaf00edebab) ``` Notes: Merged: https://github.com/ruby/ruby/pull/12089
2024-11-15Do not wait connection attempt delay without in progress fds (#12087)Misaki Shioi
Do not wait Connection Attempt Delay without in progress fds Reset Connection Attempt Delay when connection fails and there is no other socket connection in progress. This is intended to resolve an issue that was temporarily worked around in Pull Request #12062. `TCPServer::new` (used in tests such as `TestNetHTTP_v1_2_chunked#test_timeout_during_non_chunked_streamed_HTTP_session_write`) can only connect over either IPv6 or IPv4, depending on the environment. Since HEv2 attempts to connect over IPv6 first, environments where IPv6 connections are unavailable return ECONNREFUSED immediately. In such cases, the client should immediately retry the connection over IPv4. However, HEv2 includes a specification for a "Connection Attempt Delay," where it waits 250ms after the previous connection attempt before starting the next one. This delay causes Net::OpenTimeout (100ms) to be exceeded while waiting for the next connection attempt to start. With this change, when a connection attempt fails, if there are sockets still attempting to connect and there are addresses yet to be tried, the Connection Attempt Delay will be resetted, allowing the next connection attempt to start immediately. --- Additionally, the following minor fixes have been made: - The `nfds` value used for select(2) is now reset with each wait. Notes: Merged-By: shioimm <shioi.mm@gmail.com>
2024-11-12[Feature #120782] Introduction of Happy Eyeballs Version 2 (RFC8305) in ↵Misaki Shioi
TCPSocket.new (#11653) * Introduction of Happy Eyeballs Version 2 (RFC8305) in TCPSocket.new This is an implementation of Happy Eyeballs version 2 (RFC 8305) in `TCPSocket.new`. See https://github.com/ruby/ruby/pull/11653 1. Background Prior to this implementation, I implemented Happy Eyeballs Version 2 (HEv2) for `Socket.tcp` in https://github.com/ruby/ruby/pull/9374. HEv2 is an algorithm defined in [RFC 8305](https://datatracker.ietf.org/doc/html/rfc8305), aimed at improving network connectivity. For more details on the specific cases that HEv2 helps, please refer to https://bugs.ruby-lang.org/issues/20108. 2. Proposal & Outcome This proposal implements the same HEv2 algorithm in `TCPSocket.new`. Since `TCPSocket.new` is used more widely than `Socket.tcp`, this change is expected to broaden the impact of HEv2's benefits. Like `Socket.tcp`, I have also added `fast_fallback` keyword argument to `TCPSocket.new`. This option is set to true by default, enabling the HEv2 functionality. However, users can explicitly set it to false to disable HEv2 and use the previous behavior of `TCPSocket.new`. It should be noted that HEv2 is enabled only in environments where pthreads are available. This specification follows the approach taken in https://bugs.ruby-lang.org/issues/19965 , where name resolution can be interrupted. (In environments where pthreads are not available, the `fast_fallback` option is ignored.) 3. Performance Below is the benchmark of 100 requests to `www.ruby-lang.org` with the fast_fallback option set to true and false, respectively. While there is a slight performance degradation when HEv2 is enabled, the degradation is smaller compared to that seen in `Socket.tcp`. ``` ~/s/build ❯❯❯ ../install/bin/ruby ../ruby/test.rb Rehearsal -------------------------------------------------------- fast_fallback: true 0.017588 0.097045 0.114633 ( 1.460664) fast_fallback: false 0.014033 0.078984 0.093017 ( 1.413951) ----------------------------------------------- total: 0.207650sec user system total real fast_fallback: true 0.020891 0.124054 0.144945 ( 1.473816) fast_fallback: false 0.018392 0.110852 0.129244 ( 1.466014) ``` * Update debug prints Co-authored-by: Nobuyoshi Nakada <nobu.nakada@gmail.com> * Remove debug prints * misc * Disable HEv2 in Win * Raise resolution error with hostname resolution * Fix to handle errors * Remove warnings * Errors that do not need to be handled * misc * Improve doc * Fix bug on cancellation * Avoid EAI_ADDRFAMILY for resolving IPv6 * Follow upstream * misc * Refactor connection_attempt_fds management - Introduced allocate_connection_attempt_fds and reallocate_connection_attempt_fds for improved memory allocation of connection_attempt_fds - Added remove_connection_attempt_fd to resize connection_attempt_fds dynamically. - Simplified the in_progress_fds function to only check the size of connection_attempt_fds. * Rename do_pthread_create to raddrinfo_pthread_create to avoid conflicting --------- Co-authored-by: Nobuyoshi Nakada <nobu.nakada@gmail.com> Notes: Merged-By: shioimm <shioi.mm@gmail.com>
2024-10-20[DOC] Improve `Socket::tcp` with Happy Eyeballs Version 2Misaki Shioi
With the introduction of Happy Eyeballs Version 2 to `Socket::tcp`, the following areas have been mainly enhanced: - How the value specified for `connect_timeout` works - How Socket.tcp operates with Happy Eyeballs Version 2 A description for the new option `fast_fallback` has been added in https://github.com/ruby/ruby/pull/11813. Notes: Merged: https://github.com/ruby/ruby/pull/11920 Merged-By: nobu <nobu@ruby-lang.org>
2024-10-12Support `IO#timeout` for `rsock_connect`. (#11880)Samuel Williams
Notes: Merged-By: ioquatix <samuel@codeotaku.com>
2024-10-11Fix an implicit conversion that loses integer precisionNobuyoshi Nakada
Both `rb_ensure` and `udp_bind_internal` return `VALUE`.
2024-10-11Update `rsock_connect` to take `VALUE io` argument. (#11847)Samuel Williams
Notes: Merged-By: ioquatix <samuel@codeotaku.com>
2024-10-10Move directives outside `rb_ary_new3` callNobuyoshi Nakada
This function may be a macro for optimization, which will be expanded to `rb_ary_new_from_values`. ``` ext/socket/ancdata.c: In function ‘bsock_recvmsg_internal’: ext/socket/ancdata.c:1648:1: error: embedding a directive within macro arguments is not portable 1648 | #if defined(HAVE_STRUCT_MSGHDR_MSG_CONTROL) | ^ ext/socket/ancdata.c:1650:1: error: embedding a directive within macro arguments is not portable 1650 | #else | ^ ext/socket/ancdata.c:1652:1: error: embedding a directive within macro arguments is not portable 1652 | #endif | ^ ```
2024-10-09Update `Socket#recvfrom` to use `rb_io_blocking_region`.Samuel Williams
Notes: Merged: https://github.com/ruby/ruby/pull/11844
2024-10-09Update `Socket#accept` to use `rb_io_blocking_region`.Samuel Williams
Notes: Merged: https://github.com/ruby/ruby/pull/11844
2024-10-09Update `UDPSocket#send` to use `rb_io_blocking_region`.Samuel Williams
Notes: Merged: https://github.com/ruby/ruby/pull/11844
2024-10-09Update `IPSocket` to use `rb_io_descriptor` and `rb_io_mode`.Samuel Williams
Notes: Merged: https://github.com/ruby/ruby/pull/11796
2024-10-09Update `UNIXSocket` to use `rb_io_blocking_region`.Samuel Williams
Notes: Merged: https://github.com/ruby/ruby/pull/11796
2024-10-09Update `BasicSocket` to use `rb_io_blocking_region`.Samuel Williams
Notes: Merged: https://github.com/ruby/ruby/pull/11796
2024-10-07[DOC] Add some descriptions for options of `Socket::tcp`v3_4_0_preview2Misaki Shioi
Notes: Merged: https://github.com/ruby/ruby/pull/11813 Merged-By: nobu <nobu@ruby-lang.org>
2024-09-13Prevent warnings "the block passed to ... may be ignored"Yusuke Endoh
Notes: Merged: https://github.com/ruby/ruby/pull/11611
2024-09-06Prefer constants for `shutdown` over magic numbersNobuyoshi Nakada
2024-09-05Proof of Concept: Allow to prevent fork from happening in known fork unsafe APIJean Boussier
[Feature #20590] For better of for worse, fork(2) remain the primary provider of parallelism in Ruby programs. Even though it's frowned uppon in many circles, and a lot of literature will simply state that only async-signal safe APIs are safe to use after `fork()`, in practice most APIs work well as long as you are careful about not forking while another thread is holding a pthread mutex. One of the APIs that is known cause fork safety issues is `getaddrinfo`. If you fork while another thread is inside `getaddrinfo`, a mutex may be left locked in the child, with no way to unlock it. I think we could reduce the impact of these problem by preventing in for the most notorious and common cases, by locking around `fork(2)` and known unsafe APIs with a read-write lock. Notes: Merged: https://github.com/ruby/ruby/pull/10864
2024-07-30Improve Socket.tcp (#11187)Misaki Shioi
[Feature #20646]Improve Socket.tcp This is a proposed improvement to `Socket.tcp`, which has implemented Happy Eyeballs version 2 (RFC8305) in PR9374. 1. Background I implemented Happy Eyeballs version 2 (HEv2) for Socket.tcp in PR9374, but several issues have been identified: - `IO.select` waits for name resolution or connection establishment in v46w, but it does not consider the case where both events occur simultaneously when it returns a value. - In this case, Socket.tcp can only capture one event and needs to execute an unnecessary loop to capture the other one, calling `IO.select` one extra time. - `IO.select` waits for both IPv6/IPv4 name resolution (in start), but when it returns a value, it doesn't consider the case where name resolution for both address families is complete. - In this case, `Socket.tcp` can only obtain the addresses of one address family and needs to execute an unnecessary loop obtain the other addresses, calling `IO.select` one extra time. - The consideration for `connect_timeout` was insufficient. After initiating one or more connections, it raises a 'user specified timeout' after the `connect_timeout` period even if there were addresses that have been resolved and have not yet tried to connect. - It does not retry with another address in case of a connection failure. - It executes unnecessary state transitions even when an IP address is passed as the `host` argument. - The regex for IP addresses did not correctly specify the start and end. 2. Proposal & Outcome To overcome the aforementioned issues, this PR introduces the following changes: - Previously, each loop iteration represented a single state transition. This has been changed to execute all processes that meet the execution conditions within a single loop iteration. - This prevents unnecessary repeated loops and calling `IO.select` - Introduced logic to determine the timeout value set for `IO.select`. During the Resolution Delay and Connection Attempt Delay, the user-specified timeout is ignored. Otherwise, the timeout value is set to the larger of `resolv_timeout` and `connect_timeout`. - This ensures that the `connect_timeout` is only detected after attempting to connect to all resolved addresses. - Retry with another address in case of a connection failure. - This prevents unnecessary repeated loops upon connection failure. - Call `tcp_without_fast_fallback` when an IP address is passed as the host argument. - This prevents unnecessary state transitions when an IP address is passed. - Fixed regex for IP addresses. Additionally, the code has been reduced by over 100 lines, and redundancy has been minimized, which is expected to improve readability. 3. Performance No significant performance changes were observed in the happy case before and after the improvement. However, improvements in state transition deficiencies are expected to enhance performance in edge cases. ```ruby require 'socket' require 'benchmark' Benchmark.bmbm do |x| x.report('fast_fallback: true') do 30.times { Socket.tcp("www.ruby-lang.org", 80) } end x.report('fast_fallback: false') do # Ruby3.3時点と同じ 30.times { Socket.tcp("www.ruby-lang.org", 80, fast_fallback: false) } end end ``` Before: ``` ~/s/build ❯❯❯ ../install/bin/ruby ../ruby/test.rb user system total real fast_fallback: true 0.021315 0.040723 0.062038 ( 0.504866) fast_fallback: false 0.007553 0.026248 0.033801 ( 0.533211) ``` After: ``` ~/s/build ❯❯❯ ../install/bin/ruby ../ruby/test.rb user system total real fast_fallback: true 0.023081 0.040525 0.063606 ( 0.406219) fast_fallback: false 0.007302 0.025515 0.032817 ( 0.418680) ```
2024-06-23Fix dangling `else`Nobuyoshi Nakada
2024-06-22[Bug #20592] Fix segfault when sending NULL to freeaddrinfoDmitry Davydov
On alpine freeaddrinfo does not accept NULL pointer
2024-06-21retry on cancelling of `getaddrinfo`Koichi Sasada
When the registerred unblock function is called, it should retry the cancelled blocking function if possible after checkints. For example, `SIGCHLD` can cancel this method, but it should not raise any exception if there is no trap handlers. The following is repro-code: ```ruby require 'socket' PN = 10_000 1000000.times{ p _1 PN.times{ fork{ sleep rand(0.3) } } i = 0 while i<PN cpid = Process.wait -1, Process::WNOHANG if cpid # p [i, cpid] i += 1 end begin TCPServer.new(nil, 0).close rescue p $! exit! end end } ```
2024-06-03Raise EAI_SYSTEM when pthread_create fails in getaddrinfoYusuke Endoh
Previously, EAI_AGAIN was raised. In our CI, "Temporary failure in name resolution" (EAI_AGAIN) is often raised. We are not sure if this was caused by pthread_create failure or getaddrinfo failure. To make it possible to distinguish between them, this changeset raises EAI_SYSTEM instead of EAI_AGAIN on pthread_create failure.
2024-06-01Suppress -Wmaybe-uninitialized warnings with LTONobuyoshi Nakada
2024-04-27ruby tool/update-deps --fix卜部昌平
2024-03-03Drop support for old ERBNobuyoshi Nakada
2024-02-26Revise 9ec342e07df6aa5e2c2e9003517753a2f1b508fdNobuyoshi Nakada
2024-02-26[Bug #20296] Fix the default assertion messageNobuyoshi Nakada
2024-02-26Introduction of Happy Eyeballs Version 2 (RFC8305) in Socket.tcp (#9374)Misaki Shioi
* Introduction of Happy Eyeballs Version 2 (RFC8305) in Socket.tcp This is an implementation of Happy Eyeballs version 2 (RFC 8305) in Socket.tcp. [Background] Currently, `Socket.tcp` synchronously resolves names and makes connection attempts with `Addrinfo::foreach.` This implementation has the following two problems. 1. In name resolution, the program stops until the DNS server responds to all DNS queries. 2. In a connection attempt, while an IP address is trying to connect to the destination host and is taking time, the program stops, and other resolved IP addresses cannot try to connect. [Proposal] "Happy Eyeballs" ([RFC 8305](https://datatracker.ietf.org/doc/html/rfc8305)) is an algorithm to solve this kind of problem. It avoids delays to the user whenever possible and also uses IPv6 preferentially. I implemented it into `Socket.tcp` by using `Addrinfo.getaddrinfo` in each thread spawned per address family to resolve the hostname asynchronously, and using `Socket::connect_nonblock` to try to connect with multiple addrinfo in parallel. [Outcome] This change eliminates a fatal defect in the following cases. Case 1. One of the A or AAAA DNS queries does not return --- require 'socket' class Addrinfo class << self # Current Socket.tcp depends on foreach def foreach(nodename, service, family=nil, socktype=nil, protocol=nil, flags=nil, timeout: nil, &block) getaddrinfo(nodename, service, Socket::AF_INET6, socktype, protocol, flags, timeout: timeout) .concat(getaddrinfo(nodename, service, Socket::AF_INET, socktype, protocol, flags, timeout: timeout)) .each(&block) end def getaddrinfo(_, _, family, *_) case family when Socket::AF_INET6 then sleep when Socket::AF_INET then [Addrinfo.tcp("127.0.0.1", 4567)] end end end end Socket.tcp("localhost", 4567) --- Because the current `Socket.tcp` cannot resolve IPv6 names, the program stops in this case. It cannot start to connect with IPv4 address. Though `Socket.tcp` with HEv2 can promptly start a connection attempt with IPv4 address in this case. Case 2. Server does not promptly return ack for syn of either IPv4 / IPv6 address family --- require 'socket' fork do socket = Socket.new(Socket::AF_INET6, :STREAM) socket.setsockopt(:SOCKET, :REUSEADDR, true) socket.bind(Socket.pack_sockaddr_in(4567, '::1')) sleep socket.listen(1) connection, _ = socket.accept connection.close socket.close end fork do socket = Socket.new(Socket::AF_INET, :STREAM) socket.setsockopt(:SOCKET, :REUSEADDR, true) socket.bind(Socket.pack_sockaddr_in(4567, '127.0.0.1')) socket.listen(1) connection, _ = socket.accept connection.close socket.close end Socket.tcp("localhost", 4567) --- The current `Socket.tcp` tries to connect serially, so when its first name resolves an IPv6 address and initiates a connection to an IPv6 server, this server does not return an ACK, and the program stops. Though `Socket.tcp` with HEv2 starts to connect sequentially and in parallel so a connection can be established promptly at the socket that attempted to connect to the IPv4 server. In exchange, the performance of `Socket.tcp` with HEv2 will be degraded. --- 100.times { Socket.tcp("www.ruby-lang.org", 80) } --- This is due to the addition of the creation of IO objects, Thread objects, etc., and calls to `IO::select` in the implementation. * Avoid NameError of Socket::EAI_ADDRFAMILY in MinGW * Support Windows with SO_CONNECT_TIME * Improve performance I have additionally implemented the following patterns: - If the host is single-stack, name resolution is performed in the main thread. This reduces the cost of creating threads. - If an IP address is specified, name resolution is performed in the main thread. This also reduces the cost of creating threads. - If only one IP address is resolved, connect is executed in blocking mode. This reduces the cost of calling IO::select. Also, I have added a fast_fallback option for users who wish not to use HE. Here are the results of each performance test. ```ruby require 'socket' require 'benchmark' HOSTNAME = "www.ruby-lang.org" PORT = 80 ai = Addrinfo.tcp(HOSTNAME, PORT) Benchmark.bmbm do |x| x.report("Domain name") do 30.times { Socket.tcp(HOSTNAME, PORT).close } end x.report("IP Address") do 30.times { Socket.tcp(ai.ip_address, PORT).close } end x.report("fast_fallback: false") do 30.times { Socket.tcp(HOSTNAME, PORT, fast_fallback: false).close } end end ``` ``` user system total real Domain name 0.015567 0.032511 0.048078 ( 0.325284) IP Address 0.004458 0.014219 0.018677 ( 0.284361) fast_fallback: false 0.005869 0.021511 0.027380 ( 0.321891) ```` And this is the measurement result when executed in a single stack environment. ``` user system total real Domain name 0.007062 0.019276 0.026338 ( 1.905775) IP Address 0.004527 0.012176 0.016703 ( 3.051192) fast_fallback: false 0.005546 0.019426 0.024972 ( 1.775798) ``` The following is the result of the run on Ruby 3.3.0. (on Dual stack environment) ``` user system total real Ruby 3.3.0 0.007271 0.027410 0.034681 ( 0.472510) ``` (on Single stack environment) ``` user system total real Ruby 3.3.0 0.005353 0.018898 0.024251 ( 1.774535) ``` * Do not cache `Socket.ip_address_list` As mentioned in the comment at https://github.com/ruby/ruby/pull/9374#discussion_r1482269186, caching Socket.ip_address_list does not follow changes in network configuration. But if we stop caching, it becomes necessary to check every time `Socket.tcp` is called whether it's a single stack or not, which could further degrade performance in the case of a dual stack. From this, I've changed the approach so that when a domain name is passed, it doesn't check whether it's a single stack or not and resolves names in parallel each time. The performance measurement results are as follows. require 'socket' require 'benchmark' HOSTNAME = "www.ruby-lang.org" PORT = 80 ai = Addrinfo.tcp(HOSTNAME, PORT) Benchmark.bmbm do |x| x.report("Domain name") do 30.times { Socket.tcp(HOSTNAME, PORT).close } end x.report("IP Address") do 30.times { Socket.tcp(ai.ip_address, PORT).close } end x.report("fast_fallback: false") do 30.times { Socket.tcp(HOSTNAME, PORT, fast_fallback: false).close } end end user system total real Domain name 0.004085 0.011873 0.015958 ( 0.330097) IP Address 0.000993 0.004400 0.005393 ( 0.257286) fast_fallback: false 0.001348 0.008266 0.009614 ( 0.298626) * Wait forever if fallback addresses are unresolved, unless resolv_timeout Changed from waiting only 3 seconds for name resolution when there is no fallback address available, to waiting as long as there is no resolv_timeout. This is in accordance with the current `Socket.tcp` specification. * Use exact pattern to match IPv6 address format for specify address family
2024-02-23Stop using rb_str_locktmp_ensure publiclyPeter Zhu
rb_str_locktmp_ensure is a private API.