<feed xmlns='http://www.w3.org/2005/Atom'>
<title>ruby.git/ext/-test-/file/newline_conv.c, branch v4.0.2</title>
<subtitle>The Ruby Programming Language</subtitle>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/'/>
<entry>
<title>Fix CRLF -&gt; LF conversion on read for rb_io_fdopen &amp; rb_file_open</title>
<updated>2024-01-10T10:02:23+00:00</updated>
<author>
<name>KJ Tsanaktsidis</name>
<email>kj@kjtsanaktsidis.id.au</email>
</author>
<published>2023-12-28T05:08:54+00:00</published>
<link rel='alternate' type='text/html' href='https://git.ruby-lang.org/ruby.git/commit/?id=31371b2e24b03ccb0a03b622faf8c65e6cf6a31a'/>
<id>31371b2e24b03ccb0a03b622faf8c65e6cf6a31a</id>
<content type='text'>
When opening a file with `File.open`, and then setting the encoding with
`IO#set_encoding`, it still correctly performs CRLF -&gt; LF conversion on
Windows when reading files with a CRLF line ending in them (in text
mode).

However, the file is opened instead with either the `rb_io_fdopen` or
`rb_file_open` APIs from C, the CRLF conversion is _NOT_ set up
correctly; it works if the encoding is not specified, but if
`IO#set_encoding` is called, the conversion stops happening. This seems
to be because the encflags never get ECONV_DEFAULT_NEWLINE_DECORATOR
set in these codepaths.

Concretely, this means that the conversion doesn't happen in the
following circumstances:
  * When loading ruby files with require (that calls rb_io_fdopen)
  * When parsing ruuby files with RubyVM::AbstractSyntaxTree (that calls
    rb_file_open).
This then causes the ErrorHighlight tests to fail on windows if git has
checked them out with CRLF line endings - the error messages it's
testing wind up with literal \r\n sequences in them because the iseq
text from the parser contains un-newline-converted strings.

This commit fixes the problem by copy-pasting the relevant snippet which
sets this up in `rb_io_extract_modeenc` (for the File.open path) into
the relevant codepaths for `rb_io_fdopen` and `rb_file_open`.

[Bug #20101]
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
When opening a file with `File.open`, and then setting the encoding with
`IO#set_encoding`, it still correctly performs CRLF -&gt; LF conversion on
Windows when reading files with a CRLF line ending in them (in text
mode).

However, the file is opened instead with either the `rb_io_fdopen` or
`rb_file_open` APIs from C, the CRLF conversion is _NOT_ set up
correctly; it works if the encoding is not specified, but if
`IO#set_encoding` is called, the conversion stops happening. This seems
to be because the encflags never get ECONV_DEFAULT_NEWLINE_DECORATOR
set in these codepaths.

Concretely, this means that the conversion doesn't happen in the
following circumstances:
  * When loading ruby files with require (that calls rb_io_fdopen)
  * When parsing ruuby files with RubyVM::AbstractSyntaxTree (that calls
    rb_file_open).
This then causes the ErrorHighlight tests to fail on windows if git has
checked them out with CRLF line endings - the error messages it's
testing wind up with literal \r\n sequences in them because the iseq
text from the parser contains un-newline-converted strings.

This commit fixes the problem by copy-pasting the relevant snippet which
sets this up in `rb_io_extract_modeenc` (for the File.open path) into
the relevant codepaths for `rb_io_fdopen` and `rb_file_open`.

[Bug #20101]
</pre>
</div>
</content>
</entry>
</feed>
