summaryrefslogtreecommitdiff
path: root/enc/trans/newline.trans
AgeCommit message (Collapse)Author
2009-11-22* enc/trans/newline.trans (fun_so_universal_newline): generate \nakr
after \r\n detection instead of just after \r. [ruby-list:45988] [ruby-core:25881] [ruby-core:26788] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25883 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-14* transcode_data.h: return output functions ssize_t.akr
* transcode.c (transcode_restartable0): don't need to cast the result of output functions. * enc/trans/newline.trans: follow the type change. * enc/trans/escape.trans: ditto. * enc/trans/utf_16_32.trans: ditto. * enc/trans/iso2022.trans: ditto. * enc/trans/japanese.trans: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19351 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-14* transcode_data.h: output function takes output buffer size.akr
* transcode.c: give output buffer size for output functions. * enc/trans/newline.trans: follow the type change. * enc/trans/escape.trans: ditto. * enc/trans/utf_16_32.trans: ditto. * enc/trans/iso2022.trans: ditto. * enc/trans/japanese.trans: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19350 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-09* enc/trans/newline.trans (rb_universal_newline): swap src_encodingakr
and dst_encoding. * transcode.c (rb_econv_decorate_at): call get_transcoder_entry only once. (rb_econv_binmode): follow universal_newline change. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19276 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-08* include/ruby/encoding.h (rb_econv_asciicompat_encoding): renamedakr
from rb_econv_stateless_encoding to apply stateless ASCII incompatible encodings such as UTF-16BE. * io.c (make_writeconv): use rb_econv_asciicompat_encoding. * transcode_data.h (rb_transcoder_asciicompat_type_t): renamed from rb_transcoder_stateful_type_t. (rb_transcoder): use rb_transcoder_asciicompat_type_t. * transcode.c: follow the type change. (asciicompat_encoding_i): renamed from stateless_encoding_i. (rb_econv_asciicompat_encoding): renamed from rb_econv_stateless_encoding. (econv_s_asciicompat_encoding): method renamed. * tool/transcode-tblgen.rb: follow the type change. * enc/trans/utf_16_32.trans: follow the type change. rb_from_UTF_16BE to UTF-8 is asciicompat_decoder. rb_from_UTF_16LE to UTF-8 is asciicompat_decoder. rb_from_UTF_32BE to UTF-8 is asciicompat_decoder. rb_from_UTF_32LE to UTF-8 is asciicompat_decoder. UTF-8 to rb_to_UTF_16BE is asciicompat_encoder. UTF-8 to rb_to_UTF_16LE is asciicompat_encoder. UTF-8 to rb_to_UTF_32BE is asciicompat_encoder. UTF-8 to rb_to_UTF_32LE is asciicompat_encoder. * enc/trans/newline.trans: follow the type change. universal newline decoder is asciicompat_converter. * enc/trans/escape.trans: follow the type change. * enc/trans/iso2022.trans: ditto. * enc/trans/japanese.trans: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19249 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-05* enc/trans/newline.trans (universal_newline_finish): new function.akr
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19152 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-04* enc/trans/newline.trans: record newline types met in universalakr
newline decoder. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19136 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-03* transcode_data.h (rb_transcoding): remove stateful field.akr
add state field. (TRANSCODING_STATE): defined. (rb_transcoder): add fields: state_size, state_init_func, state_fini_func. change rb_transcoding* argument to void*. * transcode.c (transcode_restartable0): use TRANSCODING_STATE for first arguments of transcoder functions. (rb_transcoding_open_by_transcoder): initialize state field. (rb_transcoding_close): finalize state field. * tool/transcode-tblgen.rb: provide state size/init/fini. * enc/trans/newline.trans (universal_newline_init): defined. (fun_so_universal_newline): take void* as a state pointer. (rb_universal_newline): provide state size/init/fini. (rb_crlf_newline): ditto. (rb_cr_newline): ditto. * enc/trans/iso2022.trans (iso2022jp_init): defined. (fun_si_iso2022jp_to_eucjp): take void* as a state pointer. (fun_so_iso2022jp_to_eucjp): ditto. (fun_so_eucjp_to_iso2022jp): ditto. (iso2022jp_reset_sequence_size): ditto. (finish_eucjp_to_iso2022jp): ditto. (rb_ISO_2022_JP_to_EUC_JP): provide state size/init/fini. (rb_EUC_JP_to_ISO_2022_JP): ditto. * enc/trans/utf_16_32.trans (fun_so_from_utf_16be): take void* as a state pointer. (fun_so_to_utf_16be): ditto. (fun_so_from_utf_16le): ditto. (fun_so_to_utf_16le): ditto. (fun_so_from_utf_32be): ditto. (fun_so_to_utf_32be): ditto. (fun_so_from_utf_32le): ditto. (fun_so_to_utf_32le): ditto. (rb_from_UTF_16BE): provide state size/init/fini. (rb_to_UTF_16BE): ditto. (rb_from_UTF_16LE): ditto. (rb_to_UTF_16LE): ditto. (rb_from_UTF_32BE): ditto. (rb_to_UTF_32BE): ditto. (rb_from_UTF_32LE): ditto. (rb_to_UTF_32LE): ditto. * enc/trans/japanese.trans (fun_so_eucjp2sjis): take void* as a state pointer. (fun_so_sjis2eucjp): ditto. (rb_eucjp2sjis): provide state size/init/fini. (rb_sjis2eucjp): provide state size/init/fini. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19096 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-01* tool/transcode-tblgen.rb: define TRANSCODE_TABLE_INFO in generatedakr
code. use it in rb_transcoder. * enc/trans/newline.trans: use TRANSCODE_TABLE_INFO. * enc/trans/iso2022.trans: ditto. * enc/trans/utf_16_32.trans: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19046 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-01* tool/transcode-tblgen.rb: record offsets array as index ofakr
byte_array to avoid relocation. * transcode.c (transcode_restartable0): add byte_array to get offsets array. * transcode_data.h (BYTE_LOOKUP_BASE): change return type to uintptr_t. (rb_transcoder): add fields: byte_array, word_array and word_size. * enc/trans/newline.trans: follow rb_transcoder change. * enc/trans/iso2022.trans: ditto. * enc/trans/utf_16_32.trans: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19043 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-09-01* transcode_data.h (BYTE_LOOKUP): change to uintptr_t array.akr
(BYTE_LOOKUP_BASE): follow the type change. (BYTE_LOOKUP_INFO): ditto. (PType): ditto. (rb_transcoding): ditto. * tool/transcode-tblgen.rb: follow the type change. * transcode.c: ditto. * enc/trans/newline.trans: ditto. * enc/trans/iso2022.trans: ditto. * enc/trans/utf_16_32.trans: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19038 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-31* tool/transcode-tblgen.rb (transcode_generated_code): defined forakr
generating table at once. (transcode_tblgen): returns an empty string. (transcode_generate_node): ditto. * enc/trans/newline.trans: use transcode_generated_code. * enc/trans/iso2022.trans: ditto. * enc/trans/single_byte.trans: ditto. * enc/trans/utf_16_32.trans: ditto. * enc/trans/japanese.trans: ditto. * enc/trans/korean.trans: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19006 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-22* include/ruby/io.h (FMODE_TEXTMODE): defined.akr
* include/ruby/encoding.h (rb_econv_t): new field: flags. (rb_econv_binmode): declared. * io.c (io_unread): text mode hack removed. (NEED_NEWLINE_DECODER): defined. (NEED_NEWLINE_ENCODER): defined. (NEED_READCONV): defined. (NEED_WRITECONV): defined. (TEXTMODE_NEWLINE_ENCODER): defined for windows. (make_writeconv): setup converter with TEXTMODE_NEWLINE_ENCODER for text mode. (io_fwrite): use NEED_WRITECONV. character code conversion is disabled if fptr->writeconv_stateless is nil. (make_readconv): setup converter with ECONV_UNIVERSAL_NEWLINE_DECODER for text mode. (read_all): use NEED_READCONV. (appendline): use NEED_READCONV. (rb_io_getline_1): use NEED_READCONV. (io_getc): use NEED_READCONV. (rb_io_ungetc): use NEED_READCONV. (rb_io_binmode): OS-level text mode test removed. call rb_econv_binmode. (rb_io_binmode_m): call rb_io_binmode_m with write_io as well. (rb_io_flags_mode): return mode string including "t". (rb_io_mode_flags): detect "t" for text mode. (rb_sysopen): always specify O_BINARY. * transcode.c (rb_econv_open_by_transcoder_entries): initialize flags. (rb_econv_open): if source and destination encoding is both empty string, open newline converter. last_tc will be NULL in this case. (rb_econv_encoding_to_insert_output): last_tc may be NULL now. (rb_econv_string): ditto. (output_replacement_character): ditto. (transcode_loop): ditto. (econv_init): ditto. (econv_inspect): ditto. (rb_econv_binmode): new function. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18780 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-16* include/ruby/encoding.h (rb_econv_t): add fields: in_buf_start,akr
in_data_start, in_data_end, in_buf_end and last_trans_index. (rb_econv_output): removed. (rb_econv_insert_output): declared. (rb_econv_encoding_to_insert_output): declared. * enc/trans/newline.trans (rb_universal_newline): stateful_type changed. * transcode.c (transcode_restartable0): initialize inchar_start, tc->recognized_len and next_table at beginning of the loop. (rb_econv_open_by_transcoder_entries): initialize new fields. (rb_econv_open): setup last_trans_index. (trans_sweep): last out_buf_start can be non-NULL now. (rb_econv_convert): check last out_buf_start and in_buf_start at first. (rb_econv_output_with_destination_encoding): removed. (econv_just_convert): removed. (rb_econv_output): removed. (econv_primitive_output): method removed. (rb_econv_encoding_to_insert_output): new function. (allocate_converted_string): new function. (rb_econv_insert_output): new function. (econv_primitive_insert_output): new method. (output_replacement_character): use rb_econv_insert_output. unused arguments removed. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18654 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15* transcode_data.h (rb_transcoder_stateful_type_t): defined.akr
(rb_transcoder): add field: stateful_type. * tool/transcode-tblgen.rb: generate stateful_type field as stateless_converter. * enc/trans/iso2022.trans: follow rb_transcoder change. * enc/trans/newline.trans: ditto. * enc/trans/utf_16_32.trans: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18650 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-13fix variable name.akr
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18558 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-13* enc/trans/newline.trans (rb_crlf_newline): new transcoder.akr
(rb_cr_newline): new transcoder. * transcode.c (trans_open_i): one more exra room for input newline converter. (rb_trans_open): crlf newline and cr newline implemented. (Init_transcode): Encoding::Converter::CRLF_NEWLINE and Encoding::Converter::LF_NEWLINE defined. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18557 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-13* enc/trans/newline.trans: new file.akr
* transcode_data.h (rb_trans_t): add last_tc field. * transcode.c (UNIVERSAL_NEWLINE): defined. (CRLF_NEWLINE): defined. (CR_NEWLINE): defined. (rb_trans_open_by_transcoder_entries): initialize last_tc. (trans_open_i): allocate one more room for newline converter. (rb_trans_open): universal newline implemented. (more_output_buffer): take max_output argument instead ts. (output_replacement_character): take tc argument instead of ts. (transcode_loop): use last_tc field. (econv_init): add flags argument for rb_trans_open. (Init_transcode): Encoding::Converter::UNIVERSAL_NEWLINE defined. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18556 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-24 12:50:16 +0000