summaryrefslogtreecommitdiff
path: root/transcode_data.h
AgeCommit message (Collapse)Author
2008-08-09* transcode_data.h (rb_transcoding): new fields: next_info and next_byte.akr
* transcode.c (transcode_restartable): save/restore next_info and next_byte. sync readlen and in_p when invalid. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18454 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-09* transcode_data.h (rb_transcoding): add fields for restartable akr
transcoding. (rb_transcoder): add max_input field. from_unit_length field is renamed to input_unit_length. * tool/transcode-tblgen.rb: generate max_input field. * enc/trans/iso2022.erb.c: follow rb_transcoder change. * enc/trans/utf_16_32.erb.c: ditto. * transcode.c (PARTIAL_INPUT): new constant. (transcode_char_start): new function. (transcode_result_t): new type. (transcode_restartable): new function. (more_output_buffer): new function. (transcode_loop): use transcode_restartable. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18452 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-08* transcode_data.h (rb_transcoder): from_unit_length field added.akr
from_utf8 field removed. * tool/transcode-tblgen.rb: generate offsets range. follow rb_transcoder change. * transcode.c (transcode_loop): don't use from_utf8. make invalid region from_unit_length wise. * enc/trans/iso2022.erb.c: follow rb_transcoder and transcode_generate_node change. * enc/trans/utf_16_32.erb.c: follow rb_transcoder and transcode_generate_node change. explicit :invalid map removed. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18445 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-07* transcode_data.h (rb_transcoding): new field "stateful".akr
(rb_transcoder): preprocessor and postprocessor field removed. change arguments of func_ii, func_si, func_io and func_so. new field "finish_func". * tool/transcode-tblgen.rb: make FUNii, FUNsi and FUNio generatable. * transcode.c (transcoder_lib_table): removed. (transcoder_table): change structure. (transcoder_key): removed because the above structure change. (make_transcoder_entry): new function. (get_transcoder_entry): ditto. (rb_register_transcoder): follow the structure change. (declare_transcoder): ditto. (transcode_search_path): new function for breadth first search to find a list of converters. (transcode_search_path_i): new function. (transcode_dispatch_cb): ditto. (transcode_dispatch): use transcode_search_path. (transcode_loop): follow the argument change. (str_transcode): preprocessor and postprocessor stuff removed. * enc/trans/iso2022.erb.c: new file. ISO-2022-JP conversion re-implemented. * enc/trans/japanese.erb.c: ISO-2022-JP stuff removed. nute(23:52:53)% head -40 ChangeLog Thu Aug 7 23:43:11 2008 Tanaka Akira <akr@fsij.org> * transcode_data.h (rb_transcoding): new field "stateful". (rb_transcoder): preprocessor and postprocessor field removed. change arguments of func_ii, func_si, func_io and func_so. new field "finish_func". * tool/transcode-tblgen.rb: make FUNii, FUNsi and FUNio generatable. * transcode.c (transcoder_lib_table): removed. (transcoder_table): change structure. (transcoder_key): removed because the above structure change. (make_transcoder_entry): new function. (get_transcoder_entry): ditto. (rb_register_transcoder): follow the structure change. (declare_transcoder): ditto. (transcode_search_path): new function for breadth first search to find a list of converters. (transcode_search_path_i): new function. (transcode_dispatch_cb): ditto. (transcode_dispatch): use transcode_search_path. (transcode_loop): follow the argument change. (str_transcode): preprocessor and postprocessor stuff removed. * enc/trans/iso2022.erb.c: new file. ISO-2022-JP conversion re-implemented. * enc/trans/japanese.erb.c: ISO-2022-JP stuff removed. * enc/trans/utf_16_32.erb.c: follow argument change of FUNso. [ruby-dev:35798] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18419 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-05* transcode_data.h (TRANSCODE_ERROR): common transcode failurenobu
exception, would be changed later. * enc/trans/japanese.c (UNSUPPORTED_MODE): unsupported mode transition exception. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18363 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-07-14* transcode.c (transcode_loop): constified.nobu
* transcode.c (str_transcode): rb_str_set_len() sets a delimiter. * transcode_data.h (rb_transcoder): constified preprocessor and postprocessor input. * enc/trans/japanese.c: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18066 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-01-21* transcode.c (str_transcode): initialize transcoder innobu
rb_transcoding. [ruby-dev:33234] * transcode_data.h (rb_transcoding): transcoder constified. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15152 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-01-21* transcode.c (transcode_loop, str_transcoding_resize): use unsignednobu
char. [ruby-dev:33232] * transcode_data.h (rb_transcoding, rb_transcoder): removed callback parameters. * enc/trans/japanese.c: ditto. * enc/trans/utf_16_32.c: parenthesized bit-or operands. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15150 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-01-20* transcode.c (transcode_dispatch): constified return value.nobu
* transcode_data.h (rb_transcoding): include pointer to rb_transcoder and auxiliary data. * transcode_data.h (rb_transcoder): all callback functions shoud have their own parameters. * enc/trans/{japanese,single_byte}.c: constified. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15148 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-01-20Sun Jan 20 15:08:08 2008 Martin Duerst <duerst@it.aoyama.ac.jp>duerst
* enc/trans/utf_16_32.c: new file, currently implementing UTF-16BE conversions only. * test/ruby/test_transcode.rb: Added tests for UTF-16BE; made check_both_ways() use force_encoding differently. * transcode_data.h, transcode.c: Support for more conversion functions. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15142 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-01-06* $Date$ keyword removed to avoid inclusion of locale dependentakr
string. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14912 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-28Fri Dec 28 01:55:04 2007 Martin Duerst <duerst@it.aoyama.ac.jp>duerst
* transcode.c (transcode_dispatch): reverted some of the changes in r14746. * transcode.c, enc/trans/single_byte.c: Added conversions to/from US-ASCII and ASCII-8BIT (using data tables). * enc/trans/single_byte.c: Some spacing/ordering changes due to automatic data file generation. * transcode_data.h, transcode.c: Preliminary code for using micro-conversion functions. * test/ruby/test_transcode.rb: Added some tests for US-ASCII and ASCII-8BIT conversions. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14766 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-27* parse.y, transcode_data.h, transcode.c: change "illegal" toakr
"invalid" in a context which doesn' t against a law. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14735 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-25* common.mk (COMMONOBJS): transcode_data_*.c moved under enc/trans.nobu
* transcode_data.h (rb_transcoding, rb_transcoder): prefixed. * transcode.c (rb_register_transcoder, rb_declare_transcoder): split declaration and registration. [ruby-dev:32704] * transcode.c (transcode_dispatch): autoload pre-declared transcoder. * transcode.c (str_transcode): use rb_define_dummy_encoding(). * transcode.c (Init_transcode): initialize transcoder tables. * enc/trans/single_byte.c, enc/trans/japanese.c: moved from top. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14666 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-24* transocode.c: register_functional_transcoder() added.naruse
(init_transcoder_table(: register ISO-2022-JP. (str_transcode): add preprocessor and postprocessor. * transcode_data_japanese.c: add ISO-2022-JP support. * transcode_data.h: moved transcoder and transcoding difinition from transcode.c. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14607 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-15* transcode.c (str_transcode, transcode_dispatch): added two-stepmatz
* trancode.c: some minor formatting fixes * transcode_data.h, transcode_data_iso_8859.c: Shortened extremely frequently used macros to shorten file length. * test/ruby/test_transcode.rb: Fixed name of test class; added setup method to ensure all necessary encodings exist; split tests into more test methods; added tests; fixed ordering of arguments in assert_equal to have expected result first. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14236 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-10* string.c (rb_str_tmp_new): creates hidden temporary buffer.nobu
* transcode.c (transcoding): added a pointer to function to flush. * transcode.c (transcode_loop): do not use string internal. [ruby-dev:32512] * transcode.c (str_transcode): allow Encoding objects. * transcode_data.h (BYTE_LOOKUP): use actual struct name. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14176 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2007-12-10* transcode.c: new file to provide encoding conversion features.matz
code contributed by Martin Duerst. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14172 b2dd03c8-39d4-4d8f-98ff-823fe69b080e