ruby.git/include/ruby/oniguruma.h, branch ruby_1_9_2

* include/ruby/oniguruma.h: updated to follow Oniguruma 5.9.2.

2010-03-01T21:54:59+00:00

* re.c (make_regexp): use onig_new() instead of onig_alloc_init().

* re.c (rb_reg_to_s): ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26791 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

\d, \s and \w are now non Unicode class. [ruby-dev:39026]

2009-08-15T16:01:33+00:00

* include/ruby/oniguruma.h
  (ONIGENC_CTYPE_SPECIAL_MASK): added.
  (ONIGENC_CTYPE_D): ditto.
  (ONIGENC_CTYPE_S): ditto.
  (ONIGENC_CTYPE_W): ditto.

* regparse.c: \d, \s and \w are now non Unicode class.
  [ruby-dev:39026]
  (fetch_token_in_cc): use ONIGENC_CTYPE_[DSW] for \d/\s/\w.
  (fetch_token): ditto.
  (add_ctype_to_cc): add routines for ONIGENC_CTYPE_[DSW].
  (parse_exp): ditto.

* test/ruby/test_regexp.rb (TestRegexp#test_char_class):
  add tests for above.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24544 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Warn duplicated characters in character class of regexp. [ruby-core:24593]

2009-08-04T04:14:38+00:00

* include/ruby/oniguruma.h (ONIG_SYN_WARN_CC_DUP): defined.
* regparse.h (ScanEnv): add warnings_flag.
* regparse.c (CC_DUP_WARN): defined for warn duplicated characters in
  character class of regexp. [ruby-core:24593]
  (add_code_range_to_buf): add CC_DUP_WARN.
  (next_state_val): add CC_DUP_WARN.
  (OnigSyntaxRuby): add ONIG_SYN_WARN_CC_DUP.
  (SET_ALL_MULTI_BYTE_RANGE): add env to arguments.
  (add_code_range): ditto.
  (add_code_range_to_buf): ditto.
  (not_code_range_buf): ditto.
  (or_code_range_buf): ditto.
  (and_code_range1): ditto.
  (and_code_range_buf): ditto.
  (and_cclass): ditto.
  (or_cclass): ditto.
  (add_ctype_to_cc_by_range): ditto.
  (add_ctype_to_cc): ditto.
  (parse_char_class): ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24387 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* include/ruby/oniguruma.h, include/ruby/re.h, re.c, regcomp.c,

2009-06-30T02:08:54+00:00

  regenc.c, regerror.c, regexec.c, regint.h, regparse.c: use long.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23907 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* id.h, include/ruby/{intern,oniguruma}.h, regenc.h, regparse.h,

2008-12-09T07:14:13+00:00

  template/*.tmpl: removed trailing garbage spaces.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@20596 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* grapheme cluster implementation reverted. [ruby-dev:36375]

2008-09-18T12:53:25+00:00

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19417 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* include/ruby/oniguruma.h (OnigEncodingTypeST): add precise_ret

2008-09-16T16:48:05+00:00

  argument for mbc_to_code.
  (ONIGENC_MBC_TO_CODE): provide NULL for precise_ret.
  (ONIGENC_MBC_PRECISE_CODEPOINT): defined.

* include/ruby/encoding.h (rb_enc_mbc_precise_codepoint): defined.

* regenc.h (onigenc_single_byte_mbc_to_code): precise_ret argument
  added.
  (onigenc_mbn_mbc_to_code): ditto.

* regenc.c (onigenc_single_byte_mbc_to_code): precise_ret argument
  added.
  (onigenc_mbn_mbc_to_code): ditto.

* string.c (count_utf8_lead_bytes_with_word): removed.
  (str_utf8_nth): removed.
  (str_utf8_offset): removed.
  (str_strlen): UTF-8 codepoint oriented optimization removed.
  (rb_str_substr): ditto.
  (enc_succ_char): use rb_enc_mbc_precise_codepoint.
  (enc_pred_char): ditto.
  (rb_str_succ): ditto.

* encoding.c (rb_enc_ascget): check length with
  rb_enc_mbc_precise_codepoint.
  (rb_enc_codepoint): use rb_enc_mbc_precise_codepoint.

* regexec.c (string_cmp_ic): add text_end argument.
  (match_at): check end of character after exact string matches.

* enc/utf_8.c (graphme_table): defined for extended graphme cluster
  boundary.
  (grapheme_cmp): defined.
  (get_grapheme_properties): defined.
  (grapheme_boundary_p): defined.
  (MAX_BYTES_LENGTH): defined.
  (comb_char_enc_len): defined.
  (mbc_to_code0): extracted from mbc_to_code.
  (mbc_to_code): use mbc_to_code0.
  (left_adjust_combchar_head): defined.
  (utf_8): use a extended graphme cluster as a unit.

* enc/unicode.c (onigenc_unicode_mbc_case_fold): use
  ONIGENC_MBC_PRECISE_CODEPOINT to extract codepoints.
  (onigenc_unicode_get_case_fold_codes_by_str): ditto.

* enc/euc_jp.c (mbc_to_code): follow mbc_to_code field change.
  use onigenc_mbn_mbc_to_code.

* enc/shift_jis.c (mbc_to_code): ditto.

* enc/emacs_mule.c (mbc_to_code): ditto.

* enc/gbk.c (gbk_mbc_to_code): follow mbc_to_code field and
  onigenc_mbn_mbc_to_code change.

* enc/cp949.c (cp949_mbc_to_code): ditto.

* enc/big5.c (big5_mbc_to_code): ditto.

* enc/euc_tw.c (euctw_mbc_to_code): ditto.

* enc/euc_kr.c (euckr_mbc_to_code): ditto.

* enc/gb18030.c (gb18030_mbc_to_code): ditto.

* enc/utf_32be.c (utf32be_mbc_to_code): follow mbc_to_code field
  change.

* enc/utf_16be.c (utf16be_mbc_to_code): ditto.

* enc/utf_32le.c (utf32le_mbc_to_code): ditto.

* enc/utf_16le.c (utf16le_mbc_to_code): ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19389 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* include/ruby/oniguruma.h (OnigEncodingTypeST): add end argument for

2008-09-13T19:23:52+00:00

  left_adjust_char_head.
  (ONIGENC_LEFT_ADJUST_CHAR_HEAD): add end argument.
  (onigenc_get_left_adjust_char_head): ditto.

* include/ruby/encoding.h (rb_enc_left_char_head): add end argument.

* regenc.h (onigenc_single_byte_left_adjust_char_head): ditto.

* regenc.c (onigenc_get_right_adjust_char_head): follow the interface
  change.
  (onigenc_get_right_adjust_char_head_with_prev): ditto.
  (onigenc_get_prev_char_head): ditto.
  (onigenc_step_back): ditto.
  (onigenc_get_left_adjust_char_head): ditto.
  (onigenc_single_byte_code_to_mbc): ditto.

* re.c: ditto.

* string.c: ditto.

* io.c: ditto.

* regexec.c: ditto.

* enc/euc_jp.c: ditto.

* enc/cp949.c: ditto.

* enc/shift_jis.c: ditto.

* enc/gbk.c: ditto.

* enc/big5.c: ditto.

* enc/euc_tw.c: ditto.

* enc/euc_kr.c: ditto.

* enc/emacs_mule.c: ditto.

* enc/gb18030.c: ditto.

* enc/utf_8.c: ditto.

* enc/utf_16le.c: ditto.

* enc/utf_16be.c: ditto.

* enc/utf_32le.c: ditto.

* enc/utf_32be.c: ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19334 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* include/ruby/oniguruma.h (ONIGENC_STEP_BACK): add end argument.

2008-09-13T18:44:53+00:00

  (onigenc_step_back): ditto.

* regenc.c (onigenc_step_back): add end argument.

* regexec.c: follow the interface change.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19333 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* include/ruby/oniguruma.h (onigenc_get_prev_char_head): add end

2008-09-13T18:22:04+00:00

  argument.

* include/ruby/encoding.h (rb_enc_prev_char): ditto.

* regenc.c (onigenc_get_prev_char_head): add end argument.

* regparse.c: follow the interface change.

* regexec.c: ditto.

* string.c: ditto.

* parse.y: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19332 b2dd03c8-39d4-4d8f-98ff-823fe69b080e