Separate RDoc::TokenStream#add_tokens and #add_token

The old version of `add_tokens` accepts an array of tokens, and
multiple arguments of tokens by using `Array#flatten`.
And `add_token` was an alias to `add_tokens`.

I think it is unnecessarily flexible; in fact, all callsites of
`add_tokens` (except test) passes only an array of tokens.
And the code created a lot of temporal arrays.

This change makes `add_tokens` accept only one array of tokens,
and does `add_token` accept one token.  It is a bit faster (about
1 second in Ruby's `make rdoc`), and it ls also cleaner in my point of
view.

0 files changed, 0 insertions, 0 deletions