- 0139bc5 Introduce the english model as being on the same level as german by Akron · 1 year, 3 months ago[Renamed from testdata/tokenizer.datok]
- 6c92763 New build by Akron · 1 year, 9 months ago
- b98e4cf Improve Emoticons by Akron · 2 years, 8 months ago v0.1.5
- b428755 Support punctuation after quotes by Akron · 2 years, 8 months ago v0.1.4
- 4222ac8 Improve handling of ellipsis by Akron · 2 years, 9 months ago
- e200841 Further improve speech rule for eos with more quotation marks by Akron · 2 years, 9 months ago
- e96895f Improve handling of sentence splits including speech by Akron · 2 years, 9 months ago
- 4ec8cec Prepare first official release by Akron · 3 years ago v0.1.0
- fac8abc Reorder longest match operator and update models by Akron · 3 years, 1 month ago
- 17984c8 Improving time parsing by Akron · 3 years, 1 month ago
- a854faa Introduce EOT (end-of-transmission) marker by Akron · 3 years, 1 month ago
- 4c2a1ad Introduce XML tests by Akron · 3 years, 3 months ago
- 235ea12 Update generated tokenizers by Akron · 3 years, 3 months ago
- e184a91 Add new generated automata by Akron · 3 years, 4 months ago
- 03c92fe Support for tokenend MCS symbol by Akron · 3 years, 4 months ago
- b4bbb47 Added sentence splitter capabilities by Akron · 3 years, 4 months ago
- 3a063ef Fix loading routine by Akron · 3 years, 4 months ago