Gitiles
Code Review Sign In
korap.ids-mannheim.de / KorAP / Datok / b428755ff720aefb37ebbce920cd76df61879d30 / src / tokenizer.xfst
  1. b428755 Support punctuation after quotes by Akron · 2 years, 8 months ago v0.1.4
  2. 4222ac8 Improve handling of ellipsis by Akron · 2 years, 9 months ago
  3. ece3f01 Support quote combinations at the end of sentences by Akron · 2 years, 9 months ago
  4. e200841 Further improve speech rule for eos with more quotation marks by Akron · 2 years, 9 months ago
  5. e96895f Improve handling of sentence splits including speech by Akron · 2 years, 9 months ago
  6. b02ad07 Improve handling of apostrophes by Akron · 2 years, 10 months ago
  7. 54ed7e7 Fix handling of "z.B." by Akron · 2 years, 11 months ago
  8. 936c0f5 Support Plusampersand words in compounds by Akron · 3 years ago
  9. e62e8eb Introducing Plusampersand-Compounds by Akron · 3 years ago
  10. e87906b Minor improvements by Akron · 3 years ago
  11. fac8abc Reorder longest match operator and update models by Akron · 3 years, 1 month ago
  12. 65c0f21 Simplify tokenizer whitespace handling by Akron · 3 years, 1 month ago
  13. c840636 Separate xml rule from main script by Akron · 3 years, 1 month ago
  14. 7198645 Speed up build by Akron · 3 years, 1 month ago
  15. 6742b96 Add XML entities by Akron · 3 years, 1 month ago
  16. 11a05d9 Extend tokenizer fileending by Akron · 3 years, 1 month ago
  17. f1106ec Add single character abbreviations by Akron · 3 years, 1 month ago
  18. 17984c8 Improving time parsing by Akron · 3 years, 1 month ago
  19. 78dba06 Add time format to transducer by Akron · 3 years, 1 month ago
  20. 066d99c Fix XML empty element handling by Akron · 3 years, 1 month ago
  21. f6bdfdb Add trimming at the beginning of a text by Akron · 3 years, 1 month ago
  22. a854faa Introduce EOT (end-of-transmission) marker by Akron · 3 years, 1 month ago
  23. 4c2a1ad Introduce XML tests by Akron · 3 years, 3 months ago
  24. 3de361e Improved newline and abbreviation handling by Akron · 3 years, 3 months ago
  25. 57d0161 Add known terms with special characters by Akron · 3 years, 4 months ago
  26. e8837b5 Add file scheme by Akron · 3 years, 4 months ago
  27. fd92d7e Update abbreviations according to KorAP-Tokenizer by Akron · 3 years, 4 months ago
  28. a0bded5 Add ordinals by Akron · 3 years, 4 months ago
  29. 4af79f1 Added support for streetnames by Akron · 3 years, 4 months ago
  30. 310905f Add foma sources by Akron · 3 years, 4 months ago