Gitiles
Code Review
Sign In
korap.ids-mannheim.de
/
KorAP
/
Datok
/
b428755ff720aefb37ebbce920cd76df61879d30
/
testdata
/
tokenizer.matok
b428755
Support punctuation after quotes
by Akron
· 2 years, 8 months ago
v0.1.4
4222ac8
Improve handling of ellipsis
by Akron
· 2 years, 9 months ago
e200841
Further improve speech rule for eos with more quotation marks
by Akron
· 2 years, 9 months ago
e96895f
Improve handling of sentence splits including speech
by Akron
· 2 years, 9 months ago
4ec8cec
Prepare first official release
by Akron
· 3 years ago
v0.1.0
fac8abc
Reorder longest match operator and update models
by Akron
· 3 years, 1 month ago
17984c8
Improving time parsing
by Akron
· 3 years, 1 month ago
f6bdfdb
Add trimming at the beginning of a text
by Akron
· 3 years, 1 month ago
a854faa
Introduce EOT (end-of-transmission) marker
by Akron
· 3 years, 1 month ago
094a4e8
Use serialized matrix representation in test suite
by Akron
· 3 years, 2 months ago