Gitiles
Code Review
Sign In
korap.ids-mannheim.de
/
KorAP
/
Datok
/
96fdc9be352df941e7304b2e5ceeb8e2078d7307
/
testdata
/
tokenizer.datok
a854faa
Introduce EOT (end-of-transmission) marker
by Akron
· 3 years, 2 months ago
4c2a1ad
Introduce XML tests
by Akron
· 3 years, 4 months ago
235ea12
Update generated tokenizers
by Akron
· 3 years, 4 months ago
e184a91
Add new generated automata
by Akron
· 3 years, 5 months ago
03c92fe
Support for tokenend MCS symbol
by Akron
· 3 years, 5 months ago
b4bbb47
Added sentence splitter capabilities
by Akron
· 3 years, 5 months ago
3a063ef
Fix loading routine
by Akron
· 3 years, 5 months ago