Gitiles
Code Review
Sign In
korap.ids-mannheim.de
/
KorAP
/
KorAP-XML-TEI
/
d962747a4ac7e02a6040fad736e4a8a45a6b4431
/
t
/
tokenization.t
d962747
Establish tokenizer objects for aggressive and conservative base tokenization
by Akron
· 4 years, 5 months ago
510a88c
Minor speedup in tokenization by merging array pushes
by Akron
· 4 years, 5 months ago
eac374d
Separate dummy tokenization from main script with minimal changes
by Akron
· 4 years, 5 months ago