Gitiles
Code Review Sign In
korap.ids-mannheim.de / KorAP / KorAP-XML-TEI / 95bc98aeb0a2e0585d5f8c4a5114ea282d6697d4 / t / tokenization.t
  1. d962747 Establish tokenizer objects for aggressive and conservative base tokenization by Akron · 4 years, 5 months ago
  2. 510a88c Minor speedup in tokenization by merging array pushes by Akron · 4 years, 5 months ago
  3. eac374d Separate dummy tokenization from main script with minimal changes by Akron · 4 years, 5 months ago