Gitiles
Code Review Sign In
korap.ids-mannheim.de / KorAP / KorAP-Tokenizer / 67eed1c2ffa2a4b61958071d61f4f84d9bd10861 / src / main / java / de / ids_mannheim / korap / tokenizer
  1. 67eed1c Build language specific tokenizers: de, en by Marc Kupietz · 6 years ago
  2. 8e197f3 Allow setting input encoding explicitely whith -e <encoding> by Marc Kupietz · 6 years ago
  3. 751868b Make tokenizer implementation exchangeable by Marc Kupietz · 6 years ago
  4. b9f45e0 Rename tokenizer class to KorAPDFATokenizer by Marc Kupietz · 6 years ago
  5. c419d5b Add new command line options using picocli and sanitize code by Marc Kupietz · 6 years ago
  6. 8192509 Use original Span class and implement Tokenizer interface from OpenNLP by Marc Kupietz · 6 years ago
  7. 478632e Clean up code by Marc Kupietz · 6 years ago
  8. 45dc0fe Add Apache copyright NOTICE by Marc Kupietz · 6 years ago
  9. 3367773 Initial import from private/Ingestion by Marc Kupietz · 6 years ago