- 96bd87c Improve systematicity of options -p, -s, --[no-]tokens by Marc Kupietz · 3 years, 8 months ago
- cf9b5f5 Add heuristcis for distinguishing I. as abbrevation vs PPER / CARD by Marc Kupietz · 4 years, 1 month ago
- e3282b0 Accept URLs starting with "www." without URI scheme by Marc Kupietz · 4 years, 1 month ago
- 4fb896a Amend English abbreviation macro by Marc Kupietz · 4 years, 1 month ago
- f5a7e04 Add French tokenizer (-l fr) by Marc Kupietz · 4 years, 1 month ago
- 74141b3 Add -l command line option to choose language by Marc Kupietz · 4 years, 2 months ago
- ce48102 Recognize {LETTER}+str. as abbreviation for Straße in de-tokenizer by Marc Kupietz · 4 years, 1 month ago
- 67eed1c Build language specific tokenizers: de, en by Marc Kupietz · 4 years, 2 months ago
- 8e197f3 Allow setting input encoding explicitely whith -e <encoding> by Marc Kupietz · 4 years, 2 months ago
- 751868b Make tokenizer implementation exchangeable by Marc Kupietz · 4 years, 2 months ago
- b9f45e0 Rename tokenizer class to KorAPDFATokenizer by Marc Kupietz · 4 years, 2 months ago
- c419d5b Add new command line options using picocli and sanitize code by Marc Kupietz · 4 years, 2 months ago
- 783e2a2 Ignore quoted email names like "John Doe"@xx.com by Marc Kupietz · 4 years, 2 months ago
- 571c194 Empty text (<EOT><EOT>) -> empty output line by Marc Kupietz · 4 years, 3 months ago
- 793f85d Add first tests for IPC invocation scenario by Marc Kupietz · 4 years, 3 months ago
- b2666fc Implement sentence splitter by Marc Kupietz · 4 years, 3 months ago
- 07d9714 Move tests to proper location by Marc Kupietz · 4 years, 3 months ago
- 478632e Clean up code by Marc Kupietz · 4 years, 3 months ago
- 3367773 Initial import from private/Ingestion by Marc Kupietz · 4 years, 3 months ago