- 96fdc9b Fix TokenWriter regarding sentence boundaries and remove simple TokenWriter by Akron · 3 years, 1 month ago
- 2612f99 Improve command help page by Akron · 3 years, 1 month ago
- 0f087ea Parse command line options as bit flags by Akron · 3 years, 1 month ago
- fceddb6 Add sentence flags (for printing and offsets) by Akron · 3 years, 1 month ago
- a9e0c42 Introduce --[no]-tokens flag by Akron · 3 years, 1 month ago
- e9431ec Ignore newline after EOT with a flag by Akron · 3 years, 1 month ago
- 4f6b28c Support token offsets in token writer by Akron · 3 years, 1 month ago
- 941f215 Support both matrix and da in the command by Akron · 3 years, 2 months ago
- 7f1097f Rename datokenizer to datok by Akron · 3 years, 2 months ago
- 527c10c Replace zerolog with log by Akron · 3 years, 3 months ago
- 7e269d4 Added conversion to the command line tool by Akron · 3 years, 3 months ago
- 8e1d69b Introduced command line tool by Akron · 3 years, 3 months ago