Gitiles
Code Review
Sign In
korap.ids-mannheim.de
/
KorAP
/
Datok
/
2612f996e02898aa9cbd4964670cdd7446dd0813
/
token_writer.go
0f087ea
Parse command line options as bit flags
by Akron
· 3 years, 1 month ago
fceddb6
Add sentence flags (for printing and offsets)
by Akron
· 3 years, 1 month ago
a9e0c42
Introduce --[no]-tokens flag
by Akron
· 3 years, 1 month ago
e9431ec
Ignore newline after EOT with a flag
by Akron
· 3 years, 1 month ago
8cc2dd9
Fix buffer rewind at end of transmission
by Akron
· 3 years, 1 month ago
4f6b28c
Support token offsets in token writer
by Akron
· 3 years, 1 month ago
32416ce
Support offsets in token writer
by Akron
· 3 years, 1 month ago
a854faa
Introduce EOT (end-of-transmission) marker
by Akron
· 3 years, 1 month ago
e396a93
Introduce token_writer object
by Akron
· 3 years, 1 month ago