tree 1b91a0caa47a320a51a00bd2654086c24c635614
parent 59289de1b13d069ee2836a5acb1d1bf60e9eafbe
author Marc Kupietz <kupietz@ids-mannheim.de> 1599306682 +0200
committer Peter Harders <harders@ids-mannheim.de> 1600790549 +0200

Send <EOT>\n to external tokenizer for separating texts

This is now the standard for the KorAP tokenizer.

Change-Id: I30c2d6ca82211b1d312364899d4f56ea7908b4f8