Fix possible IO deadlocks with KorAP tokenizer
Text separators should always have a newline in front of artificial EOTs
to make sure they are recognized and to avoid them being consumed
by regular expressions for tokens.
Change-Id: I528c903904da50312a7472c7a34775476b0955be
diff --git a/Changes b/Changes
index f61104b..70a3538 100644
--- a/Changes
+++ b/Changes
@@ -3,6 +3,8 @@
- indentation of </span> tags fixed
- character entities used in DeReKo are automatically replaced by their corresponding characters
- resources defined in Makefile
+ - fixed possible IO deadlock with KorAP tokenizer
+
0.03 2021-01-12
- Update KorAP-Tokenizer to released 2.0 version
- Improve test suite for recent version