Fail hard on tokenization problems now
Change-Id: I82ba79c53b02d3335650db31a589887dba986aaf
diff --git a/Changes b/Changes
index 4054bd0..f6e51cb 100644
--- a/Changes
+++ b/Changes
@@ -1,4 +1,4 @@
-0.37 2019-02-21
+0.37 2019-03-06
- Support for 'koral:field' array.
- Support for Koral versioning.
- Added tests for english sources.
@@ -8,6 +8,8 @@
on directory archiving.
- Remove extract_text and extract_doc in
favor of extract_sigle for archives.
+ - Stop file processing when base tokenization
+ is wrong.
0.36 2019-01-22
- Support for non-word tokens (fixes #5).