| commit | d962747a4ac7e02a6040fad736e4a8a45a6b4431 | [log] [tgz] |
|---|---|---|
| author | Akron <nils@diewald-online.de> | Thu Jul 09 16:53:09 2020 +0200 |
| committer | Akron <nils@diewald-online.de> | Thu Jul 09 17:01:06 2020 +0200 |
| tree | 9f39e305982ab20660aaa09b669b8accfe00ddba | |
| parent | 95612c3d5a2aeea3d81915b7a846a8dc69d46df4 [diff] [blame] |
Establish tokenizer objects for aggressive and conservative base tokenization Change-Id: I702098185b0b6292c73217268d4516d55a2f95b5
diff --git a/.gitignore b/.gitignore index 9071e73..b0955d2 100644 --- a/.gitignore +++ b/.gitignore
@@ -6,3 +6,4 @@ *~ .* !.gitignore +/sandbox \ No newline at end of file