tree 9f39e305982ab20660aaa09b669b8accfe00ddba
parent 95612c3d5a2aeea3d81915b7a846a8dc69d46df4
author Akron <nils@diewald-online.de> 1594306389 +0200
committer Akron <nils@diewald-online.de> 1594306866 +0200

Establish tokenizer objects for aggressive and conservative base tokenization

Change-Id: I702098185b0b6292c73217268d4516d55a2f95b5