| commit | 235ea12bd2814b6cdf4bb4c275e895d3f0588ae0 | [log] [tgz] |
|---|---|---|
| author | Akron <nils@diewald-online.de> | Fri Aug 27 11:35:14 2021 +0200 |
| committer | Akron <nils@diewald-online.de> | Fri Aug 27 11:35:14 2021 +0200 |
| tree | 8bb4f025e093cbfce80cd955e633429abbd5b43e | |
| parent | 92704eb56033b5de311ca70c75f61e34def2c620 [diff] |
Update generated tokenizers
diff --git a/testdata/tokenizer.datok b/testdata/tokenizer.datok index 5dac53e..496d564 100644 --- a/testdata/tokenizer.datok +++ b/testdata/tokenizer.datok Binary files differ
diff --git a/testdata/tokenizer.fst b/testdata/tokenizer.fst index 615fe00..583b0e2 100644 --- a/testdata/tokenizer.fst +++ b/testdata/tokenizer.fst Binary files differ