Gitiles
Code Review
Sign In
korap.ids-mannheim.de
/
KorAP
/
Datok
/
df37a5595ed3e404e25918ec1c2c8d9f7a9608a5
df37a55
Fixed benchmark tests
by Akron
· 3 years, 11 months ago
4c2a1ad
Introduce XML tests
by Akron
· 3 years, 11 months ago
34dbe97
Ignore MCS transitions instead of failing
by Akron
· 3 years, 11 months ago
0630be5
Fix parsing of end states
by Akron
· 3 years, 11 months ago
235ea12
Update generated tokenizers
by Akron
· 4 years ago
92704eb
Ignore tokenend accepting transitions
by Akron
· 4 years ago
4fa28b3
Introduce TransCount method
by Akron
· 4 years ago
31f3c06
Ignore MCS in sigma if not used in the transducer
by Akron
· 4 years ago
de18e90
Minor optimization on edges
by Akron
· 4 years ago
6f1c16c
Added benchmark for double array creation
by Akron
· 4 years ago
3de361e
Improved newline and abbreviation handling
by Akron
· 4 years ago
ea46e8a
Add ASCII fast lookup to sigma
by Akron
· 4 years ago
f1a1650
Turn uint32 array in bc array
by Akron
· 4 years ago
e61380b
Added some minor comments
by Akron
· 4 years ago
91bd715
Add more reference to Readme
by Akron
· 4 years ago
31cc307
Added readme file
by Akron
· 4 years ago
1e10d00
Remove dir/Dir from abbreviation file
by Akron
· 4 years ago
527c10c
Replace zerolog with log
by Akron
· 4 years ago
bb4aac5
Optimize loading of datok files
by Akron
· 4 years ago
7e269d4
Added conversion to the command line tool
by Akron
· 4 years ago
8e1d69b
Introduced command line tool
by Akron
· 4 years ago
01912fc
Remove unnecessary allocation for buffer recasting
by Akron
· 4 years ago
4db3ecf
Change exit operations to returning nil
by Akron
· 4 years ago
bd40680
Added transducing benchmark
by Akron
· 4 years ago
e184a91
Add new generated automata
by Akron
· 4 years ago
ec835ad
Remove Match() method
by Akron
· 4 years ago
57d0161
Add known terms with special characters
by Akron
· 4 years ago
e8837b5
Add file scheme
by Akron
· 4 years ago
fd92d7e
Update abbreviations according to KorAP-Tokenizer
by Akron
· 4 years ago
a0bded5
Add ordinals
by Akron
· 4 years ago
4af79f1
Added support for streetnames
by Akron
· 4 years ago
310905f
Add foma sources
by Akron
· 4 years ago
03ca425
Adopt tokenizer tests from KorAP-Tokenizer
by Akron
· 4 years ago
6e70dc8
Fix sentence splitting tests
by Akron
· 4 years ago
1594cb8
Fix sentence splitting
by Akron
· 4 years ago
c5d8d43
Fix check on final states
by Akron
· 4 years ago
b7e1f13
Simplify transducer (single test broken)
by Akron
· 4 years ago
df0a3ef
Correctly handle final data
by Akron
· 4 years ago
439f4ec
Cleanup
by Akron
· 4 years ago
03c92fe
Support for tokenend MCS symbol
by Akron
· 4 years ago
b4bbb47
Added sentence splitter capabilities
by Akron
· 4 years ago
3610f10
Introduce buffer with single epsilon backtrack
by Akron
· 4 years ago
3a063ef
Fix loading routine
by Akron
· 4 years ago
524c543
Fix sigma to start with 1
by Akron
· 4 years ago
3f8571a
Support reader/writer in transduce and add load
by Akron
· 4 years ago
84d68e6
Support tokenend handling in transducing
by Akron
· 4 years ago
2a4b929
Switch to 2 leading bits (30 bit addresses)
by Akron
· 4 years ago
068874c
Introduce nontoken handling in preliminary transducer
by Akron
· 4 years ago
83e75a2
Introduce nontoken information
by Akron
· 4 years ago
03a3c61
Rename loadLevel to loadFactor
by Akron
· 4 years ago
3fdfec6
Turn states into uint32 pairs
by Akron
· 4 years ago
64ffd9a
Restructure and rename methods
by Akron
· 4 years ago
c17f1ca
Turn special sigma values into properties
by Akron
· 4 years ago
6247a5d
Add serialization method
by Akron
· 4 years ago
773b1ef
Cache loadlevel
by Akron
· 4 years ago
d66a926
Add load factor
by Akron
· 4 years ago
f2120ca
Split Tokenizer and DaTokenizer
by Akron
· 4 years ago
c9d84a6
Sort alphabet prior to xCheck
by Akron
· 4 years ago
740f3d7
Cleanup code
by Akron
· 4 years ago
49d27ee
Fix epsilon handling in match operation
by Akron
· 4 years ago
465a099
Add support for epsilon symbols
by Akron
· 4 years ago
730a79c
Support unknown and identity symbols
by Akron
· 4 years ago
75ebe7f
Fix foma format parser
by Akron
· 4 years ago
8ef408b
Initial commit
by Akron
· 4 years ago