blob: f5904b6525540edffb2789ba917777b1109ad580 [file] [log] [blame]
Marc Kupietzd254f5c2025-04-16 10:37:08 +02001 - Make KorAP-Tokenizer heap size configurable via environment
2 variable KORAPXMLTEI_TOKENIZER_HEAP_SIZE.
3
Marc Kupietz5b3f1d82024-07-05 17:50:55 +020042.6.0 2024-11-11
Akron132bdeb2024-06-06 14:28:56 +02005 - Add -o parameter.
Akron6b1f26b2024-09-19 11:35:32 +02006 - Add support for inline dependency relations.
Marc Kupietzfc3a0ee2024-07-05 16:58:16 +02007 - Add support for --auto-textsigle.
Marc Kupietz5b3f1d82024-07-05 17:50:55 +02008 - Add support for multiple input files.
Akron132bdeb2024-06-06 14:28:56 +02009
Akron6b1f26b2024-09-19 11:35:32 +0200102.5.0 2024-01-24
Akron568b22f2024-01-23 10:12:34 +010011 - Upgrade minimal Perl version to 5.36 to improve
12 unicode handling.
13 - Upgrade KorAP-Tokenizer to v2.2.5 and Java to 17 to
14 improve unicode handling.
15
Akronec503252023-04-24 18:03:17 +0200162.4.4 2023-04-25
17 - Allow line-breaks in text only lines.
18
Akron72f4a882023-03-02 09:48:14 +0100192.4.3 2023-03-02
20 - Allow closing elements to start with "text".
21
Akron997aa222023-02-10 11:26:28 +0100222.4.2 2023-02-10
23 - Improve checks for numerical annotation bounds.
24
Akronfcff7342023-02-07 14:05:15 +0100252.4.1 2023-02-07
26 - Fix test.
27
Akronfc2a82a2023-02-07 11:29:11 +0100282.4.0 2023-02-07
Marc Kupietza671ae52022-12-22 16:28:14 +010029 - Conversion of standard TEI P5 should now work, at least
30 in some cases.
31 - Option --xmlid-to-textsigle <from-regex>@<to-c/to-d/to-t>
32 added to convert standard P5 text id attributes to I5
33 sigles with three parts.
Akronb93fabb2023-01-13 12:05:44 +010034 - Add --no-tokenizer parameter as a requirement
35 for relying on inline tokens only.
Marc Kupietza671ae52022-12-22 16:28:14 +010036
Akron2520a342022-03-29 18:18:05 +0200372.3.4 2022-11-09
Akron85269c02022-11-07 14:03:31 +010038 - Improve stability of XML entity replacement.
Akron2520a342022-03-29 18:18:05 +020039 - Check version for script and KorAP-Tokenizer
40 library when requested.
Akron85269c02022-11-07 14:03:31 +010041
Akron2520a342022-03-29 18:18:05 +0200422.3.3 2022-03-30
Akronbd4281e2022-03-28 08:31:40 +020043 - Load KorAP-Tokenizer only on request.
44
Akrond708a612022-03-21 16:00:01 +0100452.3.2 2022-03-23
Akron540fd622022-03-21 18:20:05 +010046 - Do not reference metadata.xml
Akrond708a612022-03-21 16:00:01 +010047 - Remove schema references from header files.
Akron4ee372a2022-02-24 17:54:24 +010048 - Improve test suite for unability to use
49 KorAP-Tokenizer.
Akron540fd622022-03-21 18:20:05 +010050
Marc Kupietz0bca4f12022-01-14 13:24:22 +0100512.3.1 2022-01-14 Release
Akrona3799ce2021-10-15 16:27:30 +020052 - Improve script handling of broken data
53 - Improve handling of unknown header types
54 - Check for valid sigles to avoid broken directories
55 - Introduce exclusivity for inline tokens handling.
Akrona2cb2812021-10-30 10:29:08 +020056 - Use single dash for STDIN.
Marc Kupietz0bca4f12022-01-14 13:24:22 +010057 - Update KorAP-Tokenizer to v2.2.2 (single quote, "du." bug fixes)
Akrona3799ce2021-10-15 16:27:30 +020058
592.2.0 2021-08-26 Release
Akrond658df72021-02-18 18:58:56 +010060 - Remove unnecessary branch in recursive call
Akrondd0be8f2021-02-18 19:29:41 +010061 - Support inline-structures parameter
Akron26a71522021-02-19 10:27:37 +010062 - Introduce --base-foundry, --data-file, and --header-file parameters
Akron91705d72021-02-19 10:59:45 +010063 - Introduce --tokens-file parameter
Akron75d63142021-02-23 18:40:56 +010064 - Introduce --skip-inline-tokens parameter
Akrond3e1d282021-02-24 14:51:27 +010065 - Minor cleanups and improvements
Akron54c3ff12021-02-25 11:33:37 +010066 - Introduce --skip-inline-tags parameter
Akroneb12e232021-02-25 13:49:50 +010067 - Introduce KorAP::XML::TEI::Inline class
Akron692d17d2021-03-05 13:21:03 +010068 - Introduce --skip-inline-token-annotations parameter
69 - Deprecate KORAPXMLTEI_INLINE environment variable
70 in favor of --skip-inline-token-annotations
Akrond658df72021-02-18 18:58:56 +010071
Akrona3799ce2021-10-15 16:27:30 +0200721.0.0 2021-02-18 Release
Akrond3e1d282021-02-24 14:51:27 +010073 - -s option added that uses sentence boundaries
74 provided by the KorAP tokenizer (-tk)
Marc Kupietza1421f02021-02-18 15:32:38 +010075 - Tokenizer invocation comments removed from KorAP XML output
76 - Indentation of </span> tags fixed
Akrond3e1d282021-02-24 14:51:27 +010077 - Character entities used in DeReKo are automatically
78 replaced by their corresponding characters
Marc Kupietza1421f02021-02-18 15:32:38 +010079 - Resources defined in Makefile
80 - Fixed possible IO deadlock with KorAP tokenizer
Akron4e3c7e32021-02-18 15:19:53 +010081 - Simplified debugging by combining with X::C::T line numbers
Akron1a5271a2021-02-18 13:18:15 +010082 - Support inline-tokens parameter
Akronf8088e62021-02-18 16:18:59 +010083 - Move verbose code documentation to trailing
84 script section
Marc Kupietzeed4cb12021-02-17 19:39:32 +010085
Akronf7084c42021-01-07 10:25:22 +0100860.03 2021-01-12
Marc Kupietzb505d442021-01-06 16:40:29 +010087 - Update KorAP-Tokenizer to released 2.0 version
Akronf7084c42021-01-07 10:25:22 +010088 - Improve test suite for recent version
89 of Mojolicious.
90
Marc Kupietz44b1f252020-11-26 16:31:40 +0100910.02 2020-11-27
Akronf7084c42021-01-07 10:25:22 +010092 - Update KorAP-Tokenizer to v2.0.0.
Akroneaa96232020-10-15 17:06:15 +020093 - Switch input encoding based on XML
94 processing instruction.
Marc Kupietz44b1f252020-11-26 16:31:40 +010095 - Fix handling of UTF-8 in sigles.
Akroneaa96232020-10-15 17:06:15 +020096
Akron0c41ab32020-09-29 07:33:33 +0200970.01 2020-09-28
98 - Initial release to GitHub.