blob: 03ccb1d6b6a91fb7495244fd80440e0d376253f0 [file] [log] [blame]
Marc Kupietz4ad648e2025-12-10 10:38:46 +010012.6.2 2025-12-10
2 - Upgrade KorAP-Tokenizer to v2.3.0 (resolves issues with
3 gendersternchen after hyphens, emoji clusters, and Wikipedia templates).
4 - Upgrade Java dependency to 21.
Marc Kupietz2115ecc2025-12-10 11:37:03 +01005 - Added --progress option.
Marc Kupietz4ad648e2025-12-10 10:38:46 +01006
Marc Kupietzb6fd6bc2025-04-16 12:47:26 +020072.6.1 2025-04-16
8 - Fix ASCII entity resolution.
Marc Kupietzd254f5c2025-04-16 10:37:08 +02009 - Make KorAP-Tokenizer heap size configurable via environment
10 variable KORAPXMLTEI_TOKENIZER_HEAP_SIZE.
11
Marc Kupietz5b3f1d82024-07-05 17:50:55 +0200122.6.0 2024-11-11
Akron132bdeb2024-06-06 14:28:56 +020013 - Add -o parameter.
Akron6b1f26b2024-09-19 11:35:32 +020014 - Add support for inline dependency relations.
Marc Kupietzfc3a0ee2024-07-05 16:58:16 +020015 - Add support for --auto-textsigle.
Marc Kupietz5b3f1d82024-07-05 17:50:55 +020016 - Add support for multiple input files.
Akron132bdeb2024-06-06 14:28:56 +020017
Akron6b1f26b2024-09-19 11:35:32 +0200182.5.0 2024-01-24
Akron568b22f2024-01-23 10:12:34 +010019 - Upgrade minimal Perl version to 5.36 to improve
20 unicode handling.
21 - Upgrade KorAP-Tokenizer to v2.2.5 and Java to 17 to
22 improve unicode handling.
23
Akronec503252023-04-24 18:03:17 +0200242.4.4 2023-04-25
25 - Allow line-breaks in text only lines.
26
Akron72f4a882023-03-02 09:48:14 +0100272.4.3 2023-03-02
28 - Allow closing elements to start with "text".
29
Akron997aa222023-02-10 11:26:28 +0100302.4.2 2023-02-10
31 - Improve checks for numerical annotation bounds.
32
Akronfcff7342023-02-07 14:05:15 +0100332.4.1 2023-02-07
34 - Fix test.
35
Akronfc2a82a2023-02-07 11:29:11 +0100362.4.0 2023-02-07
Marc Kupietza671ae52022-12-22 16:28:14 +010037 - Conversion of standard TEI P5 should now work, at least
38 in some cases.
39 - Option --xmlid-to-textsigle <from-regex>@<to-c/to-d/to-t>
40 added to convert standard P5 text id attributes to I5
41 sigles with three parts.
Akronb93fabb2023-01-13 12:05:44 +010042 - Add --no-tokenizer parameter as a requirement
43 for relying on inline tokens only.
Marc Kupietza671ae52022-12-22 16:28:14 +010044
Akron2520a342022-03-29 18:18:05 +0200452.3.4 2022-11-09
Akron85269c02022-11-07 14:03:31 +010046 - Improve stability of XML entity replacement.
Akron2520a342022-03-29 18:18:05 +020047 - Check version for script and KorAP-Tokenizer
48 library when requested.
Akron85269c02022-11-07 14:03:31 +010049
Akron2520a342022-03-29 18:18:05 +0200502.3.3 2022-03-30
Akronbd4281e2022-03-28 08:31:40 +020051 - Load KorAP-Tokenizer only on request.
52
Akrond708a612022-03-21 16:00:01 +0100532.3.2 2022-03-23
Akron540fd622022-03-21 18:20:05 +010054 - Do not reference metadata.xml
Akrond708a612022-03-21 16:00:01 +010055 - Remove schema references from header files.
Akron4ee372a2022-02-24 17:54:24 +010056 - Improve test suite for unability to use
57 KorAP-Tokenizer.
Akron540fd622022-03-21 18:20:05 +010058
Marc Kupietz0bca4f12022-01-14 13:24:22 +0100592.3.1 2022-01-14 Release
Akrona3799ce2021-10-15 16:27:30 +020060 - Improve script handling of broken data
61 - Improve handling of unknown header types
62 - Check for valid sigles to avoid broken directories
63 - Introduce exclusivity for inline tokens handling.
Akrona2cb2812021-10-30 10:29:08 +020064 - Use single dash for STDIN.
Marc Kupietz0bca4f12022-01-14 13:24:22 +010065 - Update KorAP-Tokenizer to v2.2.2 (single quote, "du." bug fixes)
Akrona3799ce2021-10-15 16:27:30 +020066
672.2.0 2021-08-26 Release
Akrond658df72021-02-18 18:58:56 +010068 - Remove unnecessary branch in recursive call
Akrondd0be8f2021-02-18 19:29:41 +010069 - Support inline-structures parameter
Akron26a71522021-02-19 10:27:37 +010070 - Introduce --base-foundry, --data-file, and --header-file parameters
Akron91705d72021-02-19 10:59:45 +010071 - Introduce --tokens-file parameter
Akron75d63142021-02-23 18:40:56 +010072 - Introduce --skip-inline-tokens parameter
Akrond3e1d282021-02-24 14:51:27 +010073 - Minor cleanups and improvements
Akron54c3ff12021-02-25 11:33:37 +010074 - Introduce --skip-inline-tags parameter
Akroneb12e232021-02-25 13:49:50 +010075 - Introduce KorAP::XML::TEI::Inline class
Akron692d17d2021-03-05 13:21:03 +010076 - Introduce --skip-inline-token-annotations parameter
77 - Deprecate KORAPXMLTEI_INLINE environment variable
78 in favor of --skip-inline-token-annotations
Akrond658df72021-02-18 18:58:56 +010079
Akrona3799ce2021-10-15 16:27:30 +0200801.0.0 2021-02-18 Release
Akrond3e1d282021-02-24 14:51:27 +010081 - -s option added that uses sentence boundaries
82 provided by the KorAP tokenizer (-tk)
Marc Kupietza1421f02021-02-18 15:32:38 +010083 - Tokenizer invocation comments removed from KorAP XML output
84 - Indentation of </span> tags fixed
Akrond3e1d282021-02-24 14:51:27 +010085 - Character entities used in DeReKo are automatically
86 replaced by their corresponding characters
Marc Kupietza1421f02021-02-18 15:32:38 +010087 - Resources defined in Makefile
88 - Fixed possible IO deadlock with KorAP tokenizer
Akron4e3c7e32021-02-18 15:19:53 +010089 - Simplified debugging by combining with X::C::T line numbers
Akron1a5271a2021-02-18 13:18:15 +010090 - Support inline-tokens parameter
Akronf8088e62021-02-18 16:18:59 +010091 - Move verbose code documentation to trailing
92 script section
Marc Kupietzeed4cb12021-02-17 19:39:32 +010093
Akronf7084c42021-01-07 10:25:22 +0100940.03 2021-01-12
Marc Kupietzb505d442021-01-06 16:40:29 +010095 - Update KorAP-Tokenizer to released 2.0 version
Akronf7084c42021-01-07 10:25:22 +010096 - Improve test suite for recent version
97 of Mojolicious.
98
Marc Kupietz44b1f252020-11-26 16:31:40 +0100990.02 2020-11-27
Akronf7084c42021-01-07 10:25:22 +0100100 - Update KorAP-Tokenizer to v2.0.0.
Akroneaa96232020-10-15 17:06:15 +0200101 - Switch input encoding based on XML
102 processing instruction.
Marc Kupietz44b1f252020-11-26 16:31:40 +0100103 - Fix handling of UTF-8 in sigles.
Akroneaa96232020-10-15 17:06:15 +0200104
Akron0c41ab32020-09-29 07:33:33 +02001050.01 2020-09-28
106 - Initial release to GitHub.