blob: 86b44fb984ae083e03dea74584c6bb9801a72190 [file] [log] [blame]
Marc Kupietzff061ef2026-03-05 09:59:35 +010012.7.1 2026-03-05
2 - Fix parser error when closing body and text tags
3 appear on the same line.
4
Marc Kupietz67ee44e2026-03-03 10:04:48 +010052.7.0 2026-03-03
6 - Upgrade KorAP-Tokenizer to v2.4.0
7 with fixes for soft hyphens, thousands separators, and
8 support for German sensitive spelling forms, separeted by colons, slashes, and brackets.
9
Marc Kupietz4ad648e2025-12-10 10:38:46 +0100102.6.2 2025-12-10
11 - Upgrade KorAP-Tokenizer to v2.3.0 (resolves issues with
12 gendersternchen after hyphens, emoji clusters, and Wikipedia templates).
13 - Upgrade Java dependency to 21.
Marc Kupietz2115ecc2025-12-10 11:37:03 +010014 - Added --progress option.
Marc Kupietz4ad648e2025-12-10 10:38:46 +010015
Marc Kupietzb6fd6bc2025-04-16 12:47:26 +0200162.6.1 2025-04-16
17 - Fix ASCII entity resolution.
Marc Kupietzd254f5c2025-04-16 10:37:08 +020018 - Make KorAP-Tokenizer heap size configurable via environment
19 variable KORAPXMLTEI_TOKENIZER_HEAP_SIZE.
20
Marc Kupietz5b3f1d82024-07-05 17:50:55 +0200212.6.0 2024-11-11
Akron132bdeb2024-06-06 14:28:56 +020022 - Add -o parameter.
Akron6b1f26b2024-09-19 11:35:32 +020023 - Add support for inline dependency relations.
Marc Kupietzfc3a0ee2024-07-05 16:58:16 +020024 - Add support for --auto-textsigle.
Marc Kupietz5b3f1d82024-07-05 17:50:55 +020025 - Add support for multiple input files.
Akron132bdeb2024-06-06 14:28:56 +020026
Akron6b1f26b2024-09-19 11:35:32 +0200272.5.0 2024-01-24
Akron568b22f2024-01-23 10:12:34 +010028 - Upgrade minimal Perl version to 5.36 to improve
29 unicode handling.
30 - Upgrade KorAP-Tokenizer to v2.2.5 and Java to 17 to
31 improve unicode handling.
32
Akronec503252023-04-24 18:03:17 +0200332.4.4 2023-04-25
34 - Allow line-breaks in text only lines.
35
Akron72f4a882023-03-02 09:48:14 +0100362.4.3 2023-03-02
37 - Allow closing elements to start with "text".
38
Akron997aa222023-02-10 11:26:28 +0100392.4.2 2023-02-10
40 - Improve checks for numerical annotation bounds.
41
Akronfcff7342023-02-07 14:05:15 +0100422.4.1 2023-02-07
43 - Fix test.
44
Akronfc2a82a2023-02-07 11:29:11 +0100452.4.0 2023-02-07
Marc Kupietza671ae52022-12-22 16:28:14 +010046 - Conversion of standard TEI P5 should now work, at least
47 in some cases.
48 - Option --xmlid-to-textsigle <from-regex>@<to-c/to-d/to-t>
49 added to convert standard P5 text id attributes to I5
50 sigles with three parts.
Akronb93fabb2023-01-13 12:05:44 +010051 - Add --no-tokenizer parameter as a requirement
52 for relying on inline tokens only.
Marc Kupietza671ae52022-12-22 16:28:14 +010053
Akron2520a342022-03-29 18:18:05 +0200542.3.4 2022-11-09
Akron85269c02022-11-07 14:03:31 +010055 - Improve stability of XML entity replacement.
Akron2520a342022-03-29 18:18:05 +020056 - Check version for script and KorAP-Tokenizer
57 library when requested.
Akron85269c02022-11-07 14:03:31 +010058
Akron2520a342022-03-29 18:18:05 +0200592.3.3 2022-03-30
Akronbd4281e2022-03-28 08:31:40 +020060 - Load KorAP-Tokenizer only on request.
61
Akrond708a612022-03-21 16:00:01 +0100622.3.2 2022-03-23
Akron540fd622022-03-21 18:20:05 +010063 - Do not reference metadata.xml
Akrond708a612022-03-21 16:00:01 +010064 - Remove schema references from header files.
Akron4ee372a2022-02-24 17:54:24 +010065 - Improve test suite for unability to use
66 KorAP-Tokenizer.
Akron540fd622022-03-21 18:20:05 +010067
Marc Kupietz0bca4f12022-01-14 13:24:22 +0100682.3.1 2022-01-14 Release
Akrona3799ce2021-10-15 16:27:30 +020069 - Improve script handling of broken data
70 - Improve handling of unknown header types
71 - Check for valid sigles to avoid broken directories
72 - Introduce exclusivity for inline tokens handling.
Akrona2cb2812021-10-30 10:29:08 +020073 - Use single dash for STDIN.
Marc Kupietz0bca4f12022-01-14 13:24:22 +010074 - Update KorAP-Tokenizer to v2.2.2 (single quote, "du." bug fixes)
Akrona3799ce2021-10-15 16:27:30 +020075
762.2.0 2021-08-26 Release
Akrond658df72021-02-18 18:58:56 +010077 - Remove unnecessary branch in recursive call
Akrondd0be8f2021-02-18 19:29:41 +010078 - Support inline-structures parameter
Akron26a71522021-02-19 10:27:37 +010079 - Introduce --base-foundry, --data-file, and --header-file parameters
Akron91705d72021-02-19 10:59:45 +010080 - Introduce --tokens-file parameter
Akron75d63142021-02-23 18:40:56 +010081 - Introduce --skip-inline-tokens parameter
Akrond3e1d282021-02-24 14:51:27 +010082 - Minor cleanups and improvements
Akron54c3ff12021-02-25 11:33:37 +010083 - Introduce --skip-inline-tags parameter
Akroneb12e232021-02-25 13:49:50 +010084 - Introduce KorAP::XML::TEI::Inline class
Akron692d17d2021-03-05 13:21:03 +010085 - Introduce --skip-inline-token-annotations parameter
86 - Deprecate KORAPXMLTEI_INLINE environment variable
87 in favor of --skip-inline-token-annotations
Akrond658df72021-02-18 18:58:56 +010088
Akrona3799ce2021-10-15 16:27:30 +0200891.0.0 2021-02-18 Release
Akrond3e1d282021-02-24 14:51:27 +010090 - -s option added that uses sentence boundaries
91 provided by the KorAP tokenizer (-tk)
Marc Kupietza1421f02021-02-18 15:32:38 +010092 - Tokenizer invocation comments removed from KorAP XML output
93 - Indentation of </span> tags fixed
Akrond3e1d282021-02-24 14:51:27 +010094 - Character entities used in DeReKo are automatically
95 replaced by their corresponding characters
Marc Kupietza1421f02021-02-18 15:32:38 +010096 - Resources defined in Makefile
97 - Fixed possible IO deadlock with KorAP tokenizer
Akron4e3c7e32021-02-18 15:19:53 +010098 - Simplified debugging by combining with X::C::T line numbers
Akron1a5271a2021-02-18 13:18:15 +010099 - Support inline-tokens parameter
Akronf8088e62021-02-18 16:18:59 +0100100 - Move verbose code documentation to trailing
101 script section
Marc Kupietzeed4cb12021-02-17 19:39:32 +0100102
Akronf7084c42021-01-07 10:25:22 +01001030.03 2021-01-12
Marc Kupietzb505d442021-01-06 16:40:29 +0100104 - Update KorAP-Tokenizer to released 2.0 version
Akronf7084c42021-01-07 10:25:22 +0100105 - Improve test suite for recent version
106 of Mojolicious.
107
Marc Kupietz44b1f252020-11-26 16:31:40 +01001080.02 2020-11-27
Akronf7084c42021-01-07 10:25:22 +0100109 - Update KorAP-Tokenizer to v2.0.0.
Akroneaa96232020-10-15 17:06:15 +0200110 - Switch input encoding based on XML
111 processing instruction.
Marc Kupietz44b1f252020-11-26 16:31:40 +0100112 - Fix handling of UTF-8 in sigles.
Akroneaa96232020-10-15 17:06:15 +0200113
Akron0c41ab32020-09-29 07:33:33 +02001140.01 2020-09-28
115 - Initial release to GitHub.