tree 97dcd24f42c2cb3dcb2a5cdfece129143ac7e838
parent 394c54404a1ea6e9edb5e82d374100a047de5674
author Marc Kupietz <kupietz@ids-mannheim.de> 1783764114 +0200
committer Marc Kupietz <kupietz@ids-mannheim.de> 1783764114 +0200

Add Krill tar merge mode: update existing tars from ZIPs and stand-off metadata

Allow an existing Krill tar as input with -t krill ("merge mode"), updated
from the remaining inputs into a new tar (<name>.updated.krill.tar or -o).
This covers two scenarios: adding/fixing metadata when the source KorAP-XML
ZIPs are no longer available, and replacing single annotation foundries
(e.g. an improved wiki-taxonomy topic-domain classification) without a full
re-export.

- KrillTarMerger streams the input tar; unaffected texts are copied through
  byte-identically without being decompressed, affected texts are patched
  and recompressed in parallel with bounded memory. The input tar is never
  modified.
- KrillJsonPatcher patches documents surgically at the lexical JSON level:
  metadata fields replace same-key fields (aware of the legacy/corrected
  name pairs textClass/dmozDomain, textDomain/idsColumn) or are appended;
  a supplied annotation foundry replaces its stream annotations, sentence
  spans/counts and layerInfos/foundries entries entirely. Token offsets are
  taken from the existing stream, so the base ZIP is not needed.
- Annotation strings are built by the same code as full generation (helpers
  extracted from KrillJsonGenerator, behavior-neutral), so patched texts are
  byte-identical to a full re-export with the same inputs.
- creationDate/pubDate are only replaced when a new text-level header is
  supplied, so corpus/doc-level headers cannot clobber per-text dates.
- Texts present in the new inputs but missing from the tar are ignored with
  a warning; data/tokens/structure entries (base tokenization) are ignored
  as well. Tars produced by korapxml2krill (Perl) and LZ4-compressed tars
  are supported. The regular Krill export path is unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Change-Id: I3850b877eb0fc2eb5bc122ed13a2f60029b03ecf