Rename stand-off metadata field wikidomain to wikiDomain
Use camelCase for the Wikipedia topic-domain classification field key, to
match the casing of all other Krill metadata fields (textClass, pubDate,
corpusSigle, ...). The key derives from the metadataLayer xml:id, so this
renames the layer id in the test fixture; the taxonomy xml:id (wikitaxonomy)
and catRef/@scheme are unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Change-Id: I519efddc2bb72d44a2a8edffab05b2a946b0e9e0
diff --git a/Readme.md b/Readme.md
index 7e3ab6c..2e43976 100644
--- a/Readme.md
+++ b/Readme.md
@@ -163,6 +163,36 @@
This writes `out/krill/wud24_sample.krill.tar` plus a log file. Add more annotated KorAP-XML zips (e.g., TreeTagger, CoreNLP) to merge their layers into the same Krill export; use `--non-word-tokens` if punctuation should stay in the token stream.
+### Adding stand-off metadata (classifications, links) to the Krill export
+
+In addition to token and span annotations, text-level metadata can be supplied as
+*stand-off metadata*: one `<standOff>` XML file per corpus that assigns metadata to
+texts by their `raw_text/@docid`. This is useful for classifications that are produced
+separately (e.g. Wikipedia topic domains) or for data that changes occasionally (e.g.
+external links), without changing the source TEI files and re-running the corpus conversion.
+
+Just list such files alongside the zips — they are auto-detected by content (root
+element `<standOff>`), so no extra option is needed:
+
+```shell script
+./build/bin/korapxmltool -t krill -D out/krill \
+ app/src/test/resources/rei_sample.zip \
+ app/src/test/resources/rei_sample.domains.meta.xml
+```
+
+Each `<metadataLayer>` becomes a Krill metadata field named after the layer's
+`xml:id`: `type="classification"` layers become `type:keywords` fields, `type="links"`
+layers become `type:attachement` fields. By default every category present in the file
+is indexed, leaving the selection (e.g. top-k / threshold) to the tool that produced it.
+
+A classification file is, for example, produced by piping the NOW output through a
+classifier:
+
+```shell script
+./build/bin/korapxmltool -t now app/src/test/resources/rei_sample.zip \
+ | docker run --rm -i korap/wiki-taxonomy > rei_sample.wikiDomain.meta.xml
+```
+
## Annotation
### Tagging with integrated MarMoT POS tagger directly to a new KorAP-XML ZIP file
diff --git a/app/src/main/kotlin/de/ids_mannheim/korapxmltools/StandoffMetadata.kt b/app/src/main/kotlin/de/ids_mannheim/korapxmltools/StandoffMetadata.kt
index 4846602..cd86b76 100644
--- a/app/src/main/kotlin/de/ids_mannheim/korapxmltools/StandoffMetadata.kt
+++ b/app/src/main/kotlin/de/ids_mannheim/korapxmltools/StandoffMetadata.kt
@@ -18,7 +18,7 @@
* Layout:
*
* <standOff>
- * <metadataLayer xml:id="wikidomain" type="classification">
+ * <metadataLayer xml:id="wikiDomain" type="classification">
* <taxonomy .../> (declaration, ignored here)
* <textRef target="REI_RBR.00473"> (target == raw_text/@docid)
* <catRef target="#Language" n="1" cert="0.39"/>
diff --git a/app/src/test/kotlin/de/ids_mannheim/korapxmltools/KrillJsonGeneratorTest.kt b/app/src/test/kotlin/de/ids_mannheim/korapxmltools/KrillJsonGeneratorTest.kt
index 431e31b..20f4971 100644
--- a/app/src/test/kotlin/de/ids_mannheim/korapxmltools/KrillJsonGeneratorTest.kt
+++ b/app/src/test/kotlin/de/ids_mannheim/korapxmltools/KrillJsonGeneratorTest.kt
@@ -1238,11 +1238,11 @@
val jsonByFile = readKrillJson(generatedTar)
val rei473 = jsonByFile.entries.first { it.key.startsWith("REI-RBR-00473") }.value
- // The layer xml:id ("wikidomain") becomes the Krill field key, typed as keywords.
+ // The layer xml:id ("wikiDomain") becomes the Krill field key, typed as keywords.
val field = Regex(
- """"key":"wikidomain","@type":"koral:field","value":\[([^\]]*)\],"type":"type:keywords""""
+ """"key":"wikiDomain","@type":"koral:field","value":\[([^\]]*)\],"type":"type:keywords""""
).find(rei473)
- assertNotNull(field, "Expected a wikidomain keywords field in REI_RBR.00473")
+ assertNotNull(field, "Expected a wikiDomain keywords field in REI_RBR.00473")
// Default policy ingests everything in the file, ordered by rank (no curation
// at index time). REI_RBR.00473 has all five categories.
diff --git a/app/src/test/resources/rei_sample.domains.meta.xml b/app/src/test/resources/rei_sample.domains.meta.xml
index 4138bd1..e106a97 100644
--- a/app/src/test/resources/rei_sample.domains.meta.xml
+++ b/app/src/test/resources/rei_sample.domains.meta.xml
@@ -8,7 +8,7 @@
Conventions:
* One <metadataLayer> per logical metadata category ("foundry"); its
- xml:id is the Krill metadata field key (here: "wikidomain"). The taxonomy
+ xml:id is the Krill metadata field key (here: "wikiDomain"). The taxonomy
has its own distinct xml:id ("wikitaxonomy") so the two never collide.
* <textRef target="..."> joins on raw_text/@docid from data.xml
(e.g. REI_RBR.00473) - NOT on the slash-form textSigle. No '#':
@@ -22,7 +22,7 @@
<!-- ===================================================================
Layer 1: Wikipedia top-level topic-domain classification
=================================================================== -->
- <metadataLayer xml:id="wikidomain" type="classification" join="docid">
+ <metadataLayer xml:id="wikiDomain" type="classification" join="docid">
<!-- Provenance: this layer is machine-derived, so describe how. -->
<source tool="wiki-taxonomy" model="epoch_33-step_183090"