Gitiles
Code Review Sign In
korap.ids-mannheim.de / KorAP / deliko / 1e6bfd17a26490c777e9dd14ab6a5daebad22e3a
commit1e6bfd17a26490c777e9dd14ab6a5daebad22e3a[log] [tgz]
authorMarc Kupietz <kupietz@ids-mannheim.de>Wed Apr 10 16:33:20 2024 +0200
committerMarc Kupietz <kupietz@ids-mannheim.de>Wed Apr 10 16:33:20 2024 +0200
treebb3f3f92ed6960b3d86472411995d29358b62db7
parent8f72730a0621a009ec1e9307b43321f9138445a1 [diff]
Drop nav and audio elements

<gap> would be much better here, but is not allowed everywhere according
to the I5 DTD :/
1 file changed
tree: bb3f3f92ed6960b3d86472411995d29358b62db7
  1. lib/
  2. scripts/
  3. test/
  4. xslt/
  5. .gitignore
  6. .gitlab-ci.yml
  7. korap4dnb-compose.yml
  8. krill-korap4dnb.cfg
  9. Makefile
  10. Readme.md
Readme.md

EPub to KorAP (via TEI I5) conversion

Run

To generate I5 corpus

make -j $(nproc) target/dnb.i5.xml

To generate the KorAP-XML ZIP

Prerequisite: KorAP-XML-CoNLL-U

make -j $(nproc) target/dnb.zip

To generate Annotations

Install prerequisite korap/conllu2treetagger and korap/conllu2spacy docker imeges if not present:

docker image inspect korap/conllu2treetagger:latest || curl -Ls 'https://gitlab.ids-mannheim.de/KorAP/CoNLL-U-Treetagger/-/jobs/artifacts/master/raw/conllu2treetagger.xz?job=build-docker-image' | docker load

docker image inspect korap/conllu2spacy:latest || curl -Ls https://corpora.ids-mannheim.de/tools/conllu2spacy.tar.xz | docker load

Make annotations:

make -j $(nproc) target/dnb.marmot-malt.zip target/dnb.spacy.zip target/dnb.tree_tagger.zip

To build KorAP index (also directly)

Build KorAP all, up to the deployable index:

make -j $(nproc) all

News

  • 2024-03-24

    • slow udpipe2 dropped
    • added marmot POS and morpho-syntactic annotations
    • added malt dependency annotations

  • 2024-03-18

    • added make deploy to install new index and restart local KorAP@DNB instance (also available as ci target)
    • added show-server-logs and show-server-status make targets to monitor the local KorAP@DNB instance

  • 2024-03-17

    • added make all to build all targets, including the index

  • 2024-03-16

    • CI/CD pipeline added
    • first working pipeline for EPub ⮕ TEI I5 ⮕ KorAP-XML ⮕ (UDPipe+TreeTagger+Spacy) ⮕ Krill ⮕ KorAP-JSON

  • 2024-03-15: DNB test data added

  • 2024-03-08: example EPub and I5 added from DeReKo KJL corpus: Christiane F. ; Kai Hermann ; Horst Rieck: Wir Kinder vom Bahnhof Zoo in the folder test/resources/ – do not distribute (copyrighted data)