| commit | 96b71cf3edd7ad75605c33c1510903d54a9d88ca | [log] [tgz] |
|---|---|---|
| author | Marc Kupietz <kupietz@ids-mannheim.de> | Sat Mar 09 11:24:25 2024 +0100 |
| committer | Marc Kupietz <kupietz@ids-mannheim.de> | Sat Mar 09 12:05:55 2024 +0100 |
| tree | ba2ce62dc217e5665e5d35fb3eb2d2e3191c3311 | |
| parent | c739e9df45edc3d0226b5b0d7ef86aa0889768f0 [diff] |
Implement --extract-metadata option Change-Id: I3fb3a78787d62c88d7e885e268393d2ad4408cfa
Tool package to convert from KorAP XML format to CoNLL-U format, as well as other simple formats, including token boundary information.
Up to 200 times faster and more accurate drop-in replacement for the korapxml2conllu part of KorAP-XML-CoNLL-U.
./gradlew build
$ java -jar ./app/build/libs/korapxml2conllu.jar app/src/test/resources/wdf19.zip | head -10 # foundry = base # filename = WDF19/A0000/13072/base/tokens.xml # text_id = WDF19_A0000.13072 # start_offsets = 0 0 14 17 25 30 35 42 44 52 60 73 # end_offsets = 74 12 16 24 29 34 41 43 51 59 72 74 1 Australasien _ _ _ _ _ _ _ _ 2 on _ _ _ _ _ _ _ _ 3 devrait _ _ _ _ _ _ _ _ 4 peut _ _ _ _ _ _ _ _ 5 être _ _ _ _ _ _ _ _
$ java -jar ./app/build/libs/korapxml2conllu.jar --word2vec t/data/wdf19.zip Arts visuels Pourquoi toujours vouloir séparer BD et Manga ? Ffx 18:20 fév 25 , 2003 ( CET ) soit on ne sépara pas , soit alors on distingue aussi , le comics , le manwa , le manga .. la bd belge et touts les auteurs européens .. on commence aussi a parlé de la bd africaine et donc ... wikipedia ce prete parfaitement à ce genre de decryptage . …
java -jar ./app/build/libs/korapxml2conllu.jar -m '<textSigle>([^<]+)' -m '<creatDate>([^<]+)' --word2vec t/data/wdf19.zip
WDF19/A0000.10894 2014.08.28 Arts visuels Pourquoi toujours vouloir séparer BD et Manga ? WDF19/A0000.10894 2014.08.28 Ffx 18:20 fév 25 , 2003 ( CET ) soit on ne sépara pas , soit alors on distingue aussi , le comics , le manwa , le manga .. WDF19/A0000.10894 2014.08.28 la bd belge et touts les auteurs européens .. WDF19/A0000.10894 2014.08.28 on commence aussi a parlé de la bd africaine et donc ... WDF19/A0000.10894 2014.08.28 wikipedia ce prete parfaitement à ce genre de decryptage .
Author:
Copyright (c) 2024, Leibniz Institute for the German Language, Mannheim, Germany
This package is developed as part of the KorAP Corpus Analysis Platform at the Leibniz Institute for German Language (IDS).
It is published under the BSD 2-clause "Simplified" license.
Contributions are very welcome!
Your contributions should ideally be committed via our Gerrit server to facilitate reviewing ( see Gerrit Code Review - A Quick Introduction if you are not familiar with Gerrit). However, we are also happy to accept comments and pull requests via GitHub.