Added Stanford CoreNLP Tokenizer

Change-Id: Ib9debb34cb9a66fad041cb584d641e155b46a347
2 files changed
tree: 8944f8f024dcbb632f2b5b113ccacba522138adb
  1. benchmarks/
  2. corpus/
  3. nnsplit_bench/
  4. spacy/
  5. .gitignore
  6. Dockerfile
  7. Readme.md
Readme.md

Creating the container

To build the Docker image, run

$ docker build -f Dockerfile -t korap/euralex22 .

This will download and install an image of approximately 6GB.

It will download and install the following tokenizers in an image to your system:

...

To run the evaluation suite ...

...

Running the evaluation suite

To run the benchmark, call

$ docker run --rm -it \
  -v ${PWD}/benchmarks:/euralex/benchmarks \
  -v ${PWD}/corpus:/euralex/corpus \
  korap/euralex22 benchmarks/benchmark.pl

Licenses

For Treetagger: Please read the license terms, before you download the software! By downloading the software, you agree to the terms stated there.

Caveat

When running this benchmark using Docker you may need to run all processes privileged to get meaningful results.

docker run --privileged -v