tree dbd22e6cb3556bdcf39996e031cbdd4c259daa61
parent a9adcb22851e8cfa29cfc3f29b3915c7f46f5dd2
author Marc Kupietz <kupietz@ids-mannheim.de> 1781180742 +0200
committer Marc Kupietz <kupietz@ids-mannheim.de> 1781180868 +0200

Drop empty texts (0 tokens) from Krill output (#46)

Krill should not index a text without at least one token, so emitting
such a text only produces an empty, unusable document. Texts with no
tokens are now skipped with a per-text warning and a summary count at
the end of the run; texts with at least one token are kept.

The check is applied at the single chokepoint every Krill text passes
through before being written (enqueueKrillCompression), with a defensive
guard on the direct-write fallback (outputKrillText). A new
dropKrillText helper removes the text from all tracking structures so
the finalization pass skips it too.

Adds m21_empty_sample.zip regression fixture (one empty text, one
single-token text) and a test asserting the empty text is dropped while
the single-token text is kept.

Resolves ä46

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Change-Id: I2786557463249b105a4418d20dfbb736bfadc942