tree 63c9029b5d691871a7141ac73fd843103df0c45e
parent 9ef5dec549c076b6d4ef028f862ed6954133a2be
author Marc Kupietz <kupietz@ids-mannheim.de> 1770463753 +0100
committer Marc Kupietz <kupietz@ids-mannheim.de> 1770463930 +0100

Add lookahead to noun gender endings to prevent false matches

Forms like Innenminister/Innenministerinnen were incorrectly being
parsed as a single gender-marked token (Innenminister/Innen +
ministerinnen).

Now gender noun endings (:in, :innen, /in, /innen, (in), /frau, etc.)
use a lookahead character to verify the ending is NOT followed by a
letter - ensuring they are at a word boundary.

This correctly tokenizes:
- Innenminister/Innenministerinnen → Innenminister / Innenministerinnen
- Nutzer/in → Nutzer/in (valid gender form, unchanged)
- Kaufmann/frau → Kaufmann/frau (valid gender form, unchanged)

Adds test cases for both split and don't-split scenarios.

Change-Id: I509a6f12ec1bb5678b1d8e8a063d0164498de5de