templates/de/doc/ql/poliqarp-plus.html.ep

% layout 'main', title => 'KorAP: Poliqarp+';

<h2>Poliqarp+</h2>

<p>Das folgende Tutorial präsentiert alle Features, die unsere Version der Poliqarp Abfragesprache zur Verfügung stellt und enthält zusätzlich einige spezifische KorAP Erweiterungen.</p>
%# The following tutorial introduces all features provided by our version of the Poliqarp Query Language and some KorAP specific extensions.

<section id="segments">
  <h3>Einfache Segmente</h3>

  <p>Die einzelnen Elemente von Poliqarp sind Segmente. Meistens repräsentieren Segmente Wörter und können leicht abgefragt werden:</p>
  %# Fußnote: Im polnischen National-Korpus kann Poliqarp viele Segmente verbinden, wenn ein einzelnes Wort erkannt wird.

  %= doc_query poliqarp => 'Baum'

  <p>Abfolgen einfacher Segmenten werden durch Leerzeichen getrennt:</p>

  %= doc_query poliqarp => 'der Baum'

  <p>Einfache Segmente beziehen sich immer auf die Oberflächenform eines Wortes. Wenn Sie nach einer Oberflächenform ohne Beachtung der Groß- und Kleinschreibung suchen, können Sie <code>/i</code> anfügen.</p>

  %= doc_query poliqarp => 'laufen/i'

  <p>Die Abfrage oben findet alle Vorkommen von <code>laufen</code> unabhängig von der Großschreibung von Buchstaben, so wird <code>wir laufen</code> genauso gefunden wie <code>das Laufen</code> und sogar <code>&quot;GEH LAUFEN!&quot;</code>.

  <h4 id="regexp">Reguläre Ausdrücke</h4>

  <p>Segmente können auch durch <%= doc_link_to 'Reguläre Ausdrücke', 'ql', 'regexp' %> abgefragt werden - indem das Segment mit doppelten Anführungszeichen umschlossen wird.</p>

  %= doc_query poliqarp => '"l(au|ie)fen"'

  <p>Reguläre Ausdrücke stimmen immer mit dem gesamten Segment überein, d.h. die obige Abfrage findet Wörter, die mit <code>l</code> beginnen und mit <code>n</code> enden. Um Teilausdrücke zu unterstützen, können Sie das Flag <code>/x</code> verwenden.</p>

    %= doc_query poliqarp => '"l(au|ie)fen"/x', cutoff => 1

  <p>Das <code>/x</code> flag sucht nach allen Segmenten, die eine Sequenz von Zeichen enthalten, die mit dem regulären Ausdruck übereinstimmen. Das bedeutet, dass die obige Abfrage äquivalent zu der Folgenden ist:</p>

  %= doc_query poliqarp => '".*?l(au|ie)fen.*?"', cutoff => 1

  <p>Das <Code>/x</code> Flag kann auch in Verbindung mit exakten Ausdrücken verwendet werden, um nach Teilzeichenketten zu suchen:</p>

  %= doc_query poliqarp => 'trenn/xi', cutoff => 1

  <p>Die obige Abfrage findet alle Vorkommen von Segmenten mit der Zeichenfolge <code>trenn</code> unabhängig von Groß-Kleinschreibung, wie &quot;Trennung&quot;, &quot;unzertrennlich&quot; oder &quot;Wettrennen&quot;.</p>

  <blockquote class="warning">
    <p>Achtung: Diese Art von Abfragen (mit vorangestellten <code>.*</Code> Ausdrücken) sind extrem langsam!</p>
  </blockquote>

  <p>Sie können das <code>/i</code> Flag erneut anwenden, um unabhängig von Groß-Kleinschreibung zu suchen.</p>

  %= doc_query poliqarp => '"l(au|ie)fen"/xi', cutoff => 1

</section>

<section id="complex">
  <h3>Komplexe Segmente</h3>

  <p>Komplexe Segmente werden in eckigen Klammern ausgedrückt und enthalten zusätzliche Informationen über die Ressource des zu prüfenden Begriffs, indem sie Schlüssel/Wert-Paare enthalten, die durch ein Gleichheitszeichen getrennt sind.</p>

  <p>Die KorAP-Implementierung von Poliqarp bietet drei spezielle Segmentschlüssel: <code>orth</code> für Oberflächenformen, <code>base</code> für Lemmata und <code>pos</code> für Part-of-Speech-Annotationen. Die folgende komplexe Abfrage findet alle Oberflächenformen von <code>Baum</code>.</p>

  %# Es gibt mehr spezielle Schlüssel in Poliqarp, aber KorAP bietet sie nicht an.

  %= doc_query poliqarp => '[orth=Baum]'

  <p>Die Abfrage entspricht also:</p>

  %= doc_query poliqarp => 'Baum'

  <p>Komplexe Segmente erwarten einfache Ausdrücke als Werte, was bedeutet, dass auch der folgende Ausdruck gültig ist:</p>

  %= doc_query poliqarp => '[orth="l(au|ie)fen"/xi]', cutoff => 1

  <p>Ein weiterer spezieller Schlüssel ist <code>base</code>, bezogen auf die Lemma-Annotation der <%= doc_link_to 'Standard-Foundry', 'data', 'annotation'%>.
    Die folgende Abfrage findet alle Vorkommen von Segmenten, die als Lemma <code>Baum</code> durch die Standard-Foundry annotiert wurden.</p>

  %= doc_query poliqarp => '[base=Baum]'

  <p>Der dritte Sonderschlüssel ist <code>pos</code> und bezieht sich auf die Wortarten-Annotation der <% = doc_link_to 'Standard-Foundry', 'data', 'annotation'%>.
    Die folgende Abfrage findet alle attributiven Adjektive:</p>

  %= doc_query poliqarp => '[pos=ADJA]'

  <p>Komplexe Segmente, die weitere Token-Annotationen anfordern, können Schlüssel der <code>foundry/layer</code> Notation folgend haben.
    Zum Beispiel, um alle Vorkommen von mehreren Wörtern in der <code>mate</code> Foundry zu finden, können Sie mit der folgenden Abfrage suchen:</p>

  %= doc_query poliqarp => '[mate/m=number:pl]'

  <h4>Negation</h4>
  <p>Die Negation von Termen in komplexen Ausdrücken kann durch Voranstellen eines Ausrufezeichen vor dem Gleichheitszeichen oder dem gesamten Term ausgedrückt werden.</p>

  %= doc_query poliqarp => '[pos!=ADJA]'
  %= doc_query poliqarp => '[!pos=ADJA]'

  <blockquote class="warning">
    <p>Vorsicht: Negierte komplexe Segmente können nicht alleinstehend im Lucene-Index gesucht werden.
      Allerdings funktionieren sie, wenn sie Teil einer <%= doc_link_to 'Sequenz', 'ql', 'poliqarp-plus#syntagmatic-operators-sequence'%> sind.</p>
  </blockquote>

  <h4 id="empty-segments">Leere Segmente</h4>

  <p>Ein spezielles Segment ist das leere Segment, das jedem Wort im Index entspricht.</p>

  %= doc_query poliqarp => '[]'

  <p>Leere Segmente sind nützlich, um Abstände von Wörtern auszudrücken, indem sie <%= doc_link_to 'Wiederholungen', 'ql', 'poliqarp-plus#syntagmatic-operators-repetitions' %> verwenden.</p>

  <blockquote class="warning">
    <p>Vorsicht: Leere Segmente können nicht alleinstehend im Lucene-Index gesucht werden. 
      Allerdings arbeiten sie, falls sie Teil eines <%= doc_link_to 'sequence', 'ql', 'poliqarp-plus#syntagmatic-operators-sequence' %> sind.</p>
  </blockquote>
</section>

%# TODO:

<section id="spans">
  <h3>Span Segments</h3>

  <p>Not all segments are bound to words - some are bound to concepts spanning multiple words, for example noun phrases, sentences, or paragraphs.
Span segments can be searched for using angular brackets instead of square brackets.</p>

  %= doc_query poliqarp => '<xip/c=INFC>'
    
    <p>Otherwise they can be treated in exactly the same way as simple or complex segments.</p>
</section>

<section id="paradigmatic-operators">
  <h3>Paradigmatic Operators</h3>

  <p>A complex segment can have multiple properties a token has to fulfill. For example to search for all words with the surface form <code>laufe</code> (no matter if capitalized or not) that have the lemma <code>lauf</code> (and not, for example, <code>laufen</code>, which would indicate a verb or a gerund), you can search for:</p>

  %= doc_query poliqarp => '[orth=laufe/i & base=Lauf]'

  <p>The ampersand combines multiple properties with a logical AND.
Terms of the complex segment can be negated as introduced before.</p>

  %= doc_query poliqarp => '[orth=laufe/i & base!=Lauf]'

  <p>The following query is therefore equivalent:</p>

  %= doc_query poliqarp => '[orth=laufe & !base=Lauf]'

  <p>Alternatives can be expressed by using the pipe symbol:</p>

  %= doc_query poliqarp => '[base=laufen | base=gehen]'

  <p>All these sub expressions can be grouped using round brackets to form complex boolean expressions:</p>

  %= doc_query poliqarp => '[(base=laufen | base=gehen) & tt/pos=VVFIN]'
</section>

<section id="syntagmatic-operators">
  <h3>Syntagmatic Operators</h3>

  <h4 id="syntagmatic-operators-sequence">Sequences</h4>

  <p>Sequences can be used to search for segments in order. For example to search for the word &quot;alte&quot; preceded by &quot;der&quot; and followed by &quot;Mann&quot;, you can simple search for the sequence of simple expressions separated by whitespaces.</p>

  %= doc_query poliqarp => 'der alte Mann'

  <p>However, you can obviously search using complex segments as well:</p>

  %= doc_query poliqarp => '[orth=der][orth=alte][orth=Mann]'

  <p>Now you may see the benefit of the empty segment to search for words you don't know:</p>

  %= doc_query poliqarp => '[orth=der][][orth=Mann]'

  <p>You are also able to mix segments and spans in sequences, for example to search for the word &quot;Der&quot; at the beginning of a sentence (which can be interpreted as the first word after the end of a sentence).</p>

  %= doc_query poliqarp => '<base/s=s>[orth=Der]'

  <h4>Groups</h4>

  ...

  <h4>Alternation</h4>

  <p>Alternations allow for searching alternative segments or sequences of segments, similar to the paradigmatic operator. You already have seen that you can search for both sequences of <code>der alte Mann</code> and <code>der junge Mann</code> by typing in:</p>

  %= doc_query poliqarp => 'der [orth=alte | orth=junge] Mann'

  <p>However, this formulation has problems in case you want to search for alternations of sequences rather than terms. If you want to search for both sequences of <code>dem jungen Mann</code> and <code>der alte Mann</code> you can use syntagmatic alternations and groups:</p>

  %= doc_query poliqarp => '(dem jungen | der alte) Mann'

  <p>The pipe symbol works the same way as with the paradigmatic alternation, but supports sequences of different length as operands. The above query for <code>der alte Mann</code> and <code>der junge Mann</code> can therefor be reformulated as:</p>

  %= doc_query poliqarp => 'der (junge | alte) Mann'

  <h4 id="syntagmatic-operators-repetitions">Repetition</h4>

  <p>Repetitions in Poliqarp are realized as in <%= doc_link_to 'regular expressions', 'ql', 'regexp' %>, by giving quantifieres in curly brackets.</p>
  <p>To search for a sequence of three occurrences of <code>der</code>, you can formulate your query in any of the following ways - they will have the same results:</p>

  %= doc_query poliqarp => 'der der der'
  %= doc_query poliqarp => 'der{3}'
  %= doc_query poliqarp => '[orth=der]{3}'

  <p>In difference to regular expressions, the repetition operation won't refer to the match but to the pattern given. So the following query will give you a sequence of three words having the term <code>der</code> as a substring - but the words don't have to be identical. The following query for example will match a sequence of three words all starting with <code>la</code>.</p>

  %= doc_query poliqarp => '"la.*?"/i{3}'

  <p>The same is true for annotations. The following query will find a sequence of 3 to 4 adjectives as annotated by the TreeTagger foundry, that is preceded by the lemma <code>ein</code> as annotated by the default foundry and followed by a noun as annotated by the XIP foundry. The adjectives do not have to be identical though.</p>

  %= doc_query poliqarp => '[base=ein][tt/p=ADJA]{3,4}[xip/p=NOUN]'

  <p>In addition to numbered quantities, it is also possible to pass repetition information as Kleene operators <code>?</code>, <code>+</code>, and <code>+</code>.</p>

  <p>To search for a sequence of the lemma <code>der</code> followed by the lemma <code>baum</code> as annotated by the base foundry, but allowing an optional adjective as annotated by the TreeTagger foundry in between, you can search for:</p>

  %= doc_query poliqarp => '[base=die][tt/pos=ADJA]?[base=Baum]'

  <p>This query is identical to the numbered quantification of:</p>

  %= doc_query poliqarp => '[base=die][tt/pos=ADJA]{,1}[base=Baum]'

  <p>To search for the same sequences but with unlimited adjectives as annotated by the TreeTagger foundry in between, you can use the Kleene Star:</p>

  %= doc_query poliqarp => '[base=die][tt/pos=ADJA]*[base=Baum]'

  <p>And to search for this sequence but with at least one adjective in between, you can use the Kleene Plus (all queries are identical):</p>

  %= doc_query poliqarp => '[base=die][tt/pos=ADJA]+[base=Baum]', cutoff => 1
  %= doc_query poliqarp => '[base=die][tt/pos=ADJA]{1,}[base=Baum]', cutoff => 1
  %= doc_query poliqarp => '[base=die][tt/pos=ADJA][tt/pos=ADJA]*[base=Baum]', cutoff => 1

  <blockquote class="warning">
    <p>Repetition operators like <code>{,4}</code>, <code>?</code>, and <code>*</code> make segments or groups of segments optional. In case these queries are used separated and not as part of a sequence (and there are no mandatory segments in the query), you will be warned by the system that your query won't be treated as optional.</p>
    <p>Keep in mind that optionality may be somehow <i>inherited</i>, for example when you search for <code>(junge|alte)?|tote</code>, one segment of the alternation is optional, which makes the whole query optional as well.</p>
  </blockquote>

  <p>Repetition can also be used to express distances between segments by using <%= doc_link_to 'empty segments', 'ql', 'poliqarp-plus#empty-segments' %>.</p>

  %= doc_query poliqarp => '[base=die][][base=Baum]'
  %= doc_query poliqarp => '[base=die][]{2}[base=Baum]', cutoff => 1
  %= doc_query poliqarp => '[base=die][]{2,}[base=Baum]', cutoff => 1
  %= doc_query poliqarp => '[base=die][]{,3}[base=Baum]', cutoff => 1

  <p>Of course, Kleene operators can be used with empty segments as well.</p>

  %= doc_query poliqarp => '[base=die][]?[base=Baum]'
  %= doc_query poliqarp => '[base=die][]*[base=Baum]', cutoff => 1
  %= doc_query poliqarp => '[base=die][]+[base=Baum]', cutoff => 1
    
  <h4>Position</h4>

  <p>Sequences as shown above can all be nested in further complex queries and treated as subqueries (see <%= doc_link_to 'class operators', 'ql', 'poliqarp-plus#class-operators' %> on how to later access these subqueries directly).</p>
  <p>Positional operators compare two matches of subqueries and will match, in case a certain condition regarding the position of both is true.</p>
  <p>The <code>contains()</code> operation will match, when a second subquery matches inside the span of a first subquery.</p>

  %= doc_query poliqarp => 'contains(<base/s=s>, [tt/p=KOUS])', cutoff => 1

  <p>The <code>startsWith()</code> operation will match, when a second subquery matches at the beginning of the span of a first subquery.</p>

  %= doc_query poliqarp => 'startsWith(<base/s=s>, [tt/p=KOUS])', cutoff => 1

  <p>The <code>endsWith()</code> operation will match, when a second subquery matches at the end of the span of a first subquery.</p>

  %= doc_query poliqarp => 'endsWith(<base/s=s>, [opennlp/p=NN])', cutoff => 1

  <p>The <code>matches()</code> operation will match, when a second subquery has the exact same span of a first subquery.</p>

  %= doc_query poliqarp => 'matches(<base/s=s>,[tt/p=CARD][tt/p="N.*"])', cutoff => 1

  <p>The <code>overlaps()</code> operation will match, when a second subquery has an overlapping span with the first subquery.</p>

  %= doc_query poliqarp => 'overlaps([][tt/p=ADJA],{1:[tt/p=ADJA]}[])', cutoff => 1

  <blockquote class="warning">
    <p>Positional operators are still experimental and may change in certain aspects in the future (although the behaviour defined is intended to be stable). There is also known incorrect behaviour which will be corrected in future versions.</p>
    <p>Optional operands in position operators, like in <code>contains(&lt;s&gt;,[orth=Baum]*)</code>, have to be mandatory at the moment and will be reformulated to occur at least once.</p>
    <p>This behaviour may change in the future.</p>
  </blockquote>

  <!--
      <blockquote>
	<p>The KorAP implementation of Poliqarp also supports the postfix <code>within</code> operator, that works similar to the <code>contains()</code> operator, but is not nestable.</p>
      </blockquote>
      -->
      
</section>

<section id="class-operators">
  <h3>Class Operators</h3>

  <p>Classes are used to group sub matches by surrounding curly brackets and a class number <code>{1:...}</code>. Classes can be used to refer to sub matches in a query, similar to captures in regular expressions. In Poliqarp+ classes have multiple purposes, with highlighting being the most intuitive one:</p>

  %= doc_query poliqarp => 'der {1:{2:[]} Mann}'

  %#= doc_query poliqarp => 'der {1:{2:[]{1,4}} {3:Baum}} {4:[]}'

  <p>In KorAP classes can be defined from 1 to 128. In case a class number is dismissed, the class defaults to the class number 1: <code>{...}</code> is equal to <code>{1:...}</code>.</p>

  <h4>Match Modification</h4>

  <p>Based on classes, matches may be modified. The <code>focus()</code> operator restricts the span of a match to the boundary of a certain class.</p>

  %= doc_query poliqarp => 'focus(der {Baum})'

  <p>The query above will search for the sequence <code>der Baum</code> but the match will be limited to <code>Baum</code>. You can think of <code>der</code> in this query as a positive look-behind zero-length assertion in regular expressions.</p>

  <p>But focus is way more useful if you are searching for matches without knowing the surface form. For example, to find all terms between the words &quot;der&quot; and &quot;Mann&quot; you can search:</p>

  %= doc_query poliqarp => 'focus(der {[]} Mann)'

  <p>This will limit the match to all interesting terms in between &quot;der&quot; and &quot;Mann&quot;. Or you may want to search for all words following the sequence &quot;der alte und&quot; immediately:</p>

  %= doc_query poliqarp => 'focus(der alte und {[]})'

  <!--
      <p><code>focus()</code> is especially useful if you are searching for matches in certain areas, for example in quotes using positional operators.
While not being interested in the whole quote as a match, you can focus on what's really relevant to you.</p>

      %= doc_query poliqarp => 'focus(1:contains(er []{,10} sagte, 1{Baum}))'
      -->

  <p>In case a class number is dismissed, the focus operator defaults to the class number 1: <code>focus(...)</code> is equal to <code>focus(1: ...)</code>.</p>

  <blockquote class="warning">
    <p>As numbers in curly brackets can be ambiguous in certain circumstances, for example <code>[]{3}</code> can be read as either &quot;any word repeated three times&quot; or &quot;any word followed by the number 3 highlighted as class number 1&quot;, numbers should always be expressed as <code>[orth=3]</code> for the latter case.</p>
  </blockquote>
</section>