added dependencies to pom
removed parent from pom
PQ+ punct test
diff --git a/LICENSE b/LICENSE
index 1bda32e..1110bec 100644
--- a/LICENSE
+++ b/LICENSE
@@ -1,75 +1,5 @@
-Artistic License 2.0
-
-Copyright (c) 2000-2006, The Perl Foundation.
-
-Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
-
-Preamble
-
-This license establishes the terms under which a given free software Package may be copied, modified, distributed, and/or redistributed. The intent is that the Copyright Holder maintains some artistic control over the development of that Package while still keeping the Package available as open source and free software.
-
-You are always permitted to make arrangements wholly outside of this license directly with the Copyright Holder of a given Package. If the terms of this license do not permit the full use that you propose to make of the Package, you should contact the Copyright Holder and seek a different licensing arrangement.
-Definitions
-
-"Copyright Holder" means the individual(s) or organization(s) named in the copyright notice for the entire Package.
-
-"Contributor" means any party that has contributed code or other material to the Package, in accordance with the Copyright Holder's procedures.
-
-"You" and "your" means any person who would like to copy, distribute, or modify the Package.
-
-"Package" means the collection of files distributed by the Copyright Holder, and derivatives of that collection and/or of those files. A given Package may consist of either the Standard Version, or a Modified Version.
-
-"Distribute" means providing a copy of the Package or making it accessible to anyone else, or in the case of a company or organization, to others outside of your company or organization.
-
-"Distributor Fee" means any fee that you charge for Distributing this Package or providing support for this Package to another party. It does not mean licensing fees.
-
-"Standard Version" refers to the Package if it has not been modified, or has been modified only in ways explicitly requested by the Copyright Holder.
-
-"Modified Version" means the Package, if it has been changed, and such changes were not explicitly requested by the Copyright Holder.
-
-"Original License" means this Artistic License as Distributed with the Standard Version of the Package, in its current version or as it may be modified by The Perl Foundation in the future.
-
-"Source" form means the source code, documentation source, and configuration files for the Package.
-
-"Compiled" form means the compiled bytecode, object code, binary, or any other form resulting from mechanical transformation or translation of the Source form.
-Permission for Use and Modification Without Distribution
-
-(1) You are permitted to use the Standard Version and create and use Modified Versions for any purpose without restriction, provided that you do not Distribute the Modified Version.
-Permissions for Redistribution of the Standard Version
-
-(2) You may Distribute verbatim copies of the Source form of the Standard Version of this Package in any medium without restriction, either gratis or for a Distributor Fee, provided that you duplicate all of the original copyright notices and associated disclaimers. At your discretion, such verbatim copies may or may not include a Compiled form of the Package.
-
-(3) You may apply any bug fixes, portability changes, and other modifications made available from the Copyright Holder. The resulting Package will still be considered the Standard Version, and as such will be subject to the Original License.
-Distribution of Modified Versions of the Package as Source
-
-(4) You may Distribute your Modified Version as Source (either gratis or for a Distributor Fee, and with or without a Compiled form of the Modified Version) provided that you clearly document how it differs from the Standard Version, including, but not limited to, documenting any non-standard features, executables, or modules, and provided that you do at least ONE of the following:
-
-(a) make the Modified Version available to the Copyright Holder of the Standard Version, under the Original License, so that the Copyright Holder may include your modifications in the Standard Version.
-(b) ensure that installation of your Modified Version does not prevent the user installing or running the Standard Version. In addition, the Modified Version must bear a name that is different from the name of the Standard Version.
-(c) allow anyone who receives a copy of the Modified Version to make the Source form of the Modified Version available to others under
-(i) the Original License or
-(ii) a license that permits the licensee to freely copy, modify and redistribute the Modified Version using the same licensing terms that apply to the copy that the licensee received, and requires that the Source form of the Modified Version, and of any works derived from it, be made freely available in that license fees are prohibited but Distributor Fees are allowed.
-Distribution of Compiled Forms of the Standard Version or Modified Versions without the Source
-
-(5) You may Distribute Compiled forms of the Standard Version without the Source, provided that you include complete instructions on how to get the Source of the Standard Version. Such instructions must be valid at the time of your distribution. If these instructions, at any time while you are carrying out such distribution, become invalid, you must provide new instructions on demand or cease further distribution. If you provide valid instructions or cease distribution within thirty days after you become aware that the instructions are invalid, then you do not forfeit any of your rights under this license.
-
-(6) You may Distribute a Modified Version in Compiled form without the Source, provided that you comply with Section 4 with respect to the Source of the Modified Version.
-Aggregating or Linking the Package
-
-(7) You may aggregate the Package (either the Standard Version or Modified Version) with other packages and Distribute the resulting aggregation provided that you do not charge a licensing fee for the Package. Distributor Fees are permitted, and licensing fees for other components in the aggregation are permitted. The terms of this license apply to the use and Distribution of the Standard or Modified Versions as included in the aggregation.
-
-(8) You are permitted to link Modified and Standard Versions with other works, to embed the Package in a larger work of your own, or to build stand-alone binary or bytecode versions of applications that include the Package, and Distribute the result without restriction, provided the result does not expose a direct interface to the Package.
-Items That are Not Considered Part of a Modified Version
-
-(9) Works (including, but not limited to, modules and scripts) that merely extend or make use of the Package, do not, by themselves, cause the Package to be a Modified Version. In addition, such works are not considered parts of the Package itself, and are not subject to the terms of this license.
-General Provisions
-
-(10) Any use, modification, and distribution of the Standard or Modified Versions is governed by this Artistic License. By using, modifying or distributing the Package, you accept this license. Do not use, modify, or distribute the Package, if you do not accept this license.
-
-(11) If your Modified Version has been derived from a Modified Version made by someone other than you, you are nevertheless required to ensure that your Modified Version complies with the requirements of this license.
-
-(12) This license does not grant you the right to use any trademark, service mark, tradename, or logo of the Copyright Holder.
-
-(13) This license includes the non-exclusive, worldwide, free-of-charge patent license to make, have made, use, offer to sell, sell, import and otherwise transfer the Package with respect to any patent claims licensable by the Copyright Holder that are necessarily infringed by the Package. If you institute patent litigation (including a cross-claim or counterclaim) against any party alleging that the Package constitutes direct or contributory patent infringement, then this Artistic License to you shall terminate on the date that such litigation is filed.
-
-(14) Disclaimer of Warranty: THE PACKAGE IS PROVIDED BY THE COPYRIGHT HOLDER AND CONTRIBUTORS "AS IS' AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES. THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT ARE DISCLAIMED TO THE EXTENT PERMITTED BY YOUR LOCAL LAW. UNLESS REQUIRED BY LAW, NO COPYRIGHT HOLDER OR CONTRIBUTOR WILL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING IN ANY WAY OUT OF THE USE OF THE PACKAGE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
\ No newline at end of file
+(c) IDS Mannheim
+This package may be copied, modified, distributed, and/or redistributed under
+the terms of Perl Artistic Licence 2.0 (the Licence). However, *if you make
+any modifications to the source code, please make these modifications available
+to us under the same Licence (according to point 4(a) of the Licence).*
\ No newline at end of file
diff --git a/README.md b/README.md
index 4377b40..b169da5 100644
--- a/README.md
+++ b/README.md
@@ -1,54 +1,124 @@
-## Koral v1.0
+# Koral v0.1
Koral is a library designed for the translation of different corpus query
-languages to KoralQuery, a JSON-LD-based protocol for the representation
-of linguistic queries.
+languages to KoralQuery, a JSON-LD-based protocol for the common representation
+of linguistic queries. This work has been carried out within the KorAP
+project (see below) and forms the major part of a Master thesis that is
+due to appear. The detailed specifications of KoralQuery will be covered
+in that thesis.
-As of v1.0, the following corpus query languages (QLs) are supported:
-* [Cosmas-II QL][http://www.ids-mannheim.de/cosmas2/web-app/hilfe/suchanfrage/]
-* [ANNIS QL][http://annis-tools.org/aql.html]
-* [Poliqarp QL][http://korpus.pl/en/cheatsheet/node3.html] (extended by numerous operators to "PoliqarpPlus" QL)
-* [CQP][http://www.loc.gov/standards/sru/cql/spec.html]
-
-## Code Example
+As of v0.1, the following corpus query languages (QLs) are supported:
+* [Cosmas-II QL](http://www.ids-mannheim.de/cosmas2/web-app/hilfe/suchanfrage/)
+* [ANNIS QL](http://annis-tools.org/aql.html)
+* [Poliqarp QL](http://korpus.pl/en/cheatsheet/node3.html) (extended by numerous operators to "PoliqarpPlus" QL)
+* [CQL](http://www.loc.gov/standards/sru/cql/spec.html)
You can use the main class QuerySerializer to translate and serialize queries
-for you. The following code snippet illustrates this. Valid QL identifiers
-are `cosmas', `annis', `poliqarp', `poliqarpplus' and `cqp'.
+for you. The usage example below illustrates this. Valid QL identifiers
+are `cosmas2`, `annis`, `poliqarp`, `poliqarpplus` and `cql`.
+
+
+## Usage Example
+
```java
import de.ids_mannheim.korap.query.serialize.QuerySerialzer;
QuerySerializer qs = new QuerySerializer();
-qs.setQuery("This is a poliqarp query.", "poliqarp");
+String query = "contains(<s>,[orth=zu][pos=ADJA])";
+qs.setQuery(query, "poliqarpplus");
System.out.println(qs.toJSON());
```
-This will print out a JSON-LD string with you Koralized query.
-There is also a command line version. After installation, simply run
+This will print out the following JSON-LD string for the Koralized query.
+The query asks for a sentence element (`<s>`) that is contained in a
+sequence of the surface form *zu* and a token with the part-of-speech tag *ADJA*.
+In the KoralQuery string, a containment relation is defined over two
+operands, an *s* span and a sequence of two tokens.
+```json
+{
+ "@context": "http://ids-mannheim.de/ns/KorAP/json-ld/v0.2/context.jsonld",
+ "query": {
+ "@type": "korap:group",
+ "operation": "operation:position",
+ "frames": [
+ "frames:isAround"
+ ],
+ "operands": [
+ {
+ "@type": "korap:span",
+ "key": "s"
+ },
+ {
+ "@type": "korap:group",
+ "operation": "operation:sequence",
+ "operands": [
+ {
+ "@type": "korap:token",
+ "wrap": {
+ "@type": "korap:term",
+ "layer": "orth",
+ "key": "zu",
+ "match": "match:eq"
+ }
+ },
+ {
+ "@type": "korap:token",
+ "wrap": {
+ "@type": "korap:term",
+ "layer": "pos",
+ "key": "ADJA",
+ "match": "match:eq"
+ }
+ }
+ ]
+ }
+ ]
+ }
+}
```
-java -jar target/Koral-1.0.jar [query] [queryLanguage]
-'''
+
## Motivation
-Koral and KoralQuery have been designed and developed within the [KorAP Project][http://korap.ids-mannheim.de/].
-Through Koral, linguists can use the KorAP query engine with the QL of their
-preference. As the KorAP backend only sees the incoming KoralQuery,
-new QLs can be supported by KorAP without having to change a single line of
-code in the backend.
+Koral enables the design and implementation of corpus query systems
+independently of any specific query languages. All the system needs to do on
+the query processing side is have the query translated to KoralQuery (see usage)
+and feed the translated query to its search engine. In particular, several query
+ languages can be supported without further adjustments to the search engine.
+
+Koral and KoralQuery have been designed and developed within the
+[KorAP Project](http://korap.ids-mannheim.de/), and are used in KorAP to
+translate queries to a common format before sending them to the backend.
## Installation
-Installation is straightforward:
+Installation is straightforward (Maven3 required):
-```
-git clone https://github.com/korap/Koral [install-dir]
-cd [install-dir]
-mvn install
-'''
+ git clone https://github.com/korap/Koral [install-dir]
+ cd [install-dir]
+ mvn test
+ mvn install
+
+There is also a command line version. After installation, simply run
+
+ java -jar target/Koral-0.1.jar [query] [queryLanguage]
+
+## Authorship
+
+Koral and KoralQuery were developed by Joachim Bingel,
+Nils Diewald, Michael Hanl and Eliza Margaretha at IDS Mannheim.
+
+The ANTLR grammars for parsing ANNIS QL and COSMAS II QL were developed by
+Thomas Krause (HU Berlin) and Franck Bodmer (IDS Mannheim), respectively.
+Minor adaptations of those grammars were implemented by the Koral authors.
+
+The authors wish to thank Piotr Bański, Franck Bodmer, Elena Frick and
+Carsten Schnober for their valuable input.
## License
-Koral is published under the Perl [Artistic License][http://opensource.org/licenses/artistic-license-2.0].
+Koral is published under the Perl [Artistic License](http://opensource.org/licenses/artistic-license-2.0).
See also the attached LICENSE.
+
+The [ANNIS grammar](https://github.com/korpling/ANNIS/tree/develop/annis-service/src/main/antlr4/annis/ql) is licensed under the Apache License 2.0.
\ No newline at end of file
diff --git a/pom.xml b/pom.xml
index 216bc3b..0c2ec69 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1,15 +1,12 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
- <parent>
- <groupId>KorAP-modules</groupId>
- <artifactId>KorAP-core-modules</artifactId>
- <version>1.1</version>
- </parent>
+ <!-- <parent> <groupId>KorAP-modules</groupId> <artifactId>KorAP-core-modules</artifactId>
+ <version>1.1</version> </parent> -->
<groupId>KorAP-modules</groupId>
<artifactId>Koral</artifactId>
- <version>1.0</version>
+ <version>0.1</version>
<packaging>jar</packaging>
<name>Koral</name>
<url>http://maven.apache.org</url>
@@ -77,6 +74,31 @@
<artifactId>cql-java</artifactId>
<version>1.12</version>
</dependency>
+ <dependency>
+ <groupId>org.projectlombok</groupId>
+ <artifactId>lombok</artifactId>
+ <version>0.11.8</version>
+ </dependency>
+ <dependency>
+ <groupId>log4j</groupId>
+ <artifactId>log4j</artifactId>
+ <version>1.2.17</version>
+ </dependency>
+ <dependency>
+ <groupId>log4j</groupId>
+ <artifactId>apache-log4j-extras</artifactId>
+ <version>1.2.17</version>
+ </dependency>
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ <version>1.7.5</version>
+ </dependency>
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-log4j12</artifactId>
+ <version>1.7.5</version>
+ </dependency>
</dependencies>
<build>
<sourceDirectory>${basedir}/src/main/java</sourceDirectory>
diff --git a/src/main/antlr/poliqarpplus/PoliqarpPlusParser.g4 b/src/main/antlr/poliqarpplus/PoliqarpPlusParser.g4
index 1562637..7005b88 100644
--- a/src/main/antlr/poliqarpplus/PoliqarpPlusParser.g4
+++ b/src/main/antlr/poliqarpplus/PoliqarpPlusParser.g4
@@ -142,7 +142,7 @@
;
submatch
-: SUBMATCH_OP LRPAREN startpos COMMA (length)? COLON (segment|sequence) RRPAREN
+: SUBMATCH_OP LRPAREN startpos (COMMA length)? COLON (segment|sequence) RRPAREN
;
matching
diff --git a/src/main/java/de/ids_mannheim/korap/query/serialize/PoliqarpPlusQueryProcessor.java b/src/main/java/de/ids_mannheim/korap/query/serialize/PoliqarpPlusQueryProcessor.java
index e0ac4eb..600e4af 100644
--- a/src/main/java/de/ids_mannheim/korap/query/serialize/PoliqarpPlusQueryProcessor.java
+++ b/src/main/java/de/ids_mannheim/korap/query/serialize/PoliqarpPlusQueryProcessor.java
@@ -710,21 +710,6 @@
// process foundry
if (foundryNode != null)
term.put("foundry", foundryNode.getText());
- // process layer: map "base" -> "lemma"
- if (layerNode != null) {
- String layer = layerNode.getText();
- if (mode.equals("span")) {
- term.put("key", layer);
- } else if (mode.equals("token")) {
- if (layer.equals("base")) {
- layer = "lemma"; }
- else if (layer.equals("punct")) {
- layer = "orth";
- term.put("type", "type:punct");
- }
- term.put("layer", layer);
- }
- }
// process key: 'normal' or regex?
key = keyNode.getText();
if (getNodeCat(keyNode.getChild(0)).equals("regex")) {
@@ -737,6 +722,22 @@
term.put("value", key);
else
term.put("key", key);
+ // process layer: map "base" -> "lemma"
+ if (layerNode != null) {
+ String layer = layerNode.getText();
+ if (mode.equals("span")) {
+ term.put("key", layer);
+ } else if (mode.equals("token")) {
+ if (layer.equals("base")) {
+ layer = "lemma"; }
+ else if (layer.equals("punct")) {
+ layer = "orth";
+ // will override "type":"type:regex"
+ term.put("type", "type:punct");
+ }
+ term.put("layer", layer);
+ }
+ }
// process value
if (valueNode != null)
term.put("value", valueNode.getText());
diff --git a/src/test/java/de/ids_mannheim/korap/query/serialize/PoliqarpPlusQueryProcessorTest.java b/src/test/java/de/ids_mannheim/korap/query/serialize/PoliqarpPlusQueryProcessorTest.java
index 7368ef4..6f832e8 100644
--- a/src/test/java/de/ids_mannheim/korap/query/serialize/PoliqarpPlusQueryProcessorTest.java
+++ b/src/test/java/de/ids_mannheim/korap/query/serialize/PoliqarpPlusQueryProcessorTest.java
@@ -149,6 +149,16 @@
assertEquals("type:punct", res.at("/query/wrap/type").asText());
assertEquals("orth", res.at("/query/wrap/layer").asText());
assertEquals("match:eq", res.at("/query/wrap/match").asText());
+
+ query = "[punct=\".\"]";
+ qs.setQuery(query, "poliqarpplus");
+ res = mapper.readTree(qs.toJSON());
+ assertEquals("korap:token", res.at("/query/@type").asText());
+ assertEquals("korap:term", res.at("/query/wrap/@type").asText());
+ assertEquals(".", res.at("/query/wrap/key").asText());
+ assertEquals("type:punct", res.at("/query/wrap/type").asText());
+ assertEquals("orth", res.at("/query/wrap/layer").asText());
+ assertEquals("match:eq", res.at("/query/wrap/match").asText());
}
@Test