Fixing resource bug and adding sorting requirement flag to SpanWrapQueries
diff --git a/Changes b/Changes
index 75bd931..bac3b68 100644
--- a/Changes
+++ b/Changes
@@ -21,7 +21,9 @@
- [feature] Improved deserialization of SpanSubSpanQueries
(margaretha)
- [feature] Introducing the potential need for resorting queries
- on focussing (set by relations) (diewald)
+ on focussing (e.g., set by relations) (diewald)
+ - [bugfix] Hopefully fixing a resource related bug for
+ server instantiation (diewald)
0.49.3 2015-02-03
- [documentation] Improved documentation for API classes (diewald)
diff --git a/Readme.md b/Readme.md
index d71659f..c545522 100644
--- a/Readme.md
+++ b/Readme.md
@@ -1,9 +1,16 @@
# Krill
-A Corpus Retrieval Index using Lucene for Look-Ups
+A Corpusdata Retrieval Index using Lucene for Look-Ups
+
## Synopsis
+... TODO:
+> Adding data (JSON via server)
+> Querying data (KoralQuery)
+> Show results (JSON)
+
+
## Description
Krill is a [Lucene](https://lucene.apache.org/) based search
@@ -11,55 +18,50 @@
developed at the Institute for German Language (IDS) in Mannheim,
Germany.
+**The software is in its early stages and not stable yet**
+
+
## Features
-Krill is the reference implementation for the
-[KoralQuery](https://github.com/KorAP/Koral) protocol, covering
-most of its query features, including ...
+Krill is the reference implementation for
+[KoralQuery](https://github.com/KorAP/Koral), covering
+most of the protocols features, including ...
-### Fulltext search
-"Find all occurrences of the phrase 'sea monster'!"
+- **Fulltext search**<br>
+ "Find all occurrences of the phrase 'sea monster'!"<br>
+ "Find all case-insensitive words matching the regular expression /krak.*/"
-"Find all case-insensitive words matching the regular expression /krak.*/"
+- **Token-based annotation search**<br>
+ "Find all plural nouns in accusative!"
-### Token-based annotation search
+- **Span-based annotation search**<br>
+ "Find all nominal phrases!"
-"Find all plural nouns in accusative!"
+- **Distance search**<br>
+ ...
-### Span-based annotation search
+- **Positional search**<br>
+ ...
-"Find all nominal phrases!"
+- **Nested queries**<br>
+ ...
-### Distance search
+- **and many more ...**<br>
+ Multiple annotation resources;
+ Virtual Collections;
+ Partial highlightings;
+ Support for overlapping spans;
+ Relational queries;
+ Hierarchical queries ...
-...
-
-### Positional search
-
-...
-
-### Nested queries
-
-...
-
-### Multiple annotation resources
-
-"Find all words marked as a noun by
-[TreeTagger](http://www.ims.uni-stuttgart.de/forschung/ressourcen/werkzeuge/treetagger.html)
-and marked as an adjective by CoreNLP](https://github.com/stanfordnlp/CoreNLP)!"
-
-### and many more ...
-
-Virtual Collections;
-partial highlightings;
-Support for overlapping spans;
-relational queries;
-hierarchical queries ...
## Prerequisites
-...
+At least Java 7,
+[Git](http://git-scm.com/),
+[Maven](https://maven.apache.org/).
+Further dependencies are resolved using Maven.
## Setup
@@ -82,6 +84,17 @@
src/main/resources/korap.conf
src/test/resources/examples/
+
+## Caveats
+
+Krill operates on tokens and is limited to a single tokenization stream.
+Token annotations therefore have to rely on that tokenization,
+Span annotations have to wrap at least one token.
+Punctuations are currently not supported.
+The order of results is currently bound to the order of documents in the
+index, but this is likely to change.
+
+
## Development and License
**Authors**: [Nils Diewald](http://nils-diewald.de/), Eliza Margaretha
@@ -95,11 +108,21 @@
[Changes](https://raw.githubusercontent.com/KorAP/Krill/master/Changes)
file.
+**Contributions to Krill are very welcome!**
+Before contribution, please reformat your code according to the korap
+style guideline, provided by means of an
+[Eclipse style sheet](https://raw.githubusercontent.com/KorAP/Krill/master/korap-style.xml).
+You can either reformat using [Eclipse](http://eclipse.org/) or using
+[Maven](https://maven.apache.org/) with the command
+
+ $ mvn java-formatter:format
+
Krill is published under the
[BSD-2 License](https://raw.githubusercontent.com/KorAP/Krill/master/LICENSE).
To cite this work, please ...
+
## References and bundled Software
Named entities annotated in the test data by CoreNLP were using
diff --git a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanAlterQueryWrapper.java b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanAlterQueryWrapper.java
index 28e9c72..e60ca21 100644
--- a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanAlterQueryWrapper.java
+++ b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanAlterQueryWrapper.java
@@ -104,18 +104,25 @@
return this;
};
+ @Override
public SpanQuery toQuery() throws QueryException {
if (this.isNull || this.alternatives.size() == 0)
return (SpanQuery) null;
if (this.alternatives.size() == 1) {
- return (SpanQuery) this.alternatives.get(0).toQuery();
+ return (SpanQuery) this.alternatives.get(0).
+ retrieveNode(this.retrieveNode).
+ toQuery();
};
Iterator<SpanQueryWrapper> clause = this.alternatives.iterator();
- SpanOrQuery soquery = new SpanOrQuery( clause.next().toQuery() );
+ SpanOrQuery soquery = new SpanOrQuery(
+ clause.next().retrieveNode(this.retrieveNode).toQuery()
+ );
while (clause.hasNext()) {
- soquery.addClause( clause.next().toQuery() );
+ soquery.addClause(
+ clause.next().retrieveNode(this.retrieveNode).toQuery()
+ );
};
return (SpanQuery) soquery;
};
diff --git a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanAttributeQueryWrapper.java b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanAttributeQueryWrapper.java
index 52e2036..07ca212 100644
--- a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanAttributeQueryWrapper.java
+++ b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanAttributeQueryWrapper.java
@@ -24,7 +24,7 @@
@Override
public SpanQuery toQuery() throws QueryException {
- SpanQuery sq = subquery.toQuery();
+ SpanQuery sq = subquery.retrieveNode(this.retrieveNode).toQuery();
if (sq instanceof SpanTermQuery) {
return new SpanAttributeQuery((SpanTermQuery) sq, isNegation, true);
}
diff --git a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanClassQueryWrapper.java b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanClassQueryWrapper.java
index a10670a..96c9a3b 100644
--- a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanClassQueryWrapper.java
+++ b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanClassQueryWrapper.java
@@ -66,7 +66,7 @@
if (this.subquery.isNull())
return (SpanQuery) null;
- SpanQuery sq = (SpanQuery) this.subquery.toQuery();
+ SpanQuery sq = (SpanQuery) this.subquery.retrieveNode(this.retrieveNode).toQuery();
if (sq == null) return (SpanQuery) null;
diff --git a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanElementQueryWrapper.java b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanElementQueryWrapper.java
index 26853db..c12f1dd 100644
--- a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanElementQueryWrapper.java
+++ b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanElementQueryWrapper.java
@@ -6,6 +6,14 @@
import de.ids_mannheim.korap.query.wrap.SpanQueryWrapper;
import de.ids_mannheim.korap.util.QueryException;
+/*
+ * SpanElementQuery has to support two constructors:
+ * One respecting and adding node information to the payload,
+ * one ignoring node information.
+ * node aka depth in a tree information is only relevant for
+ * child relation queries.
+ */
+
public class SpanElementQueryWrapper extends SpanQueryWrapper {
String element;
String field;
@@ -15,10 +23,15 @@
this.element = element;
};
+
+ @Override
public SpanQuery toQuery () throws QueryException {
+ // Todo: Respect request for retrieving node data (i.e. depth information)
return (SpanQuery) new SpanElementQuery(this.field, this.element);
};
+
+ @Override
public boolean isNull () {
return false;
};
diff --git a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanMatchModifyQueryWrapper.java b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanMatchModifyQueryWrapper.java
index 4c57320..f9cafc9 100644
--- a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanMatchModifyQueryWrapper.java
+++ b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanMatchModifyQueryWrapper.java
@@ -45,9 +45,11 @@
};
public SpanQuery toQuery () throws QueryException {
- if (this.subquery.isNull())
- return (SpanQuery) null;
- return new SpanMatchModifyClassQuery(this.subquery.toQuery(), this.number);
+ if (this.subquery.isNull())
+ return (SpanQuery) null;
+ return new SpanMatchModifyClassQuery(
+ this.subquery.retrieveNode(this.retrieveNode).toQuery(), this.number
+ );
};
public boolean isOptional () {
diff --git a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanQueryWrapper.java b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanQueryWrapper.java
index 7ed43eb..5247ac4 100644
--- a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanQueryWrapper.java
+++ b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanQueryWrapper.java
@@ -34,7 +34,8 @@
isEmpty = false,
isExtended = false,
isExtendedToTheRight = false,
- maybeUnsorted = false;
+ maybeUnsorted = false,
+ retrieveNode = false;
/**
* Serialize the wrapped query and return a SpanQuery.
@@ -283,6 +284,20 @@
/**
+ * Make the query request node information in addition to
+ * span information.
+ *
+ * @param retrieve Boolean value saying the wrapper
+ * has or has not to respect node information.
+ * @return The {@link SpanQueryWrapper} object for chaining.
+ */
+ public SpanQueryWrapper retrieveNode (boolean retrieve) {
+ this.retrieveNode = retrieve;
+ return this;
+ };
+
+
+ /**
* Boolean value indicating that a wrapped query
* has a class. This is especially relevant for classed
* extension queries.
diff --git a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanRepetitionQueryWrapper.java b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanRepetitionQueryWrapper.java
index 3e829be..8785098 100644
--- a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanRepetitionQueryWrapper.java
+++ b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanRepetitionQueryWrapper.java
@@ -91,11 +91,11 @@
// The query is not a repetition query at all, but may be optional
if (this.min == 1 && this.max == 1)
- return this.subquery.toQuery();
+ return this.subquery.retrieveNode(this.retrieveNode).toQuery();
// That's a fine repetition query
return new SpanRepetitionQuery(
- this.subquery.toQuery(),
+ this.subquery.retrieveNode(this.retrieveNode).toQuery(),
this.min,
this.max,
true
diff --git a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanSequenceQueryWrapper.java b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanSequenceQueryWrapper.java
index e3cc0d2..97cb24f 100644
--- a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanSequenceQueryWrapper.java
+++ b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanSequenceQueryWrapper.java
@@ -481,7 +481,7 @@
try {
this.constraints.add(
new DistanceConstraint(
- (SpanElementQuery) unit.toQuery(),
+ (SpanElementQuery) unit.retrieveNode(this.retrieveNode).toQuery(),
min,
max,
isInOrder,
@@ -612,7 +612,7 @@
// Unproblematic single query
if (this.segments.get(0).maybeAnchor())
- return (SpanQuery) this.segments.get(0).toQuery();
+ return (SpanQuery) this.segments.get(0).retrieveNode(this.retrieveNode).toQuery();
if (this.segments.get(0).isEmpty())
throw new QueryException(
@@ -653,7 +653,7 @@
SpanQuery query = null;// = this.segments.get(0).toQuery();
int i = 0;
while (query == null && i < this.segments.size()) {
- query = this.segments.get(i).toQuery();
+ query = this.segments.get(i).retrieveNode(this.retrieveNode).toQuery();
i++;
};
@@ -664,7 +664,7 @@
if (!this.hasConstraints() && this.isInOrder()) {
for (; i < this.segments.size(); i++) {
- SpanQuery second = this.segments.get(i).toQuery();
+ SpanQuery second = this.segments.get(i).retrieveNode(this.retrieveNode).toQuery();
if (second == null)
continue;
@@ -688,7 +688,7 @@
if (this.segments.get(i).isExtended())
throw new QueryException(613, limitationError);
- SpanQuery sq = (SpanQuery) this.segments.get(i).toQuery();
+ SpanQuery sq = (SpanQuery) this.segments.get(i).retrieveNode(this.retrieveNode).toQuery();
if (sq == null) continue;
SpanDistanceQuery sdquery = new SpanDistanceQuery(
@@ -709,7 +709,7 @@
if (this.segments.get(i).isExtended())
throw new QueryException(613, limitationError);
- SpanQuery sq = (SpanQuery) this.segments.get(i).toQuery();
+ SpanQuery sq = (SpanQuery) this.segments.get(i).retrieveNode(this.retrieveNode).toQuery();
if (sq == null) continue;
SpanDistanceQuery sdquery = new SpanDistanceQuery(
@@ -732,7 +732,7 @@
if (this.segments.get(i).isExtended())
throw new QueryException(613, limitationError);
- SpanQuery sq = (SpanQuery) this.segments.get(i).toQuery();
+ SpanQuery sq = (SpanQuery) this.segments.get(i).retrieveNode(this.retrieveNode).toQuery();
if (sq == null) continue;
query = new SpanMultipleDistanceQuery(
@@ -889,7 +889,7 @@
problem.getClassNumber());
query = new SpanExpansionQuery(
- anchor.toQuery(),
+ anchor.retrieveNode(this.retrieveNode).toQuery(),
problem.getMin(),
problem.getMax(),
direction,
@@ -909,8 +909,8 @@
problem.getClassNumber());
query = new SpanExpansionQuery(
- anchor.toQuery(),
- problem.toQuery(),
+ anchor.retrieveNode(this.retrieveNode).toQuery(),
+ problem.retrieveNode(this.retrieveNode).toQuery(),
problem.getMin(),
problem.getMax(),
direction,
@@ -935,10 +935,10 @@
// [base=der][base=baum]
if (mergeLeft)
- ssqw.append(new SpanSimpleQueryWrapper(problem.toQuery()));
+ ssqw.append(new SpanSimpleQueryWrapper(problem.retrieveNode(this.retrieveNode).toQuery()));
// [base=baum][base=der]
else
- ssqw.prepend(new SpanSimpleQueryWrapper(problem.toQuery()));
+ ssqw.prepend(new SpanSimpleQueryWrapper(problem.retrieveNode(this.retrieveNode).toQuery()));
saqw.or(ssqw);
diff --git a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanSubspanQueryWrapper.java b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanSubspanQueryWrapper.java
index c907f5a..1b3d160 100644
--- a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanSubspanQueryWrapper.java
+++ b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanSubspanQueryWrapper.java
@@ -108,7 +108,7 @@
return null;
}
- SpanQuery sq = subquery.toQuery();
+ SpanQuery sq = subquery.retrieveNode(this.retrieveNode).toQuery();
if (sq == null)
return null;
if (sq instanceof SpanTermQuery) {
diff --git a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanWithinQueryWrapper.java b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanWithinQueryWrapper.java
index bb15807..a04b33a 100644
--- a/src/main/java/de/ids_mannheim/korap/query/wrap/SpanWithinQueryWrapper.java
+++ b/src/main/java/de/ids_mannheim/korap/query/wrap/SpanWithinQueryWrapper.java
@@ -93,7 +93,11 @@
// TODO: if (wrap.isNegative())
- return new SpanWithinQuery(this.element.toQuery(), this.wrap.toQuery(), this.flag);
+ return new SpanWithinQuery(
+ this.element.retrieveNode(this.retrieveNode).toQuery(),
+ this.wrap.retrieveNode(this.retrieveNode).toQuery(),
+ this.flag
+ );
};
diff --git a/src/main/resources/index.properties b/src/main/resources/index.properties
deleted file mode 100644
index 1c87567..0000000
--- a/src/main/resources/index.properties
+++ /dev/null
@@ -1,5 +0,0 @@
-lucene.version = ${project.version}
-lucene.name = ${project.name}
-lucene.index.commit.count = 134217000
-lucene.index.commit.log = log/korap.commit.log
-lucene.indexDir = /home/ndiewald/Repositories/korap/KorAP-modules/KorAP-lucene-index/sandbox/index-tanja
diff --git a/src/main/resources/server.properties.info b/src/main/resources/server.properties.info
index 2a392e8..f35a016 100644
--- a/src/main/resources/server.properties.info
+++ b/src/main/resources/server.properties.info
@@ -1,3 +1,6 @@
+lucene.version = ${project.version}
+lucene.name = ${project.name}
+
# Lucene Backend properties
lucene.properties = true
lucene.indexDir = [PATH TO INDEX DIRECTORY]
@@ -9,3 +12,6 @@
lucene.db.URL = jdbc:mysql://[DB_IP]:[DB_PORT]/[DB_NAME]
lucene.db.pwd = [DB_PWD]
lucene.db.user = [DB_USER]
+
+lucene.index.commit.count = 134217000
+lucene.index.commit.log = log/korap.commit.log
diff --git a/src/test/java/de/ids_mannheim/korap/node/TestResource.java b/src/test/java/de/ids_mannheim/korap/node/TestResource.java
index da10c71..643ec6a 100644
--- a/src/test/java/de/ids_mannheim/korap/node/TestResource.java
+++ b/src/test/java/de/ids_mannheim/korap/node/TestResource.java
@@ -1,13 +1,9 @@
package de.ids_mannheim.korap.node;
-/*
- http://harryjoy.com/2012/09/08/simple-rest-client-in-java/
-*/
import java.io.*;
import java.util.ArrayList;
import java.util.List;
-
import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.client.WebTarget;
@@ -27,13 +23,18 @@
import de.ids_mannheim.korap.response.KorapResponse;
import static de.ids_mannheim.korap.util.KorapString.*;
+
+/**
+ * @author diewald
+ */
+// http://harryjoy.com/2012/09/08/simple-rest-client-in-java/
public class TestResource {
private HttpServer server;
private WebTarget target;
@Before
- public void setUp() throws Exception {
+ public void setUp () throws Exception {
// start the server
server = KorapNode.startServer("milena", (String) null);
// create the client
@@ -57,7 +58,7 @@
* Test to see that the message "Gimme 5 minutes, please!" is sent in the response.
*/
@Test
- public void testPing() {
+ public void testPing () {
String responseMsg = target.path("ping").request().get(String.class);
assertEquals("Gimme 5 minutes, please!", responseMsg);
};
@@ -75,7 +76,10 @@
"02439"
}) {
- String json = StringfromFile(getClass().getResource("/wiki/" + i + ".json").getFile());
+ String json = StringfromFile(
+ getClass().getResource("/wiki/" + i + ".json").getFile()
+ );
+
kresp = target.path("/index/" + i).
request("application/json").
put(Entity.json(json), KorapResponse.class);