Handled invalid pipes and added tests.
Change-Id: Iba20cbbfb60fa10a769bee697295943b0e4a1772
diff --git a/core/Changes b/core/Changes
index 291e020..11265f7 100644
--- a/core/Changes
+++ b/core/Changes
@@ -6,6 +6,8 @@
resolved #47)
12/12/2019
- Added support for multiple cq parameters (margaretha, resolved #46)
+13/12/2019
+ - Handled pipe errors and added tests (margaretha)
# version 0.62.2
13/11/2019
@@ -14,7 +16,7 @@
14/11/2019
- Added a check for OAuth2 client, VC, and group name length (margaretha)
22/11/2019
- - Updated the statistic API with KoralQuery (margaretha)
+ - Updated the statistic API with KoralQuery (margaretha)
# version 0.62.1
08/07/2019
@@ -25,7 +27,7 @@
11/07/2019
- Added cq parameter to VC statistics API and deprecate corpusQuery (margaretha)
15/07/2019
- - Added backward compatibility support for corpusQuery parameter (margaretha)
+ - Added backward compatibility support for corpusQuery parameter (margaretha)
28/08/2019
- Resolved #49. Added page param check in the search api (margaretha)
23/09/2019
@@ -69,7 +71,7 @@
- Updated code structure (margaretha)
- Removed spring security libraries and ManagerInterface (margaretha)
21/01/2019
- - Removed codes related to user registration & password management (margaretha)
+ - Removed codes related to user registration & password management (margaretha)
22/01/2019
- Added create, edit, retrieve user default setting controllers (margaretha)
24/01/2019
@@ -117,12 +119,12 @@
- Moved license regex configuration to full (margaretha)
- Added defaultRewriteConstraints (margaretha)
29/10/2018
- - Added javax.servlet-api (margaretha)
- - Updated koral version (margaretha)
-
+ - Added javax.servlet-api (margaretha)
+ - Updated koral version (margaretha)
+
version 0.61.1
28/08/2018
- - Added API URL versioning (margaretha)
+ - Added API URL versioning (margaretha)
30/08/2018
- Added backwards compability for URL versioning (margaretha)
@@ -130,65 +132,65 @@
version 0.61.0
13/08/2018
- - Updated Krill and Koral versions enabling VC caching and referencing (margaretha)
- - Set kustvakt configuration as Krill properties (margaretha)
-
+ - Updated Krill and Koral versions enabling VC caching and referencing (margaretha)
+ - Set kustvakt configuration as Krill properties (margaretha)
+
version 0.60.5
09/07/2018
- - Fixed status codes (margaretha)
- - Added KustvaktException for wrapping another exception (margaretha)
- - Removed deprecated codes regarding KustvaktResource (margaretha)
-
+ - Fixed status codes (margaretha)
+ - Added KustvaktException for wrapping another exception (margaretha)
+ - Removed deprecated codes regarding KustvaktResource (margaretha)
+
version 0.60.4
25/06/2018
- - added the redirect URI property in KustvaktException (margaretha)
- - added openid related status codes (margaretha)
+ - added the redirect URI property in KustvaktException (margaretha)
+ - added openid related status codes (margaretha)
version 0.60.3
30/05/2018
- - added parameter checker for collection (margaretha)
- - updated krill version (margaretha)
-
+ - added parameter checker for collection (margaretha)
+ - updated krill version (margaretha)
+
version 0.60.2
25/04/2018
- - rearranged and cleaned up codes (margaretha)
- - generalized some KustvaktException methods (margaretha)
- - added status codes (margaretha)
- - updated FormRequestWrapper constructor (margaretha)
- - fixed get request null parameter in FormRequestWrapper (margaretha)
-
+ - rearranged and cleaned up codes (margaretha)
+ - generalized some KustvaktException methods (margaretha)
+ - added status codes (margaretha)
+ - updated FormRequestWrapper constructor (margaretha)
+ - fixed get request null parameter in FormRequestWrapper (margaretha)
+
version 0.60.1
14/03/2018
- - removed AdminHandlerIface (margaretha)
- - removed isSystemAdmin in User class (margaretha)
+ - removed AdminHandlerIface (margaretha)
+ - removed isSystemAdmin in User class (margaretha)
version 0.60
13/03/2018
- - removed old policy and deprecated code (margaretha)
- - moved authentication related code to /full (margaretha)
- - elaborated default layer names in the configuration file (margaretha)
- - added GROUP_DELETED status code (margaretha)
+ - removed old policy and deprecated code (margaretha)
+ - moved authentication related code to /full (margaretha)
+ - elaborated default layer names in the configuration file (margaretha)
+ - added GROUP_DELETED status code (margaretha)
version 0.59.10
20/02/2018
- - updated hibernate and reflection versions (margaretha)
- - added Changes file (margaretha)
- - merged BeanConfigBaseTest to BeanConfigTest in /full (margaretha)
- - added status code for already deleted entry (margaretha)
- - updated library versions and java environment (margaretha)
- - added status codes (margaretha)
- - moved validation.properties (margaretha)
- - fixed unrecognized media-type application/json (margaretha)
-
-version 0.59.9
+ - updated hibernate and reflection versions (margaretha)
+ - added Changes file (margaretha)
+ - merged BeanConfigBaseTest to BeanConfigTest in /full (margaretha)
+ - added status code for already deleted entry (margaretha)
+ - updated library versions and java environment (margaretha)
+ - added status codes (margaretha)
+ - moved validation.properties (margaretha)
+ - fixed unrecognized media-type application/json (margaretha)
+
+version 0.59.9
08/11/2017
- - fixed missing exception in JsonUtils (margaretha)
- - fixed and restructured KustvaktResponseHandler (margaretha)
- - updated status code in ParameterChecker (margaretha)
-
+ - fixed missing exception in JsonUtils (margaretha)
+ - fixed and restructured KustvaktResponseHandler (margaretha)
+ - updated status code in ParameterChecker (margaretha)
+
version 0.59.8
24/10/2017
- - restructured Kustvakt and created core project (margaretha)
- - marked loader classes as deprecated (margaretha)
- - updated Spring version (margaretha)
- - moved unnecessary dependencies (margaretha)
\ No newline at end of file
+ - restructured Kustvakt and created core project (margaretha)
+ - marked loader classes as deprecated (margaretha)
+ - updated Spring version (margaretha)
+ - moved unnecessary dependencies (margaretha)
\ No newline at end of file
diff --git a/core/src/main/java/de/ids_mannheim/korap/config/KustvaktConfiguration.java b/core/src/main/java/de/ids_mannheim/korap/config/KustvaktConfiguration.java
index 273250a..26075d1 100644
--- a/core/src/main/java/de/ids_mannheim/korap/config/KustvaktConfiguration.java
+++ b/core/src/main/java/de/ids_mannheim/korap/config/KustvaktConfiguration.java
@@ -106,11 +106,13 @@
// another variable might be needed to define which metadata fields are restricted
private boolean isMetadataRestricted = false;
+ // EM: Maybe needed we support pipe registration
+ @Deprecated
public static Map<String, String> pipes = new HashMap<>();
public KustvaktConfiguration (Properties properties) throws Exception {
load(properties);
- readPipesFile("pipes");
+// readPipesFile("pipes");
KrillProperties.setProp(properties);
}
@@ -201,6 +203,7 @@
// "accountCreation");
}
+ @Deprecated
public void readPipesFile (String filename) throws IOException {
File file = new File(filename);
if (file.exists()) {
diff --git a/core/src/main/java/de/ids_mannheim/korap/exceptions/StatusCodes.java b/core/src/main/java/de/ids_mannheim/korap/exceptions/StatusCodes.java
index a2ebe5f..52de571 100644
--- a/core/src/main/java/de/ids_mannheim/korap/exceptions/StatusCodes.java
+++ b/core/src/main/java/de/ids_mannheim/korap/exceptions/StatusCodes.java
@@ -5,7 +5,7 @@
import de.ids_mannheim.korap.config.ConfigLoader;
/**
- * @author hanl
+ * @author hanl, margaretha
* @date 07/09/2014
*/
public class StatusCodes {
@@ -43,20 +43,18 @@
/**
* 300 status codes for query language and serialization
+ * see Koral (de.ids_mannheim.korap.query.serialize.util.StatusCodes)
*/
- public static final int NO_QUERY = 301;
-// public static final int INVALID_TYPE = 302;
-// public static final int SERIALIZATION_FAILED = 300;
-
/**
- * 400 status codes for authorization and rewrite functions
+ * 400 status codes for rewrite functions
*/
- // fixme: use unsupported resource and include type in return message
- public static final int POLICY_ERROR_DEFAULT = 400;
+ public static final int REWRITE_ERROR_DEFAULT = 400;
public static final int NON_PUBLIC_FIELD_IGNORED = 401;
- public static final int UNSUPPORTED_RESOURCE = 402;
+ public static final int PIPE_FAILED = 402;
+
+// public static final int UNSUPPORTED_RESOURCE = 402;
// public static final int REWRITE_FAILED = 403;
//public static final int UNSUPPORTED_FOUNDRY = 403;
//public static final int UNSUPPORTED_CORPUS = 404;
@@ -67,10 +65,10 @@
//public static final int FOUNDRY_REWRITE = 408;
//public static final int FOUNDRY_INJECTION = 409;
// public static final int MISSING_RESOURCE = 405;
- public static final int NO_POLICY_TARGET = 406;
- public static final int NO_POLICY_CONDITION = 407;
- public static final int NO_POLICY_PERMISSION = 408;
- public static final int NO_POLICIES = 409;
+// public static final int NO_POLICY_TARGET = 406;
+// public static final int NO_POLICY_CONDITION = 407;
+// public static final int NO_POLICY_PERMISSION = 408;
+// public static final int NO_POLICIES = 409;
diff --git a/core/src/main/java/de/ids_mannheim/korap/service/SearchService.java b/core/src/main/java/de/ids_mannheim/korap/service/SearchService.java
index 0d1db18..a40cbda 100644
--- a/core/src/main/java/de/ids_mannheim/korap/service/SearchService.java
+++ b/core/src/main/java/de/ids_mannheim/korap/service/SearchService.java
@@ -12,12 +12,15 @@
import javax.ws.rs.core.MultivaluedMap;
import javax.ws.rs.core.UriBuilder;
+import org.apache.http.HttpStatus;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import com.fasterxml.jackson.databind.node.ObjectNode;
import com.sun.jersey.api.client.Client;
import com.sun.jersey.api.client.ClientResponse;
import com.sun.jersey.api.client.WebResource;
@@ -30,6 +33,7 @@
import de.ids_mannheim.korap.exceptions.StatusCodes;
import de.ids_mannheim.korap.query.serialize.MetaQueryBuilder;
import de.ids_mannheim.korap.query.serialize.QuerySerializer;
+import de.ids_mannheim.korap.response.Notifications;
import de.ids_mannheim.korap.rewrite.RewriteHandler;
import de.ids_mannheim.korap.user.User;
import de.ids_mannheim.korap.user.User.CorpusAccess;
@@ -187,22 +191,71 @@
}
- private String runPipes (String query, String[] pipeArray) {
+ /**
+ * Pipes are service URLs for modifying KoralQuery. A POST request
+ * with Content-Type application/json will be sent for each pipe.
+ * Kustvakt expects a KoralQuery in JSON format as the pipe response.
+ *
+ * @param query the original koral query
+ * @param pipeArray the pipe service URLs
+ * @param serializer the query serializer
+ * @return a modified koral query
+ * @throws KustvaktException
+ */
+ private String runPipes (String query, String[] pipeArray) throws KustvaktException {
if (pipeArray !=null){
for (int i=0; i<pipeArray.length; i++){
- String url = KustvaktConfiguration.pipes.get(pipeArray[i]);
- // update query by sending it to a pipe URL
- // NOTE: request formulation may vary depending on the service
- Client client = Client.create();
- WebResource resource = client.resource(url);
- ClientResponse response =
- resource.type(MediaType.APPLICATION_JSON)
- .post(ClientResponse.class, query);
- query = response.getEntity(String.class);
+ String pipeURL = pipeArray[i];
+ try {
+ Client client = Client.create();
+ WebResource resource = client.resource(pipeURL);
+ ClientResponse response =
+ resource.type(MediaType.APPLICATION_JSON)
+ .accept(MediaType.APPLICATION_JSON)
+ .post(ClientResponse.class, query);
+ if (response.getStatus() == HttpStatus.SC_OK) {
+ String entity = response.getEntity(String.class);
+ if (entity != null && !entity.isEmpty()) {
+ query = entity;
+ }
+ }
+ else {
+ query = handlePipeError(query, pipeURL,
+ response.getStatus() + " "
+ + response.getStatusInfo().toString());
+ }
+ }
+ catch (Exception e) {
+ query = handlePipeError(query, pipeURL,
+ e.getMessage());
+ }
}
}
return query;
}
+
+ private String handlePipeError (String query, String url,
+ String message) throws KustvaktException {
+ jlog.error("Failed running the pipe at " + url + ". Message: "+ message);
+
+ Notifications n = new Notifications();
+ n.addWarning(StatusCodes.PIPE_FAILED,
+ "Pipe failed", url, message);
+ JsonNode warning = n.toJsonNode();
+
+ ObjectNode node = (ObjectNode) JsonUtils.readTree(query);
+ if (node.has("warnings")){
+ warning = warning.at("/warnings/0");
+ ArrayNode arrayNode = (ArrayNode) node.get("warnings");
+ arrayNode.add(warning);
+ node.set("warnings", arrayNode);
+ }
+ else{
+ node.setAll((ObjectNode) warning);
+ }
+
+ return node.toString();
+ }
private void handleNonPublicFields (List<String> fieldList,
boolean accessRewriteDisabled, QuerySerializer serializer) {
diff --git a/core/src/main/java/de/ids_mannheim/korap/test/TestController.java b/core/src/main/java/de/ids_mannheim/korap/test/TestController.java
index 43144bb..03b8f96 100644
--- a/core/src/main/java/de/ids_mannheim/korap/test/TestController.java
+++ b/core/src/main/java/de/ids_mannheim/korap/test/TestController.java
@@ -1,16 +1,28 @@
package de.ids_mannheim.korap.test;
+import java.io.IOException;
import java.io.InputStream;
-import javax.ws.rs.Produces;
import javax.ws.rs.Consumes;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
+import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+
+import de.ids_mannheim.korap.exceptions.KustvaktException;
+import de.ids_mannheim.korap.utils.JsonUtils;
+import de.ids_mannheim.korap.web.KustvaktResponseHandler;
+
/**
* Controllers used only for testing
*
@@ -22,12 +34,60 @@
@Produces(MediaType.APPLICATION_JSON + ";charset=utf-8")
public class TestController {
+ @Autowired
+ private KustvaktResponseHandler kustvaktResponseHandler;
+
+ public static ObjectMapper mapper = new ObjectMapper();
+
@POST
@Path("glemm")
@Consumes(MediaType.APPLICATION_JSON)
- public Response dummyGlemm (String jsonld) {
- InputStream is = getClass().getClassLoader()
+ public Response dummyGlemm (String jsonld,
+ @QueryParam("param") String param) throws IOException {
+ InputStream is;
+ is = getClass().getClassLoader()
.getResourceAsStream("test-pipes.jsonld");
- return Response.ok(is).build();
+
+ ObjectNode newJson = (ObjectNode) mapper.readTree(is);
+
+ try {
+ JsonNode node = JsonUtils.readTree(jsonld);
+ if (node.has("warnings")) {
+ node = node.get("warnings");
+ newJson.set("warnings", node);
+ }
+ if (param != null && !param.isEmpty()) {
+ ArrayNode keys = (ArrayNode) newJson.at("/query/wrap/key");
+ keys.add("die");
+ newJson.set("key", keys);
+ }
+ }
+ catch (KustvaktException e) {
+ throw kustvaktResponseHandler.throwit(e);
+ }
+ return Response.ok(newJson.toString()).build();
+ }
+
+ @POST
+ @Path("invalid-json-pipe")
+ @Consumes(MediaType.APPLICATION_JSON)
+ public Response dummyPipe1 (String jsonld) {
+ String incorrectJson = "{blah:}";
+ return Response.ok(incorrectJson).build();
+ }
+
+ @POST
+ @Path("plain-response-pipe")
+ @Consumes(MediaType.APPLICATION_JSON)
+ public Response dummyPipe2 (String jsonld) {
+ String incorrectJson = "brumbrum";
+ return Response.ok(incorrectJson).build();
+ }
+
+ @POST
+ @Path("urlencoded-pipe")
+ @Consumes(MediaType.APPLICATION_FORM_URLENCODED)
+ public Response dummyPipe3 (String jsonld) {
+ return Response.ok().build();
}
}
diff --git a/core/src/main/resources/test-pipes.jsonld b/core/src/main/resources/test-pipes.jsonld
index eca8cec..7fde643 100644
--- a/core/src/main/resources/test-pipes.jsonld
+++ b/core/src/main/resources/test-pipes.jsonld
@@ -10,7 +10,6 @@
"match": "match:eq",
"key": [
"der",
- "die",
"das"
],
"layer": "orth",
diff --git a/full/Changes b/full/Changes
index 4ec9218..8d1347a 100644
--- a/full/Changes
+++ b/full/Changes
@@ -3,11 +3,12 @@
- Implemented pipe extension in the search API (margaretha)
11/12/2019
- Added errors when requesting VC in caching process (margaretha,
- resolved #47)
+ resolved #47)
12/12/2019
- Added support for multiple cq parameters (margaretha, resolved #46)
-
-
+13/12/2019
+ - Handled pipe errors and added tests (margaretha)
+
# version 0.62.2
17/10/2019
- Handled vulnerability CVE-2019-17195. (margaretha)
@@ -112,7 +113,7 @@
- Added aliases to some VC controllers (margaretha)
- Merged VC access list controllers (margaretha)
21/01/2019
- - Removed codes related to user registration & password management (margaretha)
+ - Removed codes related to user registration & password management (margaretha)
22/01/2019
- Added create, edit, retrieve user default setting controllers (margaretha)
22/01/2019
@@ -160,335 +161,335 @@
# version 0.61.3
17/10/2018
- - Updated NamedVCLoader to be optional (margaretha)
- - Updated annotation tables & implemented key-value structure (margaretha)
- - Added annotation parser for annotation data from kalamar (margaretha)
- - Implemented parsing free resource info from json (margaretha)
+ - Updated NamedVCLoader to be optional (margaretha)
+ - Updated annotation tables & implemented key-value structure (margaretha)
+ - Added annotation parser for annotation data from kalamar (margaretha)
+ - Implemented parsing free resource info from json (margaretha)
22/10/2018
- - Updated jetty, spring and hibernate versions (margaretha)
- - Fixed the order of annotation keys and values, and added tests (margaretha)
- - Fixed resource DAO & added tests (margaretha)
-24/10/2018
- - Fixed query serialization service (margaretha)
- - Added "highlights" parameter to matchInfo controller (margaretha)
- - Added "fields" parameter to search controllers (margaretha)
- - Integrated lite controllers, services and tests in full version (margaretha)
+ - Updated jetty, spring and hibernate versions (margaretha)
+ - Fixed the order of annotation keys and values, and added tests (margaretha)
+ - Fixed resource DAO & added tests (margaretha)
+24/10/2018
+ - Fixed query serialization service (margaretha)
+ - Added "highlights" parameter to matchInfo controller (margaretha)
+ - Added "fields" parameter to search controllers (margaretha)
+ - Integrated lite controllers, services and tests in full version (margaretha)
29/10/2018
- - Moved javax.servlet-api to core (margaretha)
+ - Moved javax.servlet-api to core (margaretha)
07/11/2018
- - OpenJDK8u181-workaround (see Debian Bug report #911925; diewald)
+ - OpenJDK8u181-workaround (see Debian Bug report #911925; diewald)
13/11/2018
- - Added Shutdown handler to Jetty server (margaretha)
- - Fixed storing VC order in NamedVCLoader (margaretha)
+ - Added Shutdown handler to Jetty server (margaretha)
+ - Fixed storing VC order in NamedVCLoader (margaretha)
# version 0.61.2
12/09/2018
- - Added various log4j2 configurations (margaretha)
+ - Added various log4j2 configurations (margaretha)
13/09/2018
- - Implemented VirtualCorpusRewrite (margaretha)
+ - Implemented VirtualCorpusRewrite (margaretha)
14/09/2018
- - Fixed SpringJerseyTest ApplicationContext (margaretha)
- - Handled VCRef with username in VirtualCorpusRewrite (margaretha)
- - Enabled VCReferenceTest in maven test suite (margaretha)
+ - Fixed SpringJerseyTest ApplicationContext (margaretha)
+ - Handled VCRef with username in VirtualCorpusRewrite (margaretha)
+ - Enabled VCReferenceTest in maven test suite (margaretha)
18/09/2018
- - Handled unique constraints / inserting duplicate items to DB (margaretha)
- - Added a controller for editing user group member roles (margaretha)
+ - Handled unique constraints / inserting duplicate items to DB (margaretha)
+ - Added a controller for editing user group member roles (margaretha)
19/09/2018
- - Fixed cached-VC ref with username (margaretha)
+ - Fixed cached-VC ref with username (margaretha)
# version 0.61.1
28/08/2018
- - Added API URL versioning (margaretha)
- - Deactivated IdRewrite (margaretha)
- - Fixed kustvakt controller (margaretha)
+ - Added API URL versioning (margaretha)
+ - Deactivated IdRewrite (margaretha)
+ - Fixed kustvakt controller (margaretha)
30/08/2018
- - Fixed root packages & added api version properties in kustvakt.conf
- (margaretha)
- - Fixed versioning in SearchController (margaretha)
- - Added API versioning tests (margaretha)
+ - Fixed root packages & added api version properties in kustvakt.conf
+ (margaretha)
+ - Fixed versioning in SearchController (margaretha)
+ - Added API versioning tests (margaretha)
# version 0.61.0
02/08/2018
- - Added VC reference tests (margaretha)
- - Implemented loading and caching named VCs (margaretha)
-03/08/2018
- - Implemented OAuth2 revoke token (margaretha)
- - Updated OAuth2 refresh token implementation (margaretha)
-14/08/2018
- - Implemented revoke all OAuth2 access tokens and authorization codes of
- client users when deregistering/deleting a client (margaretha)
- - Fixed update OAuth2 access token (margaretha)
- - Implemented reset client secret (margaretha)
- - Fixed revoking latest access token when refreshing OAuth2 access token
- (margaretha)
+ - Added VC reference tests (margaretha)
+ - Implemented loading and caching named VCs (margaretha)
+03/08/2018
+ - Implemented OAuth2 revoke token (margaretha)
+ - Updated OAuth2 refresh token implementation (margaretha)
+14/08/2018
+ - Implemented revoke all OAuth2 access tokens and authorization codes of
+ client users when deregistering/deleting a client (margaretha)
+ - Fixed update OAuth2 access token (margaretha)
+ - Implemented reset client secret (margaretha)
+ - Fixed revoking latest access token when refreshing OAuth2 access token
+ (margaretha)
15/08/2018
- - Implemented OAuth2 client info controller (margaretha)
- - Implemented update OAuth2 client privilege controller for admins
- (margaretha)
- - Implemented unlimited authorization scope for super clients with OAuth2
- password grant (margaretha)
- - Marked native clients implementation to deprecated in favour of super
- clients (margaretha)
- - Enabled using Bearer tokens as user authentication tokens (Authorization
- header value) for many controllers including OAuth2 controllers (margaretha)
+ - Implemented OAuth2 client info controller (margaretha)
+ - Implemented update OAuth2 client privilege controller for admins
+ (margaretha)
+ - Implemented unlimited authorization scope for super clients with OAuth2
+ password grant (margaretha)
+ - Marked native clients implementation to deprecated in favour of super
+ clients (margaretha)
+ - Enabled using Bearer tokens as user authentication tokens (Authorization
+ header value) for many controllers including OAuth2 controllers (margaretha)
16/08/2018
- - Implemented degrading super clients (margaretha)
- - Improved and added OAuth2 tests (margaretha)
+ - Implemented degrading super clients (margaretha)
+ - Improved and added OAuth2 tests (margaretha)
21/08/2018
- - Added VC name pattern check (margaretha)
+ - Added VC name pattern check (margaretha)
22/08/2018
- - Implemented loading VC from gz files (margaretha)
- - Updated OAuth2 authorization codes' & access tokens' expiry check
- (margaretha)
-23/08/2018
- - Updated RefreshToken implementations with separate DB tables (margaretha)
- - Allows multiple access tokens per refresh token (margaretha)
+ - Implemented loading VC from gz files (margaretha)
+ - Updated OAuth2 authorization codes' & access tokens' expiry check
+ (margaretha)
+23/08/2018
+ - Updated RefreshToken implementations with separate DB tables (margaretha)
+ - Allows multiple access tokens per refresh token (margaretha)
27/08/2018
- - Added statistic with VC reference tests (margaretha)
- - Fixed OAuth2 SQL files (margaretha)
+ - Added statistic with VC reference tests (margaretha)
+ - Fixed OAuth2 SQL files (margaretha)
28/08/2018
- - Added c3p0 datasource configuration to Spring default-config-xml
- (margaretha)
- - Added running Kustvakt server with custom spring config in the readme
- (margaretha)
- - Removed old OAuth2 codes (margaretha)
- - Moved non-config test codes to misc (margaretha)
-
+ - Added c3p0 datasource configuration to Spring default-config-xml
+ (margaretha)
+ - Added running Kustvakt server with custom spring config in the readme
+ (margaretha)
+ - Removed old OAuth2 codes (margaretha)
+ - Moved non-config test codes to misc (margaretha)
+
# version 0.60.5
09/07/2018
- - Added service layer to the search controller (margaretha)
- - Added OAuth2 scope checking in search and VC controllers (margaretha)
- - Added handling OAuth2 bearer token for VC access and User group
- controllers (margaretha)
- - Added default scope to password grant (margaretha)
+ - Added service layer to the search controller (margaretha)
+ - Added OAuth2 scope checking in search and VC controllers (margaretha)
+ - Added handling OAuth2 bearer token for VC access and User group
+ controllers (margaretha)
+ - Added default scope to password grant (margaretha)
10/07/2018
- - Made createBasicAuthorizationHeaderValue static (margaretha)
- - Added store access token in openID token service (margaretha)
- - Fixed empty scope in openID authorization and token service (margaretha)
- - Implemented storing authorization code in cache (margaretha)
-11/07/2018
- - Fixed authentication time in authentication controller (margaretha)
- - Added OAuth2 access token tests (margaretha)
-12/07/2018
- - Updated maven surefire setting for faster test suite runtime (margaretha)
- - Implemented refreshing OAuth2 access token (margaretha)
-26/07/2018
- - Fixed issue #27 (margaretha)
-02/08/2018
- - Fixed clientId encoding in OAuth2ClientControllerTest (margaretha)
+ - Made createBasicAuthorizationHeaderValue static (margaretha)
+ - Added store access token in openID token service (margaretha)
+ - Fixed empty scope in openID authorization and token service (margaretha)
+ - Implemented storing authorization code in cache (margaretha)
+11/07/2018
+ - Fixed authentication time in authentication controller (margaretha)
+ - Added OAuth2 access token tests (margaretha)
+12/07/2018
+ - Updated maven surefire setting for faster test suite runtime (margaretha)
+ - Implemented refreshing OAuth2 access token (margaretha)
+26/07/2018
+ - Fixed issue #27 (margaretha)
+02/08/2018s
+ - Fixed clientId encoding in OAuth2ClientControllerTest (margaretha)
-
+
# version 0.60.4
05/07/2018
- - implemented OAuth2 authorization code request with OpenID Authentication
- (margaretha)
- - enabled OAuth2 authorization without OpenID authentication using Nimbus
- library (margaretha)
- - implemented response handler for OpenID authentication errors in authorization
- requests (margaretha)
- - added tests regarding OpenID authentication in authorization requests
- (margaretha)
- - implemented OAuth2 authorization error response via redirect URI instead of
- JSON (margaretha)
- - added state to OAuth2 authorization error response (margaretha)
- - implemented OpenID token service for authorization code flow (margaretha)
- - implemented signed OpenID token with default algorithm RSA256 (margaretha)
- - implemented JSON Web Key (JWK) set web-controller listing kustvakt public
- keys (margaretha)
- - implemented OpenId configuration (margaretha)
- - added authentication time and support for auth_time in id_token (margaretha)
- - implemented support for nonce and max_age parameters in OpenID authentication
- (margaretha)
- - implemented OAuth2 token request with password grant using Nimbus library
- (margaretha)
- - updated redirect URI validator (margaretha)
- - updated client registration requirement to allow desktop applications
- (margaretha)
- - fixed RSA key configuration (margaretha)
- - merged OAuth2 client deregistration controllers (margaretha)
- - fixed OAuth2 client unique URL-hashcode (margaretha)
- - migrated logging to log4j 2 and adapted java.util.logging to log4j(margaretha)
- - Added support for unrestricted corpus statistics (ndiewald)
- - updated paths of user-group deletion-controllers (margaretha)
- - Do not pass broken queries to Krill (diewald)
- - added OAuth2 token request with client authentication via Authorization
- header (margaretha)
- - added port checking in test suite (margaretha)
-
+ - implemented OAuth2 authorization code request with OpenID Authentication
+ (margaretha)
+ - enabled OAuth2 authorization without OpenID authentication using Nimbus
+ library (margaretha)
+ - implemented response handler for OpenID authentication errors in authorization
+ requests (margaretha)
+ - added tests regarding OpenID authentication in authorization requests
+ (margaretha)
+ - implemented OAuth2 authorization error response via redirect URI instead of
+ JSON (margaretha)
+ - added state to OAuth2 authorization error response (margaretha)
+ - implemented OpenID token service for authorization code flow (margaretha)
+ - implemented signed OpenID token with default algorithm RSA256 (margaretha)
+ - implemented JSON Web Key (JWK) set web-controller listing kustvakt public
+ keys (margaretha)
+ - implemented OpenId configuration (margaretha)
+ - added authentication time and support for auth_time in id_token (margaretha)
+ - implemented support for nonce and max_age parameters in OpenID authentication
+ (margaretha)
+ - implemented OAuth2 token request with password grant using Nimbus library
+ (margaretha)
+ - updated redirect URI validator (margaretha)
+ - updated client registration requirement to allow desktop applications
+ (margaretha)
+ - fixed RSA key configuration (margaretha)
+ - merged OAuth2 client deregistration controllers (margaretha)
+ - fixed OAuth2 client unique URL-hashcode (margaretha)
+ - migrated logging to log4j 2 and adapted java.util.logging to log4j(margaretha)
+ - Added support for unrestricted corpus statistics (ndiewald)
+ - updated paths of user-group deletion-controllers (margaretha)
+ - Do not pass broken queries to Krill (diewald)
+ - added OAuth2 token request with client authentication via Authorization
+ header (margaretha)
+ - added port checking in test suite (margaretha)
+
# version 0.60.3
06/06/2018
- - improved user authentication by using authentication filter for authorization
- code request (margaretha)
- - limited client authentication to client id checking in authorization code
- request (margaretha)
- - added user_id in the oauth2_access_token database table (margaretha)
- - implemented OAuth2Authentication provider for token context management
- (margaretha)
- - added parameter checking for authorization DAO (margaretha)
- - added controller tests using OAuth2 access token (margaretha)
- - added database tables for MySQL (margaretha)
- - updated JWT library and related codes (margaretha)
-
+ - improved user authentication by using authentication filter for authorization
+ code request (margaretha)
+ - limited client authentication to client id checking in authorization code
+ request (margaretha)
+ - added user_id in the oauth2_access_token database table (margaretha)
+ - implemented OAuth2Authentication provider for token context management
+ (margaretha)
+ - added parameter checking for authorization DAO (margaretha)
+ - added controller tests using OAuth2 access token (margaretha)
+ - added database tables for MySQL (margaretha)
+ - updated JWT library and related codes (margaretha)
+
# version 0.60.2
03/05/2018
- - implemented OAuth2 client registration (margaretha)
- - implemented OAuth2 client authentication (margaretha)
- - changed virtual corpus search to retrieval (margaretha)
- - implemented public client deregistration task (margaretha)
- - added client registration and deregistration tests (margaretha)
- - implemented confidential client deregistration task (margaretha)
- - fixed storing client secret (margaretha)
- - implemented OAuth2 response handler (margaretha)
- - implemented OAuth2 request access token with client credentials grant
- (margaretha)
- - implemented OAuth2 request access token with resource owner password grant
- (margaretha)
- - implemented OAuth2 authorization code request (margaretha)
- - added OAuth2 error codes (margaretha)
- - added OAuth2 authorization, scope and access token tables for SQLite
- (margaretha)
- - implemented OAuth2 authorization, scope and access token DAO (margaretha)
- - implemented OAuth2 request access token with authorization code grant
- (margaretha)
- - added setting default scopes in the config file (margaretha)
- - fixed loading spring config multiple times in the test suite (margaretha)
- - added SQLite created_date trigger for access token (margaretha)
- - added a join table for access token scopes (margaretha)
- - added access scopes handling (margaretha)
- - added tests about request token with authorization code (margaretha)
-
+ - implemented OAuth2 client registration (margaretha)
+ - implemented OAuth2 client authentication (margaretha)
+ - changed virtual corpus search to retrieval (margaretha)
+ - implemented public client deregistration task (margaretha)
+ - added client registration and deregistration tests (margaretha)
+ - implemented confidential client deregistration task (margaretha)
+ - fixed storing client secret (margaretha)
+ - implemented OAuth2 response handler (margaretha)
+ - implemented OAuth2 request access token with client credentials grant
+ (margaretha)
+ - implemented OAuth2 request access token with resource owner password grant
+ (margaretha)
+ - implemented OAuth2 authorization code request (margaretha)
+ - added OAuth2 error codes (margaretha)
+ - added OAuth2 authorization, scope and access token tables for SQLite
+ (margaretha)
+ - implemented OAuth2 authorization, scope and access token DAO (margaretha)
+ - implemented OAuth2 request access token with authorization code grant
+ (margaretha)
+ - added setting default scopes in the config file (margaretha)
+ - fixed loading spring config multiple times in the test suite (margaretha)
+ - added SQLite created_date trigger for access token (margaretha)
+ - added a join table for access token scopes (margaretha)
+ - added access scopes handling (margaretha)
+ - added tests about request token with authorization code (margaretha)
+
# version 0.60.1
28/03/2018
- - added admin-related SQL codes (margaretha)
- - updated AdminDao (margaretha)
- - added optional username query parameter to group list controller (margaretha)
- - fixed non hierarchical URI of kustvakt conf files (margaretha)
- - added delete group member triggers (margaretha)
- - added list user-group by username and status for system admin (margaretha)
- - added user-group status in user-group DTO (margaretha)
- - added check for hidden groups in user-group tests (margaretha)
- - added database trigger test on deleting members when deleting group
- (margaretha)
- - renamed VC type PREDEFINED to SYSTEM (margaretha)
- - added VC list controller for system admin (margaretha)
- - added VC controller tests with for system admin (margaretha)
- - added hidden access removal when deleting published VC (margaretha)
- - added check for hidden groups in VC controller tests (margaretha)
- - added search user-group controller (margaretha)
- - removed createdBy from VirtualCorpusJson (margaretha)
- - moved member role setting from the invitation phase to the after-subscription
- phase (margaretha)
- - added member role removal after deleting members (margaretha)
- - added add and delete member role controllers (margaretha)
-
+ - added admin-related SQL codes (margaretha)
+ - updated AdminDao (margaretha)
+ - added optional username query parameter to group list controller (margaretha)
+ - fixed non hierarchical URI of kustvakt conf files (margaretha)
+ - added delete group member triggers (margaretha)
+ - added list user-group by username and status for system admin (margaretha)
+ - added user-group status in user-group DTO (margaretha)
+ - added check for hidden groups in user-group tests (margaretha)
+ - added database trigger test on deleting members when deleting group
+ (margaretha)
+ - renamed VC type PREDEFINED to SYSTEM (margaretha)
+ - added VC list controller for system admin (margaretha)
+ - added VC controller tests with for system admin (margaretha)
+ - added hidden access removal when deleting published VC (margaretha)
+ - added check for hidden groups in VC controller tests (margaretha)
+ - added search user-group controller (margaretha)
+ - removed createdBy from VirtualCorpusJson (margaretha)
+ - moved member role setting from the invitation phase to the after-subscription
+ phase (margaretha)
+ - added member role removal after deleting members (margaretha)
+ - added add and delete member role controllers (margaretha)
+
# version 0.60
14/03/2018
- - set up mail settings using localhost port 25 (margaretha)
- - added mail template in kustvakt configuration (margaretha)
- - added mail settings to readme (margaretha)
- - disabled email notification for auto group (margaretha)
- - added metadata retrieval (diewald)
- - enabled custom implementation for email address retrieval (margaretha)
- - removed old policy and deprecated code (margaretha)
- - moved authentication related code to /full (margaretha)
- - added userRoles attribute to UserGroupDto. (margaretha)
- - fixed sqlite trigger (margaretha)
- - fixed member exist error message (margaretha)
- - fixed member invitation to join deleted group (margaretha)
- - added checking deleted group (margaretha)
+ - set up mail settings using localhost port 25 (margaretha)
+ - added mail template in kustvakt configuration (margaretha)
+ - added mail settings to readme (margaretha)
+ - disabled email notification for auto group (margaretha)
+ - added metadata retrieval (diewald)
+ - enabled custom implementation for email address retrieval (margaretha)
+ - removed old policy and deprecated code (margaretha)
+ - moved authentication related code to /full (margaretha)
+ - added userRoles attribute to UserGroupDto. (margaretha)
+ - fixed sqlite trigger (margaretha)
+ - fixed member exist error message (margaretha)
+ - fixed member invitation to join deleted group (margaretha)
+ - added checking deleted group (margaretha)
# version 0.59.10
-
+
20/02/2018
- - added sort VC by id (margaretha)
- - added test cases regarding VC sharing (margaretha)
- - implemented withdraw VC from publication (margaretha)
- - added Changes file (margaretha)
- - implemented add/invite users to group (margaretha)
- - implemented delete user-group and member tasks (margaretha)
- - added userMemberStatus in group lists (margaretha)
- - updated and added SQL test data (margaretha)
- - added user group related tests (margaretha)
- - implemented custom configuration for deleting user groups and members (margaretha)
- - updated library versions and java environment (margaretha)
- - added expiration time check for member invitation (margaretha)
- - moved .properties files (margaretha)
- - merged changelog file to Changes (margaretha)
- - updated status codes and error messages to be more detailed (margaretha)
- - testing mail implementation using embedded jetty jndi (margaretha)
- - fixed collection rewrite regarding OR operation with other fields
- (margaretha)
- - implemented sending mail using spring injection and removed jetty jndi
- (margaretha)
- - fixed unrecognized application/json (margaretha)
- - fixed and updated velocity template (margaretha)
-
+ - added sort VC by id (margaretha)
+ - added test cases regarding VC sharing (margaretha)
+ - implemented withdraw VC from publication (margaretha)
+ - added Changes file (margaretha)
+ - implemented add/invite users to group (margaretha)
+ - implemented delete user-group and member tasks (margaretha)
+ - added userMemberStatus in group lists (margaretha)
+ - updated and added SQL test data (margaretha)
+ - added user group related tests (margaretha)
+ - implemented custom configuration for deleting user groups and members (margaretha)
+ - updated library versions and java environment (margaretha)
+ - added expiration time check for member invitation (margaretha)
+ - moved .properties files (margaretha)
+ - merged changelog file to Changes (margaretha)
+ - updated status codes and error messages to be more detailed (margaretha)
+ - testing mail implementation using embedded jetty jndi (margaretha)
+ - fixed collection rewrite regarding OR operation with other fields
+ (margaretha)
+ - implemented sending mail using spring injection and removed jetty jndi
+ (margaretha)
+ - fixed unrecognized application/json (margaretha)
+ - fixed and updated velocity template (margaretha)
+
# version 0.59.9
19/01/2018
- - restructured basic authentication (margaretha)
- - fixed AuthenticationException to include authentication scheme (margaretha)
- - fixed rewrite redundancy in collection rewrite (margaretha)
- - fixed foundry rewrite for constituents (margaretha)
- - introduced authentication methods, schemes and tokens (margaretha)
- - implemented collection rewrite with multiple licenses (margaretha)
- - fixed foundry rewrite for korap span without wrap node (margaretha)
- - implemented list user group (margaretha)
- - implemented delete VC task (margaretha)
- - implemented create user-group, subscribe to user-groups, unsubscribe to
- user-groups tasks(margaretha)
- - fixed handling JSON mapping exception for missing enums (margaretha)
- - implemented list VC task (margaretha)
- - added KoralQuery in VC lists (margaretha)
- - implemented edit VC task (margaretha)
- - implemented publish VC task (margaretha)
- - implemented share VC task (margaretha)
- - implemented list only owned VC task (margaretha)
- - implemented list VC access task (margaretha)
- - implemented search VC by id task (margaretha)
- - implemented delete VC access (margaretha)
- - implemented search for project VC (margaretha)
- - added search VC related tests (margaretha)
- - removed PredefinedUserGroup.ALL and related codes (margaretha)
- - implemented search for published VC (margaretha)
-
+ - restructured basic authentication (margaretha)
+ - fixed AuthenticationException to include authentication scheme (margaretha)
+ - fixed rewrite redundancy in collection rewrite (margaretha)
+ - fixed foundry rewrite for constituents (margaretha)
+ - introduced authentication methods, schemes and tokens (margaretha)
+ - implemented collection rewrite with multiple licenses (margaretha)
+ - fixed foundry rewrite for korap span without wrap node (margaretha)
+ - implemented list user group (margaretha)
+ - implemented delete VC task (margaretha)
+ - implemented create user-group, subscribe to user-groups, unsubscribe to
+ user-groups tasks(margaretha)
+ - fixed handling JSON mapping exception for missing enums (margaretha)
+ - implemented list VC task (margaretha)
+ - added KoralQuery in VC lists (margaretha)
+ - implemented edit VC task (margaretha)
+ - implemented publish VC task (margaretha)
+ - implemented share VC task (margaretha)
+ - implemented list only owned VC task (margaretha)
+ - implemented list VC access task (margaretha)
+ - implemented search VC by id task (margaretha)
+ - implemented delete VC access (margaretha)
+ - implemented search for project VC (margaretha)
+ - added search VC related tests (margaretha)
+ - removed PredefinedUserGroup.ALL and related codes (margaretha)
+ - implemented search for published VC (margaretha)
+
# version 0.59.8
21/09/2017
- - restructured statistics service (margaretha)
- - removed deprecated loader codes and tests (margaretha)
- - removed old Spring java configurations (margaretha)
- - implemented entity classes for the new database (margaretha)
- - added MySQL codes regarding VC and for testing (margaretha)
- - added dao methods regarding VC (margaretha)
- - added similar SQL codes (to MySQL) for sqlite (margaretha)
- - added dao methods regarding user groups (margaretha)
- - restructured web-service codes into controller and logic/business-service
- (margaretha)
- - implemented user role and privilege, and added tests (margaretha)
- - prepared test suite using new database (margaretha)
- - implemented UserGroupDao and tests (margaretha)
- - fixed missing exceptions in JsonUtils (margaretha)
- - restructured web filters and authentication codes (margaretha)
- - implemented create/store VC (margaretha)
- - fixed collection rewrite bug regarding availability with operation or
- (margaretha)
+ - restructured statistics service (margaretha)
+ - removed deprecated loader codes and tests (margaretha)
+ - removed old Spring java configurations (margaretha)
+ - implemented entity classes for the new database (margaretha)
+ - added MySQL codes regarding VC and for testing (margaretha)
+ - added dao methods regarding VC (margaretha)
+ - added similar SQL codes (to MySQL) for sqlite (margaretha)
+ - added dao methods regarding user groups (margaretha)
+ - restructured web-service codes into controller and logic/business-service
+ (margaretha)
+ - implemented user role and privilege, and added tests (margaretha)
+ - prepared test suite using new database (margaretha)
+ - implemented UserGroupDao and tests (margaretha)
+ - fixed missing exceptions in JsonUtils (margaretha)
+ - restructured web filters and authentication codes (margaretha)
+ - implemented create/store VC (margaretha)
+ - fixed collection rewrite bug regarding availability with operation or
+ (margaretha)
# version 0.59.7
13/10/2016
- - MOD: updated search to use new siglen (diewald)
- - MOD: fixed matchinfo retrieval in light service (diewald)
+ - MOD: updated search to use new siglen (diewald)
+ - MOD: fixed matchinfo retrieval in light service (diewald)
05/05/2015
- - ADD: rest test suite for user service (hanl)
- - MOD: setup parameter modification (hanl)
- - ADD: oauth2 client unique constraint (hanl)
+ - ADD: rest test suite for user service (hanl)
+ - MOD: setup parameter modification (hanl)
+ - ADD: oauth2 client unique constraint (hanl)
diff --git a/full/src/test/java/de/ids_mannheim/korap/web/controller/SearchPipeTest.java b/full/src/test/java/de/ids_mannheim/korap/web/controller/SearchPipeTest.java
index 9c55d6e..838b74c 100644
--- a/full/src/test/java/de/ids_mannheim/korap/web/controller/SearchPipeTest.java
+++ b/full/src/test/java/de/ids_mannheim/korap/web/controller/SearchPipeTest.java
@@ -2,72 +2,198 @@
import static org.junit.Assert.assertEquals;
-import java.io.File;
-import java.io.FileOutputStream;
import java.io.IOException;
-import java.io.OutputStreamWriter;
import org.junit.Test;
-import org.springframework.beans.factory.annotation.Autowired;
import com.fasterxml.jackson.databind.JsonNode;
import com.sun.jersey.api.client.ClientResponse;
-import de.ids_mannheim.korap.config.KustvaktConfiguration;
import de.ids_mannheim.korap.config.SpringJerseyTest;
import de.ids_mannheim.korap.exceptions.KustvaktException;
+import de.ids_mannheim.korap.exceptions.StatusCodes;
import de.ids_mannheim.korap.utils.JsonUtils;
public class SearchPipeTest extends SpringJerseyTest {
- @Autowired
- private KustvaktConfiguration config;
-
- private void setTestPipes () throws IOException {
- String filename = "test-pipes";
- File f = new File(filename);
- if (f.exists()) {
- f.delete();
- }
- f.createNewFile();
- OutputStreamWriter writer =
- new OutputStreamWriter(new FileOutputStream(f));
- writer.append("glemm\t");
- writer.append(resource().getURI().toString());
- writer.append(API_VERSION);
- writer.append("/test/glemm");
- writer.flush();
- writer.close();
-
- config.readPipesFile(filename);
- }
-
@Test
public void testSearchWithPipes () throws IOException, KustvaktException {
- setTestPipes();
+ String glemmUri =
+ resource().getURI().toString() + API_VERSION + "/test/glemm";
ClientResponse response = resource().path(API_VERSION).path("search")
.queryParam("q", "[orth=der]").queryParam("ql", "poliqarp")
- .queryParam("pipes", "glemm").get(ClientResponse.class);
+ .queryParam("pipes", glemmUri).get(ClientResponse.class);
+
+ assertEquals(ClientResponse.Status.OK.getStatusCode(),
+ response.getStatus());
String entity = response.getEntity(String.class);
JsonNode node = JsonUtils.readTree(entity);
- assertEquals(3, node.at("/query/wrap/key").size());
-
+ assertEquals(2, node.at("/query/wrap/key").size());
+
assertEquals(1, node.at("/collection/rewrites").size());
assertEquals("operation:insertion",
node.at("/collection/rewrites/0/operation").asText());
assertEquals("availability(FREE)",
node.at("/collection/rewrites/0/scope").asText());
-
+
node = node.at("/query/wrap/rewrites");
assertEquals(2, node.size());
assertEquals("Glemm", node.at("/0/src").asText());
assertEquals("operation:override", node.at("/0/operation").asText());
assertEquals("key", node.at("/0/scope").asText());
-
+
assertEquals("Kustvakt", node.at("/1/src").asText());
assertEquals("operation:injection", node.at("/1/operation").asText());
assertEquals("foundry", node.at("/1/scope").asText());
}
+
+ @Test
+ public void testSearchWithMultiplePipes () throws KustvaktException {
+ String glemmUri =
+ resource().getURI().toString() + API_VERSION + "/test/glemm";
+ String glemmUri2 = glemmUri + "?param=blah";
+ ClientResponse response = resource().path(API_VERSION).path("search")
+ .queryParam("q", "[orth=der]").queryParam("ql", "poliqarp")
+ .queryParam("pipes", glemmUri + "," + glemmUri2)
+ .get(ClientResponse.class);
+
+ assertEquals(ClientResponse.Status.OK.getStatusCode(),
+ response.getStatus());
+
+ String entity = response.getEntity(String.class);
+ JsonNode node = JsonUtils.readTree(entity);
+ assertEquals(3, node.at("/query/wrap/key").size());
+ }
+
+ @Test
+ public void testSearchWithUnknownURL ()
+ throws IOException, KustvaktException {
+ String url =
+ resource().getURI().toString() + API_VERSION + "/test/tralala";
+ ClientResponse response = resource().path(API_VERSION).path("search")
+ .queryParam("q", "[orth=der]").queryParam("ql", "poliqarp")
+ .queryParam("pipes", url).get(ClientResponse.class);
+
+ assertEquals(ClientResponse.Status.OK.getStatusCode(),
+ response.getStatus());
+ String entity = response.getEntity(String.class);
+
+ JsonNode node = JsonUtils.readTree(entity);
+ assertEquals(StatusCodes.PIPE_FAILED, node.at("/warnings/0/0").asInt());
+ }
+
+ @Test
+ public void testSearchWithUnknownHost () throws KustvaktException {
+ ClientResponse response = resource().path(API_VERSION).path("search")
+ .queryParam("q", "[orth=der]").queryParam("ql", "poliqarp")
+ .queryParam("pipes", "http://glemm").get(ClientResponse.class);
+
+ assertEquals(ClientResponse.Status.OK.getStatusCode(),
+ response.getStatus());
+
+ String entity = response.getEntity(String.class);
+ JsonNode node = JsonUtils.readTree(entity);
+
+ assertEquals(StatusCodes.PIPE_FAILED, node.at("/warnings/0/0").asInt());
+ }
+
+ @Test
+ public void testSearchWithUrlEncodedPipe () throws KustvaktException {
+ String pipe = resource().getURI().toString() + API_VERSION
+ + "/test/urlencoded-pipe";
+ ClientResponse response = resource().path(API_VERSION).path("search")
+ .queryParam("q", "[orth=der]").queryParam("ql", "poliqarp")
+ .queryParam("pipes", pipe).get(ClientResponse.class);
+
+ String entity = response.getEntity(String.class);
+ assertEquals(ClientResponse.Status.OK.getStatusCode(),
+ response.getStatus());
+
+ JsonNode node = JsonUtils.readTree(entity);
+ assertEquals(StatusCodes.PIPE_FAILED, node.at("/warnings/0/0").asInt());
+ assertEquals("415 Unsupported Media Type",
+ node.at("/warnings/0/3").asText());
+ }
+
+ @Test
+ public void testSearchWithMultiplePipeWarnings () throws KustvaktException {
+ String url =
+ resource().getURI().toString() + API_VERSION + "/test/tralala";
+ ClientResponse response = resource().path(API_VERSION).path("search")
+ .queryParam("q", "[orth=der]").queryParam("ql", "poliqarp")
+ .queryParam("pipes", url + "," + "http://glemm")
+ .get(ClientResponse.class);
+
+ assertEquals(ClientResponse.Status.OK.getStatusCode(),
+ response.getStatus());
+
+ String entity = response.getEntity(String.class);
+ JsonNode node = JsonUtils.readTree(entity);
+
+ assertEquals(2, node.at("/warnings").size());
+ assertEquals(StatusCodes.PIPE_FAILED, node.at("/warnings/0/0").asInt());
+ assertEquals(url, node.at("/warnings/0/2").asText());
+ assertEquals("404 Not Found", node.at("/warnings/0/3").asText());
+
+ assertEquals(StatusCodes.PIPE_FAILED, node.at("/warnings/1/0").asInt());
+ assertEquals("http://glemm", node.at("/warnings/1/2").asText());
+ assertEquals("java.net.UnknownHostException: glemm",
+ node.at("/warnings/1/3").asText());
+
+ }
+
+ @Test
+ public void testSearchWithInvalidJsonResponse () throws KustvaktException {
+ String pipe = resource().getURI().toString() + API_VERSION
+ + "/test/invalid-json-pipe";
+ ClientResponse response = resource().path(API_VERSION).path("search")
+ .queryParam("q", "[orth=der]").queryParam("ql", "poliqarp")
+ .queryParam("pipes", pipe).get(ClientResponse.class);
+
+ String entity = response.getEntity(String.class);
+ assertEquals(ClientResponse.Status.BAD_REQUEST.getStatusCode(),
+ response.getStatus());
+
+ JsonNode node = JsonUtils.readTree(entity);
+ assertEquals(StatusCodes.DESERIALIZATION_FAILED,
+ node.at("/errors/0/0").asInt());
+ }
+
+ @Test
+ public void testSearchWithPlainTextResponse () throws KustvaktException {
+ String pipe = resource().getURI().toString() + API_VERSION
+ + "/test/plain-response-pipe";
+ ClientResponse response = resource().path(API_VERSION).path("search")
+ .queryParam("q", "[orth=der]").queryParam("ql", "poliqarp")
+ .queryParam("pipes", pipe).get(ClientResponse.class);
+
+ String entity = response.getEntity(String.class);
+ assertEquals(ClientResponse.Status.BAD_REQUEST.getStatusCode(),
+ response.getStatus());
+
+ JsonNode node = JsonUtils.readTree(entity);
+ assertEquals(StatusCodes.DESERIALIZATION_FAILED,
+ node.at("/errors/0/0").asInt());
+ }
+
+ @Test
+ public void testSearchWithMultipleAndUnknownPipes ()
+ throws KustvaktException {
+ String glemmUri =
+ resource().getURI().toString() + API_VERSION + "/test/glemm";
+ ClientResponse response = resource().path(API_VERSION).path("search")
+ .queryParam("q", "[orth=der]").queryParam("ql", "poliqarp")
+ .queryParam("pipes", "http://unknown" + "," + glemmUri)
+ .get(ClientResponse.class);
+
+ assertEquals(ClientResponse.Status.OK.getStatusCode(),
+ response.getStatus());
+
+ String entity = response.getEntity(String.class);
+ JsonNode node = JsonUtils.readTree(entity);
+ assertEquals(StatusCodes.PIPE_FAILED, node.at("/warnings/0/0").asInt());
+ assertEquals(2, node.at("/query/wrap/key").size());
+ }
}
diff --git a/lite/Changes b/lite/Changes
index 0a06d87..287819e 100644
--- a/lite/Changes
+++ b/lite/Changes
@@ -3,7 +3,9 @@
- Implemented pipe extension in the search API (margaretha)
12/12/2019
- Added support for multiple cq parameters (margaretha, resolved #46)
-
+13/12/2019
+ - Handled pipe errors and added tests (margaretha)
+
# version 0.62.2
13/11/2019
- Added tests for issue #43 (margaretha)
@@ -17,21 +19,21 @@
version 0.62
19/03/2019
- - Added close index controller (margaretha)
+ - Added close index controller (margaretha)
version 0.61.3
05/02/2019
- - Updated kustvakt-lite.conf (margaretha)
+ - Updated kustvakt-lite.conf (margaretha)
18/02/2019
- - Updated core version (margaretha)
+ - Updated core version (margaretha)
version 0.61.2
14/11/2018
- Integrated lite and full services and controllers in core (margaretha)
- Added annotation services (margaretha)
19/12/2018
- - Fixed getUser in DummyAuthenticationManager (margaretha)
- - Fixed search test with fields parameter (margaretha)
+ - Fixed getUser in DummyAuthenticationManager (margaretha)
+ - Fixed search test with fields parameter (margaretha)
version 0.61.1
22/10/2018
@@ -39,9 +41,9 @@
29/10/2018
- Added rewrite handler post construct (margaretha)
07/11/2018
- - OpenJDK8u181-workaround (see Debian Bug report #911925; diewald)
+ - OpenJDK8u181-workaround (see Debian Bug report #911925; diewald)
13/11/2018
- - Added shutdown handler (margaretha)
+ - Added shutdown handler (margaretha)
version 0.61.0
30/08/2018
@@ -50,38 +52,38 @@
version 0.60.2
05/07/2018
- - Added support for unrestricted corpus statistics (ndiewald)
- - Do not pass broken queries to Krill (diewald)
+ - Added support for unrestricted corpus statistics (ndiewald)
+ - Do not pass broken queries to Krill (diewald)
27/08/2018
- - Added vc reference test (margaretha)
+ - Added vc reference test (margaretha)
28/08/2018
- Added API URL versioning (margaretha)
version 0.60.1
12/06/2018
- - removed kustvakt_encryption (margaretha)
- - updated kustvakt core version (margaretha)
- - adopted statistics query parameter from full (ndiewald)
-
+ - removed kustvakt_encryption (margaretha)
+ - updated kustvakt core version (margaretha)
+ - adopted statistics query parameter from full (ndiewald)
+
version 0.60
-23/03/2018
- - updated field type:date in the metadata test (margaretha)
- - removed loading application context via BeanFactory (margaretha)
- - fixed missing kustvakt-lite.conf (margaretha)
- - added support for highlights in matchinfo (diewald)
+23/03/2018
+ - updated field type:date in the metadata test (margaretha)
+ - removed loading application context via BeanFactory (margaretha)
+ - fixed missing kustvakt-lite.conf (margaretha)
+ - added support for highlights in matchinfo (diewald)
version 0.59.9
01/02/2018
- - renamed light to lite (margaretha)
- - added Changes file (margaretha)
- - updated library versions and java environment (margaretha)
- - added metadata test (diewald)
-
+ - renamed light to lite (margaretha)
+ - added Changes file (margaretha)
+ - updated library versions and java environment (margaretha)
+ - added metadata test (diewald)
+
version 0.59.8
17/01/2018
- - restructured Kustvakt and created /lite project
- - removed version from service paths (margaretha)
- - updated query serialization tests (margaretha)
- - added statistic service test (margaretha)
- - updated KustvaktResponseHandler (margaretha)
- - removed FastJerseyLight and simplified Jersey test configuration (margaretha)
\ No newline at end of file
+ - restructured Kustvakt and created /lite project
+ - removed version from service paths (margaretha)
+ - updated query serialization tests (margaretha)
+ - added statistic service test (margaretha)
+ - updated KustvaktResponseHandler (margaretha)
+ - removed FastJerseyLight and simplified Jersey test configuration (margaretha)
\ No newline at end of file
diff --git a/lite/src/test/java/de/ids_mannheim/korap/web/service/LiteSearchPipeTest.java b/lite/src/test/java/de/ids_mannheim/korap/web/service/LiteSearchPipeTest.java
index 5e8d77f..3f0c0bb 100644
--- a/lite/src/test/java/de/ids_mannheim/korap/web/service/LiteSearchPipeTest.java
+++ b/lite/src/test/java/de/ids_mannheim/korap/web/service/LiteSearchPipeTest.java
@@ -2,64 +2,191 @@
import static org.junit.Assert.assertEquals;
-import java.io.File;
-import java.io.FileOutputStream;
import java.io.IOException;
-import java.io.OutputStreamWriter;
import org.junit.Test;
-import org.springframework.beans.factory.annotation.Autowired;
import com.fasterxml.jackson.databind.JsonNode;
import com.sun.jersey.api.client.ClientResponse;
-import de.ids_mannheim.korap.config.KustvaktConfiguration;
import de.ids_mannheim.korap.config.LiteJerseyTest;
import de.ids_mannheim.korap.exceptions.KustvaktException;
+import de.ids_mannheim.korap.exceptions.StatusCodes;
import de.ids_mannheim.korap.utils.JsonUtils;
public class LiteSearchPipeTest extends LiteJerseyTest {
- @Autowired
- private KustvaktConfiguration config;
-
- private void setTestPipes () throws IOException {
- String filename = "test-pipes";
- File f = new File(filename);
- if (f.exists()) {
- f.delete();
- }
- f.createNewFile();
- OutputStreamWriter writer =
- new OutputStreamWriter(new FileOutputStream(f));
- writer.append("glemm\t");
- writer.append(resource().getURI().toString());
- writer.append(API_VERSION);
- writer.append("/test/glemm");
- writer.flush();
- writer.close();
-
- config.readPipesFile(filename);
- }
-
@Test
public void testSearchWithPipes () throws IOException, KustvaktException {
- setTestPipes();
+ String glemmUri =
+ resource().getURI().toString() + API_VERSION + "/test/glemm";
ClientResponse response = resource().path(API_VERSION).path("search")
.queryParam("q", "[orth=der]").queryParam("ql", "poliqarp")
- .queryParam("pipes", "glemm").get(ClientResponse.class);
+ .queryParam("pipes", glemmUri).get(ClientResponse.class);
+
+ assertEquals(ClientResponse.Status.OK.getStatusCode(),
+ response.getStatus());
String entity = response.getEntity(String.class);
+
JsonNode node = JsonUtils.readTree(entity);
- assertEquals(3, node.at("/query/wrap/key").size());
+ assertEquals(2, node.at("/query/wrap/key").size());
node = node.at("/query/wrap/rewrites");
assertEquals(2, node.size());
assertEquals("Glemm", node.at("/0/src").asText());
assertEquals("operation:override", node.at("/0/operation").asText());
assertEquals("key", node.at("/0/scope").asText());
-
+
assertEquals("Kustvakt", node.at("/1/src").asText());
assertEquals("operation:injection", node.at("/1/operation").asText());
assertEquals("foundry", node.at("/1/scope").asText());
}
+
+ @Test
+ public void testSearchWithMultiplePipes () throws KustvaktException {
+ String glemmUri =
+ resource().getURI().toString() + API_VERSION + "/test/glemm";
+ String glemmUri2 = glemmUri + "?param=blah";
+ ClientResponse response = resource().path(API_VERSION).path("search")
+ .queryParam("q", "[orth=der]").queryParam("ql", "poliqarp")
+ .queryParam("pipes", glemmUri + "," + glemmUri2)
+ .get(ClientResponse.class);
+
+ assertEquals(ClientResponse.Status.OK.getStatusCode(),
+ response.getStatus());
+
+ String entity = response.getEntity(String.class);
+ JsonNode node = JsonUtils.readTree(entity);
+ assertEquals(3, node.at("/query/wrap/key").size());
+ }
+
+ @Test
+ public void testSearchWithUnknownURL ()
+ throws IOException, KustvaktException {
+ String url =
+ resource().getURI().toString() + API_VERSION + "/test/tralala";
+ ClientResponse response = resource().path(API_VERSION).path("search")
+ .queryParam("q", "[orth=der]").queryParam("ql", "poliqarp")
+ .queryParam("pipes", url).get(ClientResponse.class);
+
+ assertEquals(ClientResponse.Status.OK.getStatusCode(),
+ response.getStatus());
+ String entity = response.getEntity(String.class);
+
+ JsonNode node = JsonUtils.readTree(entity);
+ assertEquals(StatusCodes.PIPE_FAILED, node.at("/warnings/0/0").asInt());
+ }
+
+ @Test
+ public void testSearchWithUnknownHost () throws KustvaktException {
+ ClientResponse response = resource().path(API_VERSION).path("search")
+ .queryParam("q", "[orth=der]").queryParam("ql", "poliqarp")
+ .queryParam("pipes", "http://glemm").get(ClientResponse.class);
+
+ assertEquals(ClientResponse.Status.OK.getStatusCode(),
+ response.getStatus());
+
+ String entity = response.getEntity(String.class);
+ JsonNode node = JsonUtils.readTree(entity);
+
+ assertEquals(StatusCodes.PIPE_FAILED, node.at("/warnings/0/0").asInt());
+ }
+
+ @Test
+ public void testSearchWithUrlEncodedPipe () throws KustvaktException {
+ String pipe = resource().getURI().toString() + API_VERSION
+ + "/test/urlencoded-pipe";
+ ClientResponse response = resource().path(API_VERSION).path("search")
+ .queryParam("q", "[orth=der]").queryParam("ql", "poliqarp")
+ .queryParam("pipes", pipe).get(ClientResponse.class);
+
+ String entity = response.getEntity(String.class);
+ assertEquals(ClientResponse.Status.OK.getStatusCode(),
+ response.getStatus());
+
+ JsonNode node = JsonUtils.readTree(entity);
+ assertEquals(StatusCodes.PIPE_FAILED, node.at("/warnings/0/0").asInt());
+ assertEquals("415 Unsupported Media Type",
+ node.at("/warnings/0/3").asText());
+ }
+
+ @Test
+ public void testSearchWithMultiplePipeWarnings () throws KustvaktException {
+ String url =
+ resource().getURI().toString() + API_VERSION + "/test/tralala";
+ ClientResponse response = resource().path(API_VERSION).path("search")
+ .queryParam("q", "[orth=der]").queryParam("ql", "poliqarp")
+ .queryParam("pipes", url + "," + "http://glemm")
+ .get(ClientResponse.class);
+
+ assertEquals(ClientResponse.Status.OK.getStatusCode(),
+ response.getStatus());
+
+ String entity = response.getEntity(String.class);
+ JsonNode node = JsonUtils.readTree(entity);
+
+ assertEquals(2, node.at("/warnings").size());
+ assertEquals(StatusCodes.PIPE_FAILED, node.at("/warnings/0/0").asInt());
+ assertEquals(url, node.at("/warnings/0/2").asText());
+ assertEquals("404 Not Found", node.at("/warnings/0/3").asText());
+
+ assertEquals(StatusCodes.PIPE_FAILED, node.at("/warnings/1/0").asInt());
+ assertEquals("http://glemm", node.at("/warnings/1/2").asText());
+ assertEquals("java.net.UnknownHostException: glemm",
+ node.at("/warnings/1/3").asText());
+
+ }
+
+ @Test
+ public void testSearchWithInvalidJsonResponse () throws KustvaktException {
+ String pipe = resource().getURI().toString() + API_VERSION
+ + "/test/invalid-json-pipe";
+ ClientResponse response = resource().path(API_VERSION).path("search")
+ .queryParam("q", "[orth=der]").queryParam("ql", "poliqarp")
+ .queryParam("pipes", pipe).get(ClientResponse.class);
+
+ String entity = response.getEntity(String.class);
+ assertEquals(ClientResponse.Status.BAD_REQUEST.getStatusCode(),
+ response.getStatus());
+
+ JsonNode node = JsonUtils.readTree(entity);
+ assertEquals(StatusCodes.DESERIALIZATION_FAILED,
+ node.at("/errors/0/0").asInt());
+ }
+
+ @Test
+ public void testSearchWithPlainTextResponse () throws KustvaktException {
+ String pipe = resource().getURI().toString() + API_VERSION
+ + "/test/plain-response-pipe";
+ ClientResponse response = resource().path(API_VERSION).path("search")
+ .queryParam("q", "[orth=der]").queryParam("ql", "poliqarp")
+ .queryParam("pipes", pipe).get(ClientResponse.class);
+
+ String entity = response.getEntity(String.class);
+ assertEquals(ClientResponse.Status.BAD_REQUEST.getStatusCode(),
+ response.getStatus());
+
+ JsonNode node = JsonUtils.readTree(entity);
+ assertEquals(StatusCodes.DESERIALIZATION_FAILED,
+ node.at("/errors/0/0").asInt());
+ }
+
+ @Test
+ public void testSearchWithMultipleAndUnknownPipes ()
+ throws KustvaktException {
+ String glemmUri =
+ resource().getURI().toString() + API_VERSION + "/test/glemm";
+ ClientResponse response = resource().path(API_VERSION).path("search")
+ .queryParam("q", "[orth=der]").queryParam("ql", "poliqarp")
+ .queryParam("pipes", "http://unknown" + "," + glemmUri)
+ .get(ClientResponse.class);
+
+ assertEquals(ClientResponse.Status.OK.getStatusCode(),
+ response.getStatus());
+
+ String entity = response.getEntity(String.class);
+ JsonNode node = JsonUtils.readTree(entity);
+ assertEquals(StatusCodes.PIPE_FAILED, node.at("/warnings/0/0").asInt());
+ assertEquals(2, node.at("/query/wrap/key").size());
+ }
}