Removed all user group and implemented search published VC.

Change-Id: I225ec59fd8f7a08b58733d06ae4062b3b7910fe3
diff --git a/README.md b/README.md
index 9d12031..319603e 100644
--- a/README.md
+++ b/README.md
@@ -2,7 +2,7 @@
 
 Kustvakt is a user and policy management component for KorAP (Diewald et al., 2016). It manages user access to resources (i.e. corpus data) that is typically bound with some licensing schemes. The licensing schemes of the IDS resources provided through KorAP (DeReKo) are very complex involving the access location and purposes (Kupietz & Lüngen, 2014). To manage user access to resources, Kustvakt performs query rewriting with document restrictions (BaƄski et al., 2014).
 
-Kustvakt acts as a middleware in KorAP binding other components, such as Koral a query serializer and Krill a search component, together. As KorAP's API provider, it provides services, e.g. searching and retrieving annotation data of a match/hit, that can be used by a client, e.g. [Kalamar](https://github.com/KorAP/Kalamar) (a KorAP web user interface) and [KorapSRU](https://github.com/KorAP/KorapSRU) (the CLARIN FCS endpoint for KorAP).
+Kustvakt acts as a middleware in KorAP binding other components, such as [Koral](https://github.com/KorAP/Koral) a query serializer and [Krill](https://github.com/KorAP/Krill) a search component, together. As KorAP's API provider, it provides services, e.g. searching and retrieving annotation data of a match/hit, that can be used by a client, e.g. [Kalamar](https://github.com/KorAP/Kalamar) (a KorAP web user interface) and [KorapSRU](https://github.com/KorAP/KorapSRU) (the CLARIN FCS endpoint for KorAP).
 
 # Versions
 * <b>Kustvakt lite version</b>
@@ -13,9 +13,9 @@
   
   provides user and policy management and extended services (e.g. resource and annotation services) in addition to the basic services. This version requires a database (Sqlite is provided) and an LDAP system for user authentication.
   
-# Services
+# Web-services
 
-Services including their usage examples are described in the [wiki](https://github.com/KorAP/Kustvakt/wiki).
+Web-services including their usage examples are described in the [wiki](https://github.com/KorAP/Kustvakt/wiki).
 
 
 # Setup
@@ -63,7 +63,8 @@
 Set krill.indexDir in the configuration file to the location of your Krill index (relative path). In Kustvakt root directory, there is a sample index, e.g.
 <pre>krill.indexDir = ../../sample-index</pre>
 
-Set the location of the ldap configuration file for Kustvakt full version. The file should contain an admin password to access an LDAP system. Without LDAP, user authentication functions and services cannot be used.
+Set the location of the LDAP configuration file for Kustvakt full version. The file should contain an admin password to access an LDAP system. Without LDAP, user authentication functions and services cannot be used. However, the authentication mechanism can be extended by implementing other authentication methods e.g. using a database. 
+
 
 <b>Optional custom configuration</b>
 
@@ -83,7 +84,7 @@
 # Running Kustvakt Server
 Requires ```kustvakt.conf``` or ```kustvakt-lite.conf``` in the same folder as the jar file. Otherwise assuming sample-index located in the parent directory of the jar file.
 
-Kustvakt full version requires an LDAP configuration file containing an admin password to access an LDAP system. You can still run Kustvakt full version without an LDAP system, but user authentication functions and services cannot be used. Only services for guest/demo user would work.
+Kustvakt full version requires an LDAP configuration file containing an admin password to access an LDAP system. You can still run Kustvakt full version without an LDAP system, but user authentication functions and services cannot be used. Only services for guest/demo user would be available.
 
 <pre>
 cd target/
@@ -93,14 +94,14 @@
 
 # Futher Setup for Developer
 
-For working with an IDE, you need to install lombok for your tool. Go to the directory of your lombok.jar, e.g \.m2\repository\org\projectlombok\lombok\1.16.6 and run
+Installing lombok is necessary when working with an IDE. Go to the directory of your lombok.jar, e.g \.m2\repository\org\projectlombok\lombok\1.16.6 and run
 <pre>
 java -jar lombok-1.16.6.jar
 </pre>
 
 Restart your IDE and clean your project.
 
-Copy ```kustvakt.conf``` or ```kustvakt-lite.conf``` from  src/main/resources to the full/ or lite folder. Then the properties in the kustvakt.conf or kustvakt-lite.conf file can be customized.
+Copy ```kustvakt.conf``` or ```kustvakt-lite.conf``` from  src/main/resources to the full/ or lite/ folder. Then the properties the configuration file can be customized.
 
 In an IDE, you can run ```KustvaktLiteServer``` or ```KustvaktServer``` as a normal Java application.
 
@@ -108,16 +109,19 @@
 
 The default Sqlite database can be switch to a MySQL database.
 
-Copy the jdbc.properties from full/src/main/resources to the full/ directory. Do not change the filename.
+Copy ```jdbc.properties``` from full/src/main/resources to the full/ directory. Do not change the filename.
 <pre>
 cp full/src/main/resources/jdbc.properties full/
 </pre>
 
 Remove or comment the Sqlite Setting.
 
-Uncomment the MySQL Setting and fill in the correct values for the jdbc.url, jdbc.username and jdbc.password.
+Uncomment the MySQL Setting and fill in the correct values for the ```jdbc.url```,
+ ```jdbc.username```
+  and ```jdbc.password```.
 
-The default setting for jdbc.schemaPath includes test data defined in ```full/src/main/resources/db/insert```
+The default setting for ```jdbc.schemaPath```
+includes test data defined in ```full/src/main/resources/db/insert```
 and some user roles defined in ```full/src/main/resources/db/predefined```. You can omit the test data by removing
  ```db.insert```.
 
@@ -137,7 +141,7 @@
 While running ```KustvaktServer``` or ```Kustvakt-full-[version].jar```,
 MySQL tables will be created to the specified database from the SQL files in 
 ```full/src/main/resources/db/new-mysql``` and other paths specified in 
-jdbc.schemaPath.
+```jdbc.schemaPath```.
 
 # Known issues
 Tests are verbose - they do not necessarily imply system errors.
diff --git a/full/src/main/java/de/ids_mannheim/korap/constant/PredefinedUserGroup.java b/full/src/main/java/de/ids_mannheim/korap/constant/PredefinedUserGroup.java
deleted file mode 100644
index f218ac4..0000000
--- a/full/src/main/java/de/ids_mannheim/korap/constant/PredefinedUserGroup.java
+++ /dev/null
@@ -1,21 +0,0 @@
-package de.ids_mannheim.korap.constant;
-
-public enum PredefinedUserGroup {
-    ALL("all users", 1);
-    
-    private String value;
-    private int id;
-
-    PredefinedUserGroup (String value, int id) {
-        this.value = value;
-        this.id = id;
-    }
-    
-    public String getValue () {
-        return value;
-    }
-    
-    public int getId () {
-        return id;
-    }
-}
diff --git a/full/src/main/java/de/ids_mannheim/korap/constant/UserGroupStatus.java b/full/src/main/java/de/ids_mannheim/korap/constant/UserGroupStatus.java
index 0390433..5cb939e 100644
--- a/full/src/main/java/de/ids_mannheim/korap/constant/UserGroupStatus.java
+++ b/full/src/main/java/de/ids_mannheim/korap/constant/UserGroupStatus.java
@@ -1,5 +1,7 @@
 package de.ids_mannheim.korap.constant;
 
 public enum UserGroupStatus {
-    ACTIVE, DELETED, HIDDEN;
+    ACTIVE, DELETED, 
+    // group members cannot see the group
+    HIDDEN;
 }
diff --git a/full/src/main/java/de/ids_mannheim/korap/constant/VirtualCorpusAccessStatus.java b/full/src/main/java/de/ids_mannheim/korap/constant/VirtualCorpusAccessStatus.java
index 2cbd167..381ae4c 100644
--- a/full/src/main/java/de/ids_mannheim/korap/constant/VirtualCorpusAccessStatus.java
+++ b/full/src/main/java/de/ids_mannheim/korap/constant/VirtualCorpusAccessStatus.java
@@ -2,5 +2,9 @@
 
 public enum VirtualCorpusAccessStatus {
 
-    ACTIVE, PENDING, DELETED, HIDDEN;
+    ACTIVE, DELETED,
+    PENDING,
+    // access for hidden group
+    // maybe not necessary?
+    HIDDEN;
 }
diff --git a/full/src/main/java/de/ids_mannheim/korap/constant/VirtualCorpusType.java b/full/src/main/java/de/ids_mannheim/korap/constant/VirtualCorpusType.java
index e51d36c..0072c67 100644
--- a/full/src/main/java/de/ids_mannheim/korap/constant/VirtualCorpusType.java
+++ b/full/src/main/java/de/ids_mannheim/korap/constant/VirtualCorpusType.java
@@ -1,8 +1,14 @@
 package de.ids_mannheim.korap.constant;
 
 public enum VirtualCorpusType {
-
-    PREDEFINED, PROJECT, PRIVATE, PUBLISHED;
+    // available for all
+    PREDEFINED, 
+    // available to project group members
+    PROJECT, 
+    // available only for the creator
+    PRIVATE, 
+    // available for all, but not listed for all
+    PUBLISHED;
     
     public String displayName () {
         return name().toLowerCase();
diff --git a/full/src/main/java/de/ids_mannheim/korap/dao/UserGroupDao.java b/full/src/main/java/de/ids_mannheim/korap/dao/UserGroupDao.java
index cb04594..958b38f 100644
--- a/full/src/main/java/de/ids_mannheim/korap/dao/UserGroupDao.java
+++ b/full/src/main/java/de/ids_mannheim/korap/dao/UserGroupDao.java
@@ -20,7 +20,6 @@
 
 import de.ids_mannheim.korap.constant.GroupMemberStatus;
 import de.ids_mannheim.korap.constant.PredefinedRole;
-import de.ids_mannheim.korap.constant.PredefinedUserGroup;
 import de.ids_mannheim.korap.constant.UserGroupStatus;
 import de.ids_mannheim.korap.constant.VirtualCorpusAccessStatus;
 import de.ids_mannheim.korap.entity.Role;
@@ -106,19 +105,6 @@
         entityManager.merge(group);
     }
 
-    public UserGroup retrieveAllUserGroup () {
-        CriteriaBuilder criteriaBuilder = entityManager.getCriteriaBuilder();
-        CriteriaQuery<UserGroup> query =
-                criteriaBuilder.createQuery(UserGroup.class);
-
-        Root<UserGroup> root = query.from(UserGroup.class);
-        query.select(root);
-        query.where(criteriaBuilder.equal(root.get(UserGroup_.id),
-                PredefinedUserGroup.ALL.getId()));
-        Query q = entityManager.createQuery(query);
-        return (UserGroup) q.getSingleResult();
-    }
-
     /** Retrieves the UserGroup by the given group id. This methods does not 
      *  fetch group members because only group admin is allowed to see them. 
      *  Group members have to be retrieved separately.
@@ -129,31 +115,11 @@
      * @throws KustvaktException 
      */
     public UserGroup retrieveGroupById (int groupId) throws KustvaktException {
-        ParameterChecker.checkIntegerValue(groupId, "groupId");
-
-        CriteriaBuilder criteriaBuilder = entityManager.getCriteriaBuilder();
-        CriteriaQuery<UserGroup> query =
-                criteriaBuilder.createQuery(UserGroup.class);
-
-        Root<UserGroup> root = query.from(UserGroup.class);
-        query.select(root);
-        query.where(criteriaBuilder.equal(root.get(UserGroup_.id), groupId));
-        Query q = entityManager.createQuery(query);
-
-        try {
-            return (UserGroup) q.getSingleResult();
-        }
-        catch (NoResultException e) {
-            throw new KustvaktException(StatusCodes.NO_RESULT_FOUND,
-                    "No result found for query: retrieve group by id "
-                            + groupId,
-                    String.valueOf(groupId), e);
-        }
+        return retrieveGroupById(groupId, false);
     }
 
-    public UserGroup retrieveGroupWithMemberById (int groupId)
+    public UserGroup retrieveGroupById (int groupId, boolean fetchMembers)
             throws KustvaktException {
-
         ParameterChecker.checkIntegerValue(groupId, "groupId");
 
         CriteriaBuilder criteriaBuilder = entityManager.getCriteriaBuilder();
@@ -161,7 +127,9 @@
                 criteriaBuilder.createQuery(UserGroup.class);
 
         Root<UserGroup> root = query.from(UserGroup.class);
-        root.fetch(UserGroup_.members);
+        if (fetchMembers) {
+            root.fetch(UserGroup_.members);
+        }
         query.select(root);
         query.where(criteriaBuilder.equal(root.get(UserGroup_.id), groupId));
         Query q = entityManager.createQuery(query);
@@ -177,6 +145,7 @@
         }
     }
 
+
     /** Retrieves only user-groups that are active (not hidden or deleted).
      * 
      * @param userId user id
@@ -246,15 +215,38 @@
         }
     }
 
-    //    public void retrieveGroupByVCId (String vcId) {
-    //        CriteriaBuilder criteriaBuilder = entityManager.getCriteriaBuilder();
-    //        CriteriaQuery<VirtualCorpusAccess> query =
-    //                criteriaBuilder.createQuery(VirtualCorpusAccess.class);
-    //
-    //        Root<VirtualCorpusAccess> root = query.from(VirtualCorpusAccess.class);
-    //
-    //
-    //    }
+    public UserGroup retrieveHiddenGroupByVC (int vcId) throws KustvaktException {
+        ParameterChecker.checkIntegerValue(vcId, "vcId");
+
+        CriteriaBuilder criteriaBuilder = entityManager.getCriteriaBuilder();
+        CriteriaQuery<UserGroup> query =
+                criteriaBuilder.createQuery(UserGroup.class);
+
+        Root<UserGroup> root = query.from(UserGroup.class);
+        Join<UserGroup, VirtualCorpusAccess> access =
+                root.join(UserGroup_.virtualCorpusAccess);
+        Join<VirtualCorpusAccess, VirtualCorpus> vc =
+                access.join(VirtualCorpusAccess_.virtualCorpus);
+
+        Predicate p = criteriaBuilder.and(
+                criteriaBuilder.equal(root.get(UserGroup_.status),
+                        UserGroupStatus.HIDDEN),
+                criteriaBuilder.equal(vc.get(VirtualCorpus_.id), vcId));
+        
+        query.select(root);
+        query.where(p);
+        Query q = entityManager.createQuery(query);
+
+        try {
+            return (UserGroup) q.getSingleResult();
+        }
+        catch (NoResultException e) {
+            throw new KustvaktException(StatusCodes.NO_RESULT_FOUND,
+                    "No hidden group found for virtual corpus with id " + vcId,
+                    String.valueOf(vcId), e);
+        }
+
+    }
 
     public void addVCToGroup (VirtualCorpus virtualCorpus, String createdBy,
             VirtualCorpusAccessStatus status, UserGroup group) {
diff --git a/full/src/main/java/de/ids_mannheim/korap/dao/VirtualCorpusAccessDao.java b/full/src/main/java/de/ids_mannheim/korap/dao/VirtualCorpusAccessDao.java
index f885417..1913d09 100644
--- a/full/src/main/java/de/ids_mannheim/korap/dao/VirtualCorpusAccessDao.java
+++ b/full/src/main/java/de/ids_mannheim/korap/dao/VirtualCorpusAccessDao.java
@@ -1,6 +1,5 @@
 package de.ids_mannheim.korap.dao;
 
-import java.util.ArrayList;
 import java.util.List;
 
 import javax.persistence.EntityManager;
@@ -16,7 +15,6 @@
 import org.springframework.stereotype.Repository;
 import org.springframework.transaction.annotation.Transactional;
 
-import de.ids_mannheim.korap.constant.PredefinedUserGroup;
 import de.ids_mannheim.korap.constant.VirtualCorpusAccessStatus;
 import de.ids_mannheim.korap.entity.UserGroup;
 import de.ids_mannheim.korap.entity.UserGroup_;
@@ -25,7 +23,6 @@
 import de.ids_mannheim.korap.entity.VirtualCorpusAccess_;
 import de.ids_mannheim.korap.entity.VirtualCorpus_;
 import de.ids_mannheim.korap.exceptions.KustvaktException;
-import de.ids_mannheim.korap.exceptions.StatusCodes;
 import de.ids_mannheim.korap.utils.ParameterChecker;
 
 @Transactional
@@ -147,7 +144,7 @@
      * @return true if there is a hidden access, false otherwise
      * @throws KustvaktException
      */
-    public List<VirtualCorpusAccess> retrieveHiddenAccess (int vcId)
+    public VirtualCorpusAccess retrieveHiddenAccess (int vcId)
             throws KustvaktException {
         ParameterChecker.checkIntegerValue(vcId, "vcId");
 
@@ -163,45 +160,11 @@
         Predicate p = builder.and(
                 builder.equal(accessVC.get(VirtualCorpus_.id), vcId),
                 builder.equal(access.get(VirtualCorpusAccess_.status),
-                        VirtualCorpusAccessStatus.HIDDEN),
-                builder.notEqual(access.get(VirtualCorpusAccess_.deletedBy),
-                        "NULL"));
-
-        query.select(access);
-        query.where(p);
-
-        try {
-            Query q = entityManager.createQuery(query);
-            return q.getResultList();
-        }
-        catch (NoResultException e) {
-            return new ArrayList<>();
-        }
-    }
-
-    public VirtualCorpusAccess retrievePublishedGroupAccess (int vcId)
-            throws KustvaktException {
-        ParameterChecker.checkIntegerValue(vcId, "vcId");
-
-        CriteriaBuilder builder = entityManager.getCriteriaBuilder();
-        CriteriaQuery<VirtualCorpusAccess> query =
-                builder.createQuery(VirtualCorpusAccess.class);
-
-        Root<VirtualCorpusAccess> access =
-                query.from(VirtualCorpusAccess.class);
-        Join<VirtualCorpusAccess, VirtualCorpus> accessVC =
-                access.join(VirtualCorpusAccess_.virtualCorpus);
-        Join<VirtualCorpusAccess, UserGroup> accessGroup =
-                access.join(VirtualCorpusAccess_.userGroup);
-
-        Predicate p = builder.and(
-                builder.equal(accessVC.get(VirtualCorpus_.id), vcId),
-                builder.equal(accessGroup.get(UserGroup_.id),
-                        PredefinedUserGroup.ALL.getId()),
-                builder.equal(access.get(VirtualCorpusAccess_.status),
-                        VirtualCorpusAccessStatus.HIDDEN),
-                builder.notEqual(access.get(VirtualCorpusAccess_.deletedBy),
-                        "NULL"));
+                        VirtualCorpusAccessStatus.HIDDEN)
+//                ,
+//                builder.notEqual(access.get(VirtualCorpusAccess_.deletedBy),
+//                        "NULL")
+                );
 
         query.select(access);
         query.where(p);
@@ -211,9 +174,7 @@
             return (VirtualCorpusAccess) q.getSingleResult();
         }
         catch (NoResultException e) {
-            throw new KustvaktException(StatusCodes.NO_RESULT_FOUND,
-                    "Auto group access  is not found for virtual corpus with id "
-                            + vcId);
+            return null;
         }
     }
 
diff --git a/full/src/main/java/de/ids_mannheim/korap/dao/VirtualCorpusDao.java b/full/src/main/java/de/ids_mannheim/korap/dao/VirtualCorpusDao.java
index 3038d72..1ca79d4 100644
--- a/full/src/main/java/de/ids_mannheim/korap/dao/VirtualCorpusDao.java
+++ b/full/src/main/java/de/ids_mannheim/korap/dao/VirtualCorpusDao.java
@@ -191,12 +191,15 @@
         Join<VirtualCorpus, VirtualCorpusAccess> access =
                 virtualCorpus.join(VirtualCorpus_.virtualCorpusAccess);
 
-        Predicate corpusStatus = builder.and(
-                builder.notEqual(access.get(VirtualCorpusAccess_.status),
-                        VirtualCorpusAccessStatus.HIDDEN),
-                builder.notEqual(access.get(VirtualCorpusAccess_.status),
-                        VirtualCorpusAccessStatus.DELETED));
+//        Predicate corpusStatus = builder.and(
+//                builder.notEqual(access.get(VirtualCorpusAccess_.status),
+//                        VirtualCorpusAccessStatus.HIDDEN),
+//                builder.notEqual(access.get(VirtualCorpusAccess_.status),
+//                        VirtualCorpusAccessStatus.DELETED));
 
+        Predicate corpusStatus = builder.notEqual(access.get(VirtualCorpusAccess_.status),
+                VirtualCorpusAccessStatus.DELETED);
+        
         Predicate userGroupStatus =
                 builder.notEqual(access.get(VirtualCorpusAccess_.userGroup)
                         .get(UserGroup_.status), UserGroupStatus.DELETED);
diff --git a/full/src/main/java/de/ids_mannheim/korap/service/UserGroupService.java b/full/src/main/java/de/ids_mannheim/korap/service/UserGroupService.java
index 612ae24..0768a62 100644
--- a/full/src/main/java/de/ids_mannheim/korap/service/UserGroupService.java
+++ b/full/src/main/java/de/ids_mannheim/korap/service/UserGroupService.java
@@ -8,7 +8,6 @@
 
 import de.ids_mannheim.korap.constant.GroupMemberStatus;
 import de.ids_mannheim.korap.constant.PredefinedRole;
-import de.ids_mannheim.korap.constant.PredefinedUserGroup;
 import de.ids_mannheim.korap.constant.UserGroupStatus;
 import de.ids_mannheim.korap.dao.RoleDao;
 import de.ids_mannheim.korap.dao.UserGroupDao;
@@ -83,17 +82,13 @@
         return userGroupDao.retrieveGroupById(groupId);
     }
 
-    public UserGroup retrieveAllUserGroup () {
-        return userGroupDao.retrieveAllUserGroup();
-    }
-
     public List<UserGroupMember> retrieveVCAccessAdmins (UserGroup userGroup)
             throws KustvaktException {
         List<UserGroupMember> groupAdmins = groupMemberDao.retrieveMemberByRole(
                 userGroup.getId(), PredefinedRole.VC_ACCESS_ADMIN.getId());
         return groupAdmins;
     }
-    
+
     public List<UserGroupMember> retrieveUserGroupAdmins (UserGroup userGroup)
             throws KustvaktException {
         List<UserGroupMember> groupAdmins = groupMemberDao.retrieveMemberByRole(
@@ -154,29 +149,29 @@
     }
 
     public int createAutoHiddenGroup (int vcId) throws KustvaktException {
-        String groupName = "auto-published-group";
+        String groupName = "auto-hidden-group";
         int groupId = userGroupDao.createGroup(groupName, "system",
                 UserGroupStatus.HIDDEN);
 
         return groupId;
     }
-    
+
     public void addUserToGroup (String username, UserGroup userGroup)
             throws KustvaktException {
-        
+
         List<Role> roles = new ArrayList<Role>(2);
         roles.add(roleDao
                 .retrieveRoleById(PredefinedRole.USER_GROUP_MEMBER.getId()));
         roles.add(roleDao
                 .retrieveRoleById(PredefinedRole.VC_ACCESS_MEMBER.getId()));
-        
+
         UserGroupMember member = new UserGroupMember();
         member.setCreatedBy("system");
         member.setGroup(userGroup);
         member.setRoles(roles);
         member.setStatus(GroupMemberStatus.ACTIVE);
         member.setUserId(username);
-        
+
         groupMemberDao.addMember(member);
     }
 
@@ -218,4 +213,8 @@
         }
         return false;
     }
+
+    public UserGroup retrieveHiddenGroup (int vcId) throws KustvaktException {
+        return userGroupDao.retrieveHiddenGroupByVC(vcId);
+    }
 }
diff --git a/full/src/main/java/de/ids_mannheim/korap/service/VirtualCorpusService.java b/full/src/main/java/de/ids_mannheim/korap/service/VirtualCorpusService.java
index 60c36bc..dcaaeb0 100644
--- a/full/src/main/java/de/ids_mannheim/korap/service/VirtualCorpusService.java
+++ b/full/src/main/java/de/ids_mannheim/korap/service/VirtualCorpusService.java
@@ -15,8 +15,6 @@
 import com.fasterxml.jackson.databind.JsonNode;
 
 import de.ids_mannheim.korap.config.FullConfiguration;
-import de.ids_mannheim.korap.constant.GroupMemberStatus;
-import de.ids_mannheim.korap.constant.PredefinedUserGroup;
 import de.ids_mannheim.korap.constant.VirtualCorpusAccessStatus;
 import de.ids_mannheim.korap.constant.VirtualCorpusType;
 import de.ids_mannheim.korap.dao.VirtualCorpusAccessDao;
@@ -25,7 +23,6 @@
 import de.ids_mannheim.korap.dto.VirtualCorpusDto;
 import de.ids_mannheim.korap.dto.converter.VirtualCorpusAccessConverter;
 import de.ids_mannheim.korap.dto.converter.VirtualCorpusConverter;
-import de.ids_mannheim.korap.entity.Role;
 import de.ids_mannheim.korap.entity.UserGroup;
 import de.ids_mannheim.korap.entity.UserGroupMember;
 import de.ids_mannheim.korap.entity.VirtualCorpus;
@@ -234,17 +231,9 @@
 
     private void publishVC (int vcId) throws KustvaktException {
 
-        List<VirtualCorpusAccess> hiddenAccess =
-                accessDao.retrieveHiddenAccess(vcId);
-
         // check if hidden access exists
-        if (hiddenAccess.isEmpty()) {
-            // assign hidden access for all users
+        if (accessDao.retrieveHiddenAccess(vcId) != null) {
             VirtualCorpus vc = vcDao.retrieveVCById(vcId);
-            UserGroup all = userGroupService.retrieveAllUserGroup();
-            accessDao.createAccessToVC(vc, all, "system",
-                    VirtualCorpusAccessStatus.HIDDEN);
-
             // create and assign a hidden group
             int groupId = userGroupService.createAutoHiddenGroup(vcId);
             UserGroup autoHidden =
@@ -405,7 +394,7 @@
         VirtualCorpus vc = vcDao.retrieveVCById(vcId);
         VirtualCorpusType type = vc.getType();
 
-        if (!user.isAdmin() || !username.equals(vc.getCreatedBy())) {
+        if (!user.isAdmin() && !username.equals(vc.getCreatedBy())) {
             if (type.equals(VirtualCorpusType.PRIVATE)
                     || (type.equals(VirtualCorpusType.PROJECT)
                             && !hasAccess(username, vcId))) {
@@ -416,10 +405,11 @@
 
             else if (VirtualCorpusType.PUBLISHED.equals(type)) {
                 // add user in the VC's auto group 
-                VirtualCorpusAccess access =
-                        accessDao.retrievePublishedGroupAccess(vcId);
-                UserGroup userGroup = access.getUserGroup();
-                if (userGroupService.isMember(username, userGroup)) {
+//                VirtualCorpusAccess access =
+//                        accessDao.retrieveHiddenAccess(vcId);
+//                UserGroup userGroup = access.getUserGroup();
+                UserGroup userGroup = userGroupService.retrieveHiddenGroup(vcId);
+                if (!userGroupService.isMember(username, userGroup)) {
                     userGroupService.addUserToGroup(username, userGroup);
                 }
             }
diff --git a/full/src/main/java/de/ids_mannheim/korap/web/controller/VirtualCorpusController.java b/full/src/main/java/de/ids_mannheim/korap/web/controller/VirtualCorpusController.java
index 1c869b3..1304b6f 100644
--- a/full/src/main/java/de/ids_mannheim/korap/web/controller/VirtualCorpusController.java
+++ b/full/src/main/java/de/ids_mannheim/korap/web/controller/VirtualCorpusController.java
@@ -39,9 +39,10 @@
  * such as creating, deleting and listing user virtual corpora.
  * 
  * This class also includes APIs related to virtual corpus access (VCA) 
- * such as sharing and publishing VCs. When a VC is published, it is shared 
+ * such as sharing and publishing VC. When a VC is published, it is shared 
  * with all users, but not always listed like system VC. It is listed for 
- * a user, once when he/she have searched for the VC.
+ * a user, once when he/she have searched for the VC. A VC can be published 
+ * by creating or editing the VC. 
  * 
  * All the APIs in this class are available to logged-in users.
  * 
@@ -89,7 +90,7 @@
         return Response.ok().build();
     }
 
-    /** Only the VC owner and system admins can edit VCs.
+    /** Only the VC owner and system admins can edit VC.
      * 
      * @param securityContext
      * @param vc a JSON object describing the virtual corpus
@@ -117,7 +118,7 @@
      * 
      * @param securityContext
      * @param vcId a virtual corpus id
-     * @return a list of VCs
+     * @return a list of VC
      */
     @GET
     @Path("search/{vcId}")
@@ -137,10 +138,10 @@
         return Response.ok(result).build();
     }
 
-    /** Lists not only private VCs but all VCs available to a user.
+    /** Lists not only private VC but all VC available to a user.
      * 
      * @param securityContext
-     * @return a list of VCs
+     * @return a list of VC
      */
     @GET
     @Path("list")
@@ -159,10 +160,10 @@
         return Response.ok(result).build();
     }
 
-    /** Lists all VCs created by a user
+    /** Lists all VC created by a user
      * 
      * @param securityContext
-     * @return a list of VCs created by the user in the security context.
+     * @return a list of VC created by the user in the security context.
      */
     @GET
     @Path("list/user")
@@ -181,8 +182,8 @@
         return Response.ok(result).build();
     }
 
-    /** Only the VC owner and system admins can delete VCs. VCA admins 
-     *  can delete VC-accesses e.g. of project VCs, but not the VCs 
+    /** Only the VC owner and system admins can delete VC. VCA admins 
+     *  can delete VC-accesses e.g. of project VC, but not the VC 
      *  themselves. 
      * 
      * @param securityContext
@@ -220,8 +221,8 @@
     //  }
 
     /** VC can only be shared with a group, not individuals. 
-     *  Only VCA admins are allowed to share VCs and 
-     *  the VCs must have been created by themselves.
+     *  Only VCA admins are allowed to share VC and 
+     *  the VC must have been created by themselves.
      * 
      * @param securityContext
      * @param vcId a virtual corpus id
diff --git a/full/src/main/resources/db/insert/V3.1__insert_virtual_corpus.sql b/full/src/main/resources/db/insert/V3.1__insert_virtual_corpus.sql
index 82ee835..62e22ed 100644
--- a/full/src/main/resources/db/insert/V3.1__insert_virtual_corpus.sql
+++ b/full/src/main/resources/db/insert/V3.1__insert_virtual_corpus.sql
@@ -2,6 +2,9 @@
 
 -- user groups
 INSERT INTO user_group(name,status,created_by) 
+	VALUES ("marlin group","ACTIVE","marlin");
+	
+INSERT INTO user_group(name,status,created_by) 
 	VALUES ("dory group","ACTIVE","dory");
 
 INSERT INTO user_group(name,status,created_by) 
@@ -17,6 +20,16 @@
 
 -- user group members
 INSERT INTO user_group_member(user_id, group_id, status, created_by)
+	SELECT "marlin",
+		(SELECT id from user_group where name = "marlin group"),
+		"ACTIVE","marlin";
+
+INSERT INTO user_group_member(user_id, group_id, status, created_by)
+	SELECT "dory",
+		(SELECT id from user_group where name = "marlin group"),
+		"ACTIVE","marlin";
+		
+INSERT INTO user_group_member(user_id, group_id, status, created_by)
 	SELECT "dory",
 		(SELECT id from user_group where name = "dory group"),
 		"ACTIVE","dory";
@@ -67,23 +80,23 @@
 		(SELECT id from user_group where name = "dory group"), 
 		"ACTIVE", "dory";
 
-INSERT INTO virtual_corpus_access(virtual_corpus_id, user_group_id, status, created_by) 
-	SELECT 
-		(SELECT id from virtual_corpus where name = "system VC"), 
-		(SELECT id from user_group where name = "all users"),
-		"ACTIVE", "system";
+--INSERT INTO virtual_corpus_access(virtual_corpus_id, user_group_id, status, created_by) 
+--	SELECT 
+--		(SELECT id from virtual_corpus where name = "system VC"), 
+--		(SELECT id from user_group where name = "all users"),
+--		"ACTIVE", "system";
 
 INSERT INTO virtual_corpus_access(virtual_corpus_id, user_group_id, status, created_by) 
 	SELECT 
 		(SELECT id from virtual_corpus where name = "published VC"),
-		(SELECT id from user_group where name = "all users"),
-		"HIDDEN", "system";
+		(SELECT id from user_group where name = "marlin group"),
+		"ACTIVE", "marlin";
 
 INSERT INTO virtual_corpus_access(virtual_corpus_id, user_group_id, status, created_by) 
 	SELECT 
 		(SELECT id from virtual_corpus where name = "published VC"),
 		(SELECT id from user_group where name = "auto group"),
-		"ACTIVE", "system";
+		"HIDDEN", "system";
 
 	
 -- Summary user VC Lists
diff --git a/full/src/main/resources/db/insert/V3.3__insert_member_roles.sql b/full/src/main/resources/db/insert/V3.3__insert_member_roles.sql
index 202ff50..611c322 100644
--- a/full/src/main/resources/db/insert/V3.3__insert_member_roles.sql
+++ b/full/src/main/resources/db/insert/V3.3__insert_member_roles.sql
@@ -1,4 +1,27 @@
 -- member roles
+
+-- marlin group
+INSERT INTO group_member_role(group_member_id,role_id)
+SELECT
+	(SELECT id FROM user_group_member WHERE user_id="marlin" AND group_id=1),
+	(SELECT id FROM role WHERE name = "group admin");
+	
+INSERT INTO group_member_role(group_member_id,role_id)
+SELECT
+	(SELECT id FROM user_group_member WHERE user_id="marlin" AND group_id=1),
+	(SELECT id FROM role WHERE name = "vc admin");
+	
+INSERT INTO group_member_role(group_member_id,role_id)
+SELECT
+	(SELECT id FROM user_group_member WHERE user_id="dory" AND group_id=1),
+	(SELECT id FROM role WHERE name = "group member");
+	
+INSERT INTO group_member_role(group_member_id,role_id)
+SELECT
+	(SELECT id FROM user_group_member WHERE user_id="dory" AND group_id=1),
+	(SELECT id FROM role WHERE name = "vc member");
+	
+	
 -- dory group
 INSERT INTO group_member_role(group_member_id,role_id)
 SELECT
@@ -20,6 +43,7 @@
 	(SELECT id FROM user_group_member WHERE user_id="nemo" AND group_id=2),
 	(SELECT id FROM role WHERE name = "vc member");
 
+
 -- auto group
 INSERT INTO group_member_role(group_member_id,role_id)
 SELECT
diff --git a/full/src/main/resources/db/predefined/V3.0__insert_predefined_user_groups.sql b/full/src/main/resources/db/predefined/V3.0__insert_predefined_user_groups.sql
deleted file mode 100644
index 9dc9e52..0000000
--- a/full/src/main/resources/db/predefined/V3.0__insert_predefined_user_groups.sql
+++ /dev/null
@@ -1,3 +0,0 @@
-
-INSERT INTO user_group(id,name,status,created_by) 
-	VALUES (1,"all users","HIDDEN","system");
\ No newline at end of file
diff --git a/full/src/test/java/de/ids_mannheim/korap/dao/UserGroupDaoTest.java b/full/src/test/java/de/ids_mannheim/korap/dao/UserGroupDaoTest.java
index 8753259..d9fdca4 100644
--- a/full/src/test/java/de/ids_mannheim/korap/dao/UserGroupDaoTest.java
+++ b/full/src/test/java/de/ids_mannheim/korap/dao/UserGroupDaoTest.java
@@ -49,7 +49,7 @@
                 UserGroupStatus.ACTIVE);
 
         // retrieve group
-        UserGroup group = userGroupDao.retrieveGroupWithMemberById(groupId);
+        UserGroup group = userGroupDao.retrieveGroupById(groupId, true);
         assertEquals(groupName, group.getName());
         assertEquals(createdBy, group.getCreatedBy());
         assertEquals(UserGroupStatus.ACTIVE, group.getStatus());
@@ -88,7 +88,7 @@
     public void retrieveGroupWithMembers () throws KustvaktException {
         // dory group
         List<UserGroupMember> members =
-                userGroupDao.retrieveGroupWithMemberById(2).getMembers();
+                userGroupDao.retrieveGroupById(2, true).getMembers();
         assertEquals(4, members.size());
 
         UserGroupMember m = members.get(1);
@@ -101,7 +101,7 @@
     @Test
     public void retrieveGroupByUserId () throws KustvaktException {
         List<UserGroup> group = userGroupDao.retrieveGroupByUserId("dory");
-        assertEquals(1, group.size());
+        assertEquals(2, group.size());
 
         group = userGroupDao.retrieveGroupByUserId("pearl");
         assertEquals(0, group.size());
@@ -140,9 +140,13 @@
         assertEquals(2, vc.size());
         assertEquals(name, vc.get(1).getName());
 
+        // delete vc from group
         userGroupDao.deleteVCFromGroup(virtualCorpus.getId(), groupId);
 
         vc = virtualCorpusDao.retrieveVCByGroup(groupId);
         assertEquals(1, vc.size());
+        
+        // delete vc
+        virtualCorpusDao.deleteVirtualCorpus(virtualCorpus.getId());
     }
 }
diff --git a/full/src/test/java/de/ids_mannheim/korap/dao/VirtualCorpusDaoTest.java b/full/src/test/java/de/ids_mannheim/korap/dao/VirtualCorpusDaoTest.java
index 199753a..d7ed3a2 100644
--- a/full/src/test/java/de/ids_mannheim/korap/dao/VirtualCorpusDaoTest.java
+++ b/full/src/test/java/de/ids_mannheim/korap/dao/VirtualCorpusDaoTest.java
@@ -72,18 +72,19 @@
 
 
     /** retrieve private and group VC
-     * excludes hidden published VC (user has never used it)
      * @throws KustvaktException
      */
     @Test
     public void retrieveVCByUserDory () throws KustvaktException {
         Set<VirtualCorpus> virtualCorpora = dao.retrieveVCByUser("dory");
-        assertEquals(3, virtualCorpora.size());
+        System.out.println(virtualCorpora);
+        assertEquals(4, virtualCorpora.size());
         // order is random
         //        Iterator<VirtualCorpus> i = virtualCorpora.iterator();
         //        assertEquals("dory VC", i.next().getName());
         //        assertEquals("system VC", i.next().getName());
         //        assertEquals("group VC", i.next().getName());
+        //      assertEquals("published VC", i.next().getName());
     }
 
 
diff --git a/full/src/test/java/de/ids_mannheim/korap/web/controller/VirtualCorpusControllerTest.java b/full/src/test/java/de/ids_mannheim/korap/web/controller/VirtualCorpusControllerTest.java
index 6afb04b..1e48889 100644
--- a/full/src/test/java/de/ids_mannheim/korap/web/controller/VirtualCorpusControllerTest.java
+++ b/full/src/test/java/de/ids_mannheim/korap/web/controller/VirtualCorpusControllerTest.java
@@ -13,6 +13,7 @@
 
 import org.apache.http.entity.ContentType;
 import org.eclipse.jetty.http.HttpHeaders;
+import org.junit.Ignore;
 import org.junit.Test;
 import org.springframework.beans.factory.annotation.Autowired;
 
@@ -97,8 +98,9 @@
     }
 
     @Test
-    public void testSearchPrivateVCUnauthorized () throws UniformInterfaceException,
-            ClientHandlerException, KustvaktException {
+    public void testSearchPrivateVCUnauthorized ()
+            throws UniformInterfaceException, ClientHandlerException,
+            KustvaktException {
         ClientResponse response = resource().path("vc").path("search").path("1")
                 .header(Attributes.AUTHORIZATION,
                         handler.createBasicAuthorizationHeaderValue(
@@ -110,7 +112,8 @@
         assertEquals(Status.UNAUTHORIZED.getStatusCode(), response.getStatus());
         assertEquals(StatusCodes.AUTHORIZATION_FAILED,
                 node.at("/errors/0/0").asInt());
-        assertEquals("Unauthorized operation for user: VirtualCorpusControllerTest",
+        assertEquals(
+                "Unauthorized operation for user: VirtualCorpusControllerTest",
                 node.at("/errors/0/1").asText());
 
         checkWWWAuthenticateHeader(response);
@@ -137,8 +140,9 @@
     }
 
     @Test
-    public void testSearchProjectVCNonActiveMember () throws UniformInterfaceException,
-            ClientHandlerException, KustvaktException {
+    public void testSearchProjectVCNonActiveMember ()
+            throws UniformInterfaceException, ClientHandlerException,
+            KustvaktException {
 
         ClientResponse response = resource().path("vc").path("search").path("2")
                 .header(Attributes.AUTHORIZATION,
@@ -157,14 +161,31 @@
         checkWWWAuthenticateHeader(response);
     }
 
-//    @Test
-//    public void testSearchPublishedVC () {
-//
-//    }
-    
+    @Test
+    public void testSearchPublishedVC () throws UniformInterfaceException,
+            ClientHandlerException, KustvaktException {
+        ClientResponse response = resource().path("vc").path("search").path("4")
+                .header(Attributes.AUTHORIZATION,
+                        handler.createBasicAuthorizationHeaderValue("VirtualCorpusControllerTest",
+                                "pass"))
+                .header(HttpHeaders.X_FORWARDED_FOR, "149.27.0.32")
+                .get(ClientResponse.class);
+        String entity = response.getEntity(String.class);
+//        System.out.println(entity);
+        assertEquals(Status.OK.getStatusCode(), response.getStatus());
+
+        JsonNode node = JsonUtils.readTree(entity);
+        assertEquals("published VC", node.at("/name").asText());
+        assertEquals(VirtualCorpusType.PUBLISHED.displayName(),
+                node.at("/type").asText());
+        
+        // EM: need admin to check if VirtualCorpusControllerTest is added to the hidden group
+    }
+
     @Test
     public void testListVC () throws UniformInterfaceException,
             ClientHandlerException, KustvaktException {
+        // dory
         ClientResponse response = resource().path("vc").path("list")
                 .header(Attributes.AUTHORIZATION,
                         handler.createBasicAuthorizationHeaderValue("dory",
@@ -176,7 +197,31 @@
         assertEquals(Status.OK.getStatusCode(), response.getStatus());
         //                System.out.println(entity);
         JsonNode node = JsonUtils.readTree(entity);
-        assertEquals(3, node.size());
+        assertEquals(4, node.size());
+
+        // nemo
+        response = resource().path("vc").path("list")
+                .header(Attributes.AUTHORIZATION,
+                        handler.createBasicAuthorizationHeaderValue("nemo",
+                                "pass"))
+                .header(HttpHeaders.X_FORWARDED_FOR, "149.27.0.32")
+                .get(ClientResponse.class);
+
+        entity = response.getEntity(String.class);
+        node = JsonUtils.readTree(entity);
+        assertEquals(2, node.size());
+
+        // pearl
+        response = resource().path("vc").path("list")
+                .header(Attributes.AUTHORIZATION,
+                        handler.createBasicAuthorizationHeaderValue("pearl",
+                                "pass"))
+                .header(HttpHeaders.X_FORWARDED_FOR, "149.27.0.32")
+                .get(ClientResponse.class);
+
+        entity = response.getEntity(String.class);
+        node = JsonUtils.readTree(entity);
+        assertEquals(2, node.size());
     }
 
     @Test
@@ -289,7 +334,7 @@
                 .get(ClientResponse.class);
 
         String entity = response.getEntity(String.class);
-        System.out.println(entity);
+        //        System.out.println(entity);
         JsonNode node = JsonUtils.readTree(entity);
         assertEquals(1, node.size());
         assertEquals("new published vc", node.get(0).get("name").asText());
@@ -531,6 +576,7 @@
     }
 
     @Test
+    @Ignore
     public void testEditPublishVC () throws KustvaktException {
 
         String json =