vmario
/
docs
mirror of https://github.com/sismics/docs.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Add doc for search syntax (#634)

This commit is contained in:
Julien Kirch 2022-04-17 13:10:01 +02:00 committed by GitHub
parent 46f6b9e537
commit bd23f14792
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
15 changed files with 144 additions and 122 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

51
README.md
View File

@ -14,8 +14,7 @@ Teedy is an open source, lightweight document management system for individuals
![New!](https://teedy.io/img/laptop-demo.png?20180301)
Demo
----
# Demo
A demo is available at [demo.teedy.io](https://demo.teedy.io)
@ -23,8 +22,7 @@ A demo is available at [demo.teedy.io](https://demo.teedy.io)
- "admin" login with "admin" password
- "demo" login with "password" password
Features
--------
# Features
- Responsive user interface
- Optical character recognition
@ -54,8 +52,7 @@ Features
- [Bulk files importer](https://github.com/sismics/docs/tree/master/docs-importer) (single or scan mode)
- Tested to one million documents
Install with Docker
-------------------
# Install with Docker
A preconfigured Docker image is available, including OCR and media conversion tools, listening on port 8080. The database is an embedded H2 database but PostgreSQL is also supported for more performance.
@ -68,7 +65,7 @@ The data directory is `/data`. Don't forget to mount a volume on it.
To build external URL, the server is expecting a `DOCS_BASE_URL` environment variable (for example https://teedy.mycompany.com)
### Available environment variables
## Available environment variables
- General
- `DOCS_BASE_URL`: The base url used by the application. Generated url's will be using this as base.
@ -94,11 +91,11 @@ To build external URL, the server is expecting a `DOCS_BASE_URL` environment var
- `DOCS_SMTP_USERNAME`: The username to be used.
- `DOCS_SMTP_PASSWORD`: The password to be used.
### Examples
## Examples
In the following examples some passwords are exposed in cleartext. This was done in order to keep the examples simple. We strongly encourage you to use variables with an `.env` file or other means to securely store your passwords.
#### Using the internal database
### Using the internal database
```yaml
version: '3'
@ -121,7 +118,7 @@ services:
- ./docs/data:/data
```
#### Using PostgreSQL
### Using PostgreSQL
```yaml
version: '3'
@ -179,10 +176,9 @@ networks:
driver: bridge
```
Manual installation
-------------------
# Manual installation
#### Requirements
## Requirements
- Java 11
- Tesseract 4 for OCR
@ -190,13 +186,12 @@ Manual installation
- mediainfo for video metadata extraction
- A webapp server like [Jetty](http://eclipse.org/jetty/) or [Tomcat](http://tomcat.apache.org/)
#### Download
## Download
The latest release is downloadable here: <https://github.com/sismics/docs/releases> in WAR format.
**The default admin password is "admin". Don't forget to change it before going to production.**
How to build Teedy from the sources
----------------------------------
## How to build Teedy from the sources
Prerequisites: JDK 11, Maven 3, NPM, Grunt, Tesseract 4
@ -209,35 +204,39 @@ Teedy is organized in several Maven modules:
First off, clone the repository: `git clone git://github.com/sismics/docs.git`
or download the sources from GitHub.
#### Launch the build
### Launch the build
From the root directory:
mvn clean -DskipTests install
```console
mvn clean -DskipTests install
```
#### Run a stand-alone version
### Run a stand-alone version
From the `docs-web` directory:
mvn jetty:run
```console
mvn jetty:run
```
#### Build a .war to deploy to your servlet container
### Build a .war to deploy to your servlet container
From the `docs-web` directory:
mvn -Pprod -DskipTests clean install
```console
mvn -Pprod -DskipTests clean install
```
You will get your deployable WAR in the `docs-web/target` directory.
Contributing
------------
# Contributing
All contributions are more than welcomed. Contributions may close an issue, fix a bug (reported or not reported), improve the existing code, add new feature, and so on.
The `master` branch is the default and base branch for the project. It is used for development and all Pull Requests should go there.
License
-------
# License
Teedy is released under the terms of the GPL license. See `COPYING` for more
information or see <http://opensource.org/licenses/GPL-2.0>.

10
docs-core/src/main/java/com/sismics/docs/core/dao/GroupDao.java
View File

@ -183,12 +183,10 @@ public class GroupDao {
}
criteriaList.add("g.GRP_DELETEDATE_D is null");
if (!criteriaList.isEmpty()) {
sb.append(" where ");
sb.append(Joiner.on(" and ").join(criteriaList));
}
sb.append(" where ");
sb.append(Joiner.on(" and ").join(criteriaList));
// Perform the search
QueryParam queryParam = QueryUtil.getSortedQueryParam(new QueryParam(sb.toString(), parameterMap), sortCriteria);
@SuppressWarnings("unchecked")

6
docs-core/src/main/java/com/sismics/docs/core/dao/MetadataDao.java
View File

@ -123,10 +123,8 @@ public class MetadataDao {
criteriaList.add("m.MET_DELETEDATE_D is null");
if (!criteriaList.isEmpty()) {
sb.append(" where ");
sb.append(Joiner.on(" and ").join(criteriaList));
}
sb.append(" where ");
sb.append(Joiner.on(" and ").join(criteriaList));
// Perform the search
QueryParam queryParam = QueryUtil.getSortedQueryParam(new QueryParam(sb.toString(), parameterMap), sortCriteria);

6
docs-core/src/main/java/com/sismics/docs/core/dao/RouteDao.java
View File

@ -64,10 +64,8 @@ public class RouteDao {
}
criteriaList.add("r.RTE_DELETEDATE_D is null");
if (!criteriaList.isEmpty()) {
sb.append(" where ");
sb.append(Joiner.on(" and ").join(criteriaList));
}
sb.append(" where ");
sb.append(Joiner.on(" and ").join(criteriaList));
// Perform the search
QueryParam queryParam = QueryUtil.getSortedQueryParam(new QueryParam(sb.toString(), parameterMap), sortCriteria);

6
docs-core/src/main/java/com/sismics/docs/core/dao/RouteModelDao.java
View File

@ -145,10 +145,8 @@ public class RouteModelDao {
criteriaList.add("rm.RTM_DELETEDATE_D is null");
if (!criteriaList.isEmpty()) {
sb.append(" where ");
sb.append(Joiner.on(" and ").join(criteriaList));
}
sb.append(" where ");
sb.append(Joiner.on(" and ").join(criteriaList));
// Perform the search
QueryParam queryParam = QueryUtil.getSortedQueryParam(new QueryParam(sb.toString(), parameterMap), sortCriteria);

6
docs-core/src/main/java/com/sismics/docs/core/dao/RouteStepDao.java
View File

@ -90,10 +90,8 @@ public class RouteStepDao {
}
criteriaList.add("rs.RTP_DELETEDATE_D is null");
if (!criteriaList.isEmpty()) {
sb.append(" where ");
sb.append(Joiner.on(" and ").join(criteriaList));
}
sb.append(" where ");
sb.append(Joiner.on(" and ").join(criteriaList));
// Perform the search
QueryParam queryParam = QueryUtil.getSortedQueryParam(new QueryParam(sb.toString(), parameterMap), sortCriteria);

1
docs-core/src/main/java/com/sismics/docs/core/dao/ShareDao.java
View File

@ -19,7 +19,6 @@ public class ShareDao {
*
* @param share Share
* @return New ID
* @throws Exception
*/
public String create(Share share) {
// Create the UUID

6
docs-core/src/main/java/com/sismics/docs/core/dao/TagDao.java
View File

@ -199,10 +199,8 @@ public class TagDao {
criteriaList.add("t.TAG_DELETEDATE_D is null");
if (!criteriaList.isEmpty()) {
sb.append(" where ");
sb.append(Joiner.on(" and ").join(criteriaList));
}
sb.append(" where ");
sb.append(Joiner.on(" and ").join(criteriaList));
// Perform the search
QueryParam queryParam = QueryUtil.getSortedQueryParam(new QueryParam(sb.toString(), parameterMap), sortCriteria);

1
docs-core/src/main/java/com/sismics/docs/core/dao/VocabularyDao.java
View File

@ -20,7 +20,6 @@ public class VocabularyDao {
*
* @param vocabulary Vocabulary
* @return New ID
* @throws Exception
*/
public String create(Vocabulary vocabulary) {
// Create the UUID

8
docs-core/src/main/java/com/sismics/docs/core/dao/WebhookDao.java
View File

@ -42,11 +42,9 @@ public class WebhookDao {
}
criteriaList.add("w.WHK_DELETEDATE_D is null");
if (!criteriaList.isEmpty()) {
sb.append(" where ");
sb.append(Joiner.on(" and ").join(criteriaList));
}
sb.append(" where ");
sb.append(Joiner.on(" and ").join(criteriaList));
// Perform the search
QueryParam queryParam = QueryUtil.getSortedQueryParam(new QueryParam(sb.toString(), parameterMap), sortCriteria);
@SuppressWarnings("unchecked")

24
docs-core/src/main/java/com/sismics/docs/core/dao/criteria/DocumentCriteria.java
View File

@ -1,5 +1,6 @@
package com.sismics.docs.core.dao.criteria;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;
@ -49,13 +50,13 @@ public class DocumentCriteria {
* Tag IDs.
* The first level list will be AND'ed and the second level list will be OR'ed.
*/
private List<List<String>> tagIdList;
private List<List<String>> tagIdList = new ArrayList<>();
/**
* Tag IDs to exclude.
* The first and second level list will be excluded.
*/
private List<List<String>> excludedTagIdList;
private List<List<String>> excludedTagIdList = new ArrayList<>();
/**
* Shared status.
@ -131,19 +132,10 @@ public class DocumentCriteria {
return tagIdList;
}
public void setTagIdList(List<List<String>> tagIdList) {
this.tagIdList = tagIdList;
}
public List<List<String>> getExcludedTagIdList() {
return excludedTagIdList;
}
public DocumentCriteria setExcludedTagIdList(List<List<String>> excludedTagIdList) {
this.excludedTagIdList = excludedTagIdList;
return this;
}
public Boolean getShared() {
return shared;
}
@ -167,11 +159,7 @@ public class DocumentCriteria {
public void setCreatorId(String creatorId) {
this.creatorId = creatorId;
}
public Boolean getActiveRoute() {
return activeRoute;
}
public Date getUpdateDateMin() {
return updateDateMin;
}
@ -188,6 +176,10 @@ public class DocumentCriteria {
this.updateDateMax = updateDateMax;
}
public Boolean getActiveRoute() {
return activeRoute;
}
public void setActiveRoute(Boolean activeRoute) {
this.activeRoute = activeRoute;
}

12
docs-core/src/main/java/com/sismics/docs/core/util/TagUtil.java
View File

@ -1,8 +1,8 @@
package com.sismics.docs.core.util;
import com.google.common.collect.Lists;
import com.sismics.docs.core.dao.dto.TagDto;
import java.util.ArrayList;
import java.util.List;
/**
@ -12,14 +12,14 @@ import java.util.List;
*/
public class TagUtil {
/**
* Recursively find children of a tags.
* Recursively find children of a tag.
*
* @param parentTagDto Parent tag
* @param allTagDtoList List of all tags
* @return Children tags
*/
public static List<TagDto> findChildren(TagDto parentTagDto, List<TagDto> allTagDtoList) {
List<TagDto> childrenTagDtoList = Lists.newArrayList();
List<TagDto> childrenTagDtoList = new ArrayList<>();
for (TagDto tagDto : allTagDtoList) {
if (parentTagDto.getId().equals(tagDto.getParentId())) {
@ -32,15 +32,15 @@ public class TagUtil {
}
/**
* Find tags by name (start with).
* Find tags by name (start with, ignore case).
*
* @param name Name
* @param allTagDtoList List of all tags
* @return List of filtered tags
*/
public static List<TagDto> findByName(String name, List<TagDto> allTagDtoList) {
List<TagDto> tagDtoList = Lists.newArrayList();
if (name == null || name.isEmpty()) {
List<TagDto> tagDtoList = new ArrayList<>();
if (name.isEmpty()) {
return tagDtoList;
}
name = name.toLowerCase();

2
docs-core/src/main/java/com/sismics/docs/core/util/indexing/LuceneIndexingHandler.java
View File

@ -299,7 +299,7 @@ public class LuceneIndexingHandler implements IndexingHandler {
criteriaList.add("d.DOC_TITLE_C = :title");
parameterMap.put("title", criteria.getTitle());
}
if (criteria.getTagIdList() != null && !criteria.getTagIdList().isEmpty()) {
if (!criteria.getTagIdList().isEmpty()) {
int index = 0;
for (List<String> tagIdList : criteria.getTagIdList()) {
List<String> tagCriteriaList = Lists.newArrayList();

84
docs-web/src/main/java/com/sismics/docs/rest/resource/DocumentResource.java
View File

@ -67,6 +67,21 @@ import java.util.*;
*/
@Path("/document")
public class DocumentResource extends BaseResource {
protected static final DateTimeParser YEAR_PARSER = DateTimeFormat.forPattern("yyyy").getParser();
protected static final DateTimeParser MONTH_PARSER = DateTimeFormat.forPattern("yyyy-MM").getParser();
protected static final DateTimeParser DAY_PARSER = DateTimeFormat.forPattern("yyyy-MM-dd").getParser();
private static final DateTimeFormatter DAY_FORMATTER = new DateTimeFormatter(null, DAY_PARSER);
private static final DateTimeFormatter MONTH_FORMATTER = new DateTimeFormatter(null, MONTH_PARSER);
private static final DateTimeFormatter YEAR_FORMATTER = new DateTimeFormatter(null, YEAR_PARSER);
private static final DateTimeParser[] DATE_PARSERS = new DateTimeParser[]{
YEAR_PARSER,
MONTH_PARSER,
DAY_PARSER};
private static final DateTimeFormatter DATE_FORMATTER = new DateTimeFormatterBuilder().append( null, DATE_PARSERS).toFormatter();
/**
* Returns a document.
*
@ -349,7 +364,7 @@ public class DocumentResource extends BaseResource {
* @apiParam {String} offset Start at this index
* @apiParam {Number} sort_column Column index to sort on
* @apiParam {Boolean} asc If true, sort in ascending order
* @apiParam {String} search Search query
* @apiParam {String} search Search query (see "Document search syntax" on the top of the page for explanations)
* @apiParam {Booleans} files If true includes files information
* @apiSuccess {Number} total Total number of documents
* @apiSuccess {Object[]} documents List of documents
@ -492,16 +507,8 @@ public class DocumentResource extends BaseResource {
TagDao tagDao = new TagDao();
List<TagDto> allTagDtoList = tagDao.findByCriteria(new TagCriteria().setTargetIdList(getTargetIdList(null)), null);
UserDao userDao = new UserDao();
DateTimeParser[] parsers = {
DateTimeFormat.forPattern("yyyy").getParser(),
DateTimeFormat.forPattern("yyyy-MM").getParser(),
DateTimeFormat.forPattern("yyyy-MM-dd").getParser() };
DateTimeFormatter yearFormatter = new DateTimeFormatter(null, parsers[0]);
DateTimeFormatter monthFormatter = new DateTimeFormatter(null, parsers[1]);
DateTimeFormatter dayFormatter = new DateTimeFormatter(null, parsers[2]);
DateTimeFormatter formatter = new DateTimeFormatterBuilder().append( null, parsers ).toFormatter();
String[] criteriaList = search.split(" *");
String[] criteriaList = search.split(" +");
List<String> query = new ArrayList<>();
List<String> fullQuery = new ArrayList<>();
for (String criteria : criteriaList) {
@ -511,20 +518,16 @@ public class DocumentResource extends BaseResource {
fullQuery.add(criteria);
continue;
}
String paramName = params[0];
String paramValue = params[1];
switch (params[0]) {
switch (paramName) {
case "tag":
case "!tag":
// New tag criteria
List<TagDto> tagDtoList = TagUtil.findByName(params[1], allTagDtoList);
if (documentCriteria.getTagIdList() == null) {
documentCriteria.setTagIdList(new ArrayList<>());
}
if (documentCriteria.getExcludedTagIdList() == null) {
documentCriteria.setExcludedTagIdList(new ArrayList<>());
}
List<TagDto> tagDtoList = TagUtil.findByName(paramValue, allTagDtoList);
if (tagDtoList.isEmpty()) {
// No tag found, the request must returns nothing
// No tag found, the request must return nothing
documentCriteria.getTagIdList().add(Lists.newArrayList(UUID.randomUUID().toString()));
} else {
List<String> tagIdList = Lists.newArrayList();
@ -535,7 +538,7 @@ public class DocumentResource extends BaseResource {
tagIdList.add(childrenTagDto.getId());
}
}
if (params[0].startsWith("!")) {
if (paramName.startsWith("!")) {
documentCriteria.getExcludedTagIdList().add(tagIdList);
} else {
documentCriteria.getTagIdList().add(tagIdList);
@ -548,9 +551,9 @@ public class DocumentResource extends BaseResource {
case "ubefore":
// New date span criteria
try {
boolean isUpdated = params[0].startsWith("u");
DateTime date = formatter.parseDateTime(params[1]);
if (params[0].endsWith("before")) {
boolean isUpdated = paramName.startsWith("u");
DateTime date = DATE_FORMATTER.parseDateTime(paramValue);
if (paramName.endsWith("before")) {
if (isUpdated) documentCriteria.setUpdateDateMax(date.toDate());
else documentCriteria.setCreateDateMax(date.toDate());
} else {
@ -566,11 +569,11 @@ public class DocumentResource extends BaseResource {
case "uat":
case "at":
// New specific date criteria
boolean isUpdated = params[0].startsWith("u");
try {
boolean isUpdated = params[0].startsWith("u");
switch (params[1].length()) {
switch (paramValue.length()) {
case 10: {
DateTime date = dayFormatter.parseDateTime(params[1]);
DateTime date = DATE_FORMATTER.parseDateTime(params[1]);
if (isUpdated) {
documentCriteria.setUpdateDateMin(date.toDate());
documentCriteria.setUpdateDateMax(date.plusDays(1).minusSeconds(1).toDate());
@ -581,7 +584,7 @@ public class DocumentResource extends BaseResource {
break;
}
case 7: {
DateTime date = monthFormatter.parseDateTime(params[1]);
DateTime date = MONTH_FORMATTER.parseDateTime(params[1]);
if (isUpdated) {
documentCriteria.setUpdateDateMin(date.toDate());
documentCriteria.setUpdateDateMax(date.plusMonths(1).minusSeconds(1).toDate());
@ -592,7 +595,7 @@ public class DocumentResource extends BaseResource {
break;
}
case 4: {
DateTime date = yearFormatter.parseDateTime(params[1]);
DateTime date = YEAR_FORMATTER.parseDateTime(params[1]);
if (isUpdated) {
documentCriteria.setUpdateDateMin(date.toDate());
documentCriteria.setUpdateDateMax(date.plusYears(1).minusSeconds(1).toDate());
@ -601,6 +604,10 @@ public class DocumentResource extends BaseResource {
documentCriteria.setCreateDateMax(date.plusYears(1).minusSeconds(1).toDate());
}
break;
} default: {
// Invalid format, returns no documents
documentCriteria.setCreateDateMin(new Date(0));
documentCriteria.setCreateDateMax(new Date(0));
}
}
} catch (IllegalArgumentException e) {
@ -611,25 +618,26 @@ public class DocumentResource extends BaseResource {
break;
case "shared":
// New shared state criteria
documentCriteria.setShared(params[1].equals("yes"));
documentCriteria.setShared(paramValue.equals("yes"));
break;
case "lang":
// New language criteria
if (Constants.SUPPORTED_LANGUAGES.contains(params[1])) {
documentCriteria.setLanguage(params[1]);
if (Constants.SUPPORTED_LANGUAGES.contains(paramValue)) {
documentCriteria.setLanguage(paramValue);
} else {
// Unsupported language, returns no documents
documentCriteria.setLanguage(UUID.randomUUID().toString());
}
break;
case "mime":
// New mime type criteria
documentCriteria.setMimeType(params[1]);
documentCriteria.setMimeType(paramValue);
break;
case "by":
// New creator criteria
User user = userDao.getActiveByUsername(params[1]);
User user = userDao.getActiveByUsername(paramValue);
if (user == null) {
// This user doesn't exists, return nothing
// This user doesn't exist, return nothing
documentCriteria.setCreatorId(UUID.randomUUID().toString());
} else {
// This user exists, search its documents
@ -638,19 +646,19 @@ public class DocumentResource extends BaseResource {
break;
case "workflow":
// New shared state criteria
documentCriteria.setActiveRoute(params[1].equals("me"));
documentCriteria.setActiveRoute(paramValue.equals("me"));
break;
case "simple":
// New simple search criteria
query.add(params[1]);
query.add(paramValue);
break;
case "full":
// New fulltext search criteria
fullQuery.add(params[1]);
fullQuery.add(paramValue);
break;
case "title":
// New title criteria
documentCriteria.setTitle(params[1]);
documentCriteria.setTitle(paramValue);
break;
default:
fullQuery.add(criteria);

43
docs-web/src/main/webapp/header.md
View File

@ -10,7 +10,7 @@ The base URL depends on your server. If your instance of Teedy is accessible thr
`https://teedy.mycompany.com`, then the base API URL is `https://teedy.mycompany.com/api`.
## Verbs and status codes
The API uses restful verbs.
The API uses RESTful verbs.
| Verb | Description |
|---|---|
@ -46,4 +46,43 @@ curl -i -X GET -H "Cookie: auth_token=64085630-2ae6-415c-9a92-4b22c107eaa4" http
A call to this API with a given `auth_token` cookie will make it unusable for other calls.
```
curl -i -X POST -H "Cookie: auth_token=64085630-2ae6-415c-9a92-4b22c107eaa4" https://docs.mycompany.com/api/user/logout
```
```
## Document search syntax
The `/api/document/list` endpoint use a String `search` parameter.
This parameter is split in segments using the space character (the other whitespace characters are not considered).
If a segment contains exactly one colon (`:`), it will used as a field criteria (see bellow).
In other cases (zero or more than one colon), the segment will be used as a search criteria for all fields including the document's files content.
### Search fields
If a search `VALUE` is considered invalid, the search result will be empty.
* Content
* `full:VALUE`: `VALUE` is used as search criteria for all fields, including the document's files content
* `simple:VALUE`: `VALUE` is used as a search criteria for all fields except the document's files content
* Date
* `after:VALUE`: the document must have been created after or at the `VALUE` moment, accepted format are `yyyy`, `yyyy-MM` and `yyyy-MM-dd`
* `at:VALUE`: the document must have been created at the `VALUE` moment, accepted format are `yyyy`, `yyyy-MM` and `yyyy-MM-dd` (for `yyyy` it must be the same year, for `yyyy-MM` the same month, for `yyyy-MM-dd` the same day)
* `before:VALUE`: the document must have been created before or at the `VALUE` moment, accepted format are `yyyy`, `yyyy-MM` and `yyyy-MM-dd`
* `uafter:VALUE`: the document must have been last updated after or at the `VALUE` moment, accepted format are `yyyy`, `yyyy-MM` and `yyyy-MM-dd`
* `at:VALUE`: the document must have been updated at the `VALUE` moment, accepted format are `yyyy`, `yyyy-MM` and `yyyy-MM-dd` (for `yyyy` it must be the same year, for `yyyy-MM` the same month, for `yyyy-MM-dd` the same day)
* `ubefore:VALUE`: the document must have been updated before or at the `VALUE` moment, accepted format are `yyyy`, `yyyy-MM` and `yyyy-MM-dd`
* Language
* `lang:VALUE`: the document must be of the specified language (example: `en`)
* Mime
* `mime:VALUE`: the document must be of the specified mime type (example: `image/png`)
* Shared
* `shared:VALUE`: if `VALUE` is `yes`the document must be shared, for other `VALUE`s the criteria is ignored
* Tags
* `tag:VALUE`: the document must contain a tag or a child of a tag that starts with `VALUE`, case is ignored
* `!tag:VALUE`: the document must not contain a tag or a child of a tag that starts with `VALUE`, case is ignored
* Title
* `title:VALUE`: the title of the document must be `VALUE`
* User
* `by:VALUE`: the document creator's username must be `VALUE` with an exact match, the user must not be deleted
* Workflow
* `workflow:VALUE`: if `VALUE` is `me` the document must have an active route, for other `VALUE`s the criteria is ignored