Chapter 15. Querying
Infinispan Query can execute Lucene queries and retrieve domain objects from a Red Hat JBoss Data Grid cache.
Procedure 15.1. Prepare and Execute a Query
- Get
SearchManagerof an indexing enabled cache as follows:SearchManager manager = Search.getSearchManager(cache);
- Create a
QueryBuilderto build queries forMyth.classas follows:final org.hibernate.search.query.dsl.QueryBuilder queryBuilder = manager.buildQueryBuilderForClass(Myth.class).get(); - Create an Apache Lucene query that queries the
Myth.classclass' atributes as follows:org.apache.lucene.search.Query query = queryBuilder.keyword() .onField("history").boostedTo(3) .matching("storm") .createQuery(); // wrap Lucene query in a org.infinispan.query.CacheQuery CacheQuery cacheQuery = manager.getQuery(query); // Get query result List<Object> result = cacheQuery.list();
15.1. Building Queries
Query Module queries are built on Lucene queries, allowing users to use any Lucene query type. When the query is built, Infinispan Query uses org.infinispan.query.CacheQuery as the query manipulation API for further query processing.
15.1.1. Building a Lucene Query Using the Lucene-based Query API
With the Lucene API, use either the query parser (simple queries) or the Lucene programmatic API (complex queries). For details, see the online Lucene documentation or a copy of Lucene in Action or Hibernate Search in Action.
15.1.2. Building a Lucene Query
Using the Lucene programmatic API, it is possible to write full-text queries. However, when using Lucene programmatic API, the parameters must be converted to their string equivalent and must also apply the correct analyzer to the right field. A ngram analyzer for example uses several ngrams as the tokens for a given word and should be searched as such. It is recommended to use the
QueryBuilder for this task.
The Lucene-based query API is fluent. This API has a following key characteristics:
- Method names are in English. As a result, API operations can be read and understood as a series of English phrases and instructions.
- It uses IDE autocompletion which helps possible completions for the current input prefix and allows the user to choose the right option.
- It often uses the chaining method pattern.
- It is easy to use and read the API operations.
To use the API, first create a query builder that is attached to a given indexed type. This
QueryBuilder knows what analyzer to use and what field bridge to apply. Several QueryBuilders (one for each type involved in the root of your query) can be created. The QueryBuilder is derived from the SearchFactory.
Search.getSearchManager(cache).buildQueryBuilderForClass(Myth.class).get();
The analyzer, used for a given field or fields can also be overridden.
SearchFactory searchFactory = Search.getSearchManager(cache).getSearchFactory();
QueryBuilder mythQB = searchFactory.buildQueryBuilder()
.forEntity(Myth.class)
.overridesForField("history","stem_analyzer_definition")
.get();
The query builder is now used to build Lucene queries.
15.1.2.1. Keyword Queries
The following example shows how to search for a specific word:
Example 15.1. Keyword Search
Query luceneQuery = mythQB.keyword().onField("history").matching("storm").createQuery();| Parameter | Description |
|---|---|
| keyword() | Use this parameter to find a specific word |
| onField() | Use this parameter to specify in which lucene field to search the word |
| matching() | use this parameter to specify the match for search string |
| createQuery() | creates the Lucene query object |
- The value "storm" is passed through the
historyFieldBridge. This is useful when numbers or dates are involved. - The field bridge value is then passed to the analyzer used to index the field
history. This ensures that the query uses the same term transformation than the indexing (lower case, ngram, stemming and so on). If the analyzing process generates several terms for a given word, a boolean query is used with theSHOULDlogic (roughly anORlogic).
To search a property that is not of type string.
@Indexed
public class Myth {
@Field(analyze = Analyze.NO)
@DateBridge(resolution = Resolution.YEAR)
public Date getCreationDate() { return creationDate; }
public Date setCreationDate(Date creationDate) { this.creationDate = creationDate; }
private Date creationDate;
}
Date birthdate = ...;
Query luceneQuery = mythQb.keyword()
.onField("creationDate")
.matching(birthdate)
.createQuery();Note
In plain Lucene, the
Date object had to be converted to its string representation (in this case the year)
This conversion works for any object, provided that the
FieldBridge has an objectToString method (and all built-in FieldBridge implementations do).
The next example searches a field that uses ngram analyzers. The ngram analyzers index succession of ngrams of words, which helps to avoid user typos. For example, the 3-grams of the word hibernate are hib, ibe, ber, rna, nat, ate.
Example 15.2. Searching Using Ngram Analyzers
@AnalyzerDef(name = "ngram",
tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class),
filters = {
@TokenFilterDef(factory = StandardFilterFactory.class),
@TokenFilterDef(factory = LowerCaseFilterFactory.class),
@TokenFilterDef(factory = StopFilterFactory.class),
@TokenFilterDef(factory = NGramFilterFactory.class,
params = {
@Parameter(name = "minGramSize", value = "3"),
@Parameter(name = "maxGramSize", value = "3")})
})
public class Myth {
@Field(analyzer = @Analyzer(definition = "ngram"))
public String getName() { return name; }
public String setName(String name) { this.name = name; }
private String name;
}
Date birthdate = ...;
Query luceneQuery = mythQb.keyword()
.onField("name")
.matching("Sisiphus")
.createQuery();
The matching word "Sisiphus" will be lower-cased and then split into 3-grams: sis, isi, sip, phu, hus. Each of these ngram will be part of the query. The user is then able to find the Sysiphus myth (with a
y). All that is transparently done for the user.
Note
If the user does not want a specific field to use the field bridge or the analyzer then the
ignoreAnalyzer() or ignoreFieldBridge() functions can be called.
To search for multiple possible words in the same field, add them all in the matching clause.
Example 15.3. Searching for Multiple Words
//search document with storm or lightning in their history
Query luceneQuery =
mythQB.keyword().onField("history").matching("storm lightning").createQuery();
To search the same word on multiple fields, use the
onFields method.
Example 15.4. Searching Multiple Fields
Query luceneQuery = mythQB
.keyword()
.onFields("history","description","name")
.matching("storm")
.createQuery();
In some cases, one field must be treated differently from another field even if searching the same term. In this case, use the
andField() method.
Example 15.5. Using the andField Method
Query luceneQuery = mythQB.keyword()
.onField("history")
.andField("name")
.boostedTo(5)
.andField("description")
.matching("storm")
.createQuery();
In the previous example, only field name is boosted to 5.
15.1.2.2. Fuzzy Queries
To execute a fuzzy query (based on the Levenshtein distance algorithm), start like a
keyword query and add the fuzzy flag.
Example 15.6. Fuzzy Query
Query luceneQuery = mythQB.keyword()
.fuzzy()
.withThreshold(.8f)
.withPrefixLength(1)
.onField("history")
.matching("starm")
.createQuery();
The
threshold is the limit above which two terms are considering matching. It is a decimal between 0 and 1 and the default value is 0.5. The prefixLength is the length of the prefix ignored by the "fuzzyness". While the default value is 0, a non zero value is recommended for indexes containing a huge amount of distinct terms.
15.1.2.3. Wildcard Queries
Wildcard queries can also be executed (queries where some of parts of the word are unknown). The
? represents a single character and * represents any character sequence. Note that for performance purposes, it is recommended that the query does not start with either ? or *.
Example 15.7. Wildcard Query
Query luceneQuery = mythQB.keyword()
.wildcard()
.onField("history")
.matching("sto*")
.createQuery();Note
Wildcard queries do not apply the analyzer on the matching terms. Otherwise the risk of
* or ? being mangled is too high.
15.1.2.4. Phrase Queries
So far we have been looking for words or sets of words, the user can also search exact or approximate sentences. Use
phrase() to do so.
Example 15.8. Phrase Query
Query luceneQuery = mythQB.phrase()
.onField("history")
.sentence("Thou shalt not kill")
.createQuery();
Approximate sentences can be searched by adding a slop factor. The slop factor represents the number of other words permitted in the sentence: this works like a within or near operator.
Example 15.9. Adding Slop Factor
Query luceneQuery = mythQB.phrase()
.withSlop(3)
.onField("history")
.sentence("Thou kill")
.createQuery();15.1.2.5. Range Queries
A range query searches for a value in between given boundaries (included or not) or for a value below or above a given boundary (included or not).
Example 15.10. Range Query
//look for 0 <= starred < 3
Query luceneQuery = mythQB.range()
.onField("starred")
.from(0).to(3).excludeLimit()
.createQuery();
//look for myths strictly BC
Date beforeChrist = ...;
Query luceneQuery = mythQB.range()
.onField("creationDate")
.below(beforeChrist).excludeLimit()
.createQuery();15.1.2.6. Combining Queries
Queries can be aggregated (combine) to create more complex queries. The following aggregation operators are available:
SHOULD: the query should contain the matching elements of the subquery.MUST: the query must contain the matching elements of the subquery.MUST NOT: the query must not contain the matching elements of the subquery.
The subqueries can be any Lucene query including a boolean query itself. Following are some examples:
Example 15.11. Combining Subqueries
//look for popular modern myths that are not urban
Date twentiethCentury = ...;
Query luceneQuery = mythQB.bool()
.must(mythQB.keyword().onField("description").matching("urban").createQuery())
.not()
.must(mythQB.range().onField("starred").above(4).createQuery())
.must(mythQB.range()
.onField("creationDate")
.above(twentiethCentury)
.createQuery())
.createQuery();
//look for popular myths that are preferably urban
Query luceneQuery = mythQB
.bool()
.should(mythQB.keyword()
.onField("description")
.matching("urban")
.createQuery())
.must(mythQB.range().onField("starred").above(4).createQuery())
.createQuery();
//look for all myths except religious ones
Query luceneQuery = mythQB.all()
.except(mythQb.keyword()
.onField("description_stem")
.matching("religion")
.createQuery())
.createQuery();15.1.2.7. Query Options
The following is a summary of query options for query types and fields:
boostedTo(on query type and on field) boosts the query or field to a provided factor.withConstantScore(on query) returns all results that match the query and have a constant score equal to the boost.filteredBy(Filter)(on query) filters query results using theFilterinstance.ignoreAnalyzer(on field) ignores the analyzer when processing this field.ignoreFieldBridge(on field) ignores the field bridge when processing this field.
The following example illustrates how to use these options:
Example 15.12. Querying Options
Query luceneQuery = mythQB
.bool()
.should(mythQB.keyword().onField("description").matching("urban").createQuery())
.should(mythQB
.keyword()
.onField("name")
.boostedTo(3)
.ignoreAnalyzer()
.matching("urban").createQuery())
.must(mythQB
.range()
.boostedTo(5)
.withConstantScore()
.onField("starred")
.above(4).createQuery())
.createQuery();
15.1.3. Build a Query with Infinispan Query
15.1.3.1. Generality
After building the Lucene query, wrap it within a Infinispan CacheQuery. The query searches all indexed entities and returns all types of indexed classes unless explicitly configured not to do so.
Example 15.13. Wrapping a Lucene Query in an Infinispan CacheQuery
CacheQuery cacheQuery = Search.getSearchManager(cache).getQuery(luceneQuery);
For improved performance, restrict the returned types as follows:
Example 15.14. Filtering the Search Result by Entity Type
CacheQuery cacheQuery =
Search.getSearchManager(cache).getQuery(luceneQuery, Customer.class);
// or
CacheQuery cacheQuery =
Search.getSearchManager(cache).getQuery(luceneQuery, Item.class, Actor.class);
The first part of the second example only returns the matching
Customer instances. The second part of the same example returns matching Actor and Item instances. The type restriction is polymorphic. As a result, if the two subclasses Salesman and Customer of the base class Person return, specify Person.class to filter based on result types.
15.1.3.2. Pagination
To avoid performance degradation, it is recommended to restrict the number of returned objects per query. A user navigating from one page to another page is a very common use case. The way to define pagination is similar to defining pagination in a plain HQL or Criteria query.
Example 15.15. Defining pagination for a search query
CacheQuery cacheQuery = Search.getSearchManager(cache)
.getQuery(luceneQuery, Customer.class);
cacheQuery.firstResult(15); //start from the 15th element
cacheQuery.maxResults(10); //return 10 elementsNote
The total number of matching elements, despite the pagination, is accessible via
cacheQuery.getResultSize().
15.1.3.3. Sorting
Apache Lucene contains a flexible and powerful result sorting mechanism. The default sorting is by relevance and is appropriate for a large variety of use cases. The sorting mechanism can be changed to sort by other properties using the Lucene Sort object to apply a Lucene sorting strategy.
Example 15.16. Specifying a Lucene Sort
org.infinispan.query.CacheQuery cacheQuery = Search.getSearchManager(cache).getQuery(luceneQuery, Book.class);
org.apache.lucene.search.Sort sort = new Sort(
new SortField("title", SortField.STRING));
cacheQuery.sort(sort);
List results = cacheQuery.list();Note
Fields used for sorting must not be tokenized. For more information about tokenizing, see Section 14.1.2, “@Field”.
15.1.3.4. Projection
In some cases, only a small subset of the properties is required. Use Infinispan Query to return a subset of properties as follows:
Example 15.17. Using Projection Instead of Returning the Full Domain Object
SearchManager searchManager = Search.getSearchManager(cache);
CacheQuery cacheQuery = searchManager.getQuery(luceneQuery, Book.class);
cacheQuery.projection("id", "summary", "body", "mainAuthor.name");
List results = cacheQuery.list();
Object[] firstResult = (Object[]) results.get(0);
Integer id = (Integer) firstResult[0];
String summary = (String) firstResult[1];
String body = (String) firstResult[2];
String authorName = (String) firstResult[3];
The Query Module extracts properties from the Lucene index and converts them to their object representation and returns a list of
Object[]. Projections prevent a time consuming database round-trip. However, they have following constraints:
- The properties projected must be stored in the index (
@Field(store=Store.YES)), which increases the index size. - The properties projected must use a
FieldBridgeimplementingorg.infinispan.query.bridge.TwoWayFieldBridgeororg.infinispan.query.bridge.TwoWayStringBridge, the latter being the simpler version.Note
All Lucene-based Query API built-in types are two-way. - Only the simple properties of the indexed entity or its embedded associations can be projected. Therefore a whole embedded entity cannot be projected.
- Projection does not work on collections or maps which are indexed via
@IndexedEmbedded
Lucene provides metadata information about query results. Use projection constants to retrieve the metadata.
Example 15.18. Using Projection to Retrieve Metadata
SearchManager searchManager = Search.getSearchManager(cache);
CacheQuery cacheQuery = searchManager.getQuery(luceneQuery, Book.class);
cacheQuery.projection("mainAuthor.name");
List results = cacheQuery.list();
Object[] firstResult = (Object[]) results.get(0);
float score = (Float) firstResult[0];
Book book = (Book) firstResult[1];
String authorName = (String) firstResult[2];
Fields can be mixed with the following projection constants:
FullTextQuery.THISreturns the initialized and managed entity as a non-projected query does.FullTextQuery.DOCUMENTreturns the Lucene Document related to the projected object.FullTextQuery.OBJECT_CLASSreturns the indexed entity's class.FullTextQuery.SCOREreturns the document score in the query. Use scores to compare one result against another for a given query. However, scores are not relevant to compare the results of two different queries.FullTextQuery.IDis the ID property value of the projected object.FullTextQuery.DOCUMENT_IDis the Lucene document ID. The Lucene document ID changes between two IndexReader openings.FullTextQuery.EXPLANATIONreturns the Lucene Explanation object for the matching object/document in the query. This is not suitable for retrieving large amounts of data. RunningFullTextQuery.EXPLANATIONis as expensive as running a Lucene query for each matching element. As a result, projection is recommended.
15.1.3.5. Limiting the Time of a Query
Limit the time a query takes in Infinispan Query as follows:
- Raise an exception when arriving at the limit.
- Limit to the number of results retrieved when the time limit is raised.
15.1.3.6. Raise an Exception on Time Limit
If a query uses more than the defined amount of time, a custom exception might be defined to be thrown.
To define the limit when using the CacheQuery API, use the following approach:
Example 15.19. Defining a Timeout in Query Execution
SearchManagerImplementor searchManager = (SearchManagerImplementor) Search.getSearchManager(cache);
searchManager.setTimeoutExceptionFactory(new MyTimeoutExceptionFactory());
CacheQuery cacheQuery = searchManager.getQuery(luceneQuery, Book.class);
//define the timeout in seconds
cacheQuery.timeout(2, TimeUnit.SECONDS)
try {
query.list();
}
catch (MyTimeoutException e) {
//do something, too slow
}
private static class MyTimeoutExceptionFactory implements TimeoutExceptionFactory {
@Override
public RuntimeException createTimeoutException(String message, Query query) {
return new MyTimeoutException();
}
}
public static class MyTimeoutException extends RuntimeException {
}
The
getResultSize(), iterate() and scroll() honor the timeout until the end of the method call. As a result, Iterable or the ScrollableResults ignore the timeout. Additionally, explain() does not honor this timeout period. This method is used for debugging and to check the reasons for slow performance of a query.
Important
The example code does not guarantee that the query stops at the specified results amount.
