查询数据网格缓存

在缓存配置中启用索引，并指定在创建索引时应包含哪些实体数据网格。

在使用查询时，您应该始终将 Data Grid 配置为索引缓存。索引为您的查询提供了显著的性能，允许您更快地了解您的数据。

protobuf 消息

Java 对象

<distributed-cache>
  <indexing>
    <indexed-entities>
      <indexed-entity>book_sample.Book</indexed-entity>
    </indexed-entities>
  </indexing>
</distributed-cache>

import org.infinispan.configuration.cache.*;

ConfigurationBuilder config=new ConfigurationBuilder();
config.indexing().enable().storage(FILESYSTEM).path("/some/folder").addIndexedEntity(Book.class);

<distributed-cache>
  <indexing storage="filesystem" path="${java.io.tmpdir}/baseDir">
    <!-- Indexing configuration goes here. -->
  </indexing>
</distributed-cache>

<distributed-cache>
  <indexing storage="local-heap">
    <!-- Additional indexing configuration goes here. -->
  </indexing>
</distributed-cache>

<distributed-cache>
  <indexing storage="filesystem" path="${java.io.tmpdir}/baseDir">
    <!-- Sets an interval of one second for the index reader. -->
    <index-reader refresh-interval="1000"/>
    <!-- Additional indexing configuration goes here. -->
  </indexing>
</distributed-cache>

<distributed-cache>
  <indexing storage="filesystem" path="${java.io.tmpdir}/baseDir">
    <index-writer commit-interval="2000"
                  low-level-trace="false"
                  max-buffered-entries="32"
                  queue-count="1"
                  queue-size="10000"
                  ram-buffer-size="400"
                  thread-pool-size="2">
      <index-merge calibrate-by-deletes="true"
                   factor="3"
                   max-entries="2000"
                   min-size="10"
                   max-size="20"/>
    </index-writer>
    <!-- Additional indexing configuration goes here. -->
  </indexing>
</distributed-cache>

当您在缓存中启用索引时，您可以将 Data Grid 配置为创建索引。您还需要为 Data Grid 提供缓存中实体的结构化表示，以便它实际上可以索引它们。

有两个注解控制 Data Grid 索引的实体和字段：

远程缓存

您可以通过两种方式为远程缓存提供索引注解：

嵌入式缓存

对于嵌入式缓存，您可以根据 Data Grid 如何存储您的条目，为 Java 类添加索引注解。

使用 @Indexed 和 @Field 注释，以及其他 Hibernate 搜索注释，如 @FullTextField。

重建索引从缓存中存储的数据重建它。当您更改索引类型或分析器的定义等内容时，您应该重建索引。同样，出于某种原因，您可以在删除索引后重建索引。

Data Grid 建议对缓存进行索引缓存，以获得最佳性能。但是，您可以查询非索引的缓存。

要使用 API，首先获取 QueryFactory 到缓存，然后调用 .create （） 方法，传递要在查询中使用的字符串。每个 QueryFactory 实例都绑定到与 Search 相同的 缓存 实例，但它是一个无状态和线程安全对象，可用于并行创建多个查询。

例如：

// Remote Query, using protobuf
QueryFactory qf = org.infinispan.client.hotrod.Search.getQueryFactory(remoteCache);
Query<Transaction> q = qf.create("from sample_bank_account.Transaction where amount > 20");

// Embedded Query using Java Objects
QueryFactory qf = org.infinispan.query.Search.getQueryFactory(cache);
Query<Transaction> q = qf.create("from org.infinispan.sample.Book where price > 20");

// Execute the query
QueryResult<Book> queryResult = q.execute();

执行查询并获取结果非常简单，就像调用 Query 对象的 execute （） 方法一样。执行后，在同一实例上调用 execute （） 将重新执行查询。

// sorted by year and match all books that have "clustering" in their title
// and return the third page of 10 results
Query<Book> query = queryFactory.create("FROM org.infinispan.sample.Book WHERE title like '%clustering%' ORDER BY year").startOffset(20).maxResults(10)

QueryFactory queryFactory = Search.getQueryFactory(cache);
// Defining a query to search for various authors and publication years
Query<Book> query = queryFactory.create("SELECT title FROM org.infinispan.sample.Book WHERE author = :authorName AND publicationYear = :publicationYear").build();

// Set actual parameter values
query.setParameter("authorName", "Doe");
query.setParameter("publicationYear", 2010);

// Execute the query
List<Book> found = query.execute().list();

Map<String, Object> parameterMap = new HashMap<>();
parameterMap.put("authorName", "Doe");
parameterMap.put("publicationYear", 2010);

query.setParameters(parameterMap);

Ickle 查询语言是 JPQL 查询语言的子集，一些用于全文本的扩展。

解析器语法有一些值得注意的规则：

# match all books that have "Data Grid" in their title
# or have an author named "Manik" and their description contains "clustering"

FROM org.infinispan.sample.Book WHERE title LIKE '%Data Grid%' OR author.name = 'Manik' AND description like '%clustering%'

# match all books that do not have "Data Grid" in their title and are authored by "Manik"
FROM org.infinispan.sample.Book WHERE title != 'Data Grid' AND author.name = 'Manik'

# match all books that have an author named "Manik" and their title contains
# "Data Grid" or their description contains "clustering"
FROM org.infinispan.sample.Book WHERE author.name = 'Manik' AND ( title like '%Data Grid%' OR description like '% clustering%')

# match all books that have "Data Grid" in their title or description
# and return only their title and publication year
SELECT title, publicationYear FROM org.infinispan.sample.Book WHERE title like '%Data Grid%' OR description like '%Data Grid%'

# match all books that have "Data Grid" in their title or description
# and return them sorted by the publication year and title
FROM org.infinispan.sample.Book WHERE title like '%Data Grid%' ORDER BY publicationYear DESC, title ASC

SELECT author, COUNT(title) FROM org.infinispan.sample.Book WHERE title LIKE '%engine%' GROUP BY author

DELETE FROM <entityName> [WHERE condition]

您可以使用 Ickle 查询语言执行全文本搜索。

FROM sample_bank_account.Transaction WHERE description : 'cofee'~2

FROM sample_bank_account.Transaction WHERE amount : [20 to 50]

FROM sample_bank_account.Transaction WHERE description : 'bus fare'

FROM sample_bank_account.Transaction WHERE description : 'canceling fee'~3

FROM sample_bank_account.Transaction where description : 'te?t'

FROM sample_bank_account.Transaction where description : 'test*'

FROM sample_library.Book  where title : /[mb]oat/

FROM sample_library.Book WHERE title : beer^3 OR wine

Data Grid 可让您通过 Hot Rod 端点，以编程方式从 Java 客户端查询远程缓存。此流程解释了如何索引查询存储书签实例的远程缓存。

Data Grid 为 @ProtoField 和 @ProtoDoc 注解提供此处理器，以便您可以生成 Protobuf 模式并执行查询。

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.infinispan</groupId>
      <artifactId>infinispan-bom</artifactId>
      <version>${version.infinispan}</version>
      <type>pom</type>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencies>
  <dependency>
    <groupId>org.infinispan.protostream</groupId>
    <artifactId>protostream-processor</artifactId>
    <scope>provided</scope>
  </dependency>
</dependencies>

<replicated-cache name="books">
  <indexing>
    <indexed-entities>
      <indexed-entity>book_sample.Book</indexed-entity>
    </indexed-entities>
  </indexing>
</replicated-cache>

以下 RemoteQuery 类执行以下操作：

package org.infinispan;

import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.List;

import org.infinispan.client.hotrod.RemoteCache;
import org.infinispan.client.hotrod.RemoteCacheManager;
import org.infinispan.client.hotrod.Search;
import org.infinispan.client.hotrod.configuration.ConfigurationBuilder;
import org.infinispan.query.dsl.Query;
import org.infinispan.query.dsl.QueryFactory;
import org.infinispan.query.remote.client.ProtobufMetadataManagerConstants;

public class RemoteQuery {

   public static void main(String[] args) throws Exception {
      ConfigurationBuilder clientBuilder = new ConfigurationBuilder();
      // RemoteQueryInitializerImpl is generated
      clientBuilder.addServer().host("127.0.0.1").port(11222)
            .security().authentication().username("user").password("user")
            .addContextInitializers(new RemoteQueryInitializerImpl());

      RemoteCacheManager remoteCacheManager = new RemoteCacheManager(clientBuilder.build());

      // Grab the generated protobuf schema and registers in the server.
      Path proto = Paths.get(RemoteQuery.class.getClassLoader()
            .getResource("proto/book.proto").toURI());
      String protoBufCacheName = ProtobufMetadataManagerConstants.PROTOBUF_METADATA_CACHE_NAME;
      remoteCacheManager.getCache(protoBufCacheName).put("book.proto", Files.readString(proto));

      // Obtain the 'books' remote cache
      RemoteCache<Object, Object> remoteCache = remoteCacheManager.getCache("books");

      // Add some Books
      Book book1 = new Book("Infinispan in Action", "Learn Infinispan with using it", 2015);
      Book book2 = new Book("Cloud-Native Applications with Java and Quarkus", "Build robust and reliable cloud applications", 2019);

      remoteCache.put(1, book1);
      remoteCache.put(2, book2);

      // Execute a full-text query
      QueryFactory queryFactory = Search.getQueryFactory(remoteCache);
      Query<Book> query = queryFactory.create("FROM book_sample.Book WHERE title:'java'");

      List<Book> list = query.execute().list(); // Voila! We have our book back from the cache!
   }
}

Data Grid Console 和 Data Grid 命令行界面(CLI)可让您查询索引和非索引的远程缓存。您还可以使用任何 HTTP 客户端通过 REST API 索引和查询缓存。

此流程解释了如何对存储 Person 实例的远程缓存进行索引和查询。

分析器将输入数据转换为您可以索引和查询的术语。您可以使用 Java 类中的 @Field 注释或直接在 Protobuf 模式中指定分析器定义。

/* @Indexed */
message TestEntity {

    /* @Field(store = Store.YES, analyze = Analyze.YES, analyzer = @Analyzer(definition = "keyword")) */
    optional string id = 1;

    /* @Field(store = Store.YES, analyze = Analyze.YES, analyzer = @Analyzer(definition = "simple")) */
    optional string name = 2;
}

@ProtoDoc("@Field(store = Store.YES, analyze = Analyze.YES, analyzer = @Analyzer(definition = \"keyword\"))")
@ProtoField(1)
final String id;

@ProtoDoc("@Field(store = Store.YES, analyze = Analyze.YES, analyzer = @Analyzer(definition = \"simple\"))")
@ProtoField(2)
final String description;

import org.apache.lucene.analysis.core.LowerCaseFilterFactory;
import org.apache.lucene.analysis.core.StopFilterFactory;
import org.apache.lucene.analysis.standard.StandardFilterFactory;
import org.apache.lucene.analysis.standard.StandardTokenizerFactory;
import org.hibernate.search.cfg.SearchMapping;
import org.infinispan.Cache;
import org.infinispan.query.spi.ProgrammaticSearchMappingProvider;

public final class MyAnalyzerProvider implements ProgrammaticSearchMappingProvider {

   @Override
   public void defineMappings(Cache cache, SearchMapping searchMapping) {
      searchMapping
            .analyzerDef("standard-with-stop", StandardTokenizerFactory.class)
               .filter(StandardFilterFactory.class)
               .filter(LowerCaseFilterFactory.class)
               .filter(StopFilterFactory.class);
   }
}

本节介绍如何使用名为"books"的示例缓存查询嵌入式缓存，该缓存存储了索引的书签实例。

在本例中，每个书签实例都定义了索引哪些属性，并使用 Hibernate Search 注解指定一些高级索引选项，如下所示：

package org.infinispan.sample;

import java.time.LocalDate;
import java.util.HashSet;
import java.util.Set;

import org.hibernate.search.mapper.pojo.mapping.definition.annotation.*;

// Annotate values with @Indexed to add them to indexes
// Annotate each fields according to how you want to index it
@Indexed
public class Book {
   @FullTextField
   String title;

   @FullTextField
   String description;

   @KeywordField
   String isbn;

   @GenericField
   LocalDate publicationDate;

   @IndexedEmbedded
   Set<Author> authors = new HashSet<Author>();
}

package org.infinispan.sample;

import org.hibernate.search.mapper.pojo.mapping.definition.annotation.FullTextField;

public class Author {
   @FullTextField
   String name;

   @FullTextField
   String surname;
}

在您的 Java 类中添加注解，将实体映射到索引。

以下小节提供了适用于用于 Data Grid 的实体映射注解的信息。

有关这些注解的完整详情，您应该参考 Hibernate Search manual。

@DocumentId

与 Hibernate Search 不同，使用 @DocumentId 将字段标记为标识符不适用于 Data Grid 值；在 Data Grid 中，所有 @Indexed 对象的标识符都是用于存储值的键。您仍然可以使用 @Transformable、自定义类型和自定义 FieldBridge 实现的组合来自定义如何索引密钥。

@Transformable key

每个值的键还需要进行索引，而键实例必须在 String 中转换。Data Grid 包括一些用于编码常见原语的默认转换例程，但要使用自定义密钥，您必须提供 org.infinispan.query.Transformer 的实现。

@Transformable(transformer = CustomTransformer.class)
public class CustomKey {
   ...
}

public class CustomTransformer implements Transformer {
   @Override
   public Object fromString(String s) {
      ...
      return new CustomKey(...);
   }

   @Override
   public String toString(Object customType) {
      CustomKey ck = (CustomKey) customType;
      return ...
   }
}

<replicated-cache name="test">
  <indexing auto-config="true">
    <key-transformers>
      <key-transformer key="com.mycompany.CustomKey"
                       transformer="com.mycompany.CustomTransformer"/>
    </key-transformers>
  </indexing>
</replicated-cache>

或者，使用 Java 配置 API (embedded 模式)：

   ConfigurationBuilder builder = ...
   builder.indexing().enable()
         .addKeyTransformer(CustomKey.class, CustomTransformer.class);

您可以将实体作为注解 Java 类的替代选择将实体映射到索引。

在以下示例中，我们映射要存储在网格中的对象 Author，并在两个属性上搜索：

import org.apache.lucene.search.Query;
import org.hibernate.search.cfg.Environment;
import org.hibernate.search.cfg.SearchMapping;
import org.hibernate.search.query.dsl.QueryBuilder;
import org.infinispan.Cache;
import org.infinispan.configuration.cache.Configuration;
import org.infinispan.configuration.cache.ConfigurationBuilder;
import org.infinispan.configuration.cache.Index;
import org.infinispan.manager.DefaultCacheManager;
import org.infinispan.query.CacheQuery;
import org.infinispan.query.Search;
import org.infinispan.query.SearchManager;

import java.io.IOException;
import java.lang.annotation.ElementType;
import java.util.Properties;

SearchMapping mapping = new SearchMapping();
mapping.entity(Author.class).indexed()
       .property("name", ElementType.METHOD).field()
       .property("surname", ElementType.METHOD).field();

Properties properties = new Properties();
properties.put(Environment.MODEL_MAPPING, mapping);
properties.put("hibernate.search.[other options]", "[...]");

Configuration infinispanConfiguration = new ConfigurationBuilder()
        .indexing().index(Index.NONE)
        .withProperties(properties)
        .build();

DefaultCacheManager cacheManager = new DefaultCacheManager(infinispanConfiguration);

Cache<Long, Author> cache = cacheManager.getCache();
SearchManager sm = Search.getSearchManager(cache);

Author author = new Author(1, "Manik", "Surtani");
cache.put(author.getId(), author);

QueryBuilder qb = sm.buildQueryBuilderForClass(Author.class).get();
Query q = qb.keyword().onField("name").matching("Manik").createQuery();
CacheQuery cq = sm.getQuery(q, Author.class);
assert cq.getResultSize() == 1;

持续查询为应用程序提供有关查询过滤的数据的实时通知。当条目与查询数据平面将更新的数据发送到任何监听程序时，这提供了事件流，而不是需要执行查询的应用程序。

持续查询可以通知应用程序有关传入匹配项、用于已加入集合的值；针对已修改并继续匹配的匹配值，请针对已设置的值进行传出匹配。

例如，持续查询可以通知应用程序所有：

持续查询的工作方式

持续查询会通知客户端监听程序，并带有以下事件：

当客户端注册持续查询监听程序时，它会立即接收与查询匹配的任何条目的 加入 事件。每次缓存操作修改与查询匹配的条目时，客户端监听程序都会接收后续的事件。

Data Grid 决定何时向客户端监听程序发送 Join、Update 或 Leave 事件，如下所示：

SELECT field1, field2 FROM Entity WHERE x AND y

您可以为远程和嵌入式缓存创建持续查询。

持续查询示例

以下代码示例演示了带有嵌入式缓存的简单持续查询。

在本例中，监听器会在 21 年龄下的任何 Person 实例添加到缓存中时收到通知。这些 Person 实例也添加到"匹配"映射中。当从缓存中删除条目或其年龄大于或等于 21 时，它们会从 "matches" 映射中删除。

import org.infinispan.query.api.continuous.ContinuousQuery;
import org.infinispan.query.api.continuous.ContinuousQueryListener;
import org.infinispan.query.Search;
import org.infinispan.query.dsl.QueryFactory;
import org.infinispan.query.dsl.Query;

import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;

[...]

// We have a cache of Person objects.
Cache<Integer, Person> cache = ...

// Create a ContinuousQuery instance on the cache.
ContinuousQuery<Integer, Person> continuousQuery = Search.getContinuousQuery(cache);

// Define a query.
// In this example, we search for Person instances under 21 years of age.
QueryFactory queryFactory = Search.getQueryFactory(cache);
Query query = queryFactory.create("FROM Person p WHERE p.age < 21");

final Map<Integer, Person> matches = new ConcurrentHashMap<Integer, Person>();

// Define the ContinuousQueryListener.
ContinuousQueryListener<Integer, Person> listener = new ContinuousQueryListener<Integer, Person>() {
    @Override
    public void resultJoining(Integer key, Person value) {
        matches.put(key, value);
    }

    @Override
    public void resultUpdated(Integer key, Person value) {
        // We do not process this event.
    }

    @Override
    public void resultLeaving(Integer key) {
        matches.remove(key);
    }
};

// Add the listener and the query.
continuousQuery.addContinuousQueryListener(query, listener);

[...]

// Remove the listener to stop receiving notifications.
continuousQuery.removeContinuousQueryListener(listener);

收集统计信息，以收集有关索引和查询性能的信息，包括索引类型和查询完成的平均时间等信息。

// Statistics for the local cluster member
SearchStatistics statistics = Search.getSearchStatistics(cache);

// Consolidated statistics for the whole cluster
CompletionStage<SearchStatisticsSnapshot> statistics = Search.getClusteredSearchStatistics(cache)

GET /v2/caches/{cacheName}/search/stats

使用以下指南帮助您提高索引操作和查询的性能。

通过检查每种查询运行所需的时间，开始调整查询性能。如果您的查询似乎较慢，您应该确保查询正在使用索引缓存，并且所有实体和字段映射都被索引。

此操作非常昂贵，因此您应该避免配置太小的时间间隔。默认值为 1000 毫秒(1 秒)。

默认值为 0， 它会在将查询写入缓存后立即返回查询中的数据。

大于 0 的值会导致一些过时的查询结果，但会显著提高吞吐量，特别是在写密集型场景中。如果您不需要在查询被写入后马上返回数据，您应该调整刷新间隔以提高查询性能。