Red Hat Summit Red Hat Summitサポートコンソール開発者トライアルの開始お問い合わせ

このコンテンツは選択した言語では利用できません。

Record Validation filter guide

Red Hat Streams for Apache Kafka 3.1

Validate producer records being sent to Kafka

Abstract

Streams for Apache Kafka Proxy is a protocol-aware proxy that extends and secures Kafka-based systems with a flexible filtering mechanism. This guide explains how to use the Record Validation filter.

Providing feedback on Red Hat documentation
リンクのコピー

We appreciate your feedback on our documentation.

To propose improvements, open a Jira issue and describe your suggested changes. Provide as much detail as possible to enable us to address your request quickly.

Prerequisite

  • You have a Red Hat Customer Portal account. This account enables you to log in to the Red Hat Jira Software instance. If you do not have an account, you will be prompted to create one.

Procedure

  1. Click Create issue.
  2. In the Summary text box, enter a brief description of the issue.

  3. In the Description text box, provide the following information:

    • The URL of the page where you found the issue.
    • A detailed description of the issue.
      You can leave the information in any other fields at their default values.
  4. Add a reporter name.
  5. Click Create to submit the Jira issue to the documentation team.

Thank you for taking the time to provide feedback.

Technology preview
リンクのコピー

The Record Validation filter is a technology preview.

Technology Preview features are not supported with Red Hat production service-level agreements (SLAs) and might not be functionally complete; therefore, Red Hat does not recommend implementing any Technology Preview features in production environments. This Technology Preview feature provides early access to upcoming product innovations, enabling you to test functionality and provide feedback during the development process. For more information about the support scope, see Technology Preview Features Support Scope.

About this guide

This guide covers using the Streams for Apache Kafka Proxy Record Validation Filter to validate records sent by Kafka client to Kafka brokers. Refer to other Streams for Apache Kafka Proxy guides for information on running the proxy or for advanced topics such as plugin development.

The Record Validation filter validates records sent by a producer. Only records that pass the validation are sent to the broker. This filter can be used to prevent poison messages—such as those containing corrupted data or invalid formats—from entering the Kafka system, which may otherwise lead to consumer failure.

The filter currently supports two modes of operation:

  1. Schema Validation ensures the content of the record conforms to a schema stored in an Apicurio Registry.
  2. JSON Syntax Validation ensures the content of the record contains syntactically valid JSON.

Validation rules can be applied to check the content of the Kafka record key or value.

If the validation fails, the product request is rejected and the producing application receives an error response. The broker will not receive the rejected records.

Note

This filter is currently in incubation and available as a preview. We would not recommend using it in a production environment.

Chapter 1. (Preview) Setting up the Record Validation filter
リンクのコピー

This procedure describes how to set up the Record Validation filter. Provide the filter configuration and rules that the filter uses to check against Kafka record keys and values.

Prerequisites

  • An instance of Streams for Apache Kafka Proxy. For information on deploying Streams for Apache Kafka Proxy, see the Deploying and Managing Streams for Apache Kafka Proxy on OpenShift guide.
  • A config map for Streams for Apache Kafka Proxy that includes the configuration for creating a virtual cluster.
  • Apicurio Registry (if wanting to use Schema validation).

Procedure

  1. Configure a RecordValidation type filter.

Replace the token <rule definition> in the YAML configuration with either a Schema Validation rule or a JSON Syntax Validation rule depending on your requirements.

Example Schema Validation Rule Definition

The Schema Validation rule validates that the key or value matches a schema identified by its global ID within an Apicurio Schema Registry.

If the key or value does not adhere to the schema, the record will be rejected.

Additionally, if the kafka producer has embedded a global ID within the record it will be validated against the global ID defined by the rule. If they do not match, the record will be rejected. See the Apicurio documentation for details on how the global ID could be embedded into the record. The filter supports extracting ID’s from either the Apicurio globalId record header or from the initial bytes of the serialized content itself. 

schemaValidationConfig:
    apicurioGlobalId: 1001
1 

    apicurioRegistryUrl: http://registry.local:8080
2 

allowNulls: true
3 

allowEmpty: true
4
Copy to Clipboard Toggle word wrap
1
Apicurio registry global ID identifying the schema that will be enforced.
2
Apicurio Registry endpoint.
3
if true, the validator allows keys and or values to be null. The default is false.
4
if true, the validator allows keys and or values to be empty. The default is false.
Note

Schema validation mode currently has the capability to enforce only JSON schemas (issue)

Example JSON Syntax Validation Rule Definition

The JSON Syntax Validation rule validates that the key or value contains only syntactically correct JSON. 

syntacticallyCorrectJson:
    validateObjectKeysUnique: true
1 

allowNulls: true
2 

allowEmpty: true
3
Copy to Clipboard Toggle word wrap
1
If true, the validator enforces that objects keys must be unique. The default is false.
2
if true, the validator allows keys and or values to be null. The default is false.
3
if true, the validator allows keys and or values to be empty. The default is false.

1.1. Example KafkaProtocolFilter resource
リンクのコピー

If your instance of Streams for Apache Kafka Proxy runs on OpenShift, you must use a KafkaProtocolFilter resource to contain the filter configuration.

Here’s a complete example of a KafkaProtocolFilter resource configured for record validation:

Example KafkaProtocolFilter resource for record validation

kind: KafkaProtocolFilter
metadata:
  name: my-validation-filter
spec:
  type: RecordValidation
  configTemplate:
    rules:
    - topicNames:
1 

        - <topic name>
      keyRule:
        <rule definition>
2 

      valueRule:
        <rule definition>
3 

    defaultRule:
4 

      keyRule:
        <rule definition>
5 

      valueRule:
        <rule definition>
6
Copy to Clipboard Toggle word wrap

1
List of topic names to which the validation rules will be applied.
2 5
Validation rules that are applied to the record’s key.
3 6
Validation rules that are applied to the record’s value.
4
(Optional) Default rule that is applied to any topics for which there is no explict rule defined.

Refer to the Deploying and Managing Streams for Apache Kafka Proxy on OpenShift guide for more information about configuration on OpenShift.

Appendix A. Using your subscription
リンクのコピー

Streams for Apache Kafka is provided through a software subscription. To manage your subscriptions, access your account at the Red Hat Customer Portal.

A.1. Accessing Your Account
リンクのコピー

  1. Go to access.redhat.com.
  2. If you do not already have an account, create one.
  3. Log in to your account.

A.2. Activating a Subscription
リンクのコピー

  1. Go to access.redhat.com.
  2. Navigate to My Subscriptions.
  3. Navigate to Activate a subscription and enter your 16-digit activation number.

A.3. Downloading Zip and Tar Files
リンクのコピー

To access zip or tar files, use the customer portal to find the relevant files for download. If you are using RPM packages, this step is not required.

  1. Open a browser and log in to the Red Hat Customer Portal Product Downloads page at access.redhat.com/downloads.
  2. Locate the Streams for Apache Kafka entries in the INTEGRATION AND AUTOMATION category.
  3. Select the desired Streams for Apache Kafka product. The Software Downloads page opens.
  4. Click the Download link for your component.

A.4. Installing packages with DNF
リンクのコピー

To install a package and all the package dependencies, use: 

dnf install <package_name>
Copy to Clipboard Toggle word wrap

To install a previously-downloaded package from a local directory, use: 

dnf install <path_to_download_package>
Copy to Clipboard Toggle word wrap

Revised on 2025-12-16 10:58:00 UTC

Legal Notice
リンクのコピー

Copyright © Red Hat.
The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat Software Collections is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.
Red Hat logoGithubredditYoutubeTwitter

詳細情報

試用、購入および販売

コミュニティー

Red Hat ドキュメントについて

Red Hat をお使いのお客様が、信頼できるコンテンツが含まれている製品やサービスを活用することで、イノベーションを行い、目標を達成できるようにします。 最新の更新を見る.

多様性を受け入れるオープンソースの強化

Red Hat では、コード、ドキュメント、Web プロパティーにおける配慮に欠ける用語の置き換えに取り組んでいます。このような変更は、段階的に実施される予定です。詳細情報: Red Hat ブログ.

会社概要

Red Hat は、企業がコアとなるデータセンターからネットワークエッジに至るまで、各種プラットフォームや環境全体で作業を簡素化できるように、強化されたソリューションを提供しています。

Theme

© 2026 Red Hatトップに戻る