Chapter 9. Bucket notification in Multicloud Object Gateway

Procedure

1. Describe connection using a json file, for example, connection.json on your local machine:

   For Kafka:

   {
     "name": "kafka_notif_conn_file" <-- any string
     "notification_protocol": "kafka",
     "metadata.broker.list": "<kafka-service-name>.<project>.svc.cluster.local:<kafka-service-port>",
     "topic": "my-topic",<--  refer to an existing KafkaTopic resource
   }

   Copy to Clipboard Toggle word wrap

   The structure under metadata.broker.list must be <service-name>.<namespace>.svc.cluster.local:9092, topic must refer to the name of an existing kafkatopic resource in the namespace.

   For HTTP(s):

   {
     "name": "http_notif_connection_config",
     "notification_protocol": "http", <-- or "https"
     "agent_request_object": {
       "host": "<http-service>.<http-server-namespace>.svc.cluster.local",
       "port": <http-server-port>
      }
   }

   Copy to Clipboard Toggle word wrap

   Additional options:

   request_options_object

   Value is a JSON that is passed to nodejs' http(s) request (optional).

   Any field supported by nodejs' http(s) request option can be used, for example:

   'path'

   Used to specify the url path

   'auth'

   Used for http simple auth. Syntax for the value of 'auth' is: <name>:<password>.

2. Create a secret from the file in the openshift-storage namespace:

   $ oc create secret generic <connection-secret> --from-file=connect.json -n openshift-storage

   Copy to Clipboard Toggle word wrap

3. Update the NooBaa CR in the openshift-storage namespace with the connection secret and an optional CephFS RWX PVC. If PVC is not provided, MCG creates one automatically as needed:

   $ oc get pvc bn-pvc -n openshift-storage
   NAME     STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS                VOLUMEATTRIBUTESCLASS   AGE
   bn-pvc   Bound    pvc-24683f8d-48e4-4c6b-b108-d507ca2f4fd1   25Gi       RWX            ocs-storagecluster-cephfs   <unset>                 25h

   Copy to Clipboard Toggle word wrap

   $ oc patch noobaa noobaa --type='merge' -n openshift-storage -p '{
     "spec": {
       "bucketNotifications": {
         "connections": [
           {
             "name": <connection-secret>,
             "namespace": "openshift-storage"
           }
         ],
         "enabled": true,
         "pvc": "bn-pvc" <-- optional
       }
     }
   }'

   Copy to Clipboard Toggle word wrap
   Important

   Names of the connection secret must be unique in the list even though the namespaces are different.

   Wait for the noobaa-core and noobaa-endpoint pods to restart before proceeding with the next step.

4. Use S3:PutBucketNotification on an MCG bucket using the noobaa-admin credentials and S3 endpoint under an S3 alias.

   $ aws_alias s3api put-bucket-notification --bucket first.bucket --notification-configuration '{
     "TopicConfiguration": {
       "Id": "<notif_event_kafka>", <-- Unique string
       "Events": ["s3:ObjectCreated:*"], <-- a filter for events
       "Topic": "<connection-secret/connect.json>"
     }
   }'

   Copy to Clipboard Toggle word wrap

Verification steps

1. Verify that the bucket notification configuration is set on the bucket:

   $ aws_alias s3api get-bucket-notification-configuration --bucket first.bucket
   {
       "TopicConfigurations": [
           {
               "Id": "notif_event_kafka",
               "TopicArn": "kafka-connection-secret/connect.json",
               "Events": [
                   "s3:ObjectCreated:*"
               ]
           }
       ]
   }

   Copy to Clipboard Toggle word wrap

2. Add some objects on the bucket:

   echo 'a' | aws_alias s3 cp - s3://first.bucket/a

   Copy to Clipboard Toggle word wrap

   Wait for a while and query the topic messages to verify the expected notification has been sent and received.

   For example:

   For Kafka

   $ oc -n your-kafka-project rsh my-cluster-kafka-0 bin/kafka-console-consumer.sh \
     --bootstrap-server my-cluster-kafka-bootstrap.myproject.svc.cluster.local:9092 \
     --topic my-topic \
     --from-beginning --timeout-ms 10000 | grep '^{.*}' | jq -c '.' | jq

   Copy to Clipboard Toggle word wrap

   For HTTP(s)

   $ oc logs deployment/http-logger -n <http-server-namespace> | grep '^{.*}' | jq

   Copy to Clipboard Toggle word wrap

   Output

   {
     "Records": [
       {
         "eventVersion": "2.3",
         "eventSource": "noobaa:s3",
         "eventTime": "2024-11-27T12:44:21.987Z",
         "s3": {
           "s3SchemaVersion": "1.0",
           "object": {
             "sequencer": 10,
             "key": "a",
             "eTag": "60b725f10c9c85c70d97880dfe8191b3"
           },
           "bucket": {
             "name": "second.bucket",
             "ownerIdentity": {
               "principalId": "admin@noobaa.io"
             },
             "arn": "arn:aws:s3:::first.bucket"
           }
         },
         "eventName": "ObjectCreated:Put",
         "userIdentity": {
           "principalId": "noobaa"
         },
         "requestParameters": {
           "sourceIPAddress": "100.64.0.3"
         },
         "responseElements": {
           "x-amz-request-id": "m3zvo0cm-5239xd-bhj",
           "x-amz-id-2": "m3zvo0cm-5239xd-bhj"
         }
       }
     ]
   }

   Copy to Clipboard Toggle word wrap