Chapter 13. Monitoring bare-metal events with the Bare Metal Event Relay

Procedure

1. Create a namespace for the Bare Metal Event Relay.

   1. Save the following YAML in the

      bare-metal-events-namespace.yaml

      file:

      apiVersion: v1
      kind: Namespace
      metadata:
        name: openshift-bare-metal-events
        labels:
          name: openshift-bare-metal-events
          openshift.io/cluster-monitoring: "true"

   2. Create the

      Namespace

      CR:

      $ oc create -f bare-metal-events-namespace.yaml

2. Create an Operator group for the Bare Metal Event Relay Operator.

   1. Save the following YAML in the

      bare-metal-events-operatorgroup.yaml

      file:

      apiVersion: operators.coreos.com/v1
      kind: OperatorGroup
      metadata:
        name: bare-metal-event-relay-group
        namespace: openshift-bare-metal-events
      spec:
        targetNamespaces:
        - openshift-bare-metal-events

   2. Create the

      OperatorGroup

      CR:

      $ oc create -f bare-metal-events-operatorgroup.yaml

3. Subscribe to the Bare Metal Event Relay.

   1. Save the following YAML in the

      bare-metal-events-sub.yaml

      file:

      apiVersion: operators.coreos.com/v1alpha1
      kind: Subscription
      metadata:
        name: bare-metal-event-relay-subscription
        namespace: openshift-bare-metal-events
      spec:
        channel: "stable"
        name: bare-metal-event-relay
        source: redhat-operators
        sourceNamespace: openshift-marketplace

   2. Create the

      Subscription

      CR:

      $ oc create -f bare-metal-events-sub.yaml

Procedure

1. Install the Bare Metal Event Relay using the OpenShift Container Platform web console:

   1. In the OpenShift Container Platform web console, click Operators OperatorHub.
   2. Choose Bare Metal Event Relay from the list of available Operators, and then click Install.
   3. On the Install Operator page, select or create a Namespace, select openshift-bare-metal-events, and then click Install.

1. Switch to the Operators Installed Operators page.

2. Ensure that Bare Metal Event Relay is listed in the project with a Status of InstallSucceeded.

   Note

   During installation an Operator might display a Failed status. If the installation later succeeds with an InstallSucceeded message, you can ignore the Failed message.

Procedure

1. Confirm that the node hardware has the Redfish

   EventService

   enabled by running the following

   curl

   command:

   curl https://<bmc_ip_address>/redfish/v1/EventService --insecure -H 'Content-Type: application/json' -u "<bmc_username>:<password>"

   where:

   bmc_ip_address

   is the IP address of the BMC where the Redfish events are generated.

   Example output

   {
      "@odata.context": "/redfish/v1/$metadata#EventService.EventService",
      "@odata.id": "/redfish/v1/EventService",
      "@odata.type": "#EventService.v1_0_2.EventService",
      "Actions": {
         "#EventService.SubmitTestEvent": {
            "EventType@Redfish.AllowableValues": ["StatusChange", "ResourceUpdated", "ResourceAdded", "ResourceRemoved", "Alert"],
            "target": "/redfish/v1/EventService/Actions/EventService.SubmitTestEvent"
         }
      },
      "DeliveryRetryAttempts": 3,
      "DeliveryRetryIntervalSeconds": 30,
      "Description": "Event Service represents the properties for the service",
      "EventTypesForSubscription": ["StatusChange", "ResourceUpdated", "ResourceAdded", "ResourceRemoved", "Alert"],
      "EventTypesForSubscription@odata.count": 5,
      "Id": "EventService",
      "Name": "Event Service",
      "ServiceEnabled": true,
      "Status": {
         "Health": "OK",
         "HealthRollup": "OK",
         "State": "Enabled"
      },
      "Subscriptions": {
         "@odata.id": "/redfish/v1/EventService/Subscriptions"
      }
   }

2. Get the Bare Metal Event Relay service route for the cluster by running the following command:

   $ oc get route -n openshift-bare-metal-events

   Example output

   NAME             HOST/PORT                                                                                           PATH   SERVICES                 PORT   TERMINATION   WILDCARD
   hw-event-proxy   hw-event-proxy-openshift-bare-metal-events.apps.compute-1.example.com          hw-event-proxy-service   9087   edge          None

3. Create a

   BMCEventSubscription

   resource to subscribe to the Redfish events:

   1. Save the following YAML in the

      bmc_sub.yaml

      file:

      apiVersion: metal3.io/v1alpha1
      kind: BMCEventSubscription
      metadata:
        name: sub-01
        namespace: openshift-machine-api
      spec:
         hostName: <hostname> 

      1

      
         destination: <proxy_service_url> 

      2

      
         context: ''

      1

      Specifies the name or UUID of the worker node where the Redfish events are generated.

      2

      Specifies the bare-metal event proxy service, for example, https://hw-event-proxy-openshift-bare-metal-events.apps.compute-1.example.com/webhook.

   2. Create the

      BMCEventSubscription

      CR:

      $ oc create -f bmc_sub.yaml

4. Optional: To delete the BMC event subscription, run the following command:

   $ oc delete -f bmc_sub.yaml

5. Optional: To manually create a Redfish event subscription without creating a

   BMCEventSubscription

   CR, run the following

   curl

   command, specifying the BMC username and password.

   $ curl -i -k -X POST -H "Content-Type: application/json"  -d '{"Destination": "https://<proxy_service_url>", "Protocol" : "Redfish", "EventTypes": ["Alert"], "Context": "root"}' -u <bmc_username>:<password> 'https://<bmc_ip_address>/redfish/v1/EventService/Subscriptions' –v

   where:

   proxy_service_url

   is the bare-metal event proxy service, for example, https://hw-event-proxy-openshift-bare-metal-events.apps.compute-1.example.com/webhook.

   bmc_ip_address

   is the IP address of the BMC where the Redfish events are generated.

   Example output

   HTTP/1.1 201 Created
   Server: AMI MegaRAC Redfish Service
   Location: /redfish/v1/EventService/Subscriptions/1
   Allow: GET, POST
   Access-Control-Allow-Origin: *
   Access-Control-Expose-Headers: X-Auth-Token
   Access-Control-Allow-Headers: X-Auth-Token
   Access-Control-Allow-Credentials: true
   Cache-Control: no-cache, must-revalidate
   Link: <http://redfish.dmtf.org/schemas/v1/EventDestination.v1_6_0.json>; rel=describedby
   Link: <http://redfish.dmtf.org/schemas/v1/EventDestination.v1_6_0.json>
   Link: </redfish/v1/EventService/Subscriptions>; path=
   ETag: "1651135676"
   Content-Type: application/json; charset=UTF-8
   OData-Version: 4.0
   Content-Length: 614
   Date: Thu, 28 Apr 2022 08:47:57 GMT

Procedure

1. Check the current subscriptions for the BMC by running the following

   curl

   command:

   $ curl --globoff -H "Content-Type: application/json" -k -X GET --user <bmc_username>:<password> https://<bmc_ip_address>/redfish/v1/EventService/Subscriptions

   where:

   bmc_ip_address

   is the IP address of the BMC where the Redfish events are generated.

   Example output

   % Total % Received % Xferd Average Speed Time Time Time Current
   Dload Upload Total Spent Left Speed
   100 435 100 435 0 0 399 0 0:00:01 0:00:01 --:--:-- 399
   {
     "@odata.context": "/redfish/v1/$metadata#EventDestinationCollection.EventDestinationCollection",
     "@odata.etag": ""
     1651137375 "",
     "@odata.id": "/redfish/v1/EventService/Subscriptions",
     "@odata.type": "#EventDestinationCollection.EventDestinationCollection",
     "Description": "Collection for Event Subscriptions",
     "Members": [
     {
       "@odata.id": "/redfish/v1/EventService/Subscriptions/1"
     }],
     "Members@odata.count": 1,
     "Name": "Event Subscriptions Collection"
   }

   In this example, a single subscription is configured:

   /redfish/v1/EventService/Subscriptions/1

   .

2. Optional: To remove the

   /redfish/v1/EventService/Subscriptions/1

   subscription with

   curl

   , run the following command, specifying the BMC username and password:

   $ curl --globoff -L -w "%{http_code} %{url_effective}\n" -k -u <bmc_username>:<password >-H "Content-Type: application/json" -d '{}' -X DELETE https://<bmc_ip_address>/redfish/v1/EventService/Subscriptions/1

   where:

   bmc_ip_address

   is the IP address of the BMC where the Redfish events are generated.

Procedure

1. Create the

   HardwareEvent

   custom resource (CR):

   1. Save the following YAML in the

      hw-event.yaml

      file:

      apiVersion: "event.redhat-cne.org/v1alpha1"
      kind: "HardwareEvent"
      metadata:
        name: "hardware-event"
      spec:
        nodeSelector:
          node-role.kubernetes.io/hw-event: "" 

      1

      
        transportHost: "amqp://amq-router-service-name.amq-namespace.svc.cluster.local" 

      2

      
        logLevel: "debug" 

      3

      
        msgParserTimeout: "10" 

      4

      1

      Required. Use the nodeSelector field to target nodes with the specified label, for example, node-role.kubernetes.io/hw-event: "".

      2

      Required. AMQP host that delivers the events at the transport layer using the AMQP protocol.

      3

      Optional. The default value is debug. Sets the log level in hw-event-proxy logs. The following log levels are available: fatal, error, warning, info, debug, trace.

      4

      Optional. Sets the timeout value in milliseconds for the Message Parser. If a message parsing request is not responded to within the timeout duration, the original hardware event message is passed to the cloud native event framework. The default value is 10.

   2. Create the

      HardwareEvent

      CR:

      $ oc create -f hardware-event.yaml

2. Create a BMC username and password

   Secret

   CR that enables the hardware events proxy to access the Redfish message registry for the bare-metal host.

   1. Save the following YAML in the

      hw-event-bmc-secret.yaml

      file:

      apiVersion: v1
      kind: Secret
      metadata:
        name: redfish-basic-auth
      type: Opaque
      stringData: 

      1

      
        username: <bmc_username>
        password: <bmc_password>
        # BMC host DNS or IP address
        hostaddr: <bmc_host_ip_address>

      1

      Enter plain text values for the various items under stringData.

   2. Create the

      Secret

      CR:

      $ oc create -f hw-event-bmc-secret.yaml