Red Hat Summit Red Hat SummitSupportConsoleDéveloppeursCommencer un essaiContact

Ce contenu n'est pas disponible dans la langue sélectionnée.

Chapter 12. Monitoring Tools Configuration

Monitoring tools are an optional suite of tools designed to help operators maintain an OpenStack environment. The tools perform the following functions:

  • Centralized logging: Allows you gather logs from all components in the OpenStack environment in one central location. You can identify problems across all nodes and services, and optionally, export the log data to Red Hat for assistance in diagnosing problems.
  • Availability monitoring: Allows you to monitor all components in the OpenStack environment and determine if any components are currently experiencing outages or are otherwise not functional. You can also receive notifications when problems occur to optimize your response time.

12.1. Architecture
Copier lien

Monitoring tools use a client-server model with the client deployed onto the Red Hat OpenStack Platform overcloud nodes. The Fluentd service provides client-side centralized logging (CL) and the Sensu client service provides client-side availability monitoring (AM).

12.1.1. Centralized Logging
Copier lien

Centralized logging allows you to have one central place to view logs across your entire OpenStack environment. These logs come from the operating system, such as syslog and audit log files, infrastructure components such as RabbitMQ and MariaDB, and OpenStack services such as Identity, Compute, and others.

The centralized logging toolchain consists of a number of components, including:

  • Log Collection Agent (Fluentd)
  • Log Relay/Transformer (Fluentd)
  • Data Store (Elasticsearch)
  • API/Presentation Layer (Kibana)
Note

The director does not deploy the server-side components for centralized logging. Red Hat does not support server-side components including the Elasticsearch database, Kibana, and Fluentd with plugins running as a log aggregator.

The centralized logging components and their interactions are laid out in the following diagrams:

Note

Items shown in blue denote Red Hat-supported components.

Figure 12.1. Centralized logging architecture at a high level

centralised logging arch
View larger image
centralised logging arch

Figure 12.2. Single-node deployment for Red Hat OpenStack Platform

centralised logging single node fluentd
View larger image
centralised logging single node fluentd

Figure 12.3. HA deployment for Red Hat OpenStack Platform

centralised logging ha fluentd
View larger image
centralised logging ha fluentd

12.1.2. Availability Monitoring
Copier lien

Availability monitoring allows you to have one central place to monitor the high-level functionality of all components across your entire OpenStack environment.

The availability monitoring toolchain consists of a number of components, including:

  • Monitoring Agent (Sensu client)
  • Monitoring Relay/Proxy (RabbitMQ)
  • Monitoring Controller/Server (Sensu server)
  • API/Presentation Layer (Uchiwa)
Note

The director does not deploy the server-side components for availability monitoring. Red Hat does not support server-side components including Uchiwa, Sensu Server, the Sensu API plus RabbitMQ, and a Redis instance running on a monitoring node.

The availability monitoring components and their interactions are laid out in the following diagrams:

Note

Items shown in blue denote Red Hat-supported components.

Figure 12.4. Availability monitoring architecture at a high level

availability monitoring arch
View larger image
availability monitoring arch

Figure 12.5. Single-node deployment for Red Hat OpenStack Platform

availability monitoring single node sensu
View larger image
availability monitoring single node sensu

Figure 12.6. HA deployment for Red Hat OpenStack Platform

availability monitoring ha sensu
View larger image
availability monitoring ha sensu

12.2. Install the Client-Side Tools
Copier lien

Prior to overcloud deployment, you need to determine the configuration settings to apply to each client. Copy the example environment files from the director’s Heat template collection and modify them to suit your environment.

12.2.1. Set Centralized Logging Client Parameters
Copier lien

For Fluentd configuration settings, copy /usr/share/openstack-tripleo-heat-templates/environments/logging-environment.yaml and modify the file to suit your environment. For example:

Simple configuration

resource_registry:
  OS::TripleO::Services::FluentdClient: ../puppet/services/logging/fluentd-client.yaml

parameter_defaults:
  LoggingServers:
    - host: log0.example.com
      port: 24224
    - host: log1.example.com
      port: 24224
Copy to Clipboard Toggle word wrap

Example SSL configuration

## (note the use of port 24284 for ssl connections)

resource_registry:
  OS::TripleO::Services::FluentdClient: ../puppet/services/logging/fluentd-client.yaml

parameter_defaults:
  LoggingServers:
    - host: 192.0.2.11
      port: 24284
  LoggingUsesSSL: true
  LoggingSharedKey: secret
  LoggingSSLCertificate: |
    -----BEGIN CERTIFICATE-----
    ...certificate data here...
    -----END CERTIFICATE-----
Copy to Clipboard Toggle word wrap

  • LoggingServers - The destination system that will receive Fluentd log messages.
  • LoggingUsesSSL - Setting that determines whether secure_forward is used when forwarding log messages.
  • LoggingSharedKey - The shared secret used by secure_forward.
  • LoggingSSLCertificate - The PEM-encoded contents of the SSL CA certificate.

12.2.2. Set Availability Monitoring Client Parameters
Copier lien

For the Sensu client configuration settings, copy /usr/share/openstack-tripleo-heat-templates/environments/monitoring-environment.yaml and modify the file to suit your environment. For example: 

resource_registry:
  OS::TripleO::Services::SensuClient: ../puppet/services/monitoring/sensu-client.yaml

parameter_defaults:
  MonitoringRabbitHost: 10.10.10.10
  MonitoringRabbitPort: 5672
  MonitoringRabbitUserName: sensu
  MonitoringRabbitPassword: sensu
  MonitoringRabbitUseSSL: false
  MonitoringRabbitVhost: "/sensu"
  SensuClientCustomConfig:
    api:
      warning: 10
      critical: 20
Copy to Clipboard Toggle word wrap
  • MonitoringRabbit* - These parameters connect the Sensu client services to the RabbitMQ instance that runs on the monitoring server.
  • MonitoringRabbitUseSSL - This parameter is not currently available for availability monitoring.
  • SensuClientCustomConfig - Specify additional Sensu client configuration. Defines the OpenStack credentials to be used, including username/password, auth_url, tenant, and region.

12.2.3. Install Operational Tools on Overcloud Nodes
Copier lien

Include the modified YAML files with your openstack overcloud deploy command to install the Sensu client and Fluentd tools on all overcloud nodes. For example: 

$ openstack overcloud deploy --templates  -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e network-environment.yaml -e ~/templates/monitoring-environment.yaml -e ~/templates/logging-environment.yaml --control-scale 3 --compute-scale 1 --ntp-server 192.168.122.10
Copy to Clipboard Toggle word wrap

12.3. Install the Server-Side Components
Copier lien

Note

Red Hat does not support the server-side components and their deployment process.

You can use the opstools-ansible playbook to install the server-side components onto Red Hat Enterprise Linux 7. These server-side components include availability monitoring and centralized logging services that complement the Red Hat-supported client-side components. The most tested opstools-ansible scenario is the deployment of server-side components onto CentOS 7. Detailed instructions can be found in the README.md: https://github.com/centos-opstools/opstools-ansible

12.4. Monitor the OpenStack Platform
Copier lien

See the Sensu documentation for further details about the Sensu stack infrastructure: https://sensuapp.org/docs/latest/overview/architecture.html

Red Hat supplies a set of check scripts in the osops-tools-monitoring-oschecks package. The majority of the check scripts only check the API connection to the OpenStack component. However, certain scripts also perform additional OpenStack resource tests for OpenStack Compute (nova), OpenStack Block Storage (cinder), OpenStack Image (glance), and OpenStack Networking (neutron). For example, the OpenStack Identity (keystone) API check gives the following result when keystone is running: 

OK: Got a token, Keystone API is working.
Copy to Clipboard Toggle word wrap

12.5. Validate the Sensu Client Installation
Copier lien

  1. Check the status of the sensu-client on each overcloud node: 

    # systemctl status sensu-client
    Copy to Clipboard Toggle word wrap
  2. Review the error log for any issues: /var/log/sensu/sensu-client.log
  3. Verify that each overcloud node has the /etc/sensu/conf.d/rabbitmq.json file that sets the IP address of the monitoring server.

12.6. Review the State of a Node
Copier lien

If you have a deployment of the Uchiwa dashboard, you can use it with the Sensu server to review the state of your nodes:

  1. Login to the Uchiwa dashboard and click the Data Center tab to confirm that the Data Center is operational. 

    http://<SERVER_IP_ADDRESS>/uchiwa
    Copy to Clipboard Toggle word wrap
  2. Check that all overcloud nodes are in a Connected state.
  3. At a suitable time, reboot one of the overcloud nodes and review the rebooted node’s status in the Uchiwa dashboard. After the reboot completes, verify that the node successfully re-connects to the Sensu server and starts executing checks.

12.7. Review the State of an OpenStack Service
Copier lien

This example tests the monitoring of the openstack-ceilometer-central service.

  1. Confirm that the openstack-ceilometer-central service is running: 

    systemctl status openstack-ceilometer-central.service
    Copy to Clipboard Toggle word wrap
  2. Connect to the Uchiwa dashboard and confirm that a successful ceilometer check is present and running as defined in the ceilometer JSON file.

  3. Stop the openstack-ceilometer-central service.

    Note

    This may disrupt services, so run this test at an appropriate time. 

    systemctl stop openstack-ceilometer-central.service
    Copy to Clipboard Toggle word wrap
  4. Log in to the Uchiwa dashboard and confirm that the failed ceilometer check has been reported.

  5. Start the openstack-ceilometer-central service: 

    systemctl start openstack-ceilometer-central.service
    Copy to Clipboard Toggle word wrap
  6. Log in to the Uchiwa dashboard and view the time interval between the ceilometer check reports to confirm that the check runs in the time interval defined in the ceilometer JSON file.
Retour au début
Red Hat logoGithubredditYoutubeTwitter

Apprendre

Essayez, achetez et vendez

Communautés

À propos de la documentation Red Hat

Nous aidons les utilisateurs de Red Hat à innover et à atteindre leurs objectifs grâce à nos produits et services avec un contenu auquel ils peuvent faire confiance. Découvrez nos récentes mises à jour.

Rendre l’open source plus inclusif

Red Hat s'engage à remplacer le langage problématique dans notre code, notre documentation et nos propriétés Web. Pour plus de détails, consultez le Blog Red Hat.

À propos de Red Hat

Nous proposons des solutions renforcées qui facilitent le travail des entreprises sur plusieurs plates-formes et environnements, du centre de données central à la périphérie du réseau.

Theme

© 2025 Red Hat