SupportConsoleDevelopersStart a trialContact

Chapter 4. Managing operational measurements

4.1. Modifying the environment variables based on your deployment

Procedure

  1. Copy the /usr/share/openstack-tripleo-heat-templates/environments/gnocchi-environment.yaml file to your home directory.

  2. Modify the parameters to suit your environment. You can modify the following primary parameters in the YAML file:

  3. Add gnocchi-environment.yaml to the overcloud deploy command with any other environment files that are relevant to your environment and deploy. Replace <existing_overcloud_environment_files> with the list of environment files that are part of your existing deployment: 

    $ openstack overcloud deploy \
<existing_overcloud_environment_files> \
-e ~gnocchi-environment.yaml \
...

4.1.1. Running metricd workers

By default, the gnocchi-metricd daemon spans your CPU power to maximize CPU use when computing metric aggregation.

Procedure

  • Use the openstack metric status command to query the HTTP API and retrieve the status for metric processing: 

    # openstack metric status

    The command output shows you the processing backlog for the gnocchi-metrid daemon. As long as this backlog is not continuously increasing, it means that gnocchi-metricd can cope with the amount of metrics that are being gathered. If the number of measures to process is continuously increasing, increase the number of gnocchi-metricd daemons. You can run any number of metricd daemons on any number of servers.

4.1.2. Number of sacks

Incoming metrics data in Gnocchi are pushed into different sacks and each sack is assigned to one or more gnocchi-metricd daemons for processing. The number of sacks depends on the active metrics the system captures.

Red Hat recommends that the number of sacks is greater than the total number of active gnocchi-metricd workers.

4.1.3. Changing the sack size

You can change the sack size if you want to collect more metrics than originally anticipated.

Measurement data that is pushed to Gnocchi is divided into sacks for better distribution. Incoming metrics are pushed to specific sacks and each sack is assigned to one or more gnocchi-metricd daemons for processing. To set the number of sacks, use the number of active metrics the system captures. The number of sacks should be higher than the total number of active gnocchi-metricd workers.

Procedure

  • To determine the appropriate sacks value to set, use the following equation:

    sacks value = number of active metrics / 300

    Note

    If the estimated number of metrics is the absolute maximum, divide the value by 500. If the estimated number of active metrics is conservative and expected to grow, divide the value by 100 to accommodate growth.

4.2. Monitoring the time-series database service

The /v1/status endpoint of the HTTP API returns information, such as the number of measures to process (measures backlog), which you can monitor. The following conditions indicate a healthy system:

  • The HTTP server and gnocchi-metricd are running
  • The HTTP server and gnocchi-metricd are not writing error messages to the log files.

Procedure

  • View the status of the time-series database: 

    # openstack metric status

    The output shows the status of the time-series database, as well as the number of metrics to process. Lower values are better here, ideally, they are near 0.

4.3. Backing up and restoring the time-series database

To be able to recover from an unfortunate event, backup the index and the storage. You must create a database dump using PostgreSQL or MySQL, and take snapshots or copies of your data storage, using Ceph, Swift, or your file system.

Procedure

  1. Restore your index and storage backups.
  2. Reinstall Gnocchi, if necessary.
  3. Restart Gnocchi.

4.4. Viewing measures

You can view a list of measures for a particular resource:

Procedure

  1. Use the metric measures command: 

    # openstack metric measures show --resource-id UUID <METER_NAME>

  2. List measures for a particular resource within a range of timestamps: 

    # openstack metric measures show --aggregation mean --start <START_TIME> --stop <STOP_TIME> --resource-id UUID <METER_NAME>

    Timestamp variables <START_TIME> and <END_TIME> use the format iso-dateThh:mm:ss.

4.5. Managing resource types

You can create, view, and delete resource types. The default resource type is generic, but you can create your own resource type with any extra number of attributes.

Procedure

  1. Create a new resource type: 

    $ openstack metric resource-type create testResource01 -a bla:string:True:min_length=123
----------------------------------------------------------------------------+
| Field          | Value                                                      |
----------------------------------------------------------------------------+
| attributes/bla | max_length=255, min_length=123, required=True, type=string |
| name           | testResource01                                             |
| state          | active                                                     |
----------------------------------------------------------------------------+

  2. Review the configuration of the resource type: 

    $ openstack metric resource-typegnocchi resource-type show testResource01
----------------------------------------------------------------------------+
| Field          | Value                                                      |
----------------------------------------------------------------------------+
| attributes/bla | max_length=255, min_length=123, required=True, type=string |
| name           | testResource01                                             |
| state          | active                                                     |
----------------------------------------------------------------------------+

  3. Delete the resource type: 

    $ openstack metric resource-type delete testResource01
Note

You cannot delete a resource type if a resource is using it.

4.6. Viewing cloud usage measures

Procedure

  • View the average memory usage of all instances for each project: 

    openstack metrics measures aggregation --resource-type instance --groupby project_id -m “memoryView L3” --resource-id UUID

4.6.1. Enabling L3 cache monitoring

If your Intel hardware and libvirt version supports Cache Monitoring Technology (CMT), you can use the cpu_l3_cache meter to monitor the amount of L3 cache used by an instance.

To monitor the L3 cache, you must have the following parameters and files:

  • cmt in the LibvirtEnabledPerfEvents parameter.
  • cpu_l3_cache in the gnocchi_resources.yaml file.
  • cpu_l3_cache in the Ceilometer polling.yaml file.

Procedure

  1. Create a YAML file for Telemetry, for example, ceilometer-environment.yaml.
  2. In the ceilometer-environment.yaml file, add cmt to the LibvirtEnabledPerfEvents parameter. For more information, see /usr/share/openstack-triple-heat-templates/puppet/services/nova_libvirt.yaml.

  3. Deploy the overcloud with this YAML file. Replace <existing_overcloud_environment_files> with the list of environment files that are part of your existing deployment: 

     #!/bin/bash

  openstack overcloud deploy \
  --templates \
  <existing_overcloud_environment_files> \
  -e  /home/stack/ceilometer-environment.yaml \
  ...

  4. Verify that cpu_l3_cache is enabled in Gnocchi on the Compute node. 

    $ sudo -i
# docker exec -ti ceilometer_agent_compute cat /etc/ceilometer/gnocchi_resources.yaml | grep cpu_l3_cache
//Verify that cpu_l3_cache is enabled for Telemetry polling.
# docker exec -ti ceilometer_agent_compute cat /etc/ceilometer/polling.yaml  | grep cpu_l3_cache
//If cpu_l3_cache is not enabled for Telemetry, enable it and restart the service.
# docker exec -ti ceilometer_agent_compute echo "        - cpu_l3_cache" >> /etc/ceilometer/polling.yaml
# docker exec -ti ceilometer_agent_compute pkill -HUP -f "ceilometer.*master process"
    Note

    Changing the setting in the container image does not persist over a reboot.

  5. After you launch a guest instance on this Compute node, monitor the CMT metrics: 

    (overcloud) [stack@undercloud-0 ~]$ openstack metric measures show --resource-id a6491d92-b2c8-4f6d-94ba-edc9dfde23ac cpu_l3_cache
---------------------------------------------------
| timestamp                 | granularity |     value |
---------------------------------------------------
| 2017-10-25T09:40:00+00:00 |       300.0 | 1966080.0 |
| 2017-10-25T09:45:00+00:00 |       300.0 | 1933312.0 |
| 2017-10-25T09:50:00+00:00 |       300.0 | 2129920.0 |
| 2017-10-25T09:55:00+00:00 |       300.0 | 1966080.0 |
| 2017-10-25T10:00:00+00:00 |       300.0 | 1933312.0 |
| 2017-10-25T10:05:00+00:00 |       300.0 | 2195456.0 |
| 2017-10-25T10:10:00+00:00 |       300.0 | 1933312.0 |
| 2017-10-25T10:15:00+00:00 |       300.0 | 1998848.0 |
| 2017-10-25T10:20:00+00:00 |       300.0 | 2097152.0 |
| 2017-10-25T10:25:00+00:00 |       300.0 | 1933312.0 |
| 2017-10-25T10:30:00+00:00 |       300.0 | 1966080.0 |
| 2017-10-25T10:35:00+00:00 |       300.0 | 1933312.0 |
| 2017-10-25T10:40:00+00:00 |       300.0 | 1933312.0 |
| 2017-10-25T10:45:00+00:00 |       300.0 | 1933312.0 |
| 2017-10-25T10:50:00+00:00 |       300.0 | 2850816.0 |
| 2017-10-25T10:55:00+00:00 |       300.0 | 2359296.0 |
| 2017-10-25T11:00:00+00:00 |       300.0 | 2293760.0 |
---------------------------------------------------

4.7. Upgrading Gnocchi

By default, upgrading your deployment using Red Hat OpenStack Platform director upgrades Gnocchi. For information on upgrading your deployment, see Upgrading Red Hat OpenStack Platform. If you are using Red Hat OpenStack Platform 10 and want to upgrade to Red Hat OpenStack Platform 13, see Fast Forward Upgrades.

Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.