Search
SupportConsoleDevelopersStart a trialContact

Chapter 10. Configuring Quality of Service (QoS) policies

download PDF

You can offer varying service levels for instances by using quality of service (QoS) policies to apply rate limits to egress and ingress traffic.

You can apply QoS policies to individual ports. You can also apply QoS policies to a project network, where ports with no specific policy attached inherit the policy.

Note

Internal network owned ports, such as DHCP and internal router ports, are excluded from network policy application.

You can apply, modify, or remove QoS policies dynamically. However, for guaranteed minimum bandwidth QoS policies, you can only apply modifications when there are no instances that use any of the ports the policy is assigned to.

10.1. QoS rules

The following rule types define the limits of a particular quality of service (QoS) policy:

  • bandwidth_limit: Provides bandwidth limitations on networks, ports or floating IPs. If implemented, any traffic that exceeds the specified rate is dropped.
  • minimum_bandwidth: Provides minimum bandwidth constraints on certain types of traffic. If implemented, best efforts are made to provide no less than the specified bandwidth to each port on which the rule is applied.
  • dscp_marking: Marks network traffic with a Differentiated Services Code Point (DSCP) value.

Related procedures

The following procedures describe how to create a QoS policy with each rule type:

10.2. Creating and applying a QoS policy and rule

To create a quality of service (QoS) policy and rule and apply the policy to a port, complete the following steps:

Procedure

  1. Identify the ID of the project you want to create the QoS policy for: 

    (overcloud) $ openstack project list
    -------------------------------------------+
| ID                               | Name    |
-------------------------------------------+
| 8c409e909fb34d69bc896ab358317d60 | admin   |
| 92b6c16c7c7244378a062be0bfd55fa0 | service |
-------------------------------------------+

  2. Create the new QoS policy: 

    (overcloud) $ openstack network qos policy create --share --project <project_ID> <policy_name>

  3. Create new rules for the QoS policy: 

    (overcloud) $ openstack network qos rule create --type <rule-type> [rule properties] <policy_name>
    Table 10.1. Rule properties
    PropertyDescription

    max_kbps

    The maximum rate (in Kbps) that the instance can send.

    max_burst_kbps

    The maximum amount of data (in kbits) that the port can send in an instant if the token buffer is full. The token buffer replenishes at a "max_kbps" rate.

    The burst value for TCP traffic can be set as 80% of desired bandwidth limit value. For example, if the bandwidth limit is set to 1000kbps then a burst value of 800kbps is sufficient.

    Note
    • If the burst value is set too low, bandwidth usage is throttled even with a proper bandwidth limit setting, resulting in a lower than expected bandwidth.
    • If the configured burst value is too high, too few packets could be limited, resulting in a higher than expected bandwidth limit.

    min-kbps

    The minimum bandwidth (in Kbps) guaranteed to an instance.

    ingress/egress

    The direction of traffic the rule is applied to. From the perspective of a cloud server, ingress indicates download, and egress indicates upload.

    dscp-mark

    Specifies the decimal value of a DSCP mark.

  4. Configure a port or network to apply the policy to. You can update an existing port or network, or create a new port or network to apply the policy to:

    • Apply the policy to an existing port: 

      (overcloud) $ openstack port set --qos-policy <policy_name> <port_name|port_ID>

    • Create a new port: 

      (overcloud) $ openstack port create --qos-policy <policy_name> --network <network_name|network_ID> <port_name|port_ID>

    • Apply the policy to an existing network: 

      (overcloud) $ openstack network set --qos-policy <policy_name> <network_name|network_ID>

    • Create a new network: 

      (overcloud) $ openstack network create --qos-policy <policy_name> <network_name>

10.2.1. Creating and applying a bandwidth limit QoS policy and rule

You can create a QoS policy that limits the bandwidth on networks, ports or floating IPs, and drops any traffic that exceeds the specified rate. To create and apply a bandwidth limit QoS policy and rule, complete the following steps:

Procedure

  1. If the qos extension is not already enabled for OpenStack Networking in /etc/neutron/plugins/ml2/<agent_name>_agent.ini, then perform the following steps:

    1. Create a custom environment file, and add the following configuration: 

      parameter_defaults:
  NeutronSriovAgentExtensions: 'qos'

    2. To apply this configuration, deploy the overcloud, adding your custom environment file to the stack along with your other environment files: 

      (undercloud) $ openstack overcloud deploy --templates \
  -e [your environment files]
  -e /home/stack/templates/<custom-environment-file>.yaml

      For more information, see "Modifying the Overcloud Environment" in the Director Installation and Usage guide.

  2. Identify the ID of the project you want to create the QoS policy for: 

    (overcloud) $ openstack project list
+----------------------------------+----------+
| ID                               | Name     |
+----------------------------------+----------+
| 4b0b98f8c6c040f38ba4f7146e8680f5 | auditors |
| 519e6344f82e4c079c8e2eabb690023b | services |
| 80bf5732752a41128e612fe615c886c6 | demo     |
| 98a2f53c20ce4d50a40dac4a38016c69 | admin    |
+----------------------------------+----------+

  3. Create a QoS policy named "bw-limiter" in the admin project: 

    (overcloud) $ openstack network qos policy create --share --project 98a2f53c20ce4d50a40dac4a38016c69 bw-limiter

  4. Configure the rules for the "bw-limiter" policy: 

    (overcloud) $ openstack network qos rule create --type bandwidth-limit --max_kbps 3000 --max_burst_kbps 300 bw-limiter

  5. Configure a port to apply the "bw-limiter" policy to: 

    (overcloud) $ openstack port set --qos-policy bw-limiter <port_name|port_ID>

10.2.2. Creating and applying a guaranteed minimum bandwidth QoS policy and rule

You can request bandwidth guarantees for ports on networks with segmentation_type=flat or segmentation_type=vlan, which are backed by a physical network (physnet).

Note
  • Do not mix ports with and without bandwidth guarantees on the same physical interface, as the ports without a guarantee may starve. Create host aggregates to separate ports with bandwidth guarantees from those without.
  • You can only modify a guaranteed minimum bandwidth QoS policy when there are no instances using any of the ports the policy is assigned to.

Supported drivers and agents¶

  • SR-IOV (sriovnicswitch) vnic_types: direct, macvtap

  • ML2/OVS (openvswitch) vnic_types: normal, direct

    Note

    ML2/OVN does not support minimum bandwidth.

Prerequisites

  • The Placement service must support microversion 1.29.
  • The Compute (nova) service must support microversion 2.72.

  • The Networking (neutron) service must support the following API extensions:

    • agent-resources-synced
    • port-resource-request
    • qos-bw-minimum-ingress
  • To query placement information using the OpenStack CLI, install the Placement service package, python3-osc-placement, on the undercloud.

Procedure

  1. If the Placement service plugin is not already configured for Openstack networking, then perform the following steps:

    1. If the NeutronServicePlugins is already provided in a custom environment file, update the parameter to include "placement", otherwise create a custom environment file and add the following configuration: 

      parameter_defaults:
	NeutronServicePlugins: 'router,qos,segments,trunk,placement'

    2. To apply this configuration, deploy the overcloud, adding your custom environment file to the stack along with your other environment files: 

      (undercloud) $ openstack overcloud deploy --templates \
  -e [your environment files]
  -e /home/stack/templates/network-environment.yaml

      For more information, see "Modifying the Overcloud Environment" in the Director Installation and Usage guide.

  2. (Optional) To black list vnic_types when multiple ML2 mechanism drivers support them by default and multiple agents are being tracked in Placement, add vnic_type_blacklist to /etc/neutron/plugins/ml2/ml2_conf.ini and restart the agent: 

    [ovs_driver]
vnic_type_blacklist = direct
[sriov_driver]
#vnic_type_blacklist = direct

  3. Configure the resource provider ingress and egress bandwidths for the relevant agents on each Compute node that needs to provide a minimum bandwidth. You can configure only ingress or egress, or both, using the following formats:

    • Configure only egress bandwidth, in kbps: 

      resource_provider_bandwidths = <bridge0>:<egress_kbps>:,<bridge1>:<egress_kbps>:,...,<bridgeN>:<egress_kbps>:

    • Configure only ingress bandwidth, in kbps: 

      resource_provider_bandwidths = <bridge0>::<ingress_kbps>,<bridge1>::<ingress_kbps>,...,<bridgeN>::<ingress_kbps>

    • Configure both egress and ingress bandwidth, in kbps: 

      resource_provider_bandwidths = <bridge0>:<egress_kbps>:<ingress_kbps>,<bridge1>:<egress_kbps>:<ingress_kbps>,...,<bridgeN>:<egress_kbps>:<ingress_kbps>

      For example:

      • To configure the resource provider ingress and egress bandwidths for the OVS agent, add resource_provider_bandwidths to /etc/neutron/plugins/ml2/openvswitch_agent.ini: 

        [ovs]
bridge_mappings = physnet0:br-physnet0
resource_provider_bandwidths = br-physnet0:10000000:10000000

      • To configure the resource provider ingress and egress bandwidths for the SRIOV agent, add resource_provider_bandwidths to /etc/neutron/plugins/ml2/sriov_agent.ini: 

        [sriov_nic]
physical_device_mappings = physnet0:ens5,physnet0:ens6
resource_provider_bandwidths = ens5:40000000:40000000,ens6:40000000:40000000

    To implement the resource provider bandwidths, restart the configured agents.

  4. Identify the ID of the project you want to create the QoS policy for: 

    (overcloud) $ openstack project list
+----------------------------------+----------+
| ID                               | Name     |
+----------------------------------+----------+
| 4b0b98f8c6c040f38ba4f7146e8680f5 | auditors |
| 519e6344f82e4c079c8e2eabb690023b | services |
| 80bf5732752a41128e612fe615c886c6 | demo     |
| 98a2f53c20ce4d50a40dac4a38016c69 | admin    |
+----------------------------------+----------+

  5. Create a QoS policy named "guaranteed_min_bw" in the admin project: 

    (overcloud) $ openstack network qos policy create --share --project 98a2f53c20ce4d50a40dac4a38016c69 guaranteed_min_bw

  6. Configure the rules for the "guaranteed_min_bw" policy: 

    (overcloud) $ openstack network qos rule create --type minimum-bandwidth --min-kbps 40000000 --ingress guaranteed_min_bw
(overcloud) $ openstack network qos rule create --type minimum-bandwidth --min-kbps 40000000 --egress guaranteed_min_bw

  7. Configure a port to apply the "guaranteed_min_bw" policy to: 

    (overcloud) $ openstack port set --qos-policy guaranteed_min_bw <port_name|port_ID>

Verification

  1. List all of the available resource providers: 

    (undercloud) $ openstack --os-placement-api-version 1.17 resource provider list

    Example output: 

    -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| uuid                                 | name                                                | generation | root_provider_uuid                   | parent_provider_uuid                 |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 31d3d88b-bc3a-41cd-9dc0-fda54028a882 | dell-r730-014.localdomain                           |         28 | 31d3d88b-bc3a-41cd-9dc0-fda54028a882 | None                                 |
| 6b15ddce-13cf-4c85-a58f-baec5b57ab52 | dell-r730-063.localdomain                           |         18 | 6b15ddce-13cf-4c85-a58f-baec5b57ab52 | None                                 |
| e2f5082a-c965-55db-acb3-8daf9857c721 | dell-r730-063.localdomain:NIC Switch agent          |          0 | 6b15ddce-13cf-4c85-a58f-baec5b57ab52 | 6b15ddce-13cf-4c85-a58f-baec5b57ab52 |
| d2fb0ef4-2f45-53a8-88be-113b3e64ba1b | dell-r730-014.localdomain:NIC Switch agent          |          0 | 31d3d88b-bc3a-41cd-9dc0-fda54028a882 | 31d3d88b-bc3a-41cd-9dc0-fda54028a882 |
| f1ca35e2-47ad-53a0-9058-390ade93b73e | dell-r730-063.localdomain:NIC Switch agent:enp6s0f1 |         13 | 6b15ddce-13cf-4c85-a58f-baec5b57ab52 | e2f5082a-c965-55db-acb3-8daf9857c721 |
| e518d381-d590-5767-8f34-c20def34b252 | dell-r730-014.localdomain:NIC Switch agent:enp6s0f1 |         19 | 31d3d88b-bc3a-41cd-9dc0-fda54028a882 | d2fb0ef4-2f45-53a8-88be-113b3e64ba1b |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

  2. Check the bandwidth a specific resource provides: 

    (undercloud) $ openstack --os-placement-api-version 1.17 resource provider inventory list <rp_uuid>

    The following example output shows the bandwidth provided by interface enp6s0f1 on dell-r730-014: 

    [stack@dell-r730-014 nova]$ openstack --os-placement-api-version 1.17 resource provider inventory list e518d381-d590-5767-8f34-c20def34b252
---------------------------------------------------------------------------------------------------
| resource_class             | allocation_ratio | min_unit |   max_unit | reserved | step_size |    total |
---------------------------------------------------------------------------------------------------
| NET_BW_EGR_KILOBIT_PER_SEC |              1.0 |        1 | 2147483647 |        0 |         1 | 10000000 |
| NET_BW_IGR_KILOBIT_PER_SEC |              1.0 |        1 | 2147483647 |        0 |         1 | 10000000 |
---------------------------------------------------------------------------------------------------

  3. To check claims against the resource provider when instances are running, run the following command: 

    (undercloud) $ openstack --os-placement-api-version 1.17 resource provider show --allocations  <rp_uuid>

    Example output: 

    [stack@dell-r730-014 nova]$ openstack --os-placement-api-version 1.17 resource provider show --allocations  e518d381-d590-5767-8f34-c20def34b252 -f value -c allocations
{3cbb9e07-90a8-4154-8acd-b6ec2f894a83: {resources: {NET_BW_EGR_KILOBIT_PER_SEC: 1000000, NET_BW_IGR_KILOBIT_PER_SEC: 1000000}}, 8848b88b-4464-443f-bf33-5d4e49fd6204: {resources: {NET_BW_EGR_KILOBIT_PER_SEC: 1000000, NET_BW_IGR_KILOBIT_PER_SEC: 1000000}}, 9a29e946-698b-4731-bc28-89368073be1a: {resources: {NET_BW_EGR_KILOBIT_PER_SEC: 1000000, NET_BW_IGR_KILOBIT_PER_SEC: 1000000}}, a6c83b86-9139-4e98-9341-dc76065136cc: {resources: {NET_BW_EGR_KILOBIT_PER_SEC: 3000000, NET_BW_IGR_KILOBIT_PER_SEC: 3000000}}, da60e33f-156e-47be-a632-870172ec5483: {resources: {NET_BW_EGR_KILOBIT_PER_SEC: 1000000, NET_BW_IGR_KILOBIT_PER_SEC: 1000000}}, eb582a0e-8274-4f21-9890-9a0d55114663: {resources: {NET_BW_EGR_KILOBIT_PER_SEC: 3000000, NET_BW_IGR_KILOBIT_PER_SEC: 3000000}}}

10.2.3. DSCP marking for egress traffic

You can use differentiated services code point (DSCP) to implement quality-of-service (QoS) policies on your network by embedding relevant values in the IP headers. OpenStack Networking (neutron) QoS policies can use DSCP marking to manage egress traffic on neutron ports and networks. At present, DSCP is available only for VLAN and flat provider networks using Open vSwitch (OVS).

Complete the steps in this example procedure to create a policy, define DSCP rules, and apply the rules to the policy. These rules use the --dscp-mark parameter, which specifies the decimal value of a DSCP mark.

  1. Create a new QoS policy: 

    openstack network qos policy create --project 98a2f53c20ce4d50a40dac4a38016c69 qos_policy_name

  2. Create a DSCP rule and apply it to the qos-web-servers policy, using DSCP mark 18: 

    openstack network qos rule create --type dscp-marking --dscp-mark 18 qos_policy_name
Created a new dscp_marking_rule:
+-----------+--------------------------------------+
| Field     | Value                                |
+-----------+--------------------------------------+
| dscp_mark | 18                                   |
| id        | d7f976ec-7fab-4e60-af70-f59bf88198e6 |
+-----------+--------------------------------------+

  3. View the DSCP rules for QoS policy, qos-web-servers: 

    openstack network qos rule list qos-web-servers
+-----------+--------------------------------------+
| dscp_mark | id                                   |
+-----------+--------------------------------------+
|        18 | d7f976ec-7fab-4e60-af70-f59bf88198e6 |
+-----------+--------------------------------------+

  4. View the details of the DSCP rule assigned to policy, qos-web-servers: 

    openstack network qos rule show qos-web-servers d7f976ec-7fab-4e60-af70-f59bf88198e6
+-----------+--------------------------------------+
| Field     | Value                                |
+-----------+--------------------------------------+
| dscp_mark | 18                                   |
| id        | d7f976ec-7fab-4e60-af70-f59bf88198e6 |
+-----------+--------------------------------------+

  5. Change the DSCP value assigned to a rule: 

    openstack network qos rule set --dscp-mark 22 qos-web-servers d7f976ec-7fab-4e60-af70-f59bf88198e6

  6. Delete a DSCP rule: 

    openstack network qos rule delete qos-web-servers d7f976ec-7fab-4e60-af70-f59bf88198e6

10.2.4. How to verify the application of your QoS policy and rules

Use the following commands to verify the creation and application of your quality of service (QoS) policy and rules.

ActionCommand

List the available QoS policies

$ openstack network qos policy list

Show details of a specific QoS policy

$ openstack network qos policy show <policy_name>

List the available QoS rules

$ openstack network qos rule type list

List the rules of a specific QoS policy

$ openstack network qos rule list <policy_name>

Show details of a specific rule

$ openstack network qos rule type show <rule_id>

List the available ports

$ openstack port list

Show details of a specific port

$ openstack port show <port_ID/port_name>

10.3. RBAC for QoS Policies

You can add a role-based access control (RBAC) for quality-of-service (QoS) policies. As a result, you can now make QoS policies available to certain projects.

For example, you can now create a QoS policy that allows for lower-priority network traffic, and have it only apply to certain projects. Run the following command to assign the bw-limiter policy to the project, demo: 

# openstack network rbac create --type qos_policy --target-project 80bf5732752a41128e612fe615c886c6 --action access_as_shared bw-limiter
Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.