Managing cost data using tagging
Organize resources and allocate costs with tags
Abstract
Chapter 1. Planning your tagging strategy
1.1. Using tags
Tags, which are also known as labels in OpenShift, are strings of custom metadata that you can assign to resources. In cost management, tags enable you to allocate costs to different parts of your environment and get a more accurate view of your cost data.
You can use tags in conjunction with hierarchies, such as projects, in your clouds and clusters. Cloud providers place limits on the number of tags that you can associate with resources, so you should plan a tagging strategy for cost management.
Tags can also help you achieve the following goals:
- Map business concepts to reports
- Show costs grouped by business concepts
- Define operations like business automation, operational profiles, or access and security controls
- Apply policies based on tags
- Split resources into smaller units when it is not possible to use projects or subprojects
- Identify relationships between integrations and group applications across multiple clusters with the same environment, cost center, or team
- Identify dependencies when there is no direct link between a resource, such as an RDS database and the OpenShift project that uses it
1.2. Creating a strategy for tags
When you plan a strategy for your tags, you should consider how you will organize and report costs for your integrations. Start with the minimum number of tags that you think you need to accomplish your organizational objectives. If you need more, slowly build over time.
The following sections outline some key considerations to make before you begin creating your tags:
- Value precedence
When multiple values are associated with the same key for a specific resource, Value precedence refers to the criteria that determines which value takes precedence. To avoid duplicating costs when you group by a tag, each tag or label key must be unique for every resource.
To learn more about how to consider value precedence, see Section 1.3, “Understanding value precedence in tags”.
- Map business to reporting
- Define the business perspectives you want to report on. For example, your taxonomy for cost management could consider these different perspectives:
- Ownership and usage
- Defining the owner and the user of the resource: for example, the unique identifier of the user who requested the resource, and the one that is actually consuming the resource.
- Tenancy
- If your environment is shared, it can be beneficial to understand which group or business unit has requested the resource. When the user is part of different groups, one group needs to be selected. For cost reporting, you can achieve this in many cases using cost center; but department, project, or partner are also good candidates.
- Location
- For organizations with software deployed globally, cloud providers already identify the region where your resources are running, but your private cloud may be different.
- Environment or stage
- You may want to differentiate between development and production, so that different cost decisions can be made depending on the environment where you are creating or running the resources. If your development pipeline already includes stages, such as development, testing, staging, pre-production and production, this is a good candidate.
- Application / Project / Service / Event
- Perhaps your environment is providing a service, such as a group of transient resources for an event (for example, your yearly customer-focused conference). You could even include the application version.
- Standardize your labels
Consistency is the most important aspect of a tagging strategy to deliver accurate and comparable cost reporting results.
Create a clear tagging policy that defines what resources need to be tagged, what tags are mandatory and what tags are optional, making sure that there is no room for interpretation.
If the values need to be chosen between a list, verify that those values are defined, consistent, and easily accessible, or that the list is presented to the user. For example, if you are defining development with the key “Development”, do not also use variations such as “Dev”, “DEV”, or “R&D” to also identify resources as “Development”.
- Tag all elements on your integrations (manually or through automation)
Because untagged resources cannot be reported, tag as many elements as possible, ideally using automation to prevent human error. Integrations have different automation features to take advantage of for tagging:
- In Azure, you can use Azure Policies to enforce tagging rules and conventions and avoid resources being deployed that do not comply with your expectations. You can create a policy that automatically applies necessary tags during provisioning, that enforces a predefined format for dates, or that makes some tags mandatory for some resource type.
- In AWS, you can use IAM policies for the same. Additionally, you can use an automation tool such as Ansible to add the necessary tags during provisioning and to verify that all the resources have been properly tagged.
- OpenShift Container Platform does not presently have a method of automating labelling.
- Review your tags often and refine as needed
Define your tags and use them as early as possible with cost management, even if you need to adjust your tagging scheme afterwards.
Review the resulting reports with your business owner and stakeholders early on to ensure your tags are helping you to generate the required reports, and review your tagging strategy every few weeks to optimize it.
- Select your tag terminology
Name your resources with names that allow you to identify your resources without accessing metadata, and then continue by adding metadata to it. Many clouds have a guide about how to do this properly; see Chapter 5, Additional resources for links.
Map your resources into keys and values. Keys will map to perspectives, while values will define the different options allowed for each key. In some cases, the value will be Null.
Not all integrations allow the same identifiers, and have different limitations. See Section 5.1, “Tag specifications by integration type” for limitations by integration.
1.3. Understanding value precedence in tags
When you associate multiple values with the same key for a specific resource, value precedence refers to the criteria that determines which value cost management prioritizes. To avoid duplicating costs when you group by a tag, each tag or label key should be unique for every resource.
1.3.1. OpenShift value precedence
In OpenShift, create a protocol that maintains the most specific value possible.
1.3.1.1. Namespace, node, and pod labels
If each resource has the same key, pod values override both node and namespace values. Cost management prioritizes pod values because the labels at these levels are more specific to the workload that is running inside of them. Node labels and namespace labels are more generic and describe characteristics at a higher level.
Namespace, node, and pod labels have the following value precedence:
- Pod labels
- Namespace labels
- Node labels
1.3.1.2. Persistent Volumes and Persistent Volume Claims labels
Persistent Volumes (PVs) are resources in the cluster that exist independently from the node or namespace. Pods create Persistent Volume Claims (PVCs) to make requests for those resources from the available PVs. When a request matches the criteria for one of these PVs, the pod uses these claims as volumes.
PVs and PVCs have the following value precedence:
- Persistent Volume Claim labels
- Persistent Volume labels
- Pod labels
- Namespace labels
- Node labels
1.3.1.3. Cloud data that is filtered by Openshift
When something is filtered by OpenShift, it indicates what amount of the cloud provider’s cost is associated with running an OpenShift cluster. When a cloud provider and an OpenShift source have matching tags or resource IDs in the cost reports, cost management correlates the two reports. This correlation calculates how much of the cloud provider cost is related to running OpenShift.
We combine the tags and labels of the two reports into one, but prioritize the cloud instance’s tags—rather than the Openshift labels—because the cloud instance is the original source of the data:
- Cloud instance tags
- OpenShift labels
Chapter 2. Using tags to manage cost data
Learn how tags work in cost management and how you can use them to best organize and view your resources to manage your costs.
2.1. Enabling and creating a tag mapping
Tag mapping is when you combine multiple tags across your cloud integrations. Tag mapping enables you to group and filter similar tags with one tag key.
To map a tag, you must first enable it. Cost management has a limit of 200 tags that you can enable. Complete the following steps:
- In cost management, click .
- Click the header tab, Tags and labels.
- Click the drop-down menu, Enable tags and labels.
- Select the tags that you want to enable. Clear a tag to disable it.
- Next, click the Map tags and labels drop-down menu and click .
- In the wizard that opens, select the tags that you want to make child tags. Then click .
- Select one tag that you want to be the parent tag. This action will map the parent tag to the children tags that you selected in the previous step. Click .
- Review your selections and click .
2.1.1. Troubleshooting duplicate keys
For every resource, each tag key must be unique and have only one value. However, when you map tags, you can unintentionally create scenarios that violate this rule and create multiple values.
Ordinarily, having more than one value would duplicate your costs. However, to avoid duplication, cost management prioritizes one key’s value. To understand how cost management prioritizes values and plan accordingly, see Section 1.3, “Understanding value precedence in tags”.
2.1.1.1. Troubleshooting example
Consider the following example where you are running an EC2 instance on AWS. You tagged this instance with the following key > value pairs:
-
app>cost-management -
App>Insights
In cost management, you mapped app to App. Therefore, the same EC2 instance has the following key > value pairs:
-
App>cost-management -
App>Insights
In this situation, cost management prioritizes the pre-existing value of the key App > Insights. Cost management also removes the association of the key app and its value cost-management from the AWS resource to prevent duplicate costs. App=cost-management is added to the App=insight cost.
Because App is set as the parent key, cost management prioritizes its value over the value of app in the tag mapping. Therefore, you should consider your tag mapping strategy before you set parent tags.
To troubleshoot issues with duplicate keys, ensure that your tag keys are unique and have only one value. Alternatively, to learn how to prioritize your keys, see Section 1.3, “Understanding value precedence in tags”.
Chapter 3. Configuring tags and labels in cost management
You must configure tags in each integration before cost management can use the tags to automatically organize your cost data.
After adding your integrations to cost management:
- Tag or label resources on each of your integrations. See Section 3.2, “Configuring tags on your integrations”.
- Refine and add to your tags to optimize your view of cost data. See Section 1.2, “Creating a strategy for tags”.
See the Getting started with cost management guide for instructions on configuring integrations.
3.1. How cost management associates tags
Tags in AWS and Microsoft Azure and labels in OpenShift consist of key:value pairs. When the key:value pairs match, the AWS/Azure and OpenShift costs are automatically associated by cost management. Tag matching in cost management is not case sensitive: for example, an AWS resource tagged APP and an OpenShift resource tagged app are a match:
| Source and resource type | Key | Value |
|---|---|---|
| AWS resource (RDS) | APP | Cost-Management |
| OpenShift pod | app | cost-management |
If an AWS resource tag matches with multiple OpenShift projects, the cost and usage of that resource are split evenly between the matched projects.
This is not the case with AWS compute resources that are matched through the instance ID-node relationship. In that case, cost and usage are broken down using information about a project’s resource consumption within the OpenShift cluster.
By default, cost management tracks AWS compute usage and costs by associating the Amazon EC2 instance ID or Microsoft Azure virtual machine instance ID with the OpenShift Container Platform node running on that instance.
3.1.1. Tag matching hierarchy in cost management
To identify your OpenShift resources running on AWS or Azure instances, cost management matches tags between integrations in the following order:
- Direct resource matching (AWS EC2 instance ID or Azure virtual machine instance ID)
- Special OpenShift tags
- Custom tags
3.1.2. OpenShift label inheritance in cost management
OpenShift labels follow an inheritance pattern from cluster to node, and from project to pod. You can associate costs at the node or project level without labeling every pod in your cluster.
Key-value pairs from node and project labels are inherited at the pod level for pod metrics in cost management. Key-value pairs from the cluster and node labels are inherited at the project level by the persistent volume claims (PVC) at each level. You can group by cluster, node, or project labels to see relevant PVCs in those workloads.
If a key already exists in the pod, then the value for that key in the pod remains. Cost management does not overwrite the pod value with the project or node value. A similar process happens from node to project.
Consider the following examples.
Example 1: Your organization assigns the label app and the value costpod1 to a pod. The project for this pod has the label app and the value cost-project. These resources are running on a node with the label us-east-1. The label app and the value costpod1 remain associated with the pod.
Example 2: Your organization has a project with the label app and the value cost-project. The project has three pods running and they are not labeled. Cost management associates the label app and the value cost-project with these pods.
3.1.3. Direct resource matching (instance ID)
The integrations apply these identifiers automatically. This form of tagging provides a direct link between Microsoft Azure or AWS instances and OpenShift nodes.
AWS assigns every EC2 instance a resource identifier (a number such as i-01f44b3d90ef90055). OpenShift nodes are matched directly to the AWS EC2 instance the cluster is running on using the AWS resource identifier. The OpenShift reports in cost management (generated from Prometheus data) include this identifier for nodes. Similarly in Microsoft Azure, each virtual machine instance ID is included in the OpenShift reports for cost management.
3.1.4. Special OpenShift tags
There are three special-case AWS tags you can use to associate cost with OpenShift:
-
openshift_cluster -
openshift_node -
openshift_project
These tags have matching priority over custom tags, and are especially useful in differentiating the costs of different OpenShift clusters running on the same AWS instance.
To use this tagging method to identify an OpenShift cluster, tag your AWS instance with the key openshift_cluster, and provide the OpenShift integration name as the value. In the following example, the name of OpenShift integration in the cost management application is dev-cluster:
| Source and resource type | Key | Value |
|---|---|---|
| AWS resource (RDS) |
|
|
| OpenShift cluster |
No tags needed. This will match if the name of the OpenShift integration in cost management is | No tags needed. |
3.1.5. Custom tags
You can use any key:value combination as tags, and cost management will associate identical tag key and values together. You can then group costs by tag key, account, service, region, and more to view your costs and charge for that tag.
| Source and resource type | Key | Value |
|---|---|---|
| AWS resource (RDS) |
|
|
| OpenShift pod |
|
|
3.2. Configuring tags on your integrations
To control the tags that cost management imports, activate or enable the tags that you want to view for each integration:
- You must activate AWS tags, and are then selected and exported to cost management in the data export. For instructions, see Activating AWS tags for cost management in the Adding an Amazon Web Services (AWS) source guide.
- Microsoft Azure tags are exported to cost management in the cost export report configured in Configuring a daily Azure data export schedule.
- OpenShift Container Platform labels are exported by the Cost Management Metrics Operator and included in the metrics reports that cost management uses as input.
3.2.1. Adding tags to an AWS resource
Amazon creates certain identifiers automatically, such as the EC2 instance resource identifier, or a number such as i-123456789, which cost management uses similarly.
You can also add your own tags at the individual resource level. These tags must be activated for Cost and Usage Reporting to export them to the cost management application.
Configure AWS tags for cost management using the following steps:
Procedure
Create and apply tags to your AWS resources.
NoteIf you converted from a compatible third-party Linux distribution to Red Hat Enterprise Linux (RHEL) and purchased the RHEL for third-party migration listing in AWS, activate the cost allocation tags for your RHEL systems on the AWS Cost Allocation tags page. Create
com_redhat_rhel_conversionand set the tag value totrue. If you are using ELS (Extended Lifecycle Support), createcom_redhat_rhel_addonand set the value toELS. Finally, createcom_redhat_rheland set the tag value to7or8to match your version of RHEL. The changes will be reflected in cost management the next time cost management downloads data.Do not use host metering if you plan on tagging items for RHEL metering. Your instances could be double-billed.
For instructions in the AWS documentation, see User-Defined Cost Allocation Tags.
Activate the tags you want to be collected by the cost management application through the data export. In the AWS Billing console, select the tags that you want to activate from the Cost Allocation Tags area.
For instructions in the AWS documentation, see Activating the AWS-Generated Cost Allocation Tags.
3.2.2. Adding tags to a Microsoft Azure resource
To create identifiers for virtual machine instances automatically, add a Microsoft Azure integration, which cost management uses similarly to tags to associate Microsoft Azure resources to related OpenShift resources. Add your own tags in Microsoft Azure at the individual resource level.
If you converted from a compatible third-party Linux distribution to Red Hat Enterprise Linux (RHEL) and purchased the RHEL for third party migration listing in Microsoft Azure, label the VMs for your RHEL systems. Create com_redhat_rhel_conversion and set the tag value to true. If you are using ELS (Extended Lifecycle Support), create com_redhat_rhel_addon and set the value to ELS. Finally, create com_redhat_rhel and set the tag value to 7 or 8 to match your version of RHEL. The changes will be reflected in cost management the next time cost management downloads data.
Do not use host metering if you plan on tagging items for RHEL metering. If you plan to tag, this could cause instances to be double-billed.
Create and apply Microsoft Azure tags for cost management using the instructions in the Microsoft Azure documentation: Use tags to organize your Azure resources and management hierarchy.
3.2.3. Adding tags to a Google Cloud resource
You can apply custom labels to Google cloud resources, such as virtual machine instances, images, and persistent disks. These labels are automatically added to your BigQuery export and sent to cost management.
Procedure
Create and apply labels to your Google Cloud resources.
See Creating and managing labels in the Google Cloud documentation for instructions.
3.2.4. Viewing labels in an OpenShift namespace
The AWS or Microsoft Azure tag equivalent in OpenShift is a label, which also consists of a key:value pair. The cost management service collects OpenShift tag data from nodes, pods, and persistent volumes (or persistent volume claims) using Prometheus metrics and Cost Management Metrics Operator.
To view the available tags, navigate to a resource in the OpenShift web console. Any assigned labels are listed under the Labels heading, for example: openshift.io/cluster-monitoring=true.
3.2.5. Enabling and Disabling tags in cost management
All cloud provider tags are activated in cost management by default. Sometimes having too many resource tags can affect cost management performance. Enabled tags are limited to 200 per account. Unnecessary tags can also make managing your costs more complicated when grouping tags and matching key:value pairs. Disable tags you are not actively using to reduce these potential issues.
Prerequisites
- You must have Organization Administrator or Cost Price List Administrator privileges to change these settings in cost management. See Limiting access to cost management resources in Getting started with cost management for more information about user roles and access.
Procedure
- From cost management, click → .
- Click the Tags and labels tab.
- Select the tags you want to disable.
Click .
This tag is now deactivated for the cost management application. You can enable tags you have disabled on the same page by selecting the tags you want to enable and clicking Enable tags.
3.2.6. Configuring Amazon Web Services cost categories in cost management
You can enable or disable Amazon Web Services (AWS) cost categories in the cost management service. AWS cost categories allow your organization to group meaningful cost and usage information in addition to tags. In order to use cost categories in cost management, they must first be configured in the AWS Console. The following procedure describes how to enable cost categories in the cost management service.
Prerequisites
- You must have Organization Administrator or Cost Price List Administrator privileges to change these settings in cost management. See Limiting access to cost management resources in Getting started with cost management for more information about user roles and access.
- You have an Amazon Web Services integration added to cost management with cost categories enabled through the Amazon Web Services Console.
Procedure
- From cost management, click → .
- Click the Cost categories tab.
- Select the cost category to enable. You can select more than one.
Click .
The selected cost categories are now enabled for the cost management application. You can also disable cost categories by selecting the cost categories you want to disable and clicking Disable categories.
Chapter 4. Viewing and exporting your cost data
4.1. Filtering your cost data view
Tags allow you to customize your view of cost data. You can view resources by type (for example, project, node, cluster) or tag or label to investigate why certain resources show an increase in cost, or when data looks abnormal.
This example shows how to see how much each OpenShift project within a cluster is costing.
Prerequisites
- Your OpenShift cluster added as a cost management data integration. See Integrating OpenShift Container Platform data into cost management in Getting started with cost management for instructions.
- Your cloud infrastructure account added as a cost management data integration. See Getting started with cost management for instructions for your cloud provider type.
- Configure tags on your integrations. For tips and configuration instructions, see Section 3.2, “Configuring tags on your integrations”.
Procedure
- From the OpenShift details menu, click the filter button and select Tag.
- In the Choose key dropdown list, select the key to filter by. For example, select environment to view clusters with the environment tag. Selecting a tag key reveals another dropdown to choose the value to filter by.
- In the Choose value dropdown list, select one or more values to filter by. For example, select qe and dev to show cost data for OpenShift projects with these tags.
To view more information about each project:
- Click the arrow icon for each resource to see more information such as the cluster(s) the resource belongs to, and CPU and memory usage, limits, and requests.
Click
to reveal more viewing options:
- Click to see rates applied to the OpenShift metrics to calculate the costs.
- Click to open the daily usage comparison view, which compares usage, request, and limits by day between months for that resource.
- Click or View all tags to see related resources and metadata.
- Click to reset your OpenShift details view.
4.2. Grouping cost data by tag category
You can group resources by tag category to further investigate your cost data.
Grouping and filtering is useful for finding the root cause of a cost or problem, or investigating part of the environment that acts independently of the rest, such as a cost center or an specific environment.
This allows you to hide information about the rest of the environment to help avoid unneeded complexity in cost data results and allow you to find the desired information, which can otherwise be hidden among other data.
This example shows how an educational course provider who is running a lab environment on OpenShift Container Platform can use tag grouping to filter cost information by student and course.
Prerequisites
- Your OpenShift cluster added as a cost management data integration. See Integrating OpenShift Container Platform data into cost management in Getting started with cost management for instructions.
- Your cloud infrastructure account added as a cost management data integration. See Getting started with cost management for instructions for your cloud provider type.
- Configure tags on your integrations. For tips and configuration instructions, see Section 3.2, “Configuring tags on your integrations”.
Procedure
- From the OpenShift details page in the Group cost by: field, select the tag key to group the cost by. In this case, select Tag Key:user to show results grouped by student user.
- In the filter area, select Tag.
- In the Choose key list, select the tag key user.
- In the Choose value dropdown list, check the values course_id and course_type to identify how many students have taken the course X and the costs for that course.
To view more information about each resource, for example, how much course X has cost:
- Click the arrow icon for each resource to see more information such as the cluster(s) the resource belongs to, and CPU and memory usage, limits, and requests.
- Click to open the daily usage comparison view, which compares by month the usage, request, and limits per day for that resource.
Click
to reveal more viewing options:
- Click to open the daily usage comparison view, which compares usage, request, and limits by day between months for that resource.
- Click to create a .CSV file for reporting. Specify a daily or monthly aggregate and click .
- Click to reset your OpenShift details view.
4.3. Exporting cost data to a reporting tool
Tags allow you to customize your view of cost data. This is useful when you want to investigate further why certain resources show an increase in cost, or data looks abnormal.
This example shows how to view data for specific OpenShift resources, and export the data to your desired reporting tool.
Prerequisites
- Your OpenShift cluster added as a cost management data integration. See Integrating OpenShift Container Platform data into cost management in Getting started with cost management for instructions.
- Your cloud infrastructure account added as a cost management data integration. See Getting started with cost management for instructions for your cloud provider type.
- Configure tags on your integrations. For tips and configuration instructions, see Section 3.2, “Configuring tags on your integrations”.
Procedure
- From the OpenShift details menu, click the filter button and select Tag.
- In the Choose key dropdown list, select the key to filter by. For example, select version. Selecting a tag key reveals another dropdown to choose the value to filter by.
- In the Choose value dropdown list, select one or more values to filter by. For example, select qe and dev to show cost data for OpenShift resources with these tags.
- To export data about your resources, check the boxes next to each resource you want to export data for. Click to open the export options dialog.
- Specify a daily or monthly aggregate and click .
The CSV file will download to your local system, and you can use it in your desired reporting tool.
You can also export your data as a .CSV file from the More options
> Export data menu on each resource.
Click to reset your OpenShift details view.
Chapter 5. Additional resources
5.1. Tag specifications by integration type
Tagging standards differ between integration types. To use the same tags/labels across integrations, you must use the most common of all the restrictions across the different providers.
The following table summarizes tagging and labelling criteria across AWS, Azure and OpenShift Container Platform 4:
| Criteria | AWS | Azure | Google Cloud | Red Hat OpenShift |
|---|---|---|---|---|
| Name | Tags | Tags | Labels | Labels |
| Format | Key & value | Name & value | Key & value | Key & value Keys: [prefix/]name Prefix: must be a DNS subdomain |
| Allows empty value | Yes | Yes | Yes | Yes |
| Unique label per key | Yes | Yes | Yes | Yes |
| Case sensitive | Yes | No | Only lowercase letters | Yes |
| Limit per resource | 50 | 50 (15 for storage) | 64 | N/A |
| Length of key | 128 | 512 (128 for storage) | 63 | 253(prefix) / 63(name) |
| Length of value | 256 | 256 | 63 | 63 |
| Allowed characters | Letters, numbers, and spaces representable in UTF-8, and the following characters: + - = . _ : / @ | Tag names cannot contain these characters: <, >, %, &, \, ?, / | Only lowercase letters, numeric characters, underscores, and dashes. | The name segment is required and must be 63 characters or less, beginning and ending with an alphanumeric character ([a-z0-9A-Z]) with dashes (-), underscores (_), dots (.), and alphanumerics between |
| Restrictions | The prefix “aws:” is reserved. Tags applied to EC2 can use any character. Not all resource types support tags. | Not all resource types support tags. Generalized VMs do not support tags. Tags applied to the resource group are not inherited by the resources. | Keys must start with a lowercase letter or international character. | Prefixes kubernetes.io/ and k8s.io/ are reserved. Not all resource types support tags. |
| Notes | You need to select the tag keys that will be included in cost and usage files and billing reports. | You can use a JSON string to go over the limit of keys. | There is no limit on how many labels you can apply across all resources within a project. | If the prefix is omitted, the label Key is presumed to be private to the user. |
5.2. Further reading
The following links provide further guidance on tagging for each integration type.
AWS:
OpenShift:
Microsoft Azure:
Google Cloud:
Providing feedback on Red Hat documentation
We appreciate and prioritize your feedback regarding our documentation. Provide as much detail as possible, so that your request can be quickly addressed.
Prerequisites
- You are logged in to the Red Hat Customer Portal.
Procedure
To provide feedback, perform the following steps:
- Click the following link: Create Issue.
- Describe the issue or enhancement in the Summary text box.
- Provide details about the issue or requested enhancement in the Description text box.
- Type your name in the Reporter text box.
- Click the Create button.
This action creates a documentation ticket and routes it to the appropriate documentation team. Thank you for taking the time to provide feedback.
