About
Introduction to OpenShift Container Platform
Abstract
Chapter 1. OpenShift Container Platform 4.15 Documentation
Table of Contents
Welcome to the official OpenShift Container Platform 4.15 documentation, where you can learn about OpenShift Container Platform and start exploring its features.
To navigate the OpenShift Container Platform 4.15 documentation, you can use one of the following methods:
- Use the left navigation bar to browse the documentation.
- Select the task that interests you from the contents of this Welcome page.
Start with Architecture and Security and compliance. Next, view the release notes.
1.1. Cluster installer activities
Explore the following OpenShift Container Platform installation tasks:
- OpenShift Container Platform installation overview: Depending on the platform, you can install OpenShift Container Platform on installer-provisioned or user-provisioned infrastructure. The OpenShift Container Platform installation program provides the flexibility to deploy OpenShift Container Platform on a range of different platforms.
- Install a cluster on Alibaba: On Alibaba Cloud, you can install OpenShift Container Platform on installer-provisioned infrastructure. This is currently a Technology Preview feature only.
- Install a cluster on AWS: On AWS, you can install OpenShift Container Platform on installer-provisioned infrastructure or user-provisioned infrastructure.
- Install a cluster on Azure: On Microsoft Azure, you can install OpenShift Container Platform on installer-provisioned infrastructure or user-provisioned infrastructure.
- Install a cluster on Azure Stack Hub: On Microsoft Azure Stack Hub, you can install OpenShift Container Platform on installer-provisioned infrastructure or user-provisioned infrastructure.
- Installing OpenShift Container Platform with the Assisted Installer: The Assisted Installer is an installation solution that is provided on the Red Hat Red Hat Hybrid Cloud Console. The Assisted Installer supports installing an OpenShift Container Platform cluster on many platforms, but with a focus on bare metal, Nutanix, and VMware vSphere infrastructures.
- Installing OpenShift Container Platform with the Agent-based Installer: You can use the Agent-based Installer to generate a bootable ISO image that contains the Assisted discovery agent, the Assisted Service, and all the other information required to deploy an OpenShift Container Platform cluster. The Agent-based Installer leverages the advantages of the Assisted Installer in a disconnected environment
- Install a cluster on bare metal: On bare metal, you can install OpenShift Container Platform on installer-provisioned infrastructure or user-provisioned infrastructure. If none of the available platform and cloud provider deployment options meet your needs, consider using bare metal user-provisioned infrastructure.
- Install a cluster on GCP: On Google Cloud Platform (GCP) you can install OpenShift Container Platform on installer-provisioned infrastructure or user-provisioned infrastructure.
- Install a cluster on IBM Cloud®: On IBM Cloud®, you can install OpenShift Container Platform on installer-provisioned infrastructure.
- Install a cluster on IBM Power® Virtual Server: On IBM Power® Virtual Server, you can install OpenShift Container Platform on installer-provisioned infrastructure.
- Install a cluster on IBM Power®: On IBM Power®, you can install OpenShift Container Platform on user-provisioned infrastructure.
- Install a cluster on IBM Z® and IBM® LinuxONE: On IBM Z® and IBM® LinuxONE, you can install OpenShift Container Platform on user-provisioned infrastructure.
- Install a cluster on Oracle® Cloud Infrastructure (OCI): You can use the Assisted Installer or the Agent-based Installer to install a cluster on OCI. This means that you can run cluster workloads on infrastructure that supports dedicated, hybrid, public, and multiple cloud environments. See Installing a cluster on Oracle Cloud Infrastructure (OCI) by using the Assisted Installer and Installing a cluster on Oracle Cloud Infrastructure (OCI) by using the Agent-based Installer.
- Install a cluster on Nutanix: On Nutanix, you can install a cluster on your OpenShift Container Platform on installer-provisioned infrastructure.
- Install a cluster on Red Hat OpenStack Platform (RHOSP): On RHOSP, you can install OpenShift Container Platform on installer-provisioned infrastructure or user-provisioned infrastructure.
- Install a cluster on VMware vSphere: You can install OpenShift Container Platform on supported versions of vSphere.
1.2. Other cluster installer activities
Install a cluster in a restricted network: If your cluster uses user-provisioned infrastructure on AWS, GCP, vSphere, IBM Cloud®, IBM Z® and IBM® LinuxONE, IBM Power®, or bare metal and the cluster does not have full access to the internet, you must mirror the OpenShift Container Platform installation images. To do this action, use one of the following methods, so that you can install a cluster in a restricted network.
- Install a cluster in an existing network: If you use an existing Virtual Private Cloud (VPC) in AWS or GCP or an existing VNet on Microsoft Azure, you can install a cluster. Also consider Installing a cluster on GCP into a shared VPC
- Install a private cluster: If your cluster does not require external internet access, you can install a private cluster on AWS, Azure, GCP, or IBM Cloud®. Internet access is still required to access the cloud APIs and installation media.
- Check installation logs: Access installation logs to evaluate issues that occur during OpenShift Container Platform installation.
- Access OpenShift Container Platform: Use credentials output at the end of the installation process to log in to the OpenShift Container Platform cluster from the command line or web console.
- Install Red Hat OpenShift Data Foundation: You can install Red Hat OpenShift Data Foundation as an Operator to provide highly integrated and simplified persistent storage management for containers.
- Red Hat Enterprise Linux CoreOS (RHCOS) image layering: As a post-installation task, you can add new images on top of the base RHCOS image. This layering does not modify the base RHCOS image. Instead, the layering creates a custom layered image that includes all RHCOS functions and adds additional functions to specific nodes in the cluster.
1.3. Developer activities
Develop and deploy containerized applications with OpenShift Container Platform. OpenShift Container Platform is a platform for developing and deploying containerized applications. Read the following OpenShift Container Platform documentation, so that you can better understand OpenShift Container Platform functions:
- Understand OpenShift Container Platform development: Learn the different types of containerized applications, from simple containers to advanced Kubernetes deployments and Operators.
-
Work with projects: Create projects from the OpenShift Container Platform web console or OpenShift CLI (
oc) to organize and share the software you develop. - Creating applications using the Developer perspective: Use the Developer perspective in the OpenShift Container Platform web console to easily create and deploy applications.
- Viewing application composition using the Topology view: Use the Topology view to visually interact with your applications, monitor status, connect and group components, and modify your code base.
- Understanding Service Binding Operator: With the Service Binding Operator, an application developer can bind workloads with Operator-managed backing services by automatically collecting and sharing binding data with the workloads. The Service Binding Operator improves the development lifecycle with a consistent and declarative service binding method that prevents discrepancies in cluster environments.
- Create CI/CD Pipelines: Pipelines are serverless, cloud-native, continuous integration and continuous deployment systems that run in isolated containers. Pipelines use standard Tekton custom resources to automate deployments and are designed for decentralized teams that work on microservice-based architecture.
- Manage your infrastructure and application configurations: GitOps is a declarative way to implement continuous deployment for cloud native applications. GitOps defines infrastructure and application definitions as code. GitOps uses this code to manage multiple workspaces and clusters to simplify the creation of infrastructure and application configurations. GitOps also handles and automates complex deployments at a fast pace, which saves time during deployment and release cycles.
- Deploy Helm charts: Helm is a software package manager that simplifies deployment of applications and services to OpenShift Container Platform clusters. Helm uses a packaging format called charts. A Helm chart is a collection of files that describes the OpenShift Container Platform resources.
- Understand image builds: Choose from different build strategies (Docker, S2I, custom, and pipeline) that can include different kinds of source materials, such as Git repositories, local binary inputs, and external artifacts. You can follow examples of build types from basic builds to advanced builds.
- Create container images: A container image is the most basic building block in OpenShift Container Platform and Kubernetes applications. By defining image streams, you can gather multiple versions of an image in one place as you continue to develop the image stream. With S2I containers, you can insert your source code into a base container. The base container is configured to run code of a particular type, such as Ruby, Node.js, or Python.
-
Create deployments: Use
Deploymentobjects to exert fine-grained management over applications. Deployments create replica sets according to the rollout strategy, which orchestrates pod lifecycles. - Create templates: Use existing templates or create your own templates that describe how an application is built or deployed. A template can combine images with descriptions, parameters, replicas, exposed ports and other content that defines how an application can be run or built.
- Understand Operators: Operators are the preferred method for creating on-cluster applications for OpenShift Container Platform 4.15. Learn about the Operator Framework and how to deploy applications by using installed Operators into your projects.
- Develop Operators: Operators are the preferred method for creating on-cluster applications for OpenShift Container Platform 4.15. Learn the workflow for building, testing, and deploying Operators. You can then create your own Operators based on Ansible or Helm, or configure built-in Prometheus monitoring by using the Operator SDK.
- Reference the REST API index: Learn about OpenShift Container Platform application programming interface endpoints.
- Software Supply Chain Security enhancements: The PipelineRun details page in the Developer or Administrator perspective of the web console provides a visual representation of identified vulnerabilities, which are categorized by severity. Additionally, these enhancements provide an option to download or view Software Bill of Materials (SBOMs) for enhanced transparency and control within your supply chain. Learn about setting up OpenShift Pipelines in the web console to view Software Supply Chain Security elements.
1.4. Cluster administrator activities
Manage machines, provide services to users, and follow monitoring and logging reports. Read the following OpenShift Container Platform documentation, so that you can better understand OpenShift Container Platform functions:
- Understand OpenShift Container Platform management: Learn about components of the OpenShift Container Platform 4.15 control plane. See how OpenShift Container Platform control plane and compute nodes are managed and updated through the Machine API and Operators.
- Cluster capabilities: As a cluster administrator, you can enable cluster capabilities that were disabled before installation.
1.4.1. Manage cluster components
- Manage machines: Manage compute and control plane machines in your cluster with machine sets, by deploying health checks, and applying autoscaling.
- Manage container registries: Each OpenShift Container Platform cluster includes a built-in container registry for storing its images. You can also configure a separate Red Hat Quay registry to use with OpenShift Container Platform. The Quay.io website provides a public container registry that stores OpenShift Container Platform containers and Operators.
- Manage users and groups: Add users and groups with different levels of permissions to use or modify clusters.
- Manage authentication: Learn how user, group, and API authentication works in OpenShift Container Platform. OpenShift Container Platform supports multiple identity providers.
- Manage ingress, API server, and service certificates: OpenShift Container Platform creates certificates by default for the Ingress Operator, the API server, and for services needed by complex middleware applications that require encryption. You might need to change, add, or rotate these certificates.
-
Manage networking: The cluster network in OpenShift Container Platform is managed by the Cluster Network Operator (CNO). The CNO uses
iptablesrules in kube-proxy to direct traffic between nodes and pods running on those nodes. The Multus Container Network Interface adds the capability to attach multiple network interfaces to a pod. By using network policy features, you can isolate your pods or permit selected traffic. - Manage storage: With OpenShift Container Platform, a cluster administrator can configure persistent storage by using Red Hat OpenShift Data Foundation, AWS Elastic Block Store, NFS, iSCSI, Container Storage Interface (CSI), and more. You can expand persistent volumes, configure dynamic provisioning, and use CSI to configure, clone, and use snapshots of persistent storage.
- Manage Operators: Lists of Red Hat, ISV, and community Operators can be reviewed by cluster administrators and installed on their clusters. After you install them, you can run, upgrade, back up, or otherwise manage the Operator on your cluster.
- Understanding Windows container workloads. You can use the Red Hat OpenShift support for Windows Containers feature to run Windows compute nodes in an OpenShift Container Platform cluster. This is possible by using the Red Hat Windows Machine Config Operator (WMCO) to install and manage Windows nodes.
1.4.2. Change cluster components
- Use custom resource definitions (CRDs) to modify the cluster: Cluster features implemented with Operators can be modified with CRDs. Learn to create a CRD and manage resources from CRDs.
- Set resource quotas: Choose from CPU, memory, and other system resources to set quotas.
- Prune and reclaim resources: Reclaim space by pruning unneeded Operators, groups, deployments, builds, images, registries, and cron jobs.
- Scale and tune clusters: Set cluster limits, tune nodes, scale cluster monitoring, and optimize networking, storage, and routes for your environment.
-
Update a cluster: Use the Cluster Version Operator (CVO) to upgrade your OpenShift Container Platform cluster. If an update is available from the OpenShift Update Service (OSUS), you apply that cluster update from the OpenShift Container Platform web console or the OpenShift CLI (
oc). - Using the OpenShift Update Service in a disconnected environment: You can use the OpenShift Update Service for recommending OpenShift Container Platform updates in disconnected environments.
- Improving cluster stability in high latency environments by using worker latency profiles: If your network has latency issues, you can use one of three worker latency profiles to help ensure that your control plane does not accidentally evict pods in case it cannot reach a worker node. You can configure or modify the profile at any time during the life of the cluster.
1.4.3. Observe a cluster
- OpenShift Logging: Learn about logging and configure different logging components, such as log storage, log collectors, and the logging web console plugin.
- Red Hat OpenShift distributed tracing platform: Store and visualize large volumes of requests passing through distributed systems, across the whole stack of microservices, and under heavy loads. Use the distributed tracing platform for monitoring distributed transactions, gathering insights into your instrumented services, network profiling, performance and latency optimization, root cause analysis, and troubleshooting the interaction between components in modern cloud-native microservices-based applications.
- Red Hat build of OpenTelemetry: Instrument, generate, collect, and export telemetry traces, metrics, and logs to analyze and understand your software’s performance and behavior. Use open source backends like Tempo or Prometheus, or use commercial offerings. Learn a single set of APIs and conventions, and own the data that you generate.
- Network Observability: Observe network traffic for OpenShift Container Platform clusters by using eBPF technology to create and enrich network flows. You can view dashboards, customize alerts, and analyze network flow information for further insight and troubleshooting.
- In-cluster monitoring: Learn to configure the monitoring stack. After configuring monitoring, use the web console to access monitoring dashboards. In addition to infrastructure metrics, you can also scrape and view metrics for your own services.
- Remote health monitoring: OpenShift Container Platform collects anonymized aggregated information about your cluster. By using Telemetry and the Insights Operator, this data is received by Red Hat and used to improve OpenShift Container Platform. You can view the data collected by remote health monitoring.
- Power monitoring for Red Hat OpenShift (Technology Preview): You can use power monitoring for Red Hat OpenShift to monitor the power usage and identify power-consuming containers running in an OpenShift Container Platform cluster. Power monitoring collects and exports energy-related system statistics from various components, such as CPU and DRAM. Power monitoring provides granular power consumption data for Kubernetes pods, namespaces, and nodes.
1.5. Hosted control plane activities
Support for bare metal and OpenShift Virtualization: Hosted control planes for OpenShift Container Platform is now Generally Available on bare metal and OpenShift Virtualization platforms. For more information, see the following documentation:
Technology Preview features: Hosted control planes remains available as a Technology Preview feature on the Amazon Web Services, IBM Power®, and IBM Z® platforms. You can now provision a hosted control plane cluster by using the non bare metal agent machines. For more information, see the following documentation:
- Configuring the hosting cluster on AWS (Technology Preview)
- Configuring the hosting cluster on a 64-bit x86 OpenShift Container Platform cluster to create hosted control planes for IBM Power® compute nodes (Technology Preview)
- Configuring the hosted cluster on 64-bit x86 bare metal for IBM Z® compute nodes (Technology Preview)
- Configuring hosted control plane clusters using non bare metal agent machines (Technology Preview)
Chapter 2. Learn more about OpenShift Container Platform
Use the following sections to find content to help you learn about and use OpenShift Container Platform.
2.1. Architect
| Learn about OpenShift Container Platform | Plan an OpenShift Container Platform deployment | Additional resources |
|---|---|---|
2.2. Cluster Administrator
| Learn about OpenShift Container Platform | Deploy OpenShift Container Platform | Manage OpenShift Container Platform | Additional resources |
|---|---|---|---|
2.3. Application Site Reliability Engineer (App SRE)
| Learn about OpenShift Container Platform | Deploy and manage applications | Additional resources |
|---|---|---|
2.4. Developer
| Learn about application development in OpenShift Container Platform | Deploy applications |
|---|---|
| Getting started with OpenShift for developers (interactive tutorial) | |
| Red Hat OpenShift Dev Spaces (formerly Red Hat CodeReady Workspaces) | |
Chapter 3. About OpenShift Kubernetes Engine
As of 27 April 2020, Red Hat has decided to rename Red Hat OpenShift Container Engine to Red Hat OpenShift Kubernetes Engine to better communicate what value the product offering delivers.
Red Hat OpenShift Kubernetes Engine is a product offering from Red Hat that lets you use an enterprise class Kubernetes platform as a production platform for launching containers. You download and install OpenShift Kubernetes Engine the same way as OpenShift Container Platform as they are the same binary distribution, but OpenShift Kubernetes Engine offers a subset of the features that OpenShift Container Platform offers.
3.1. Similarities and differences
You can see the similarities and differences between OpenShift Kubernetes Engine and OpenShift Container Platform in the following table:
| OpenShift Kubernetes Engine | OpenShift Container Platform | ||
|---|---|---|---|
| Fully Automated Installers | Yes | Yes | |
| Over the Air Smart Upgrades | Yes | Yes | |
| Enterprise Secured Kubernetes | Yes | Yes | |
| Kubectl and oc automated command line | Yes | Yes | |
| Operator Lifecycle Manager (OLM) | Yes | Yes | |
| Administrator Web console | Yes | Yes | |
| OpenShift Virtualization | Yes | Yes | |
| User Workload Monitoring | Yes | ||
| Cluster Monitoring | Yes | Yes | |
| Cost Management SaaS Service | Yes | Yes | |
| Platform Logging | Yes | ||
| Developer Web Console | Yes | ||
| Developer Application Catalog | Yes | ||
| Source to Image and Builder Automation (Tekton) | Yes | ||
| OpenShift Service Mesh (Maistra, Kiali, and Jaeger) | Yes | ||
| OpenShift distributed tracing (Jaeger) | Yes | ||
| OpenShift Serverless (Knative) | Yes | ||
| OpenShift Pipelines (Jenkins and Tekton) | Yes | ||
| Embedded Component of IBM Cloud® Pak and RHT MW Bundles | Yes | ||
| OpenShift sandboxed containers | Yes | ||
3.1.1. Core Kubernetes and container orchestration
OpenShift Kubernetes Engine offers full access to an enterprise-ready Kubernetes environment that is easy to install and offers an extensive compatibility test matrix with many of the software elements that you might use in your data center.
OpenShift Kubernetes Engine offers the same service level agreements, bug fixes, and common vulnerabilities and errors protection as OpenShift Container Platform. OpenShift Kubernetes Engine includes a Red Hat Enterprise Linux (RHEL) Virtual Datacenter and Red Hat Enterprise Linux CoreOS (RHCOS) entitlement that allows you to use an integrated Linux operating system with container runtime from the same technology provider.
The OpenShift Kubernetes Engine subscription is compatible with the Red Hat OpenShift support for Windows Containers subscription.
3.1.2. Enterprise-ready configurations
OpenShift Kubernetes Engine uses the same security options and default settings as the OpenShift Container Platform. Default security context constraints, pod security policies, best practice network and storage settings, service account configuration, SELinux integration, HAproxy edge routing configuration, and all other standard protections that OpenShift Container Platform offers are available in OpenShift Kubernetes Engine. OpenShift Kubernetes Engine offers full access to the integrated monitoring solution that OpenShift Container Platform uses, which is based on Prometheus and offers deep coverage and alerting for common Kubernetes issues.
OpenShift Kubernetes Engine uses the same installation and upgrade automation as OpenShift Container Platform.
3.1.3. Standard infrastructure services
With an OpenShift Kubernetes Engine subscription, you receive support for all storage plugins that OpenShift Container Platform supports.
In terms of networking, OpenShift Kubernetes Engine offers full and supported access to the Kubernetes Container Network Interface (CNI) and therefore allows you to use any third-party SDN that supports OpenShift Container Platform. It also allows you to use the included Open vSwitch software defined network to its fullest extent. OpenShift Kubernetes Engine allows you to take full advantage of the OVN Kubernetes overlay, Multus, and Multus plugins that are supported on OpenShift Container Platform. OpenShift Kubernetes Engine allows customers to use a Kubernetes Network Policy to create microsegmentation between deployed application services on the cluster.
You can also use the Route API objects that are found in OpenShift Container Platform, including its sophisticated integration with the HAproxy edge routing layer as an out of the box Kubernetes Ingress Controller.
3.1.4. Core user experience
OpenShift Kubernetes Engine users have full access to Kubernetes Operators, pod deployment strategies, Helm, and OpenShift Container Platform templates. OpenShift Kubernetes Engine users can use both the oc and kubectl command line interfaces. OpenShift Kubernetes Engine also offers an administrator web-based console that shows all aspects of the deployed container services and offers a container-as-a service experience. OpenShift Kubernetes Engine grants access to the Operator Life Cycle Manager that helps you control access to content on the cluster and life cycle operator-enabled services that you use. With an OpenShift Kubernetes Engine subscription, you receive access to the Kubernetes namespace, the OpenShift Project API object, and cluster-level Prometheus monitoring metrics and events.
3.1.5. Maintained and curated content
With an OpenShift Kubernetes Engine subscription, you receive access to the OpenShift Container Platform content from the Red Hat Ecosystem Catalog and Red Hat Connect ISV marketplace. You can access all maintained and curated content that the OpenShift Container Platform eco-system offers.
3.1.6. OpenShift Data Foundation compatible
OpenShift Kubernetes Engine is compatible and supported with your purchase of OpenShift Data Foundation.
3.1.7. Red Hat Middleware compatible
OpenShift Kubernetes Engine is compatible and supported with individual Red Hat Middleware product solutions. Red Hat Middleware Bundles that include OpenShift embedded in them only contain OpenShift Container Platform.
3.1.8. OpenShift Serverless
OpenShift Kubernetes Engine does not include OpenShift Serverless support. Use OpenShift Container Platform for this support.
3.1.9. Quay Integration compatible
OpenShift Kubernetes Engine is compatible and supported with a Red Hat Quay purchase.
3.1.10. OpenShift Virtualization
OpenShift Kubernetes Engine includes support for the Red Hat product offerings derived from the kubevirt.io open source project.
3.1.11. Advanced cluster management
OpenShift Kubernetes Engine is compatible with your additional purchase of Red Hat Advanced Cluster Management (RHACM) for Kubernetes. An OpenShift Kubernetes Engine subscription does not offer a cluster-wide log aggregation solution or support Elasticsearch, Fluentd, or Kibana-based logging solutions. Red Hat OpenShift Service Mesh capabilities derived from the open-source istio.io and kiali.io projects that offer OpenTracing observability for containerized services on OpenShift Container Platform are not supported in OpenShift Kubernetes Engine.
3.1.12. Advanced networking
The standard networking solutions in OpenShift Container Platform are supported with an OpenShift Kubernetes Engine subscription. The OpenShift Container Platform Kubernetes CNI plugin for automation of multi-tenant network segmentation between OpenShift Container Platform projects is entitled for use with OpenShift Kubernetes Engine. OpenShift Kubernetes Engine offers all the granular control of the source IP addresses that are used by application services on the cluster. Those egress IP address controls are entitled for use with OpenShift Kubernetes Engine. OpenShift Container Platform offers ingress routing to on cluster services that use non-standard ports when no public cloud provider is in use via the VIP pods found in OpenShift Container Platform. That ingress solution is supported in OpenShift Kubernetes Engine. OpenShift Kubernetes Engine users are supported for the Kubernetes ingress control object, which offers integrations with public cloud providers. Red Hat Service Mesh, which is derived from the istio.io open source project, is not supported in OpenShift Kubernetes Engine. Also, the Kourier Ingress Controller found in OpenShift Serverless is not supported on OpenShift Kubernetes Engine.
3.1.13. OpenShift sandboxed containers
OpenShift Kubernetes Engine does not include OpenShift sandboxed containers. Use OpenShift Container Platform for this support.
3.1.14. Developer experience
With OpenShift Kubernetes Engine, the following capabilities are not supported:
- The OpenShift Container Platform developer experience utilities and tools, such as Red Hat OpenShift Dev Spaces.
- The OpenShift Container Platform pipeline feature that integrates a streamlined, Kubernetes-enabled Jenkins and Tekton experience in the user’s project space.
- The OpenShift Container Platform source-to-image feature, which allows you to easily deploy source code, dockerfiles, or container images across the cluster.
- Build strategies, builder pods, or Tekton for end user container deployments.
-
The
ododeveloper command line. - The developer persona in the OpenShift Container Platform web console.
3.1.15. Feature summary
The following table is a summary of the feature availability in OpenShift Kubernetes Engine and OpenShift Container Platform. Where applicable, it includes the name of the Operator that enables a feature.
| Feature | OpenShift Kubernetes Engine | OpenShift Container Platform | Operator name |
|---|---|---|---|
| Fully Automated Installers (IPI) | Included | Included | N/A |
| Customizable Installers (UPI) | Included | Included | N/A |
| Disconnected Installation | Included | Included | N/A |
| Red Hat Enterprise Linux (RHEL) or Red Hat Enterprise Linux CoreOS (RHCOS) entitlement | Included | Included | N/A |
| Existing RHEL manual attach to cluster (BYO) | Included | Included | N/A |
| CRIO Runtime | Included | Included | N/A |
| Over the Air Smart Upgrades and Operating System (RHCOS) Management | Included | Included | N/A |
| Enterprise Secured Kubernetes | Included | Included | N/A |
|
Kubectl and | Included | Included | N/A |
| Auth Integrations, RBAC, SCC, Multi-Tenancy Admission Controller | Included | Included | N/A |
| Operator Lifecycle Manager (OLM) | Included | Included | N/A |
| Administrator web console | Included | Included | N/A |
| OpenShift Virtualization | Included | Included | OpenShift Virtualization Operator |
| Compliance Operator provided by Red Hat | Included | Included | Compliance Operator |
| File Integrity Operator | Included | Included | File Integrity Operator |
| Gatekeeper Operator | Not Included - Requires separate subscription | Not Included - Requires separate subscription | Gatekeeper Operator |
| Klusterlet | Not Included - Requires separate subscription | Not Included - Requires separate subscription | N/A |
| Kube Descheduler Operator provided by Red Hat | Included | Included | Kube Descheduler Operator |
| Local Storage provided by Red Hat | Included | Included | Local Storage Operator |
| Node Feature Discovery provided by Red Hat | Included | Included | Node Feature Discovery Operator |
| Performance Profile controller | Included | Included | N/A |
| PTP Operator provided by Red Hat | Included | Included | PTP Operator |
| Service Telemetry Operator provided by Red Hat | Not Included | Included | Service Telemetry Operator |
| SR-IOV Network Operator | Included | Included | SR-IOV Network Operator |
| Vertical Pod Autoscaler | Included | Included | Vertical Pod Autoscaler |
| Cluster Monitoring (Prometheus) | Included | Included | Cluster Monitoring |
| Device Manager (for example, GPU) | Included | Included | N/A |
| Log Forwarding | Included | Included | Red Hat OpenShift Logging Operator |
| Telemeter and Insights Connected Experience | Included | Included | N/A |
| Feature | OpenShift Kubernetes Engine | OpenShift Container Platform | Operator name |
| OpenShift Cloud Manager SaaS Service | Included | Included | N/A |
| OVS and OVN SDN | Included | Included | N/A |
| MetalLB | Included | Included | MetalLB Operator |
| HAProxy Ingress Controller | Included | Included | N/A |
| Ingress Cluster-wide Firewall | Included | Included | N/A |
| Egress Pod and Namespace Granular Control | Included | Included | N/A |
| Ingress Non-Standard Ports | Included | Included | N/A |
| Multus and Available Multus Plugins | Included | Included | N/A |
| Network Policies | Included | Included | N/A |
| IPv6 Single and Dual Stack | Included | Included | N/A |
| CNI Plugin ISV Compatibility | Included | Included | N/A |
| CSI Plugin ISV Compatibility | Included | Included | N/A |
| RHT and IBM® middleware à la carte purchases (not included in OpenShift Container Platform or OpenShift Kubernetes Engine) | Included | Included | N/A |
| ISV or Partner Operator and Container Compatibility (not included in OpenShift Container Platform or OpenShift Kubernetes Engine) | Included | Included | N/A |
| Embedded OperatorHub | Included | Included | N/A |
| Embedded Marketplace | Included | Included | N/A |
| Quay Compatibility (not included) | Included | Included | N/A |
| OpenShift API for Data Protection (OADP) | Included | Included | OADP Operator |
| RHEL Software Collections and RHT SSO Common Service (included) | Included | Included | N/A |
| Embedded Registry | Included | Included | N/A |
| Helm | Included | Included | N/A |
| User Workload Monitoring | Not Included | Included | N/A |
| Cost Management SaaS Service | Included | Included | Cost Management Metrics Operator |
| Platform Logging | Not Included | Included | Red Hat OpenShift Logging Operator |
| OpenShift Elasticsearch Operator provided by Red Hat | Not Included | Cannot be run standalone | N/A |
| Developer Web Console | Not Included | Included | N/A |
| Developer Application Catalog | Not Included | Included | N/A |
| Source to Image and Builder Automation (Tekton) | Not Included | Included | N/A |
| OpenShift Service Mesh | Not Included | Included | OpenShift Service Mesh Operator |
| Service Binding Operator | Not Included | Included | Service Binding Operator |
| Feature | OpenShift Kubernetes Engine | OpenShift Container Platform | Operator name |
| Red Hat OpenShift Serverless | Not Included | Included | OpenShift Serverless Operator |
| Web Terminal provided by Red Hat | Not Included | Included | Web Terminal Operator |
| Red Hat OpenShift Pipelines Operator | Not Included | Included | OpenShift Pipelines Operator |
| Embedded Component of IBM Cloud® Pak and RHT MW Bundles | Not Included | Included | N/A |
| Red Hat OpenShift GitOps | Not Included | Included | OpenShift GitOps |
| Red Hat OpenShift Dev Spaces | Not Included | Included | Red Hat OpenShift Dev Spaces |
| Red Hat OpenShift Local | Not Included | Included | N/A |
| Quay Bridge Operator provided by Red Hat | Not Included | Included | Quay Bridge Operator |
| Quay Container Security provided by Red Hat | Not Included | Included | Quay Operator |
| Red Hat OpenShift distributed tracing platform | Not Included | Included | Red Hat OpenShift distributed tracing platform Operator |
| Red Hat OpenShift Kiali | Not Included | Included | Kiali Operator |
| Metering provided by Red Hat (deprecated) | Not Included | Included | N/A |
| Migration Toolkit for Containers Operator | Not Included | Included | Migration Toolkit for Containers Operator |
| Cost management for OpenShift | Not included | Included | N/A |
| JBoss Web Server provided by Red Hat | Not included | Included | JWS Operator |
| Red Hat Build of Quarkus | Not included | Included | N/A |
| Kourier Ingress Controller | Not included | Included | N/A |
| RHT Middleware Bundles Sub Compatibility (not included in OpenShift Container Platform) | Not included | Included | N/A |
| IBM Cloud® Pak Sub Compatibility (not included in OpenShift Container Platform) | Not included | Included | N/A |
|
OpenShift Do ( | Not included | Included | N/A |
| Source to Image and Tekton Builders | Not included | Included | N/A |
| OpenShift Serverless FaaS | Not included | Included | N/A |
| IDE Integrations | Not included | Included | N/A |
| OpenShift sandboxed containers | Not included | Not included | OpenShift sandboxed containers Operator |
| Windows Machine Config Operator | Community Windows Machine Config Operator included - no subscription required | Red Hat Windows Machine Config Operator included - Requires separate subscription | Windows Machine Config Operator |
| Red Hat Quay | Not Included - Requires separate subscription | Not Included - Requires separate subscription | Quay Operator |
| Red Hat Advanced Cluster Management | Not Included - Requires separate subscription | Not Included - Requires separate subscription | Advanced Cluster Management for Kubernetes |
| Red Hat Advanced Cluster Security | Not Included - Requires separate subscription | Not Included - Requires separate subscription | N/A |
| OpenShift Data Foundation | Not Included - Requires separate subscription | Not Included - Requires separate subscription | OpenShift Data Foundation |
| Feature | OpenShift Kubernetes Engine | OpenShift Container Platform | Operator name |
| Ansible Automation Platform Resource Operator | Not Included - Requires separate subscription | Not Included - Requires separate subscription | Ansible Automation Platform Resource Operator |
| Business Automation provided by Red Hat | Not Included - Requires separate subscription | Not Included - Requires separate subscription | Business Automation Operator |
| Data Grid provided by Red Hat | Not Included - Requires separate subscription | Not Included - Requires separate subscription | Data Grid Operator |
| Red Hat Integration provided by Red Hat | Not Included - Requires separate subscription | Not Included - Requires separate subscription | Red Hat Integration Operator |
| Red Hat Integration - 3Scale provided by Red Hat | Not Included - Requires separate subscription | Not Included - Requires separate subscription | 3scale |
| Red Hat Integration - 3Scale APICast gateway provided by Red Hat | Not Included - Requires separate subscription | Not Included - Requires separate subscription | 3scale APIcast |
| Red Hat Integration - AMQ Broker | Not Included - Requires separate subscription | Not Included - Requires separate subscription | AMQ Broker |
| Red Hat Integration - AMQ Broker LTS | Not Included - Requires separate subscription | Not Included - Requires separate subscription | |
| Red Hat Integration - AMQ Interconnect | Not Included - Requires separate subscription | Not Included - Requires separate subscription | AMQ Interconnect |
| Red Hat Integration - AMQ Online | Not Included - Requires separate subscription | Not Included - Requires separate subscription | |
| Red Hat Integration - AMQ Streams | Not Included - Requires separate subscription | Not Included - Requires separate subscription | AMQ Streams |
| Red Hat Integration - Camel K | Not Included - Requires separate subscription | Not Included - Requires separate subscription | Camel K |
| Red Hat Integration - Fuse Console | Not Included - Requires separate subscription | Not Included - Requires separate subscription | Fuse Console |
| Red Hat Integration - Fuse Online | Not Included - Requires separate subscription | Not Included - Requires separate subscription | Fuse Online |
| Red Hat Integration - Service Registry Operator | Not Included - Requires separate subscription | Not Included - Requires separate subscription | Service Registry |
| API Designer provided by Red Hat | Not Included - Requires separate subscription | Not Included - Requires separate subscription | API Designer |
| JBoss EAP provided by Red Hat | Not Included - Requires separate subscription | Not Included - Requires separate subscription | JBoss EAP |
| Smart Gateway Operator | Not Included - Requires separate subscription | Not Included - Requires separate subscription | Smart Gateway Operator |
| Kubernetes NMState Operator | Included | Included | N/A |
3.2. Subscription limitations
OpenShift Kubernetes Engine is a subscription offering that provides OpenShift Container Platform with a limited set of supported features at a lower list price. OpenShift Kubernetes Engine and OpenShift Container Platform are the same product and, therefore, all software and features are delivered in both. There is only one download, OpenShift Container Platform. OpenShift Kubernetes Engine uses the OpenShift Container Platform documentation and support services and bug errata for this reason.
