Chapter 2. Windows Container Support for Red Hat OpenShift release notes

2.1. About Windows Container Support for Red Hat OpenShift

Red Hat OpenShift support for Windows Containers enables running Windows compute nodes in an OpenShift Container Platform cluster. Running Windows workloads is possible by using the Red Hat Windows Machine Config Operator (WMCO) to install and manage Windows nodes. With Windows nodes available, you can run Windows container workloads in OpenShift Container Platform.

The release notes for Red Hat OpenShift support for Windows Containers tracks the development of the WMCO, which provides all Windows container workload capabilities in OpenShift Container Platform.

2.2. Getting support

Windows Container Support for Red Hat OpenShift is provided and available as an optional, installable component. Windows Container Support for Red Hat OpenShift is not part of the OpenShift Container Platform subscription. It requires an additional Red Hat subscription and is supported according to the Scope of coverage and Service level agreements.

You must have a separate subscription to receive support for Windows Container Support for Red Hat OpenShift. Without this additional Red Hat subscription, deploying Windows container workloads in production clusters is not supported. You can request support through the Red Hat Customer Portal.

For more information, see the Red Hat OpenShift Container Platform Life Cycle Policy document for Red Hat OpenShift support for Windows Containers.

If you do not have this additional Red Hat subscription, you can use the Community Windows Machine Config Operator, which is a distribution that lacks official support.

2.3. Windows Machine Config Operator prerequisites

The following information details the supported cloud provider versions, Windows Server versions, and networking configurations for the Windows Machine Config Operator. See the vSphere documentation for any information that is relevant to only that platform.

2.3.1. Supported cloud providers based on OpenShift Container Platform and WMCO versions

Cloud provider	Supported OpenShift Container Platform version	Supported WMCO version
Amazon Web Services (AWS)	4.6+	WMCO 1.0+
Microsoft Azure	4.6+	WMCO 1.0+
VMware vSphere	4.7+	WMCO 2.0+

2.3.2. Supported Windows Server versions

The following table lists the supported Windows Server version based on the applicable cloud provider. Any unlisted Windows Server version is not supported and will cause errors. To prevent these errors, only use the appropriate version according to the cloud provider in use.

Cloud provider	Supported Windows Server version
Amazon Web Services (AWS)	Windows Server Long-Term Servicing Channel (LTSC): Windows Server 2019
Microsoft Azure	Windows Server Long-Term Servicing Channel (LTSC): Windows Server 2019
VMware vSphere	Windows Server Semi-Annual Channel (SAC): Windows Server 20H2

2.3.3. Supported networking

Hybrid networking with OVN-Kubernetes is the only supported networking configuration. See the additional resources below for more information on this functionality. The following tables outline the type of networking configuration and Windows Server versions to use based on your cloud provider. You must specify the network configuration when you install the cluster. Be aware that OpenShift SDN networking is the default network for OpenShift Container Platform clusters. However, OpenShift SDN is not supported by WMCO.

Table 2.1. Cloud provider networking support
Cloud provider	Supported networking
Amazon Web Services (AWS)	Hybrid networking with OVN-Kubernetes
Microsoft Azure	Hybrid networking with OVN-Kubernetes
VMware vSphere	Hybrid networking with OVN-Kubernetes with a custom VXLAN port

Table 2.2. Hybrid OVN-Kubernetes Windows Server support
Hybrid networking with OVN-Kubernetes	Supported Windows Server version
Default VXLAN port	Windows Server Long-Term Servicing Channel (LTSC): Windows Server 2019
Custom VXLAN port	Windows Server Semi-Annual Channel (SAC): Windows Server 20H2

2.3.4. Supported installation method

The installer-provisioned infrastructure installation method is the only supported installation method. This is consistent across all supported cloud providers. User-provisioned infrastructure installation method is not supported.

2.4. Release notes for Red Hat Windows Machine Config Operator 2.0.5

Issued: 2022-05-02

The WMCO 2.0.5 is now available with bug fixes. The components of the WMCO were released in link:RHSA-2022:93074.

2.5. Release notes for Red Hat Windows Machine Config Operator 2.0.4

Issued: 2022-02-16

The WMCO 2.0.4 is now available with bug fixes. The components of the WMCO were released in RHBA-2022:0241.

2.5.1. Bug fixes

For clusters installed on VMware vSphere, the WMCO ignored the Deleting phase notification event, leaving incorrect node information in the windows-exporter metrics endpoint. This resulted in an invalid mapping for the Prometheus metrics endpoint. This bug has been fixed; the WMCO now recognizes the Deleting phase notification event and maps the Prometheus metrics endpoint appropriately. (BZ#1995340)

2.6. Release notes for Red Hat Windows Machine Config Operator 2.0.3

Issued: 2021-07-28

The WMCO 2.0.3 is now available with bug fixes. The components of the WMCO were released in RHBA-2021:2926.

2.6.1. Bug fixes

This WMCO release fixes a bug that prevented users from upgrading to WMCO 3.0.0. Users should upgrade to WMCO 2.0.3 before upgrading to OpenShift Container Platform 4.8, which only supports WMCO 3.0.0. (BZ#1985349)

2.7. Release notes for Red Hat Windows Machine Config Operator 2.0.2

Issued: 2021-07-08

The WMCO 2.0.2 is now available with bug fixes. The components of the WMCO were released in RHBA-2021:2671.

Important

Users who are running a version of WMCO prior to 2.0.3 should first upgrade to WMCO 2.0.3 prior to upgrading to WMCO 3.0.0. (BZ#1983153)

2.7.1. Bug fixes

OpenShift Container Platform 4.8 enables the BoundServiceAccountTokenVolume option by default. This option attaches the projected volumes to all of the pods. In addition, OpenShift Container Platform 4.8 adds the RunAsUser option to the SecurityContext. This combination results in Windows pods being stuck in the ContainerCreating status. To work around this issue, you should upgrade to WMCO 2.0.2 before upgrading your cluster to OpenShift Container Platform 4.8. (BZ#1975553)

2.8. Release notes for Red Hat Windows Machine Config Operator 2.0.1

Issued: 2021-06-23

The WMCO 2.0.1 is now available with bug fixes. The components of the WMCO were released in RHSA-2021:2130.

2.8.1. New features and improvements

This release adds the following new features and improvements.

2.8.1.1. Increased image pull time-out duration

Image pull time-out has been increased to 30 minutes.

2.8.1.2. Autoscaling for Windows instances

Cluster autoscaling is now supported for Windows instances. You can complete the following actions for Windows nodes:

Define and deploy a cluster autoscaler.
Create a Windows node using a Windows machine set.
Define and deploy a machine autoscaler, referencing a Windows machine set.

2.8.2. Bug fixes

Previously, when using the Windows kube-proxy component on an AWS installation, when you created a LoadBalancer service, packets would be misrouted and reached an unintended destination. Now, packets are no longer wrongly routed to unintended destinations. (BZ#1946538)
Previously, Windows nodes were not reporting some key node-level metrics via telemetry monitoring. The windows_exporter reports various metrics as windows_* instead of the node_exporter equivalent of node_*. Now, the telemetry results cover all of the expected metrics. (BZ#1955319)
Previously, when the WMCO configured Windows instances, if the hybrid-overlay or kube-proxy components failed, the node might report itself as Ready. Now, the error is detected and the node reports itself as NotReady. (BZ#1956412)
Previously, the kube-proxy service would terminate unexpectedly after the load balancer is created if you created the load balancer after the Windows pods begin running. Now, the kube-proxy service does not crash when recreating the load balancer service. (BZ#1939968)

2.8.3. RHSA-2021:2130 - Windows Container support for OpenShift Container Platform security update

As part of the previously noted bug fix (BZ#1946538), an update for Windows kube-proxy is now available for Red Hat Windows Machine Config Operator 2.0.1. Details of the update are documented in the RHSA-2021:2130 advisory.

2.9. Release notes for Red Hat Windows Machine Config Operator 2.0.0

This release of the WMCO provides bug fixes and enhancements for running Windows compute nodes in an OpenShift Container Platform cluster. The components of the WMCO 2.0.0 were released in RHBA-2021:0440.

Important

Running Windows container workloads is not supported for clusters in a restricted network or disconnected environment.

Version 2.x of the WMCO is only compatible with OpenShift Container Platform 4.7.

2.9.1. New features and improvements

This release adds the following new features and improvements.

2.9.1.1. Support for clusters running on VMware vSphere

You can now run Windows nodes on a cluster installed on VMware vSphere version 6.5, 6.7, or 7.0. You can create a Windows MachineSet object on vSphere to host Windows Server compute nodes. For more information, see Creating a Windows MachineSet object on vSphere.

2.9.1.2. Enhanced Windows node monitoring

Windows nodes are now fully integrated with most of the monitoring capabilities provided by the web console. However, it is not possible to view workload graphs for pods running on Windows nodes in this release.

2.10. Known issues

When you create Windows pods with RunAsUserName set in its "SecurityContext" with a projected volume associated with these pods, the file ownership permissions for the projected entities are ignored, resulting in incorrectly configured ownership permissions.
The filesystem graphs available in the web console do not display for Windows nodes. This is caused by changes in the filesystem queries. This will be fixed in a future release of WMCO. (BZ#1930347)
The Prometheus windows_exporter used by the WMCO currently collects metrics through HTTP, so it is considered unsafe. You must ensure that only trusted users can retrieve metrics from the endpoint. The windows_exporter feature recently added support for HTTPS configuration, but this configuration has not been implemented for WMCO. Support for HTTPS configuration in the WMCO will be added in a future release.
When the RunAsUser permission is set in the security context of a Linux-based pod, the projected files have the correct permissions set, including container user ownership. However, when the Windows equivalent RunAsUsername permission is set in a Windows pod, the kubelet is prevented from setting correct ownership on the files in the projected volume. This problem can get exacerbated when used in conjunction with a hostPath volume where best practices are not followed. For example, giving a pod access to the C:\var\lib\kubelet\pods\ folder results in that pod being able to access service account tokens from other pods.
By default, the projected files will have the following ownership, as shown in this example Windows projected volume file:
```
Path   : Microsoft.PowerShell.Core\FileSystem::C:\var\run\secrets\kubernetes.io\serviceaccount\..2021_08_31_22_22_18.318230061\ca.crt
Owner  : BUILTIN\Administrators
Group  : NT AUTHORITY\SYSTEM
Access : NT AUTHORITY\SYSTEM Allow  FullControl
         BUILTIN\Administrators Allow  FullControl
         BUILTIN\Users Allow  ReadAndExecute, Synchronize
Audit  :
Sddl   : O:BAG:SYD:AI(A;ID;FA;;;SY)(A;ID;FA;;;BA)(A;ID;0x1200a9;;;BU)
```
This indicates all administrator users, such as someone with the ContainerAdministrator role, have read, write, and execute access, while non-administrator users have read and execute access.
Important
OpenShift Container Platform applies the RunAsUser security context to all pods irrespective of its operating system. This means Windows pods automatically have the RunAsUser permission applied to its security context.
In addition, if a Windows pod is created with a projected volume with the default RunAsUser permission set, the pod remains in the ContainerCreating phase.
To handle these issues, OpenShift Container Platform forces the file permission handling in projected service account volumes set in the security context of the pod to not be honored for projected volumes on Windows. Note that this behavior for Windows pods is how file permission handling used to work for all pod types prior to OpenShift Container Platform 4.7. (BZ#1971745)

2.11. Known limitations

Note the following limitations when working with Windows nodes managed by the WMCO (Windows nodes):

The following OpenShift Container Platform features are not supported on Windows nodes:
- Red Hat OpenShift Developer CLI (odo)
- Image builds
- OpenShift Pipelines
- OpenShift Service Mesh
- OpenShift monitoring of user-defined projects
- OpenShift Serverless
- Horizontal Pod Autoscaling
- Vertical Pod Autoscaling
The following Red Hat features are not supported on Windows nodes:
- Red Hat cost management
- Red Hat OpenShift Local
Windows nodes do not support pulling container images from private registries. You can use images from public registries or pre-pull the images.
Windows nodes do not support workloads created by using deployment configs. You can use a deployment or other method to deploy workloads.
Windows nodes are not supported in clusters that use a cluster-wide proxy. This is because the WMCO is not able to route traffic through the proxy connection for the workloads.
Windows nodes are not supported in clusters that are in a disconnected environment.
Red Hat OpenShift support for Windows Containers supports only in-tree storage drivers for all cloud providers.
Kubernetes has identified the following node feature limitations :
- Huge pages are not supported for Windows containers.
- Privileged containers are not supported for Windows containers.
- Pod termination grace periods require the containerd container runtime to be installed on the Windows node.
Kubernetes has identified several API compatibility issues.