Este contenido no está disponible en el idioma seleccionado.
Chapter 2. Primary networks
2.1. About user-defined networks Copiar enlaceEnlace copiado en el portapapeles!
Before the implementation of user-defined networks (UDN), the OVN-Kubernetes CNI plugin for OpenShift Container Platform only supported a Layer 3 topology on the primary or main network. Due to Kubernetes design principles: all pods are attached to the main network, all pods communicate with each other by their IP addresses, and inter-pod traffic is restricted according to network policy.
While the Kubernetes design is useful for simple deployments, this Layer 3 topology restricts customization of primary network segment configurations, especially for modern multi-tenant deployments.
UDN improves the flexibility and segmentation capabilities of the default Layer 3 topology for a Kubernetes pod network by enabling custom Layer 2 and Layer 3 network segments, where all these segments are isolated by default. These segments act as either primary or secondary networks for container pods and virtual machines that use the default OVN-Kubernetes CNI plugin. UDNs enable a wide range of network architectures and topologies, enhancing network flexibility, security, and performance.
A cluster administrator can use a UDN to create and define primary or secondary networks that span multiple namespaces at the cluster level by leveraging the ClusterUserDefinedNetwork
custom resource (CR). Additionally, a cluster administrator or a cluster user can use a UDN to define secondary networks at the namespace level with the UserDefinedNetwork
CR.
The following sections further emphasize the benefits and limitations of user-defined networks, the best practices when creating a ClusterUserDefinedNetwork
or UserDefinedNetwork
CR, how to create the CR, and additional configuration details that might be relevant to your deployment.
2.1.1. Benefits of a user-defined network Copiar enlaceEnlace copiado en el portapapeles!
User-defined networks provide the following benefits:
Enhanced network isolation for security
- Tenant isolation: Namespaces can have their own isolated primary network, similar to how tenants are isolated in Red Hat OpenStack Platform (RHOSP). This improves security by reducing the risk of cross-tenant traffic.
Network flexibility
- Layer 2 and layer 3 support: Cluster administrators can configure primary networks as layer 2 or layer 3 network types.
Simplified network management
- Reduced network configuration complexity: With user-defined networks, the need for complex network policies are eliminated because isolation can be achieved by grouping workloads in different networks.
Advanced capabilities
- Consistent and selectable IP addressing: Users can specify and reuse IP subnets across different namespaces and clusters, providing a consistent networking environment.
- Support for multiple networks: The user-defined networking feature allows administrators to connect multiple namespaces to a single network, or to create distinct networks for different sets of namespaces.
Simplification of application migration from Red Hat OpenStack Platform (RHOSP)
- Network parity: With user-defined networking, the migration of applications from OpenStack to OpenShift Container Platform is simplified by providing similar network isolation and configuration options.
Developers and administrators can create a user-defined network that is namespace scoped using the custom resource. An overview of the process is as follows:
-
An administrator creates a namespace for a user-defined network with the
k8s.ovn.org/primary-user-defined-network
label. -
The
UserDefinedNetwork
CR is created by either the cluster administrator or the user. - The user creates pods in the namespace.
2.1.2. Limitations of a user-defined network Copiar enlaceEnlace copiado en el portapapeles!
While user-defined networks (UDN) offer highly customizable network configuration options, there are limitations that cluster administrators and developers should be aware of when implementing and managing these networks. Consider the following limitations before implementing a UDN.
DNS limitations:
- DNS lookups for pods resolve to the pod’s IP address on the cluster default network. Even if a pod is part of a user-defined network, DNS lookups will not resolve to the pod’s IP address on that user-defined network. However, DNS lookups for services and external entities will function as expected.
- When a pod is assigned to a primary UDN, it can access the Kubernetes API (KAPI) and DNS services on the cluster’s default network.
- Initial network assignment: You must create the namespace and network before creating pods. Assigning a namespace with pods to a new network or creating a UDN in an existing namespace will not be accepted by OVN-Kubernetes.
- Health check limitations: Kubelet health checks are performed by the cluster default network, which does not confirm the network connectivity of the primary interface on the pod. Consequently, scenarios where a pod appears healthy by the default network, but has broken connectivity on the primary interface, are possible with user-defined networks.
- Network policy limitations: Network policies that enable traffic between namespaces connected to different user-defined primary networks are not effective. These traffic policies do not take effect because there is no connectivity between these isolated networks.
-
Creation and modification limitation: The
ClusterUserDefinedNetwork
CR and theUserDefinedNetwork
CR cannot be modified after being created. -
Default network service access: A user-defined network pod is isolated from the default network, which means that most default network services are inaccessible. For example, a user-defined network pod cannot currently access the OpenShift Container Platform image registry. Because of this limitation, source-to-image builds do not work in a user-defined network namespace. Additionally, other functions do not work, including functions to create applications based on the source code in a Git repository, such as
oc new-app <command>
, and functions to create applications from an OpenShift Container Platform template that use source-to-image builds. This limitation might also affect otheropenshift-*.svc
services. - Connectivity limitation: NodePort services on user-defined networks are not guaranteed isolation. For example, NodePort traffic from a pod to a service on the same node is not accessible, whereas traffic from a pod on a different node succeeds.
-
Unclear error message for IP address exhaustion: When the subnet of a user-defined network runs out of available IP addresses, new pods fail to start. When this occurs, the following error is returned:
Warning: Failed to create pod sandbox
. This error message does not clearly specify that IP depletion is the cause. To confirm the issue, you can check the Events page in the pod’s namespace on the OpenShift Container Platform web console, where an explicit message about subnet exhaustion is reported.
2.1.3. Layer 2 and layer 3 topologies Copiar enlaceEnlace copiado en el portapapeles!
A flat layer 2 topology creates a virtual switch that is distributed across all nodes in a cluster. Virtual machines and pods connect to this virtual switch so that all these components can communicate with each other within the same subnet. A flat layer 2 topology is useful for live migration of virtual machines across nodes that exist in a cluster. The following diagram shows a flat layer 2 topology with two nodes that use the virtual switch for live migration purposes:
Figure 2.1. A flat layer 2 topology that uses a virtual switch for component communication
If you decide not to specify a layer 2 subnet, then you must manually configure IP addresses for each pod in your cluster. When you do not specify a layer 2 subnet, port security is limited to preventing Media Access Control (MAC) spoofing only, and does not include IP spoofing. A layer 2 topology creates a single broadcast domain that can be challenging in large network environments, where the topology might cause a broadcast storm that can degrade network performance.
To access more configurable options for your network, you can integrate a layer 2 topology with a user-defined network (UDN). The following diagram shows two nodes that use a UDN with a layer 2 topology that includes pods that exist on each node. Each node includes two interfaces:
- A node interface, which is a compute node that connects networking components to the node.
-
An Open vSwitch (OVS) bridge such as
br-ex
, which creates an layer 2 OVN switch so that pods can communicate with each other and share resources.
An external switch connects these two interfaces, while the gateway or router handles routing traffic between the external switch and the layer 2 OVN switch. VMs and pods in a node can use the UDN to communicate with each other. The layer 2 OVN switch handles node traffic over a UDN so that live migrate of a VM from one node to another is possible.
Figure 2.2. A user-defined network (UDN) that uses a layer 2 topology
A layer 3 topology creates a unique layer 2 segment for each node in a cluster. The layer 3 routing mechanism interconnects these segments so that virtual machines and pods that are hosted on different nodes can communicate with each other. A layer 3 topology can effectively manage large broadcast domains by assigning each domain to a specific node, so that broadcast traffic has a reduced scope. To configure a layer 3 topology, you must configure cidr
and hostSubnet
parameters.
2.1.4. About the ClusterUserDefinedNetwork CR Copiar enlaceEnlace copiado en el portapapeles!
The ClusterUserDefinedNetwork
(UDN) custom resource (CR) provides cluster-scoped network segmentation and isolation for administrators only.
The following diagram demonstrates how a cluster administrator can use the ClusterUserDefinedNetwork
CR to create network isolation between tenants. This network configuration allows a network to span across many namespaces. In the diagram, network isolation is achieved through the creation of two user-defined networks, udn-1
and udn-2
. These networks are not connected and the spec.namespaceSelector.matchLabels
field is used to select different namespaces. For example, udn-1
configures and isolates communication for namespace-1
and namespace-2
, while udn-2
configures and isolates communication for namespace-3
and namespace-4
. Isolated tenants (Tenants 1 and Tenants 2) are created by separating namespaces while also allowing pods in the same namespace to communicate.
Figure 2.3. Tenant isolation using a ClusterUserDefinedNetwork CR
2.1.4.1. Best practices for ClusterUserDefinedNetwork CRs Copiar enlaceEnlace copiado en el portapapeles!
Before setting up a ClusterUserDefinedNetwork
custom resource (CR), users should consider the following information:
-
A
ClusterUserDefinedNetwork
CR is intended for use by cluster administrators and should not be used by non-administrators. If used incorrectly, it might result in security issues with your deployment, cause disruptions, or break the cluster network. -
ClusterUserDefinedNetwork
CRs should not select thedefault
namespace. This can result in no isolation and, as a result, could introduce security risks to the cluster. -
ClusterUserDefinedNetwork
CRs should not selectopenshift-*
namespaces. OpenShift Container Platform administrators should be aware that all namespaces of a cluster are selected when one of the following conditions are met:
-
The
matchLabels
selector is left empty. -
The
matchExpressions
selector is left empty. -
The
namespaceSelector
is initialized, but does not specifymatchExpressions
ormatchLabel
. For example:namespaceSelector: {}
.
-
The
For primary networks, the namespace used for the
ClusterUserDefinedNetwork
CR must include thek8s.ovn.org/primary-user-defined-network
label. This label cannot be updated, and can only be added when the namespace is created. The following conditions apply with thek8s.ovn.org/primary-user-defined-network
namespace label:-
If the namespace is missing the
k8s.ovn.org/primary-user-defined-network
label and a pod is created, the pod attaches itself to the default network. -
If the namespace is missing the
k8s.ovn.org/primary-user-defined-network
label and a primaryClusterUserDefinedNetwork
CR is created that matches the namespace, an error is reported and the network is not created. -
If the namespace is missing the
k8s.ovn.org/primary-user-defined-network
label and a primaryClusterUserDefinedNetwork
CR already exists, a pod in the namespace is created and attached to the default network. -
If the namespace has the label, and a primary
ClusterUserDefinedNetwork
CR does not exist, a pod in the namespace is not created until theClusterUserDefinedNetwork
CR is created.
-
If the namespace is missing the
When using the
ClusterUserDefinedNetwork
CR to createlocalnet
topology, the following are best practices for administrators:-
You must make sure that the
spec.network.physicalNetworkName
parameter matches the parameter that you configured in the Open vSwitch (OVS) bridge mapping when you create your CUDN CR. This ensures that you are bridging to the intended segment of your physical network. If you intend to deploy multiple CUDN CR using the same bridge mapping, you must ensure that the samephysicalNetworkName
parameter is used. -
Avoid overlapping subnets between your physical network and your other network interfaces. Overlapping network subnets can cause routing conflicts and network instability. To prevent conflicts when using the
spec.network.localnet.subnets
parameter, you might use thespec.network.localnet.excludeSubnets
parameter. -
When you configure a Virtual Local Area Network (VLAN), you must ensure that both your underlying physical infrastructure (switches, routers, and so on) and your nodes are properly configured to accept VLAN IDs (VIDs). This means that you configure the physical network interface, for example
eth1
, as an access port for the VLAN, for example20
, that you are connecting to through the physical switch. In addition, you must verify that an OVS bridge mapping, for exampleeth1
, exists on your nodes to ensure that the physical interface is properly connected with OVN-Kubernetes.
-
You must make sure that the
2.1.4.2. Creating a ClusterUserDefinedNetwork CR by using the CLI Copiar enlaceEnlace copiado en el portapapeles!
The following procedure creates a ClusterUserDefinedNetwork
custom resource (CR) by using the CLI. Based upon your use case, create your request using either the cluster-layer-two-udn.yaml
example for a Layer2
topology type or the cluster-layer-three-udn.yaml
example for a Layer3
topology type.
-
The
ClusterUserDefinedNetwork
CR is intended for use by cluster administrators and should not be used by non-administrators. If used incorrectly, it might result in security issues with your deployment, cause disruptions, or break the cluster network. -
OpenShift Virtualization only supports the
Layer2
andLocalnet
topologies.
Prerequisites
-
You have logged in as a user with
cluster-admin
privileges.
Procedure
Optional: For a
ClusterUserDefinedNetwork
CR that uses a primary network, create a namespace with thek8s.ovn.org/primary-user-defined-network
label by entering the following command:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a cluster-wide user-defined network for either a
Layer2
orLayer3
topology type:Create a YAML file, such as
cluster-layer-two-udn.yaml
, to define your request for aLayer2
topology as in the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- Name of your
ClusterUserDefinedNetwork
CR. - 2
- A label query over the set of namespaces that the CUDN CR applies to. Uses the standard Kubernetes
MatchLabel
selector. Must not point todefault
oropenshift-*
namespaces. - 3
- Uses the
matchLabels
selector type, where terms are evaluated with anAND
relationship. - 4 5
- In this example, the CUDN CR is deployed to namespaces that contain both
<label_1_key>=<label_1_value>
and<label_2_key>=<label_2_value>
labels. - 6
- Describes the network configuration.
- 7
- The
topology
field describes the network configuration; accepted values areLayer2
andLayer3
. Specifying aLayer2
topology type creates one logical switch that is shared by all nodes. - 8
- This field specifies the topology configuration. It can be
layer2
orlayer3
. - 9
- Specifies
Primary
orSecondary
.Primary
is the onlyrole
specification supported in 4.19. - 10
- For
Layer2
topology types the following specifies config details for thesubnet
field:- The subnets field is optional.
-
The subnets field is of type
string
and accepts standard CIDR formats for both IPv4 and IPv6. -
The subnets field accepts one or two items. For two items, they must be of a different family. For example, subnets values of
10.100.0.0/16
and2001:db8::/64
. -
Layer2
subnets can be omitted. If omitted, users must configure static IP addresses for the pods. As a consequence, port security only prevents MAC spoofing. For more information, see "Configuring pods with a static IP address".
Create a YAML file, such as
cluster-layer-three-udn.yaml
, to define your request for aLayer3
topology as in the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- Name of your
ClusterUserDefinedNetwork
CR. - 2
- A label query over the set of namespaces that the cluster UDN applies to. Uses the standard Kubernetes
MatchLabel
selector. Must not point todefault
oropenshift-*
namespaces. - 3
- Uses the
matchExpressions
selector type, where terms are evaluated with anOR
relationship. - 4
- Specifies the label key to match.
- 5
- Specifies the operator. Valid values include:
In
,NotIn
,Exists
, andDoesNotExist
. - 6
- Because the
matchExpressions
type is used, provisions namespaces matching either<example_namespace_one>
or<example_namespace_two>
. - 7
- Describes the network configuration.
- 8
- The
topology
field describes the network configuration; accepted values areLayer2
andLayer3
. Specifying aLayer3
topology type creates a layer 2 segment per node, each with a different subnet. Layer 3 routing is used to interconnect node subnets. - 9
- This field specifies the topology configuration. Valid values are
layer2
orlayer3
. - 10
- Specifies a
Primary
orSecondary
role.Primary
is the onlyrole
specification supported in 4.19. - 11
- For
Layer3
topology types the following specifies config details for thesubnet
field:-
The
subnets
field is mandatory. The type for the
subnets
field iscidr
andhostSubnet
:-
cidr
is the cluster subnet and accepts a string value. -
hostSubnet
specifies the nodes subnet prefix that the cluster subnet is split to. -
For IPv6, only a
/64
length is supported forhostSubnet
.
-
-
The
Apply your request by running the following command:
oc create --validate=true -f <example_cluster_udn>.yaml
$ oc create --validate=true -f <example_cluster_udn>.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Where
<example_cluster_udn>.yaml
is the name of yourLayer2
orLayer3
configuration file.Verify that your request is successful by running the following command:
oc get clusteruserdefinednetwork <cudn_name> -o yaml
$ oc get clusteruserdefinednetwork <cudn_name> -o yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Where
<cudn_name>
is the name you created of your cluster-wide user-defined network.Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
2.1.4.3. Creating a ClusterUserDefinedNetwork CR for a Localnet topology Copiar enlaceEnlace copiado en el portapapeles!
A Localnet
topology connects the secondary network to the physical underlay. This enables both east-west cluster traffic and access to services running outside the cluster. This topology type requires the additional configuration of the underlying Open vSwitch (OVS) system on cluster nodes.
Prerequisites
-
You are logged in as a user with
cluster-admin
privileges. - You created and configured the Open vSwitch (OVS) bridge mapping to associate the logical OVN-Kubernetes network with the physical node network through the OVS bridge. For more information, see "Configuration for a localnet switched topology".
Procedure
Create a cluster-wide user-defined network with a
Localnet
topology:Create a YAML file, such as
cluster-udn-localnet.yaml
, to define your request for aLocalnet
topology as in the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- Name of your
ClusterUserDefinedNetwork
(CUDN) CR. - 2
- A label query over the set of namespaces that the cluster CUDN CR applies to. Uses the standard Kubernetes
MatchLabel
selector. Must not point todefault
,openshift-*
, or any other system namespaces. - 3
- Uses the
matchLabels
selector type, where terms are evaluated with anAND
relationship. - 4 5
- In this example, the CUDN CR is deployed to namespaces that contain both
<label_1_key>=<label_1_value>
and<label_2_key>=<label_2_value>
labels. - 6
- Describes the network configuration.
- 7
- Specifying a
Localnet
topology type creates one logical switch that is directly bridged to one provider network. - 8
- This field specifies the
localnet
topology. - 9
- Specifies the
role
for the network configuration.Secondary
is the onlyrole
specification supported for thelocalnet
topology. - 10
- For
Localnet
topology types the following specifies config details for thesubnet
field:- The subnets field is optional.
-
The subnets field is of type
string
and accepts standard CIDR formats for both IPv4 and IPv6. -
The subnets field accepts one or two items. For two items, they must be of a different IP family. For example, subnets values of
10.100.0.0/16
and2001:db8::/64
. -
localnet
subnets can be omitted. If omitted, users must configure static IP addresses for the pods. As a consequence, port security only prevents MAC spoofing. For more information, see "Configuring pods with a static IP address".
Apply your request by running the following command:
oc create --validate=true -f <example_cluster_udn>.yaml
$ oc create --validate=true -f <example_cluster_udn>.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow where:
<example_cluster_udn>.yaml
-
Is the name of your
Localnet
configuration file.
Verify that your request is successful by running the following command:
oc get clusteruserdefinednetwork <cudn_name> -o yaml
$ oc get clusteruserdefinednetwork <cudn_name> -o yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow where:
<cudn_name>
- Is the name you created of your cluster-wide user-defined network.
Example 2.1. Example output
2.1.4.4. Creating a ClusterUserDefinedNetwork CR by using the web console Copiar enlaceEnlace copiado en el portapapeles!
You can create a ClusterUserDefinedNetwork
custom resource (CR) with a Layer2
topology in the OpenShift Container Platform web console.
Currently, creation of a ClusterUserDefinedNetwork
CR with a Layer3
topology is not supported when using the OpenShift Container Platform web console.
Prerequisites
-
You have access to the OpenShift Container Platform web console as a user with
cluster-admin
permissions. -
You have created a namespace and applied the
k8s.ovn.org/primary-user-defined-network
label.
Procedure
-
From the Administrator perspective, click Networking
UserDefinedNetworks. - Click ClusterUserDefinedNetwork.
- In the Name field, specify a name for the cluster-scoped UDN.
- Specify a value in the Subnet field.
- In the Project(s) Match Labels field, add the appropriate labels to select namespaces that the cluster UDN applies to.
- Click Create. The cluster-scoped UDN serves as the default primary network for pods located in namespaces that contain the labels that you specified in step 5.
2.1.5. About the UserDefinedNetwork CR Copiar enlaceEnlace copiado en el portapapeles!
The UserDefinedNetwork
(UDN) custom resource (CR) provides advanced network segmentation and isolation for users and administrators.
The following diagram shows four cluster namespaces, where each namespace has a single assigned user-defined network (UDN), and each UDN has an assigned custom subnet for its pod IP allocations. The OVN-Kubernetes handles any overlapping UDN subnets. Without using the Kubernetes network policy, a pod attached to a UDN can communicate with other pods in that UDN. By default, these pods are isolated from communicating with pods that exist in other UDNs. For microsegmentation, you can apply network policy within a UDN. You can assign one or more UDNs to a namespace, with a limitation of only one primary UDN to a namespace, and one or more namespaces to a UDN.
Figure 2.4. Namespace isolation using a UserDefinedNetwork CR
2.1.5.1. Best practices for UserDefinedNetwork CRs Copiar enlaceEnlace copiado en el portapapeles!
Before setting up a UserDefinedNetwork
custom resource (CR), you should consider the following information:
-
openshift-*
namespaces should not be used to set up aUserDefinedNetwork
CR. -
UserDefinedNetwork
CRs should not be created in the default namespace. This can result in no isolation and, as a result, could introduce security risks to the cluster. For primary networks, the namespace used for the
UserDefinedNetwork
CR must include thek8s.ovn.org/primary-user-defined-network
label. This label cannot be updated, and can only be added when the namespace is created. The following conditions apply with thek8s.ovn.org/primary-user-defined-network
namespace label:-
If the namespace is missing the
k8s.ovn.org/primary-user-defined-network
label and a pod is created, the pod attaches itself to the default network. -
If the namespace is missing the
k8s.ovn.org/primary-user-defined-network
label and a primaryUserDefinedNetwork
CR is created that matches the namespace, a status error is reported and the network is not created. -
If the namespace is missing the
k8s.ovn.org/primary-user-defined-network
label and a primaryUserDefinedNetwork
CR already exists, a pod in the namespace is created and attached to the default network. -
If the namespace has the label, and a primary
UserDefinedNetwork
CR does not exist, a pod in the namespace is not created until theUserDefinedNetwork
CR is created.
-
If the namespace is missing the
2 masquerade IP addresses are required for user defined networks. You must reconfigure your masquerade subnet to be large enough to hold the required number of networks.
Important-
For OpenShift Container Platform 4.17 and later, clusters use
169.254.0.0/17
for IPv4 andfd69::/112
for IPv6 as the default masquerade subnet. These ranges should be avoided by users. For updated clusters, there is no change to the default masquerade subnet. -
Changing the cluster’s masquerade subnet is unsupported after a user-defined network has been configured for a project. Attempting to modify the masquerade subnet after a
UserDefinedNetwork
CR has been set up can disrupt the network connectivity and cause configuration issues.
-
For OpenShift Container Platform 4.17 and later, clusters use
-
Ensure tenants are using the
UserDefinedNetwork
resource and not theNetworkAttachmentDefinition
(NAD) CR. This can create security risks between tenants. -
When creating network segmentation, you should only use the
NetworkAttachmentDefinition
CR if user-defined network segmentation cannot be completed using theUserDefinedNetwork
CR. -
The cluster subnet and services CIDR for a
UserDefinedNetwork
CR cannot overlap with the default cluster subnet CIDR. OVN-Kubernetes network plugin uses100.64.0.0/16
as the default join subnet for the network. You must not use that value to configure aUserDefinedNetwork
CR’sjoinSubnets
field. If the default address values are used anywhere in the network for the cluster you must override the default values by setting thejoinSubnets
field. For more information, see "Additional configuration details for user-defined networks".
2.1.5.2. Creating a UserDefinedNetwork CR by using the CLI Copiar enlaceEnlace copiado en el portapapeles!
The following procedure creates a UserDefinedNetwork
CR that is namespace scoped. Based upon your use case, create your request by using either the my-layer-two-udn.yaml
example for a Layer2
topology type or the my-layer-three-udn.yaml
example for a Layer3
topology type.
Perquisites
-
You have logged in with
cluster-admin
privileges, or you haveview
andedit
role-based access control (RBAC).
Procedure
Optional: For a
UserDefinedNetwork
CR that uses a primary network, create a namespace with thek8s.ovn.org/primary-user-defined-network
label by entering the following command:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a user-defined network for either a
Layer2
orLayer3
topology type:Create a YAML file, such as
my-layer-two-udn.yaml
, to define your request for aLayer2
topology as in the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- Name of your
UserDefinedNetwork
resource. This should not bedefault
or duplicate any global namespaces created by the Cluster Network Operator (CNO). - 2
- The
topology
field describes the network configuration; accepted values areLayer2
andLayer3
. Specifying aLayer2
topology type creates one logical switch that is shared by all nodes. - 3
- This field specifies the topology configuration. It can be
layer2
orlayer3
. - 4
- Specifies a
Primary
orSecondary
role. - 5
- For
Layer2
topology types the following specifies config details for thesubnet
field:- The subnets field is optional.
-
The subnets field is of type
string
and accepts standard CIDR formats for both IPv4 and IPv6. -
The subnets field accepts one or two items. For two items, they must be of a different family. For example, subnets values of
10.100.0.0/16
and2001:db8::/64
. -
Layer2
subnets can be omitted. If omitted, users must configure IP addresses for the pods. As a consequence, port security only prevents MAC spoofing. -
The
Layer2
subnets
field is mandatory when theipamLifecycle
field is specified.
Create a YAML file, such as
my-layer-three-udn.yaml
, to define your request for aLayer3
topology as in the following example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- Name of your
UserDefinedNetwork
resource. This should not bedefault
or duplicate any global namespaces created by the Cluster Network Operator (CNO). - 2
- The
topology
field describes the network configuration; accepted values areLayer2
andLayer3
. Specifying aLayer3
topology type creates a layer 2 segment per node, each with a different subnet. Layer 3 routing is used to interconnect node subnets. - 3
- This field specifies the topology configuration. Valid values are
layer2
orlayer3
. - 4
- Specifies a
Primary
orSecondary
role. - 5
- For
Layer3
topology types the following specifies config details for thesubnet
field:-
The
subnets
field is mandatory. The type for the
subnets
field iscidr
andhostSubnet
:-
cidr
is equivalent to theclusterNetwork
configuration settings of a cluster. The IP addresses in the CIDR are distributed to pods in the user defined network. This parameter accepts a string value. -
hostSubnet
defines the per-node subnet prefix. -
For IPv6, only a
/64
length is supported forhostSubnet
.
-
-
The
Apply your request by running the following command:
oc apply -f <my_layer_two_udn>.yaml
$ oc apply -f <my_layer_two_udn>.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Where
<my_layer_two_udn>.yaml
is the name of yourLayer2
orLayer3
configuration file.Verify that your request is successful by running the following command:
oc get userdefinednetworks udn-1 -n <some_custom_namespace> -o yaml
$ oc get userdefinednetworks udn-1 -n <some_custom_namespace> -o yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Where
some_custom_namespace
is the namespace you created for your user-defined network.Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
2.1.5.3. Creating a UserDefinedNetwork CR by using the web console Copiar enlaceEnlace copiado en el portapapeles!
You can create a UserDefinedNetwork
custom resource (CR) with a Layer2
topology and Primary
role by using the OpenShift Container Platform web console.
Currently, creation of a UserDefinedNetwork
CR with a Layer3
topology or a Secondary
role are not supported when using the OpenShift Container Platform web console.
Prerequisites
-
You have access to the OpenShift Container Platform web console as a user with
cluster-admin
permissions. -
You have created a namespace and applied the
k8s.ovn.org/primary-user-defined-network
label.
Procedure
-
From the Administrator perspective, click Networking
UserDefinedNetworks. - Click Create UserDefinedNetwork.
- From the Project name list, select the namespace that you previously created.
- Specify a value in the Subnet field.
- Click Create. The user-defined network serves as the default primary network for pods that you create in this namespace.
2.1.6. Additional configuration details for user-defined networks Copiar enlaceEnlace copiado en el portapapeles!
The following table explains additional configurations for ClusterUserDefinedNetwork
and UserDefinedNetwork
custom resources (CRs) that are optional. It is not recommended to set these fields without explicit need and understanding of OVN-Kubernetes network topology.
- Optional configurations for user-defined networks
CUDN field | UDN field | Type | Description |
|
| object |
When omitted, the platform sets default values for the
The |
|
| string |
Specifies a list of CIDRs to be removed from the specified CIDRs in the
When deploying a secondary network with |
|
| object |
The
Setting a value of Persistent is only supported when |
|
| object |
The
Enabled:
Disabled: |
|
| integer |
The maximum transmission units (MTU). The default value is |
| N/A | object | This field is optional and configures the virtual local area network (VLAN) tagging and allows you to segment the physical network into multiple independent broadcast domains. |
| N/A | object |
Acceptable values are |
| N/A | string |
Specifies the name for a physical network interface. The value you specify must match the |
where:
<topology>
-
Can be either
layer2
orlayer3
for theUserDefinedNetwork
CR. For theClusterUserDefinedNetwork
CR the topology can also beLocalnet
.
2.1.7. User-defined network status condition types Copiar enlaceEnlace copiado en el portapapeles!
The following tables explain the status condition types returned for ClusterUserDefinedNetwork
and UserDefinedNetwork
CRs when describing the resource. These conditions can be used to troubleshoot your deployment.
Condition type | Status | Reason and Message | |
---|---|---|---|
|
|
When | |
Reason | Message | ||
| 'NetworkAttachmentDefinition has been created in following namespaces: [example-namespace-1, example-namespace-2, example-namespace-3]'` | ||
|
|
When | |
Reason | Message | ||
|
| ||
|
| ||
|
| ||
|
| ||
|
| ||
|
| ||
|
|
Condition type | Status | Reason and Message | |
---|---|---|---|
|
|
When | |
Reason | Message | ||
|
| ||
|
|
When | |
Reason | Message | ||
|
|
Condition type | Reason, Message, Resolution | ||
---|---|---|---|
|
One of the following messages is returned when the | ||
Reason | Message | Resolution | |
The |
|
You must set the | |
The |
|
You must set the | |
The |
|
You must set the |
Condition type | Reason, Message, Resolution | ||
---|---|---|---|
|
One of the following messages is returned when the | ||
Reason | Message | Resolution | |
The name of the physical network is not set. |
|
You must set the | |
The name of the physical network does not meet minimum length requirements. |
| You must set physical network name to be at least one character in length. | |
The name of the physical network exceeds the maximum character limit of 253. |
| You must set physical network name to not exceed the 253 character in length. | |
The name of the physical network must not contain |
|
You must remove the |
Condition type | Reason, Message, Resolution | ||
---|---|---|---|
|
One of the following messages is returned when the | ||
Reason | Message | Resolution | |
The |
|
You must set the | |
|
|
You must set the |
Condition type | Reason, Message, Resolution | ||
---|---|---|---|
|
One of the following messages is returned when either the | ||
Reason | Message | Resolution | |
The optional fields, |
|
You must set the | |
The |
|
You must set an acceptable value for | |
The |
|
You must set the | |
The |
|
You must set the value for the | |
The CIDR range is invalid. |
|
You must set an acceptable CIDR range for | |
You must set the |
|
You must set the | |
Setting two CIDR ranges for |
| You must change one of your CIDR ranges to a different IP family. | |
The |
|
You must set the |
Condition type | Reason, Message, Resolution | ||
---|---|---|---|
|
One of the following messages is returned when the | ||
Reason | Message | Resolution | |
The |
|
You must set the | |
The |
|
You must set | |
The |
|
You must set a value for | |
Acceptable values for |
|
You must set a value of 1 or greater for | |
Acceptable values for |
|
You must set a value of 4094 or less for |
2.1.8. Opening default network ports on user-defined network pods Copiar enlaceEnlace copiado en el portapapeles!
By default, pods on a user-defined network (UDN) are isolated from the default network. This means that default network pods, such as those running monitoring services (Prometheus or Alertmanager) or the OpenShift Container Platform image registry, cannot initiate connections to UDN pods.
To allow default network pods to connect to a user-defined network pod, you can use the k8s.ovn.org/open-default-ports
annotation. This annotation opens specific ports on the user-defined network pod for access from the default network.
The following pod specification allows incoming TCP connections on port 80
and UDP traffic on port 53
from the default network:
Open ports are accessible on the pod’s default network IP, not its UDN network IP.
2.2. Creating primary networks using a NetworkAttachmentDefinition Copiar enlaceEnlace copiado en el portapapeles!
The following sections explain how to create and manage primary networks using the NetworkAttachmentDefinition
(NAD) resource.
2.2.1. Approaches to managing a primary network Copiar enlaceEnlace copiado en el portapapeles!
You can manage the life cycle of a primary network created by NAD with one of the following two approaches:
-
By modifying the Cluster Network Operator (CNO) configuration. With this method, the CNO automatically creates and manages the
NetworkAttachmentDefinition
object. In addition to managing the object lifecycle, the CNO ensures that a DHCP is available for a primary network that uses a DHCP assigned IP address. -
By applying a YAML manifest. With this method, you can manage the primary network directly by creating an
NetworkAttachmentDefinition
object. This approach allows for the invocation of multiple CNI plugins in order to attach primary network interfaces in a pod.
Each approach is mutually exclusive and you can only use one approach for managing a primary network at a time. For either approach, the primary network is managed by a Container Network Interface (CNI) plugin that you configure.
When deploying OpenShift Container Platform nodes with multiple network interfaces on Red Hat OpenStack Platform (RHOSP) with OVN SDN, DNS configuration of the secondary interface might take precedence over the DNS configuration of the primary interface. In this case, remove the DNS nameservers for the subnet ID that is attached to the secondary interface by running the following command:
openstack subnet set --dns-nameserver 0.0.0.0 <subnet_id>
$ openstack subnet set --dns-nameserver 0.0.0.0 <subnet_id>
2.2.2. Creating a primary network attachment with the Cluster Network Operator Copiar enlaceEnlace copiado en el portapapeles!
The Cluster Network Operator (CNO) manages additional network definitions. When you specify a primary network to create, the CNO creates the NetworkAttachmentDefinition
custom resource definition (CRD) automatically.
Do not edit the NetworkAttachmentDefinition
CRDs that the Cluster Network Operator manages. Doing so might disrupt network traffic on your primary network.
Prerequisites
-
Install the OpenShift CLI (
oc
). -
Log in as a user with
cluster-admin
privileges.
Procedure
Optional: Create the namespace for the primary networks:
oc create namespace <namespace_name>
$ oc create namespace <namespace_name>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow To edit the CNO configuration, enter the following command:
oc edit networks.operator.openshift.io cluster
$ oc edit networks.operator.openshift.io cluster
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Modify the CR that you are creating by adding the configuration for the primary network that you are creating, as in the following example CR.
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Save your changes and quit the text editor to commit your changes.
Verification
Confirm that the CNO created the
NetworkAttachmentDefinition
CRD by running the following command. A delay might exist before the CNO creates the CRD. The expected output shows the name of the NAD CRD and the creation age in minutes.oc get network-attachment-definitions -n <namespace>
$ oc get network-attachment-definitions -n <namespace>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow where:
<namespace>
- Specifies the namespace for the network attachment that you added to the CNO configuration.
2.2.2.1. Configuration for a primary network attachment Copiar enlaceEnlace copiado en el portapapeles!
A primary network is configured by using the NetworkAttachmentDefinition
API in the k8s.cni.cncf.io
API group.
The configuration for the API is described in the following table:
Field | Type | Description |
---|---|---|
|
| The name for the primary network. |
|
| The namespace that the object is associated with. |
|
| The CNI plugin configuration in JSON format. |
2.2.3. Creating a primary network attachment by applying a YAML manifest Copiar enlaceEnlace copiado en el portapapeles!
Prerequisites
-
You have installed the OpenShift CLI (
oc
). -
You have logged in as a user with
cluster-admin
privileges. - You are working in the namespace where the NAD is to be deployed.
Procedure
Create a YAML file with your primary network configuration, such as in the following example:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- Optional: You can specify a namespace to which the NAD is applied. If you are working in the namespace where the NAD is to be deployed, this spec is not necessary.
To create the primary network, enter the following command:
oc apply -f <file>.yaml
$ oc apply -f <file>.yaml
Copy to Clipboard Copied! Toggle word wrap Toggle overflow where:
<file>
- Specifies the name of the file contained the YAML manifest.