Ce contenu n'est pas disponible dans la langue sélectionnée.
Chapter 2. Admin network policy
2.1. OVN-Kubernetes AdminNetworkPolicy Copier lienLien copié sur presse-papiers!
2.1.1. AdminNetworkPolicy Copier lienLien copié sur presse-papiers!
An AdminNetworkPolicy (ANP) is a cluster-scoped custom resource definition (CRD). As a OpenShift Container Platform administrator, you can use ANP to secure your network by creating network policies before creating namespaces. Additionally, you can create network policies on a cluster-scoped level that is non-overridable by NetworkPolicy objects.
The key difference between AdminNetworkPolicy and NetworkPolicy objects are that the former is for administrators and is cluster scoped while the latter is for tenant owners and is namespace scoped.
An ANP allows administrators to specify the following:
-
A
priorityvalue that determines the order of its evaluation. The lower the value the higher the precedence. - A set of pods that consists of a set of namespaces or namespace on which the policy is applied.
-
A list of ingress rules to be applied for all ingress traffic towards the
subject. -
A list of egress rules to be applied for all egress traffic from the
subject.
2.1.1.1. AdminNetworkPolicy example Copier lienLien copié sur presse-papiers!
Example 2.1. Example YAML file for an ANP
apiVersion: policy.networking.k8s.io/v1alpha1
kind: AdminNetworkPolicy
metadata:
name: sample-anp-deny-pass-rules
spec:
priority: 50
subject:
namespaces:
matchLabels:
kubernetes.io/metadata.name: example.name
ingress:
- name: "deny-all-ingress-tenant-1"
action: "Deny"
from:
- pods:
namespaceSelector:
matchLabels:
custom-anp: tenant-1
podSelector:
matchLabels:
custom-anp: tenant-1
egress:
- name: "pass-all-egress-to-tenant-1"
action: "Pass"
to:
- pods:
namespaceSelector:
matchLabels:
custom-anp: tenant-1
podSelector:
matchLabels:
custom-anp: tenant-1
- 1
- Specify a name for your ANP.
- 2
- The
spec.priorityfield supports a maximum of 100 ANPs in the range of values0-99in a cluster. The lower the value, the higher the precedence because the range is read in order from the lowest to highest value. Because there is no guarantee which policy takes precedence when ANPs are created at the same priority, set ANPs at different priorities so that precedence is deliberate. - 3
- Specify the namespace to apply the ANP resource.
- 4
- ANP have both ingress and egress rules. ANP rules for
spec.ingressfield accepts values ofPass,Deny, andAllowfor theactionfield. - 5
- Specify a name for the
ingress.name. - 6
- Specify
podSelector.matchLabelsto select pods within the namespaces selected bynamespaceSelector.matchLabelsas ingress peers. - 7
- ANPs have both ingress and egress rules. ANP rules for
spec.egressfield accepts values ofPass,Deny, andAllowfor theactionfield.
Additional resources
2.1.1.2. AdminNetworkPolicy actions for rules Copier lienLien copié sur presse-papiers!
As an administrator, you can set Allow, Deny, or Pass as the action field for your AdminNetworkPolicy rules. Because OVN-Kubernetes uses a tiered ACLs to evaluate network traffic rules, ANP allow you to set very strong policy rules that can only be changed by an administrator modifying them, deleting the rule, or overriding them by setting a higher priority rule.
2.1.1.2.1. AdminNetworkPolicy Allow example Copier lienLien copié sur presse-papiers!
The following ANP that is defined at priority 9 ensures all ingress traffic is allowed from the monitoring namespace towards any tenant (all other namespaces) in the cluster.
Example 2.2. Example YAML file for a strong Allow ANP
apiVersion: policy.networking.k8s.io/v1alpha1
kind: AdminNetworkPolicy
metadata:
name: allow-monitoring
spec:
priority: 9
subject:
namespaces: {} # Use the empty selector with caution because it also selects OpenShift namespaces as well.
ingress:
- name: "allow-ingress-from-monitoring"
action: "Allow"
from:
- namespaces:
matchLabels:
kubernetes.io/metadata.name: monitoring
# ...
This is an example of a strong Allow ANP because it is non-overridable by all the parties involved. No tenants can block themselves from being monitored using NetworkPolicy objects and the monitoring tenant also has no say in what it can or cannot monitor.
2.1.1.2.2. AdminNetworkPolicy Deny example Copier lienLien copié sur presse-papiers!
The following ANP that is defined at priority 5 ensures all ingress traffic from the monitoring namespace is blocked towards restricted tenants (namespaces that have labels security: restricted).
Example 2.3. Example YAML file for a strong Deny ANP
apiVersion: policy.networking.k8s.io/v1alpha1
kind: AdminNetworkPolicy
metadata:
name: block-monitoring
spec:
priority: 5
subject:
namespaces:
matchLabels:
security: restricted
ingress:
- name: "deny-ingress-from-monitoring"
action: "Deny"
from:
- namespaces:
matchLabels:
kubernetes.io/metadata.name: monitoring
# ...
This is a strong Deny ANP that is non-overridable by all the parties involved. The restricted tenant owners cannot authorize themselves to allow monitoring traffic, and the infrastructure’s monitoring service cannot scrape anything from these sensitive namespaces.
When combined with the strong Allow example, the block-monitoring ANP has a lower priority value giving it higher precedence, which ensures restricted tenants are never monitored.
2.1.1.2.3. AdminNetworkPolicy Pass example Copier lienLien copié sur presse-papiers!
The following ANP that is defined at priority 7 ensures all ingress traffic from the monitoring namespace towards internal infrastructure tenants (namespaces that have labels security: internal) are passed on to tier 2 of the ACLs and evaluated by the namespaces’ NetworkPolicy objects.
Example 2.4. Example YAML file for a strong Pass ANP
apiVersion: policy.networking.k8s.io/v1alpha1
kind: AdminNetworkPolicy
metadata:
name: pass-monitoring
spec:
priority: 7
subject:
namespaces:
matchLabels:
security: internal
ingress:
- name: "pass-ingress-from-monitoring"
action: "Pass"
from:
- namespaces:
matchLabels:
kubernetes.io/metadata.name: monitoring
# ...
This example is a strong Pass action ANP because it delegates the decision to NetworkPolicy objects defined by tenant owners. This pass-monitoring ANP allows all tenant owners grouped at security level internal to choose if their metrics should be scraped by the infrastructures' monitoring service using namespace scoped NetworkPolicy objects.
2.2. OVN-Kubernetes BaselineAdminNetworkPolicy Copier lienLien copié sur presse-papiers!
2.2.1. BaselineAdminNetworkPolicy Copier lienLien copié sur presse-papiers!
BaselineAdminNetworkPolicy (BANP) is a cluster-scoped custom resource definition (CRD). As a OpenShift Container Platform administrator, you can use BANP to setup and enforce optional baseline network policy rules that are overridable by users using NetworkPolicy objects if need be. Rule actions for BANP are allow or deny.
The BaselineAdminNetworkPolicy resource is a cluster singleton object that can be used as a guardrail policy incase a passed traffic policy does not match any NetworkPolicy objects in the cluster. A BANP can also be used as a default security model that provides guardrails that intra-cluster traffic is blocked by default and a user will need to use NetworkPolicy objects to allow known traffic. You must use default as the name when creating a BANP resource.
A BANP allows administrators to specify:
-
A
subjectthat consists of a set of namespaces or namespace. -
A list of ingress rules to be applied for all ingress traffic towards the
subject. -
A list of egress rules to be applied for all egress traffic from the
subject.
2.2.1.1. BaselineAdminNetworkPolicy example Copier lienLien copié sur presse-papiers!
Example 2.5. Example YAML file for BANP
apiVersion: policy.networking.k8s.io/v1alpha1
kind: BaselineAdminNetworkPolicy
metadata:
name: default
spec:
subject:
namespaces:
matchLabels:
kubernetes.io/metadata.name: example.name
ingress:
- name: "deny-all-ingress-from-tenant-1"
action: "Deny"
from:
- pods:
namespaceSelector:
matchLabels:
custom-banp: tenant-1
podSelector:
matchLabels:
custom-banp: tenant-1
egress:
- name: "allow-all-egress-to-tenant-1"
action: "Allow"
to:
- pods:
namespaceSelector:
matchLabels:
custom-banp: tenant-1
podSelector:
matchLabels:
custom-banp: tenant-1
- 1
- The policy name must be
defaultbecause BANP is a singleton object. - 2
- Specify the namespace to apply the ANP to.
- 3
- BANP have both ingress and egress rules. BANP rules for
spec.ingressandspec.egressfields accepts values ofDenyandAllowfor theactionfield. - 4
- Specify a name for the
ingress.name - 5
- Specify the namespaces to select the pods from to apply the BANP resource.
- 6
- Specify
podSelector.matchLabelsname of the pods to apply the BANP resource.
2.2.1.2. BaselineAdminNetworkPolicy Deny example Copier lienLien copié sur presse-papiers!
The following BANP singleton ensures that the administrator has set up a default deny policy for all ingress monitoring traffic coming into the tenants at internal security level. When combined with the "AdminNetworkPolicy Pass example", this deny policy acts as a guardrail policy for all ingress traffic that is passed by the ANP pass-monitoring policy.
Example 2.6. Example YAML file for a guardrail Deny rule
apiVersion: policy.networking.k8s.io/v1alpha1
kind: BaselineAdminNetworkPolicy
metadata:
name: default
spec:
subject:
namespaces:
matchLabels:
security: internal
ingress:
- name: "deny-ingress-from-monitoring"
action: "Deny"
from:
- namespaces:
matchLabels:
kubernetes.io/metadata.name: monitoring
# ...
You can use an AdminNetworkPolicy resource with a Pass value for the action field in conjunction with the BaselineAdminNetworkPolicy resource to create a multi-tenant policy. This multi-tenant policy allows one tenant to collect monitoring data on their application while simultaneously not collecting data from a second tenant.
As an administrator, if you apply both the "AdminNetworkPolicy Pass action example" and the "BaselineAdminNetwork Policy Deny example", tenants are then left with the ability to choose to create a NetworkPolicy resource that will be evaluated before the BANP.
For example, Tenant 1 can set up the following NetworkPolicy resource to monitor ingress traffic:
Example 2.7. Example NetworkPolicy
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-monitoring
namespace: tenant 1
spec:
podSelector:
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: monitoring
# ...
In this scenario, Tenant 1’s policy would be evaluated after the "AdminNetworkPolicy Pass action example" and before the "BaselineAdminNetwork Policy Deny example", which denies all ingress monitoring traffic coming into tenants with security level internal. With Tenant 1’s NetworkPolicy object in place, they will be able to collect data on their application. Tenant 2, however, who does not have any NetworkPolicy objects in place, will not be able to collect data. As an administrator, you have not by default monitored internal tenants, but instead, you created a BANP that allows tenants to use NetworkPolicy objects to override the default behavior of your BANP.
2.3. Monitoring ANP and BANP Copier lienLien copié sur presse-papiers!
AdminNetworkPolicy and BaselineAdminNetworkPolicy resources have metrics that can be used for monitoring and managing your policies. See the following table for more details on the metrics.
2.3.1. Metrics for AdminNetworkPolicy Copier lienLien copié sur presse-papiers!
| Name | Description | Explanation |
|---|---|---|
|
| Not applicable |
The total number of |
|
| Not applicable |
The total number of |
|
|
|
The total number of rules across all ANP policies in the cluster grouped by |
|
|
|
The total number of rules across all BANP policies in the cluster grouped by |
|
|
|
The total number of OVN Northbound database (nbdb) objects that are created by all the ANP in the cluster grouped by the |
|
|
|
The total number of OVN Northbound database (nbdb) objects that are created by all the BANP in the cluster grouped by the |
2.4. Egress nodes and networks peer for AdminNetworkPolicy Copier lienLien copié sur presse-papiers!
This section explains nodes and networks peers. Administrators can use the examples in this section to design AdminNetworkPolicy and BaselineAdminNetworkPolicy to control northbound traffic in their cluster.
2.4.1. Northbound traffic controls for AdminNetworkPolicy and BaselineAdminNetworkPolicy Copier lienLien copié sur presse-papiers!
In addition to supporting east-west traffic controls, ANP and BANP also allow administrators to control their northbound traffic leaving the cluster or traffic leaving the node to other nodes in the cluster. End-users can do the following:
-
Implement egress traffic control towards cluster nodes using
nodesegress peer -
Implement egress traffic control towards Kubernetes API servers using
nodesornetworksegress peers -
Implement egress traffic control towards external destinations outside the cluster using
networkspeer
For ANP and BANP, nodes and networks peers can be specified for egress rules only.
2.4.1.1. Using nodes peer to control egress traffic to cluster nodes Copier lienLien copié sur presse-papiers!
Using the nodes peer administrators can control egress traffic from pods to nodes in the cluster. A benefit of this is that you do not have to change the policy when nodes are added to or deleted from the cluster.
The following example allows egress traffic to the Kubernetes API server on port 6443 by any of the namespaces with a restricted, confidential, or internal level of security using the node selector peer. It also denies traffic to all worker nodes in your cluster from any of the namespaces with a restricted, confidential, or internal level of security.
Example 2.8. Example of ANP Allow egress using nodes peer
apiVersion: policy.networking.k8s.io/v1alpha1
kind: AdminNetworkPolicy
metadata:
name: egress-security-allow
spec:
egress:
- action: Deny
to:
- nodes:
matchExpressions:
- key: node-role.kubernetes.io/worker
operator: Exists
- action: Allow
name: allow-to-kubernetes-api-server-and-engr-dept-pods
ports:
- portNumber:
port: 6443
protocol: TCP
to:
- nodes:
matchExpressions:
- key: node-role.kubernetes.io/control-plane
operator: Exists
- pods:
namespaceSelector:
matchLabels:
dept: engr
podSelector: {}
priority: 55
subject:
namespaces:
matchExpressions:
- key: security
operator: In
values:
- restricted
- confidential
- internal
- 1
- Specifies a node or set of nodes in the cluster using the
matchExpressionsfield. - 2
- Specifies all the pods labeled with
dept: engr. - 3
- Specifies the subject of the ANP which includes any namespaces that match the labels used by the network policy. The example matches any of the namespaces with
restricted,confidential, orinternallevel ofsecurity. - 4
- Specifies key/value pairs for
matchExpressionsfield.
2.4.1.2. Using networks peer to control egress traffic towards external destinations Copier lienLien copié sur presse-papiers!
Cluster administrators can use CIDR ranges in networks peer and apply a policy to control egress traffic leaving from pods and going to a destination configured at the IP address that is within the CIDR range specified with networks field.
The following example uses networks peer and combines ANP and BANP policies to restrict egress traffic.
Use the empty selector ({}) in the namespace field for ANP and BANP with caution. When using an empty selector, it also selects OpenShift namespaces.
If you use values of 0.0.0.0/0 in a ANP or BANP Deny rule, you must set a higher priority ANP Allow rule to necessary destinations before setting the Deny to 0.0.0.0/0.
Example 2.9. Example of ANP and BANP using networks peers
apiVersion: policy.networking.k8s.io/v1alpha1
kind: AdminNetworkPolicy
metadata:
name: network-as-egress-peer
spec:
priority: 70
subject:
namespaces: {} # Use the empty selector with caution because it also selects OpenShift namespaces as well.
egress:
- name: "deny-egress-to-external-dns-servers"
action: "Deny"
to:
- networks:
- 8.8.8.8/32
- 8.8.4.4/32
- 208.67.222.222/32
ports:
- portNumber:
protocol: UDP
port: 53
- name: "allow-all-egress-to-intranet"
action: "Allow"
to:
- networks:
- 89.246.180.0/22
- 60.45.72.0/22
- name: "allow-all-intra-cluster-traffic"
action: "Allow"
to:
- namespaces: {} # Use the empty selector with caution because it also selects OpenShift namespaces as well.
- name: "pass-all-egress-to-internet"
action: "Pass"
to:
- networks:
- 0.0.0.0/0
---
apiVersion: policy.networking.k8s.io/v1alpha1
kind: BaselineAdminNetworkPolicy
metadata:
name: default
spec:
subject:
namespaces: {} # Use the empty selector with caution because it also selects OpenShift namespaces as well.
egress:
- name: "deny-all-egress-to-internet"
action: "Deny"
to:
- networks:
- 0.0.0.0/0
---
- 1
- Use
networksto specify a range of CIDR networks outside of the cluster. - 2
- Specifies the CIDR ranges for the intra-cluster traffic from your resources.
- 3 4
- Specifies a
Denyegress to everything by settingnetworksvalues to0.0.0.0/0. Make sure you have a higher priorityAllowrule to necessary destinations before setting aDenyto0.0.0.0/0because this will deny all traffic including to Kubernetes API and DNS servers.
Collectively the network-as-egress-peer ANP and default BANP using networks peers enforces the following egress policy:
- All pods cannot talk to external DNS servers at the listed IP addresses.
- All pods can talk to rest of the company’s intranet.
- All pods can talk to other pods, nodes, and services.
-
All pods cannot talk to the internet. Combining the last ANP
Passrule and the strong BANPDenyrule a guardrail policy is created that secures traffic in the cluster.
2.4.1.3. Using nodes peer and networks peer together Copier lienLien copié sur presse-papiers!
Cluster administrators can combine nodes and networks peer in your ANP and BANP policies.
Example 2.10. Example of nodes and networks peer
apiVersion: policy.networking.k8s.io/v1alpha1
kind: AdminNetworkPolicy
metadata:
name: egress-peer-1
spec:
egress:
- action: "Allow"
name: "allow-egress"
to:
- nodes:
matchExpressions:
- key: worker-group
operator: In
values:
- workloads # Egress traffic from nodes with label worker-group: workloads is allowed.
- networks:
- 104.154.164.170/32
- pods:
namespaceSelector:
matchLabels:
apps: external-apps
podSelector:
matchLabels:
app: web # This rule in the policy allows the traffic directed to pods labeled apps: web in projects with apps: external-apps to leave the cluster.
- action: "Deny"
name: "deny-egress"
to:
- nodes:
matchExpressions:
- key: worker-group
operator: In
values:
- infra # Egress traffic from nodes with label worker-group: infra is denied.
- networks:
- 104.154.164.160/32 # Egress traffic to this IP address from cluster is denied.
- pods:
namespaceSelector:
matchLabels:
apps: internal-apps
podSelector: {}
- action: "Pass"
name: "pass-egress"
to:
- nodes:
matchExpressions:
- key: node-role.kubernetes.io/worker
operator: Exists # All other egress traffic is passed to NetworkPolicy or BANP for evaluation.
priority: 30
subject:
namespaces:
matchLabels:
apps: all-apps
- 1
- Specifies the name of the policy.
- 2
- For
nodesandnetworkspeers, you can only use northbound traffic controls in ANP asegress. - 3
- Specifies the priority of the ANP, determining the order in which they should be evaluated. Lower priority rules have higher precedence. ANP accepts values of 0-99 with 0 being the highest priority and 99 being the lowest.
- 4
- Specifies the set of pods in the cluster on which the rules of the policy are to be applied. In the example, any pods with the
apps: all-appslabel across all namespaces are thesubjectof the policy.
2.5. Troubleshooting AdminNetworkPolicy Copier lienLien copié sur presse-papiers!
2.5.1. Checking creation of ANP Copier lienLien copié sur presse-papiers!
To check that your AdminNetworkPolicy (ANP) and BaselineAdminNetworkPolicy (BANP) are created correctly, check the status outputs of the following commands: oc describe anp or oc describe banp.
A good status indicates OVN DB plumbing was successful and the SetupSucceeded.
Example 2.11. Example ANP with a good status
...
Conditions:
Last Transition Time: 2024-06-08T20:29:00Z
Message: Setting up OVN DB plumbing was successful
Reason: SetupSucceeded
Status: True
Type: Ready-In-Zone-ovn-control-plane Last Transition Time: 2024-06-08T20:29:00Z
Message: Setting up OVN DB plumbing was successful
Reason: SetupSucceeded
Status: True
Type: Ready-In-Zone-ovn-worker
Last Transition Time: 2024-06-08T20:29:00Z
Message: Setting up OVN DB plumbing was successful
Reason: SetupSucceeded
Status: True
Type: Ready-In-Zone-ovn-worker2
...
If plumbing is unsuccessful, an error is reported from the respective zone controller.
Example 2.12. Example of an ANP with a bad status and error message
...
Status:
Conditions:
Last Transition Time: 2024-06-25T12:47:44Z
Message: error attempting to add ANP cluster-control with priority 600 because, OVNK only supports priority ranges 0-99
Reason: SetupFailed
Status: False
Type: Ready-In-Zone-example-worker-1.example.example-org.net
Last Transition Time: 2024-06-25T12:47:45Z
Message: error attempting to add ANP cluster-control with priority 600 because, OVNK only supports priority ranges 0-99
Reason: SetupFailed
Status: False
Type: Ready-In-Zone-example-worker-0.example.example-org.net
Last Transition Time: 2024-06-25T12:47:44Z
Message: error attempting to add ANP cluster-control with priority 600 because, OVNK only supports priority ranges 0-99
Reason: SetupFailed
Status: False
Type: Ready-In-Zone-example-ctlplane-1.example.example-org.net
Last Transition Time: 2024-06-25T12:47:44Z
Message: error attempting to add ANP cluster-control with priority 600 because, OVNK only supports priority ranges 0-99
Reason: SetupFailed
Status: False
Type: Ready-In-Zone-example-ctlplane-2.example.example-org.net
Last Transition Time: 2024-06-25T12:47:44Z
Message: error attempting to add ANP cluster-control with priority 600 because, OVNK only supports priority ranges 0-99
Reason: SetupFailed
Status: False
Type: Ready-In-Zone-example-ctlplane-0.example.example-org.net
```
See the following section for nbctl commands to help troubleshoot unsuccessful policies.
2.5.1.1. Using nbctl commands for ANP and BANP Copier lienLien copié sur presse-papiers!
To troubleshoot an unsuccessful setup, start by looking at OVN Northbound database (nbdb) objects including ACL, AdressSet, and Port_Group. To view the nbdb, you need to be inside the pod on that node to view the objects in that node’s database.
Prerequisites
-
Access to the cluster as a user with the
cluster-adminrole. -
The OpenShift CLI (
oc) installed.
To run ovn nbctl commands in a cluster, you must open a remote shell into the `nbdb`on the relevant node.
The following policy was used to generate outputs.
Example 2.13. AdminNetworkPolicy used to generate outputs
apiVersion: policy.networking.k8s.io/v1alpha1
kind: AdminNetworkPolicy
metadata:
name: cluster-control
spec:
priority: 34
subject:
namespaces:
matchLabels:
anp: cluster-control-anp # Only namespaces with this label have this ANP
ingress:
- name: "allow-from-ingress-router" # rule0
action: "Allow"
from:
- namespaces:
matchLabels:
policy-group.network.openshift.io/ingress: ""
- name: "allow-from-monitoring" # rule1
action: "Allow"
from:
- namespaces:
matchLabels:
kubernetes.io/metadata.name: openshift-monitoring
ports:
- portNumber:
protocol: TCP
port: 7564
- namedPort: "scrape"
- name: "allow-from-open-tenants" # rule2
action: "Allow"
from:
- namespaces: # open tenants
matchLabels:
tenant: open
- name: "pass-from-restricted-tenants" # rule3
action: "Pass"
from:
- namespaces: # restricted tenants
matchLabels:
tenant: restricted
- name: "default-deny" # rule4
action: "Deny"
from:
- namespaces: {} # Use the empty selector with caution because it also selects OpenShift namespaces as well.
egress:
- name: "allow-to-dns" # rule0
action: "Allow"
to:
- pods:
namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: openshift-dns
podSelector:
matchLabels:
app: dns
ports:
- portNumber:
protocol: UDP
port: 5353
- name: "allow-to-kapi-server" # rule1
action: "Allow"
to:
- nodes:
matchExpressions:
- key: node-role.kubernetes.io/control-plane
operator: Exists
ports:
- portNumber:
protocol: TCP
port: 6443
- name: "allow-to-splunk" # rule2
action: "Allow"
to:
- namespaces:
matchLabels:
tenant: splunk
ports:
- portNumber:
protocol: TCP
port: 8991
- portNumber:
protocol: TCP
port: 8992
- name: "allow-to-open-tenants-and-intranet-and-worker-nodes" # rule3
action: "Allow"
to:
- nodes: # worker-nodes
matchExpressions:
- key: node-role.kubernetes.io/worker
operator: Exists
- networks: # intranet
- 172.29.0.0/30
- 10.0.54.0/19
- 10.0.56.38/32
- 10.0.69.0/24
- namespaces: # open tenants
matchLabels:
tenant: open
- name: "pass-to-restricted-tenants" # rule4
action: "Pass"
to:
- namespaces: # restricted tenants
matchLabels:
tenant: restricted
- name: "default-deny"
action: "Deny"
to:
- networks:
- 0.0.0.0/0
Procedure
List pods with node information by running the following command:
$ oc get pods -n openshift-ovn-kubernetes -owideExample output
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES ovnkube-control-plane-5c95487779-8k9fd 2/2 Running 0 34m 10.0.0.5 ci-ln-0tv5gg2-72292-6sjw5-master-0 <none> <none> ovnkube-control-plane-5c95487779-v2xn8 2/2 Running 0 34m 10.0.0.3 ci-ln-0tv5gg2-72292-6sjw5-master-1 <none> <none> ovnkube-node-524dt 8/8 Running 0 33m 10.0.0.4 ci-ln-0tv5gg2-72292-6sjw5-master-2 <none> <none> ovnkube-node-gbwr9 8/8 Running 0 24m 10.0.128.4 ci-ln-0tv5gg2-72292-6sjw5-worker-c-s9gqt <none> <none> ovnkube-node-h4fpx 8/8 Running 0 33m 10.0.0.5 ci-ln-0tv5gg2-72292-6sjw5-master-0 <none> <none> ovnkube-node-j4hzw 8/8 Running 0 24m 10.0.128.2 ci-ln-0tv5gg2-72292-6sjw5-worker-a-hzbh5 <none> <none> ovnkube-node-wdhgv 8/8 Running 0 33m 10.0.0.3 ci-ln-0tv5gg2-72292-6sjw5-master-1 <none> <none> ovnkube-node-wfncn 8/8 Running 0 24m 10.0.128.3 ci-ln-0tv5gg2-72292-6sjw5-worker-b-5bb7f <none> <none>Navigate into a pod to look at the northbound database by running the following command:
$ oc rsh -c nbdb -n openshift-ovn-kubernetes ovnkube-node-524dtRun the following command to look at the ACLs nbdb:
$ ovn-nbctl find ACL 'external_ids{>=}{"k8s.ovn.org/owner-type"=AdminNetworkPolicy,"k8s.ovn.org/name"=cluster-control}'- Where, cluster-control
-
Specifies the name of the
AdminNetworkPolicyyou are troubleshooting. - AdminNetworkPolicy
-
Specifies the type:
AdminNetworkPolicyorBaselineAdminNetworkPolicy.
Example 2.14. Example output for ACLs
_uuid : 0d5e4722-b608-4bb1-b625-23c323cc9926 action : allow-related direction : to-lport external_ids : {direction=Ingress, gress-index="2", "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Ingress:2:None", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy, port-policy-protocol=None} label : 0 log : false match : "outport == @a14645450421485494999 && ((ip4.src == $a13730899355151937870))" meter : acl-logging name : "ANP:cluster-control:Ingress:2" options : {} priority : 26598 severity : [] tier : 1 _uuid : b7be6472-df67-439c-8c9c-f55929f0a6e0 action : drop direction : from-lport external_ids : {direction=Egress, gress-index="5", "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Egress:5:None", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy, port-policy-protocol=None} label : 0 log : false match : "inport == @a14645450421485494999 && ((ip4.dst == $a11452480169090787059))" meter : acl-logging name : "ANP:cluster-control:Egress:5" options : {apply-after-lb="true"} priority : 26595 severity : [] tier : 1 _uuid : 5a6e5bb4-36eb-4209-b8bc-c611983d4624 action : pass direction : to-lport external_ids : {direction=Ingress, gress-index="3", "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Ingress:3:None", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy, port-policy-protocol=None} label : 0 log : false match : "outport == @a14645450421485494999 && ((ip4.src == $a764182844364804195))" meter : acl-logging name : "ANP:cluster-control:Ingress:3" options : {} priority : 26597 severity : [] tier : 1 _uuid : 04f20275-c410-405c-a923-0e677f767889 action : pass direction : from-lport external_ids : {direction=Egress, gress-index="4", "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Egress:4:None", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy, port-policy-protocol=None} label : 0 log : false match : "inport == @a14645450421485494999 && ((ip4.dst == $a5972452606168369118))" meter : acl-logging name : "ANP:cluster-control:Egress:4" options : {apply-after-lb="true"} priority : 26596 severity : [] tier : 1 _uuid : 4b5d836a-e0a3-4088-825e-f9f0ca58e538 action : drop direction : to-lport external_ids : {direction=Ingress, gress-index="4", "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Ingress:4:None", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy, port-policy-protocol=None} label : 0 log : false match : "outport == @a14645450421485494999 && ((ip4.src == $a13814616246365836720))" meter : acl-logging name : "ANP:cluster-control:Ingress:4" options : {} priority : 26596 severity : [] tier : 1 _uuid : 5d09957d-d2cc-4f5a-9ddd-b97d9d772023 action : allow-related direction : from-lport external_ids : {direction=Egress, gress-index="2", "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Egress:2:tcp", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy, port-policy-protocol=tcp} label : 0 log : false match : "inport == @a14645450421485494999 && ((ip4.dst == $a18396736153283155648)) && tcp && tcp.dst=={8991,8992}" meter : acl-logging name : "ANP:cluster-control:Egress:2" options : {apply-after-lb="true"} priority : 26598 severity : [] tier : 1 _uuid : 1a68a5ed-e7f9-47d0-b55c-89184d97e81a action : allow-related direction : from-lport external_ids : {direction=Egress, gress-index="1", "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Egress:1:tcp", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy, port-policy-protocol=tcp} label : 0 log : false match : "inport == @a14645450421485494999 && ((ip4.dst == $a10706246167277696183)) && tcp && tcp.dst==6443" meter : acl-logging name : "ANP:cluster-control:Egress:1" options : {apply-after-lb="true"} priority : 26599 severity : [] tier : 1 _uuid : aa1a224d-7960-4952-bdfb-35246bafbac8 action : allow-related direction : to-lport external_ids : {direction=Ingress, gress-index="1", "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Ingress:1:tcp", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy, port-policy-protocol=tcp} label : 0 log : false match : "outport == @a14645450421485494999 && ((ip4.src == $a6786643370959569281)) && tcp && tcp.dst==7564" meter : acl-logging name : "ANP:cluster-control:Ingress:1" options : {} priority : 26599 severity : [] tier : 1 _uuid : 1a27d30e-3f96-4915-8ddd-ade7f22c117b action : allow-related direction : from-lport external_ids : {direction=Egress, gress-index="3", "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Egress:3:None", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy, port-policy-protocol=None} label : 0 log : false match : "inport == @a14645450421485494999 && ((ip4.dst == $a10622494091691694581))" meter : acl-logging name : "ANP:cluster-control:Egress:3" options : {apply-after-lb="true"} priority : 26597 severity : [] tier : 1 _uuid : b23a087f-08f8-4225-8c27-4a9a9ee0c407 action : allow-related direction : from-lport external_ids : {direction=Egress, gress-index="0", "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Egress:0:udp", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy, port-policy-protocol=udp} label : 0 log : false match : "inport == @a14645450421485494999 && ((ip4.dst == $a13517855690389298082)) && udp && udp.dst==5353" meter : acl-logging name : "ANP:cluster-control:Egress:0" options : {apply-after-lb="true"} priority : 26600 severity : [] tier : 1 _uuid : d14ed5cf-2e06-496e-8cae-6b76d5dd5ccd action : allow-related direction : to-lport external_ids : {direction=Ingress, gress-index="0", "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Ingress:0:None", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy, port-policy-protocol=None} label : 0 log : false match : "outport == @a14645450421485494999 && ((ip4.src == $a14545668191619617708))" meter : acl-logging name : "ANP:cluster-control:Ingress:0" options : {} priority : 26600 severity : [] tier : 1NoteThe outputs for ingress and egress show you the logic of the policy in the ACL. For example, every time a packet matches the provided
matchtheactionis taken.Examine the specific ACL for the rule by running the following command:
$ ovn-nbctl find ACL 'external_ids{>=}{"k8s.ovn.org/owner-type"=AdminNetworkPolicy,direction=Ingress,"k8s.ovn.org/name"=cluster-control,gress-index="1"}'- Where,
cluster-control -
Specifies the
nameof your ANP. Ingress-
Specifies the
directionof traffic either of typeIngressorEgress. 1- Specifies the rule you want to look at.
For the example ANP named
cluster-controlatpriority34, the following is an example output forIngressrule1:Example 2.15. Example output
_uuid : aa1a224d-7960-4952-bdfb-35246bafbac8 action : allow-related direction : to-lport external_ids : {direction=Ingress, gress-index="1", "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Ingress:1:tcp", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy, port-policy-protocol=tcp} label : 0 log : false match : "outport == @a14645450421485494999 && ((ip4.src == $a6786643370959569281)) && tcp && tcp.dst==7564" meter : acl-logging name : "ANP:cluster-control:Ingress:1" options : {} priority : 26599 severity : [] tier : 1- Where,
Run the following command to look at address sets in the nbdb:
$ ovn-nbctl find Address_Set 'external_ids{>=}{"k8s.ovn.org/owner-type"=AdminNetworkPolicy,"k8s.ovn.org/name"=cluster-control}'Example 2.16. Example outputs for
Address_Set_uuid : 56e89601-5552-4238-9fc3-8833f5494869 addresses : ["192.168.194.135", "192.168.194.152", "192.168.194.193", "192.168.194.254"] external_ids : {direction=Egress, gress-index="1", ip-family=v4, "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Egress:1:v4", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy} name : a10706246167277696183 _uuid : 7df9330d-380b-4bdb-8acd-4eddeda2419c addresses : ["10.132.0.10", "10.132.0.11", "10.132.0.12", "10.132.0.13", "10.132.0.14", "10.132.0.15", "10.132.0.16", "10.132.0.17", "10.132.0.5", "10.132.0.7", "10.132.0.71", "10.132.0.75", "10.132.0.8", "10.132.0.81", "10.132.0.9", "10.132.2.10", "10.132.2.11", "10.132.2.12", "10.132.2.14", "10.132.2.15", "10.132.2.3", "10.132.2.4", "10.132.2.5", "10.132.2.6", "10.132.2.7", "10.132.2.8", "10.132.2.9", "10.132.3.64", "10.132.3.65", "10.132.3.72", "10.132.3.73", "10.132.3.76", "10.133.0.10", "10.133.0.11", "10.133.0.12", "10.133.0.13", "10.133.0.14", "10.133.0.15", "10.133.0.16", "10.133.0.17", "10.133.0.18", "10.133.0.19", "10.133.0.20", "10.133.0.21", "10.133.0.22", "10.133.0.23", "10.133.0.24", "10.133.0.25", "10.133.0.26", "10.133.0.27", "10.133.0.28", "10.133.0.29", "10.133.0.30", "10.133.0.31", "10.133.0.32", "10.133.0.33", "10.133.0.34", "10.133.0.35", "10.133.0.36", "10.133.0.37", "10.133.0.38", "10.133.0.39", "10.133.0.40", "10.133.0.41", "10.133.0.42", "10.133.0.44", "10.133.0.45", "10.133.0.46", "10.133.0.47", "10.133.0.48", "10.133.0.5", "10.133.0.6", "10.133.0.7", "10.133.0.8", "10.133.0.9", "10.134.0.10", "10.134.0.11", "10.134.0.12", "10.134.0.13", "10.134.0.14", "10.134.0.15", "10.134.0.16", "10.134.0.17", "10.134.0.18", "10.134.0.19", "10.134.0.20", "10.134.0.21", "10.134.0.22", "10.134.0.23", "10.134.0.24", "10.134.0.25", "10.134.0.26", "10.134.0.27", "10.134.0.28", "10.134.0.30", "10.134.0.31", "10.134.0.32", "10.134.0.33", "10.134.0.34", "10.134.0.35", "10.134.0.36", "10.134.0.37", "10.134.0.38", "10.134.0.4", "10.134.0.42", "10.134.0.9", "10.135.0.10", "10.135.0.11", "10.135.0.12", "10.135.0.13", "10.135.0.14", "10.135.0.15", "10.135.0.16", "10.135.0.17", "10.135.0.18", "10.135.0.19", "10.135.0.23", "10.135.0.24", "10.135.0.26", "10.135.0.27", "10.135.0.29", "10.135.0.3", "10.135.0.4", "10.135.0.40", "10.135.0.41", "10.135.0.42", "10.135.0.43", "10.135.0.44", "10.135.0.5", "10.135.0.6", "10.135.0.7", "10.135.0.8", "10.135.0.9"] external_ids : {direction=Ingress, gress-index="4", ip-family=v4, "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Ingress:4:v4", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy} name : a13814616246365836720 _uuid : 84d76f13-ad95-4c00-8329-a0b1d023c289 addresses : ["10.132.3.76", "10.135.0.44"] external_ids : {direction=Egress, gress-index="4", ip-family=v4, "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Egress:4:v4", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy} name : a5972452606168369118 _uuid : 0c53e917-f7ee-4256-8f3a-9522c0481e52 addresses : ["10.132.0.10", "10.132.0.11", "10.132.0.12", "10.132.0.13", "10.132.0.14", "10.132.0.15", "10.132.0.16", "10.132.0.17", "10.132.0.5", "10.132.0.7", "10.132.0.71", "10.132.0.75", "10.132.0.8", "10.132.0.81", "10.132.0.9", "10.132.2.10", "10.132.2.11", "10.132.2.12", "10.132.2.14", "10.132.2.15", "10.132.2.3", "10.132.2.4", "10.132.2.5", "10.132.2.6", "10.132.2.7", "10.132.2.8", "10.132.2.9", "10.132.3.64", "10.132.3.65", "10.132.3.72", "10.132.3.73", "10.132.3.76", "10.133.0.10", "10.133.0.11", "10.133.0.12", "10.133.0.13", "10.133.0.14", "10.133.0.15", "10.133.0.16", "10.133.0.17", "10.133.0.18", "10.133.0.19", "10.133.0.20", "10.133.0.21", "10.133.0.22", "10.133.0.23", "10.133.0.24", "10.133.0.25", "10.133.0.26", "10.133.0.27", "10.133.0.28", "10.133.0.29", "10.133.0.30", "10.133.0.31", "10.133.0.32", "10.133.0.33", "10.133.0.34", "10.133.0.35", "10.133.0.36", "10.133.0.37", "10.133.0.38", "10.133.0.39", "10.133.0.40", "10.133.0.41", "10.133.0.42", "10.133.0.44", "10.133.0.45", "10.133.0.46", "10.133.0.47", "10.133.0.48", "10.133.0.5", "10.133.0.6", "10.133.0.7", "10.133.0.8", "10.133.0.9", "10.134.0.10", "10.134.0.11", "10.134.0.12", "10.134.0.13", "10.134.0.14", "10.134.0.15", "10.134.0.16", "10.134.0.17", "10.134.0.18", "10.134.0.19", "10.134.0.20", "10.134.0.21", "10.134.0.22", "10.134.0.23", "10.134.0.24", "10.134.0.25", "10.134.0.26", "10.134.0.27", "10.134.0.28", "10.134.0.30", "10.134.0.31", "10.134.0.32", "10.134.0.33", "10.134.0.34", "10.134.0.35", "10.134.0.36", "10.134.0.37", "10.134.0.38", "10.134.0.4", "10.134.0.42", "10.134.0.9", "10.135.0.10", "10.135.0.11", "10.135.0.12", "10.135.0.13", "10.135.0.14", "10.135.0.15", "10.135.0.16", "10.135.0.17", "10.135.0.18", "10.135.0.19", "10.135.0.23", "10.135.0.24", "10.135.0.26", "10.135.0.27", "10.135.0.29", "10.135.0.3", "10.135.0.4", "10.135.0.40", "10.135.0.41", "10.135.0.42", "10.135.0.43", "10.135.0.44", "10.135.0.5", "10.135.0.6", "10.135.0.7", "10.135.0.8", "10.135.0.9"] external_ids : {direction=Egress, gress-index="2", ip-family=v4, "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Egress:2:v4", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy} name : a18396736153283155648 _uuid : 5228bf1b-dfd8-40ec-bfa8-95c5bf9aded9 addresses : [] external_ids : {direction=Ingress, gress-index="0", ip-family=v4, "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Ingress:0:v4", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy} name : a14545668191619617708 _uuid : 46530d69-70da-4558-8c63-884ec9dc4f25 addresses : ["10.132.2.10", "10.132.2.5", "10.132.2.6", "10.132.2.7", "10.132.2.8", "10.132.2.9", "10.133.0.47", "10.134.0.33", "10.135.0.10", "10.135.0.11", "10.135.0.12", "10.135.0.19", "10.135.0.24", "10.135.0.7", "10.135.0.8", "10.135.0.9"] external_ids : {direction=Ingress, gress-index="1", ip-family=v4, "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Ingress:1:v4", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy} name : a6786643370959569281 _uuid : 65fdcdea-0b9f-4318-9884-1b51d231ad1d addresses : ["10.132.3.72", "10.135.0.42"] external_ids : {direction=Ingress, gress-index="2", ip-family=v4, "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Ingress:2:v4", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy} name : a13730899355151937870 _uuid : 73eabdb0-36bf-4ca3-b66d-156ac710df4c addresses : ["10.0.32.0/19", "10.0.56.38/32", "10.0.69.0/24", "10.132.3.72", "10.135.0.42", "172.29.0.0/30", "192.168.194.103", "192.168.194.2"] external_ids : {direction=Egress, gress-index="3", ip-family=v4, "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Egress:3:v4", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy} name : a10622494091691694581 _uuid : 50cdbef2-71b5-474b-914c-6fcd1d7712d3 addresses : ["10.132.0.10", "10.132.0.11", "10.132.0.12", "10.132.0.13", "10.132.0.14", "10.132.0.15", "10.132.0.16", "10.132.0.17", "10.132.0.5", "10.132.0.7", "10.132.0.71", "10.132.0.75", "10.132.0.8", "10.132.0.81", "10.132.0.9", "10.132.2.10", "10.132.2.11", "10.132.2.12", "10.132.2.14", "10.132.2.15", "10.132.2.3", "10.132.2.4", "10.132.2.5", "10.132.2.6", "10.132.2.7", "10.132.2.8", "10.132.2.9", "10.132.3.64", "10.132.3.65", "10.132.3.72", "10.132.3.73", "10.132.3.76", "10.133.0.10", "10.133.0.11", "10.133.0.12", "10.133.0.13", "10.133.0.14", "10.133.0.15", "10.133.0.16", "10.133.0.17", "10.133.0.18", "10.133.0.19", "10.133.0.20", "10.133.0.21", "10.133.0.22", "10.133.0.23", "10.133.0.24", "10.133.0.25", "10.133.0.26", "10.133.0.27", "10.133.0.28", "10.133.0.29", "10.133.0.30", "10.133.0.31", "10.133.0.32", "10.133.0.33", "10.133.0.34", "10.133.0.35", "10.133.0.36", "10.133.0.37", "10.133.0.38", "10.133.0.39", "10.133.0.40", "10.133.0.41", "10.133.0.42", "10.133.0.44", "10.133.0.45", "10.133.0.46", "10.133.0.47", "10.133.0.48", "10.133.0.5", "10.133.0.6", "10.133.0.7", "10.133.0.8", "10.133.0.9", "10.134.0.10", "10.134.0.11", "10.134.0.12", "10.134.0.13", "10.134.0.14", "10.134.0.15", "10.134.0.16", "10.134.0.17", "10.134.0.18", "10.134.0.19", "10.134.0.20", "10.134.0.21", "10.134.0.22", "10.134.0.23", "10.134.0.24", "10.134.0.25", "10.134.0.26", "10.134.0.27", "10.134.0.28", "10.134.0.30", "10.134.0.31", "10.134.0.32", "10.134.0.33", "10.134.0.34", "10.134.0.35", "10.134.0.36", "10.134.0.37", "10.134.0.38", "10.134.0.4", "10.134.0.42", "10.134.0.9", "10.135.0.10", "10.135.0.11", "10.135.0.12", "10.135.0.13", "10.135.0.14", "10.135.0.15", "10.135.0.16", "10.135.0.17", "10.135.0.18", "10.135.0.19", "10.135.0.23", "10.135.0.24", "10.135.0.26", "10.135.0.27", "10.135.0.29", "10.135.0.3", "10.135.0.4", "10.135.0.40", "10.135.0.41", "10.135.0.42", "10.135.0.43", "10.135.0.44", "10.135.0.5", "10.135.0.6", "10.135.0.7", "10.135.0.8", "10.135.0.9"] external_ids : {direction=Egress, gress-index="0", ip-family=v4, "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Egress:0:v4", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy} name : a13517855690389298082 _uuid : 32a42f32-2d11-43dd-979d-a56d7ee6aa57 addresses : ["10.132.3.76", "10.135.0.44"] external_ids : {direction=Ingress, gress-index="3", ip-family=v4, "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Ingress:3:v4", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy} name : a764182844364804195 _uuid : 8fd3b977-6e1c-47aa-82b7-e3e3136c4a72 addresses : ["0.0.0.0/0"] external_ids : {direction=Egress, gress-index="5", ip-family=v4, "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Egress:5:v4", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy} name : a11452480169090787059Examine the specific address set of the rule by running the following command:
$ ovn-nbctl find Address_Set 'external_ids{>=}{"k8s.ovn.org/owner-type"=AdminNetworkPolicy,direction=Egress,"k8s.ovn.org/name"=cluster-control,gress-index="5"}'Example 2.17. Example outputs for
Address_Set_uuid : 8fd3b977-6e1c-47aa-82b7-e3e3136c4a72 addresses : ["0.0.0.0/0"] external_ids : {direction=Egress, gress-index="5", ip-family=v4, "k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control:Egress:5:v4", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy} name : a11452480169090787059
Run the following command to look at the port groups in the nbdb:
$ ovn-nbctl find Port_Group 'external_ids{>=}{"k8s.ovn.org/owner-type"=AdminNetworkPolicy,"k8s.ovn.org/name"=cluster-control}'Example 2.18. Example outputs for
Port_Group_uuid : f50acf71-7488-4b9a-b7b8-c8a024e99d21 acls : [04f20275-c410-405c-a923-0e677f767889, 0d5e4722-b608-4bb1-b625-23c323cc9926, 1a27d30e-3f96-4915-8ddd-ade7f22c117b, 1a68a5ed-e7f9-47d0-b55c-89184d97e81a, 4b5d836a-e0a3-4088-825e-f9f0ca58e538, 5a6e5bb4-36eb-4209-b8bc-c611983d4624, 5d09957d-d2cc-4f5a-9ddd-b97d9d772023, aa1a224d-7960-4952-bdfb-35246bafbac8, b23a087f-08f8-4225-8c27-4a9a9ee0c407, b7be6472-df67-439c-8c9c-f55929f0a6e0, d14ed5cf-2e06-496e-8cae-6b76d5dd5ccd] external_ids : {"k8s.ovn.org/id"="default-network-controller:AdminNetworkPolicy:cluster-control", "k8s.ovn.org/name"=cluster-control, "k8s.ovn.org/owner-controller"=default-network-controller, "k8s.ovn.org/owner-type"=AdminNetworkPolicy} name : a14645450421485494999 ports : [5e75f289-8273-4f8a-8798-8c10f7318833, de7e1b71-6184-445d-93e7-b20acadf41ea]
2.6. Best practices for AdminNetworkPolicy Copier lienLien copié sur presse-papiers!
This section provides best practices for the AdminNetworkPolicy and BaselineAdminNetworkPolicy resources.
2.6.1. Designing AdminNetworkPolicy Copier lienLien copié sur presse-papiers!
When building AdminNetworkPolicy (ANP) resources, you might consider the following when creating your policies:
- You can create ANPs that have the same priority. If you do create two ANPs at the same priority, ensure that they do not apply overlapping rules to the same traffic. Only one rule per value is applied and there is no guarantee which rule is applied when there is more than one at the same priority value. Because there is no guarantee which policy takes precedence when overlapping ANPs are created, set ANPs at different priorities so that precedence is well defined.
- Administrators must create ANP that apply to user namespaces not system namespaces.
Applying ANP and BaselineAdminNetworkPolicy (BANP) to system namespaces (default, kube-system, any namespace whose name starts with openshift-, etc) is not supported, and this can leave your cluster unresponsive and in a non-functional state.
-
Because
0-100is the supported priority range, you might design your ANP to use a middle range like30-70. This leaves some placeholder for priorities before and after. Even in the middle range, you might want to leave gaps so that as your infrastructure requirements evolve over time, you are able to insert new ANPs when needed at the right priority level. If you pack your ANPs, then you might need to recreate all of them to accommodate any changes in the future. -
When using
0.0.0.0/0or::/0to create a strongDenypolicy, ensure that you have higher priorityAlloworPassrules for essential traffic. -
Use
Allowas youractionfield when you want to ensure that a connection is allowed no matter what. AnAllowrule in an ANP means that the connection will always be allowed, andNetworkPolicywill be ignored. -
Use
Passas youractionfield to delegate the policy decision of allowing or denying the connection to theNetworkPolicylayer. - Ensure that the selectors across multiple rules do not overlap so that the same IPs do not appear in multiple policies, which can cause performance and scale limitations.
-
Avoid using
namedPortsin conjunction withPortNumberandPortRangebecause this creates 6 ACLs and cause inefficiencies in your cluster.
2.6.1.1. Considerations for using BaselineAdminNetworkPolicy Copier lienLien copié sur presse-papiers!
You can define only a single
BaselineAdminNetworkPolicy(BANP) resource within a cluster. The following are supported uses for BANP that administrators might consider in designing their BANP:-
You can set a default deny policy for cluster-local ingress in user namespaces. This BANP will force developers to have to add
NetworkPolicyobjects to allow the ingress traffic that they want to allow, and if they do not add network policies for ingress it will be denied. -
You can set a default deny policy for cluster-local egress in user namespaces. This BANP will force developers to have to add
NetworkPolicyobjects to allow the egress traffic that they want to allow, and if they do not add network policies it will be denied. -
You can set a default allow policy for egress to the in-cluster DNS service. Such a BANP ensures that the namespaced users do not have to set an allow egress
NetworkPolicyto the in-cluster DNS service. -
You can set an egress policy that allows internal egress traffic to all pods but denies access to all external endpoints (i.e
0.0.0.0/0and::/0). This BANP allows user workloads to send traffic to other in-cluster endpoints, but not to external endpoints by default.NetworkPolicycan then be used by developers in order to allow their applications to send traffic to an explicit set of external services.
-
You can set a default deny policy for cluster-local ingress in user namespaces. This BANP will force developers to have to add
-
Ensure you scope your BANP so that it only denies traffic to user namespaces and not to system namespaces. This is because the system namespaces do not have
NetworkPolicyobjects to override your BANP.
2.6.1.2. Differences to consider between AdminNetworkPolicy and NetworkPolicy Copier lienLien copié sur presse-papiers!
-
Unlike
NetworkPolicyobjects, you must use explicit labels to reference your workloads within ANP and BANP rather than using the empty ({}) catch all selector to avoid accidental traffic selection.
An empty namespace selector applied to a infrastructure namespace can make your cluster unresponsive and in a non-functional state.
-
In API semantics for ANP, you have to explicitly define allow or deny rules when you create the policy, unlike
NetworkPolicyobjects which have an implicit deny. -
Unlike
NetworkPolicyobjects,AdminNetworkPolicyobjects ingress rules are limited to in-cluster pods and namespaces so you cannot, and do not need to, set rules for ingress from the host network.