Advanced networking

Procedure

1. Prepare your configuration for the hardware MTU:

   • If your hardware MTU is specified with DHCP, update your DHCP configuration such as with the following dnsmasq configuration:

     dhcp-option-force=26,<mtu>

     Copy to Clipboard Toggle word wrap

     where:

     <mtu>

     Specifies the hardware MTU for the DHCP server to advertise.

   • If your hardware MTU is specified with a kernel command line with PXE, update that configuration accordingly.

   • If your hardware MTU is specified in a NetworkManager connection configuration, complete the following steps. This approach is the default for OpenShift Container Platform if you do not explicitly specify your network configuration with DHCP, a kernel command line, or some other method. Your cluster nodes must all use the same underlying network configuration for the following procedure to work unmodified.

     1. Find the primary network interface by entering the following command:

        $ oc debug node/<node_name> -- chroot /host nmcli -g connection.interface-name c show ovs-if-phys0

        Copy to Clipboard Toggle word wrap

        where:

        <node_name>

        Specifies the name of a node in your cluster.

     2. Create the following NetworkManager configuration in the <interface>-mtu.conf file:

        [connection-<interface>-mtu]
        match-device=interface-name:<interface>
        ethernet.mtu=<mtu>

        Copy to Clipboard Toggle word wrap

        where:

        <interface>

        Specifies the primary network interface name.

        <mtu>

        Specifies the new hardware MTU value.

Procedure

1. Create two MachineConfig objects, one for the control plane nodes and another for the worker nodes in your cluster:

   1. Create the following Butane config in the control-plane-interface.bu file:

      Note

      The Butane version you specify in the config file should match the OpenShift Container Platform version and always ends in 0. For example, 4.17.0. See "Creating machine configs with Butane" for information about Butane.

      variant: openshift
      version: 4.17.0
      metadata:
        name: 01-control-plane-interface
        labels:
          machineconfiguration.openshift.io/role: master
      storage:
        files:
          - path: /etc/NetworkManager/conf.d/99-<interface>-mtu.conf 

      1

      
            contents:
              local: <interface>-mtu.conf 

      2

      
            mode: 0600

      Copy to Clipboard Toggle word wrap

      1

      Specifies the NetworkManager connection name for the primary network interface.

      2

      Specifies the local filename for the updated NetworkManager configuration file from an earlier step.

   2. Create the following Butane config in the worker-interface.bu file:

      Note

      The Butane version you specify in the config file should match the OpenShift Container Platform version and always ends in 0. For example, 4.17.0. See "Creating machine configs with Butane" for information about Butane.

      variant: openshift
      version: 4.17.0
      metadata:
        name: 01-worker-interface
        labels:
          machineconfiguration.openshift.io/role: worker
      storage:
        files:
          - path: /etc/NetworkManager/conf.d/99-<interface>-mtu.conf 

      1

      
            contents:
              local: <interface>-mtu.conf 

      2

      
            mode: 0600

      Copy to Clipboard Toggle word wrap

      1

      Specifies the NetworkManager connection name for the primary network interface.

      2

      Specifies the local filename for the updated NetworkManager configuration file from an earlier step.

2. Create MachineConfig objects from the Butane configs by running the following command:

   $ for manifest in control-plane-interface worker-interface; do
       butane --files-dir . $manifest.bu > $manifest.yaml
     done

   Copy to Clipboard Toggle word wrap
   Warning

   Do not apply these machine configs until explicitly instructed later in this procedure. Applying these machine configs now causes a loss of stability for the cluster.

Procedure

1. To begin the MTU migration, specify the migration configuration by entering the following command. The Machine Config Operator performs a rolling reboot of the nodes in the cluster in preparation for the MTU change.

   $ oc patch Network.operator.openshift.io cluster --type=merge --patch \
     '{"spec": { "migration": { "mtu": { "network": { "from": <overlay_from>, "to": <overlay_to> } , "machine": { "to" : <machine_to> } } } } }'

   Copy to Clipboard Toggle word wrap

   where:

   <overlay_from>

   Specifies the current cluster network MTU value.

   <overlay_to>

   Specifies the target MTU for the cluster network. This value is set relative to the value of <machine_to>. For OVN-Kubernetes, this value must be 100 less than the value of <machine_to>.

   <machine_to>

   Specifies the MTU for the primary network interface on the underlying host network.

   $ oc patch Network.operator.openshift.io cluster --type=merge --patch \
     '{"spec": { "migration": { "mtu": { "network": { "from": 1400, "to": 9000 } , "machine": { "to" : 9100} } } } }'

   Copy to Clipboard Toggle word wrap

2. As the Machine Config Operator updates machines in each machine config pool, the Operator reboots each node one by one. You must wait until all the nodes are updated. Check the machine config pool status by entering the following command:

   $ oc get machineconfigpools

   Copy to Clipboard Toggle word wrap

   A successfully updated node has the following status: UPDATED=true, UPDATING=false, DEGRADED=false.

   Note

   By default, the Machine Config Operator updates one machine per pool at a time, causing the total time the migration takes to increase with the size of the cluster.

Procedure

1. Update the underlying network interface MTU value:

   • If you are specifying the new MTU with a NetworkManager connection configuration, enter the following command. The MachineConfig Operator automatically performs a rolling reboot of the nodes in your cluster.

     $ for manifest in control-plane-interface worker-interface; do
         oc create -f $manifest.yaml
       done

     Copy to Clipboard Toggle word wrap
   • If you are specifying the new MTU with a DHCP server option or a kernel command line and PXE, make the necessary changes for your infrastructure.

2. As the Machine Config Operator updates machines in each machine config pool, the Operator reboots each node one by one. You must wait until all the nodes are updated. Check the machine config pool status by entering the following command:

   $ oc get machineconfigpools

   Copy to Clipboard Toggle word wrap

   A successfully updated node has the following status: UPDATED=true, UPDATING=false, DEGRADED=false.

   Note

   By default, the Machine Config Operator updates one machine per pool at a time, causing the total time the migration takes to increase with the size of the cluster.

3. Confirm the status of the new machine configuration on the hosts:

   1. To list the machine configuration state and the name of the applied machine configuration, enter the following command:

      $ oc describe node | egrep "hostname|machineconfig"

      Copy to Clipboard Toggle word wrap

      Example output

      kubernetes.io/hostname=master-0
      machineconfiguration.openshift.io/currentConfig: rendered-master-c53e221d9d24e1c8bb6ee89dd3d8ad7b
      machineconfiguration.openshift.io/desiredConfig: rendered-master-c53e221d9d24e1c8bb6ee89dd3d8ad7b
      machineconfiguration.openshift.io/reason:
      machineconfiguration.openshift.io/state: Done

      Copy to Clipboard Toggle word wrap

      Verify that the following statements are true:

      • The value of machineconfiguration.openshift.io/state field is Done.
      • The value of the machineconfiguration.openshift.io/currentConfig field is equal to the value of the machineconfiguration.openshift.io/desiredConfig field.

   2. To confirm that the machine config is correct, enter the following command:

      $ oc get machineconfig <config_name> -o yaml | grep path:

      Copy to Clipboard Toggle word wrap

      where:

      <config_name>

      Specifies the name of the machine config from the machineconfiguration.openshift.io/currentConfig field.

      If the machine config is successfully deployed, the previous output contains the /etc/NetworkManager/conf.d/99-<interface>-mtu.conf file path and the ExecStart=/usr/local/bin/mtu-migration.sh line.

Procedure

1. To finalize the MTU migration, enter the following command for the OVN-Kubernetes network plugin:

   $ oc patch Network.operator.openshift.io cluster --type=merge --patch \
     '{"spec": { "migration": null, "defaultNetwork":{ "ovnKubernetesConfig": { "mtu": <mtu> }}}}'

   Copy to Clipboard Toggle word wrap

   where:

   <mtu>

   Specifies the new cluster network MTU that you specified with <overlay_to>.

2. After finalizing the MTU migration, each machine config pool node is rebooted one by one. You must wait until all the nodes are updated. Check the machine config pool status by entering the following command:

   $ oc get machineconfigpools

   Copy to Clipboard Toggle word wrap

   A successfully updated node has the following status: UPDATED=true, UPDATING=false, DEGRADED=false.

Verification

1. To get the current MTU for the cluster network, enter the following command:

   $ oc describe network.config cluster

   Copy to Clipboard Toggle word wrap

2. Get the current MTU for the primary network interface of a node:

   1. To list the nodes in your cluster, enter the following command:

      $ oc get nodes

      Copy to Clipboard Toggle word wrap

   2. To obtain the current MTU setting for the primary network interface on a node, enter the following command:

      $ oc adm node-logs <node> -u ovs-configuration | grep configure-ovs.sh | grep mtu | grep <interface> | head -1

      Copy to Clipboard Toggle word wrap

      where:

      <node>

      Specifies a node from the output from the previous step.

      <interface>

      Specifies the primary network interface name for the node.

      Example output

      ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8051

      Copy to Clipboard Toggle word wrap

Procedure

1. Create a namespace for the PTP Operator.

   1. Save the following YAML in the ptp-namespace.yaml file:

      apiVersion: v1
      kind: Namespace
      metadata:
        name: openshift-ptp
        annotations:
          workload.openshift.io/allowed: management
        labels:
          name: openshift-ptp
          openshift.io/cluster-monitoring: "true"

      Copy to Clipboard Toggle word wrap

   2. Create the Namespace CR:

      $ oc create -f ptp-namespace.yaml

      Copy to Clipboard Toggle word wrap

2. Create an Operator group for the PTP Operator.

   1. Save the following YAML in the ptp-operatorgroup.yaml file:

      apiVersion: operators.coreos.com/v1
      kind: OperatorGroup
      metadata:
        name: ptp-operators
        namespace: openshift-ptp
      spec:
        targetNamespaces:
        - openshift-ptp

      Copy to Clipboard Toggle word wrap

   2. Create the OperatorGroup CR:

      $ oc create -f ptp-operatorgroup.yaml

      Copy to Clipboard Toggle word wrap

3. Subscribe to the PTP Operator.

   1. Save the following YAML in the ptp-sub.yaml file:

      apiVersion: operators.coreos.com/v1alpha1
      kind: Subscription
      metadata:
        name: ptp-operator-subscription
        namespace: openshift-ptp
      spec:
        channel: "stable"
        name: ptp-operator
        source: redhat-operators
        sourceNamespace: openshift-marketplace

      Copy to Clipboard Toggle word wrap

   2. Create the Subscription CR:

      $ oc create -f ptp-sub.yaml

      Copy to Clipboard Toggle word wrap

4. To verify that the Operator is installed, enter the following command:

   $ oc get csv -n openshift-ptp -o custom-columns=Name:.metadata.name,Phase:.status.phase

   Copy to Clipboard Toggle word wrap

   Example output

   Name                         Phase
   4.17.0-202301261535          Succeeded

   Copy to Clipboard Toggle word wrap

Procedure

1. Install the PTP Operator using the OpenShift Container Platform web console:

   1. In the OpenShift Container Platform web console, click Operators → OperatorHub.
   2. Choose PTP Operator from the list of available Operators, and then click Install.
   3. On the Install Operator page, under A specific namespace on the cluster select openshift-ptp. Then, click Install.

2. Optional: Verify that the PTP Operator installed successfully:

   1. Switch to the Operators → Installed Operators page.

   2. Ensure that PTP Operator is listed in the openshift-ptp project with a Status of InstallSucceeded.

      Note

      During installation an Operator might display a Failed status. If the installation later succeeds with an InstallSucceeded message, you can ignore the Failed message.

      If the Operator does not appear as installed, to troubleshoot further:

      • Go to the Operators → Installed Operators page and inspect the Operator Subscriptions and Install Plans tabs for any failure or errors under Status.
      • Go to the Workloads → Pods page and check the logs for pods in the openshift-ptp project.

Procedure

1. Create the PtpConfig CR. For example:

   1. Depending on your requirements, use one of the following T-GM configurations for your deployment. Save the YAML in the grandmaster-clock-ptp-config.yaml file:

      Example 5.1. PTP grandmaster clock configuration for E810 NIC

      apiVersion: ptp.openshift.io/v1
      kind: PtpConfig
      metadata:
        name: grandmaster
        namespace: openshift-ptp
        annotations: {}
      spec:
        profile:
          - name: "grandmaster"
            ptp4lOpts: "-2 --summary_interval -4"
            phc2sysOpts: -r -u 0 -m -N 8 -R 16 -s $iface_master -n 24
            ptpSchedulingPolicy: SCHED_FIFO
            ptpSchedulingPriority: 10
            ptpSettings:
              logReduce: "true"
            plugins:
              e810:
                enableDefaultConfig: false
                settings:
                  LocalMaxHoldoverOffSet: 1500
                  LocalHoldoverTimeout: 14400
                  MaxInSpecOffset: 1500
                pins: $e810_pins
                #  "$iface_master":
                #    "U.FL2": "0 2"
                #    "U.FL1": "0 1"
                #    "SMA2": "0 2"
                #    "SMA1": "0 1"
                ublxCmds:
                  - args: #ubxtool -P 29.20 -z CFG-HW-ANT_CFG_VOLTCTRL,1
                      - "-P"
                      - "29.20"
                      - "-z"
                      - "CFG-HW-ANT_CFG_VOLTCTRL,1"
                    reportOutput: false
                  - args: #ubxtool -P 29.20 -e GPS
                      - "-P"
                      - "29.20"
                      - "-e"
                      - "GPS"
                    reportOutput: false
                  - args: #ubxtool -P 29.20 -d Galileo
                      - "-P"
                      - "29.20"
                      - "-d"
                      - "Galileo"
                    reportOutput: false
                  - args: #ubxtool -P 29.20 -d GLONASS
                      - "-P"
                      - "29.20"
                      - "-d"
                      - "GLONASS"
                    reportOutput: false
                  - args: #ubxtool -P 29.20 -d BeiDou
                      - "-P"
                      - "29.20"
                      - "-d"
                      - "BeiDou"
                    reportOutput: false
                  - args: #ubxtool -P 29.20 -d SBAS
                      - "-P"
                      - "29.20"
                      - "-d"
                      - "SBAS"
                    reportOutput: false
                  - args: #ubxtool -P 29.20 -t -w 5 -v 1 -e SURVEYIN,600,50000
                      - "-P"
                      - "29.20"
                      - "-t"
                      - "-w"
                      - "5"
                      - "-v"
                      - "1"
                      - "-e"
                      - "SURVEYIN,600,50000"
                    reportOutput: true
                  - args: #ubxtool -P 29.20 -p MON-HW
                      - "-P"
                      - "29.20"
                      - "-p"
                      - "MON-HW"
                    reportOutput: true
                  - args: #ubxtool -P 29.20 -p CFG-MSG,1,38,248
                      - "-P"
                      - "29.20"
                      - "-p"
                      - "CFG-MSG,1,38,248"
                    reportOutput: true
            ts2phcOpts: " "
            ts2phcConf: |
              [nmea]
              ts2phc.master 1
              [global]
              use_syslog  0
              verbose 1
              logging_level 7
              ts2phc.pulsewidth 100000000
              #cat /dev/GNSS to find available serial port
              #example value of gnss_serialport is /dev/ttyGNSS_1700_0
              ts2phc.nmea_serialport $gnss_serialport
              [$iface_master]
              ts2phc.extts_polarity rising
              ts2phc.extts_correction 0
            ptp4lConf: |
              [$iface_master]
              masterOnly 1
              [$iface_master_1]
              masterOnly 1
              [$iface_master_2]
              masterOnly 1
              [$iface_master_3]
              masterOnly 1
              [global]
              #
              # Default Data Set
              #
              twoStepFlag 1
              priority1 128
              priority2 128
              domainNumber 24
              #utc_offset 37
              clockClass 6
              clockAccuracy 0x27
              offsetScaledLogVariance 0xFFFF
              free_running 0
              freq_est_interval 1
              dscp_event 0
              dscp_general 0
              dataset_comparison G.8275.x
              G.8275.defaultDS.localPriority 128
              #
              # Port Data Set
              #
              logAnnounceInterval -3
              logSyncInterval -4
              logMinDelayReqInterval -4
              logMinPdelayReqInterval 0
              announceReceiptTimeout 3
              syncReceiptTimeout 0
              delayAsymmetry 0
              fault_reset_interval -4
              neighborPropDelayThresh 20000000
              masterOnly 0
              G.8275.portDS.localPriority 128
              #
              # Run time options
              #
              assume_two_step 0
              logging_level 6
              path_trace_enabled 0
              follow_up_info 0
              hybrid_e2e 0
              inhibit_multicast_service 0
              net_sync_monitor 0
              tc_spanning_tree 0
              tx_timestamp_timeout 50
              unicast_listen 0
              unicast_master_table 0
              unicast_req_duration 3600
              use_syslog 1
              verbose 0
              summary_interval -4
              kernel_leap 1
              check_fup_sync 0
              clock_class_threshold 7
              #
              # Servo Options
              #
              pi_proportional_const 0.0
              pi_integral_const 0.0
              pi_proportional_scale 0.0
              pi_proportional_exponent -0.3
              pi_proportional_norm_max 0.7
              pi_integral_scale 0.0
              pi_integral_exponent 0.4
              pi_integral_norm_max 0.3
              step_threshold 2.0
              first_step_threshold 0.00002
              clock_servo pi
              sanity_freq_limit  200000000
              ntpshm_segment 0
              #
              # Transport options
              #
              transportSpecific 0x0
              ptp_dst_mac 01:1B:19:00:00:00
              p2p_dst_mac 01:80:C2:00:00:0E
              udp_ttl 1
              udp6_scope 0x0E
              uds_address /var/run/ptp4l
              #
              # Default interface options
              #
              clock_type BC
              network_transport L2
              delay_mechanism E2E
              time_stamping hardware
              tsproc_mode filter
              delay_filter moving_median
              delay_filter_length 10
              egressLatency 0
              ingressLatency 0
              boundary_clock_jbod 0
              #
              # Clock description
              #
              productDescription ;;
              revisionData ;;
              manufacturerIdentity 00:00:00
              userDescription ;
              timeSource 0x20
        recommend:
          - profile: "grandmaster"
            priority: 4
            match:
              - nodeLabel: "node-role.kubernetes.io/$mcp"

      Copy to Clipboard Toggle word wrap
      Note

      For E810 Westport Channel NICs, set the value for ts2phc.nmea_serialport to /dev/gnss0.

   2. Create the CR by running the following command:

      $ oc create -f grandmaster-clock-ptp-config.yaml

      Copy to Clipboard Toggle word wrap

Verification

1. Check that the PtpConfig profile is applied to the node.

   1. Get the list of pods in the openshift-ptp namespace by running the following command:

      $ oc get pods -n openshift-ptp -o wide

      Copy to Clipboard Toggle word wrap

      Example output

      NAME                          READY   STATUS    RESTARTS   AGE     IP             NODE
      linuxptp-daemon-74m2g         3/3     Running   3          4d15h   10.16.230.7    compute-1.example.com
      ptp-operator-5f4f48d7c-x7zkf  1/1     Running   1          4d15h   10.128.1.145   compute-1.example.com

      Copy to Clipboard Toggle word wrap

   2. Check that the profile is correct. Examine the logs of the linuxptp daemon that corresponds to the node you specified in the PtpConfig profile. Run the following command:

      $ oc logs linuxptp-daemon-74m2g -n openshift-ptp -c linuxptp-daemon-container

      Copy to Clipboard Toggle word wrap

      Example output

      ts2phc[94980.334]: [ts2phc.0.config] nmea delay: 98690975 ns
      ts2phc[94980.334]: [ts2phc.0.config] ens3f0 extts index 0 at 1676577329.999999999 corr 0 src 1676577330.901342528 diff -1
      ts2phc[94980.334]: [ts2phc.0.config] ens3f0 master offset         -1 s2 freq      -1
      ts2phc[94980.441]: [ts2phc.0.config] nmea sentence: GNRMC,195453.00,A,4233.24427,N,07126.64420,W,0.008,,160223,,,A,V
      phc2sys[94980.450]: [ptp4l.0.config] CLOCK_REALTIME phc offset       943 s2 freq  -89604 delay    504
      phc2sys[94980.512]: [ptp4l.0.config] CLOCK_REALTIME phc offset      1000 s2 freq  -89264 delay    474

      Copy to Clipboard Toggle word wrap

Procedure

1. Configure automatic leap second handling in the phc2sysOpts section of the PtpConfig CR. Set the following options:

   phc2sysOpts: -r -u 0 -m -N 8 -R 16 -S 2 -s ens2f0 -n 24 

   1

   Copy to Clipboard Toggle word wrap
   Note

   Previously, the T-GM required an offset adjustment in the phc2sys configuration (-O -37) to account for historical leap seconds. This is no longer needed.

2. Configure the Intel e810 NIC to enable periodical reporting of NAV-TIMELS messages by the GPS receiver in the spec.profile.plugins.e810.ublxCmds section of the PtpConfig CR. For example:

   - args: #ubxtool -P 29.20 -p CFG-MSG,1,38,248
       - "-P"
       - "29.20"
       - "-p"
       - "CFG-MSG,1,38,248"

   Copy to Clipboard Toggle word wrap

Verification

1. Validate that the configured T-GM is receiving NAV-TIMELS messages from the connected GPS. Run the following command:

   $ oc -n openshift-ptp -c linuxptp-daemon-container exec -it $(oc -n openshift-ptp get pods -o name | grep daemon) -- ubxtool -t -p NAV-TIMELS -P 29.20

   Copy to Clipboard Toggle word wrap

   Example output

   1722509534.4417
   UBX-NAV-STATUS:
     iTOW 384752000 gpsFix 5 flags 0xdd fixStat 0x0 flags2 0x8
     ttff 18261, msss 1367642864
   
   1722509534.4419
   UBX-NAV-TIMELS:
     iTOW 384752000 version 0 reserved2 0 0 0 srcOfCurrLs 2
     currLs 18 srcOfLsChange 2 lsChange 0 timeToLsEvent 70376866
     dateOfLsGpsWn 2441 dateOfLsGpsDn 7 reserved2 0 0 0
     valid x3
   
   1722509534.4421
   UBX-NAV-CLOCK:
     iTOW 384752000 clkB 784281 clkD 435 tAcc 3 fAcc 215
   
   1722509535.4477
   UBX-NAV-STATUS:
     iTOW 384753000 gpsFix 5 flags 0xdd fixStat 0x0 flags2 0x8
     ttff 18261, msss 1367643864
   
   1722509535.4479
   UBX-NAV-CLOCK:
     iTOW 384753000 clkB 784716 clkD 435 tAcc 3 fAcc 218

   Copy to Clipboard Toggle word wrap

2. Validate that the leap-configmap resource has been successfully generated by the PTP Operator and is up to date with the latest version of the leap-seconds.list. Run the following command:

   $ oc -n openshift-ptp get configmap leap-configmap -o jsonpath='{.data.<node_name>}' 

   1

   Copy to Clipboard Toggle word wrap

   1 1

   Replace <node_name> with the node where you have installed and configured the PTP T-GM clock with automatic leap second management. Escape special characters in the node name. For example, node-1\.example\.com.

   Example output

   # Do not edit
   # This file is generated automatically by linuxptp-daemon
   #$  3913697179
   #@  4291747200
   2272060800     10    # 1 Jan 1972
   2287785600     11    # 1 Jul 1972
   2303683200     12    # 1 Jan 1973
   2335219200     13    # 1 Jan 1974
   2366755200     14    # 1 Jan 1975
   2398291200     15    # 1 Jan 1976
   2429913600     16    # 1 Jan 1977
   2461449600     17    # 1 Jan 1978
   2492985600     18    # 1 Jan 1979
   2524521600     19    # 1 Jan 1980
   2571782400     20    # 1 Jul 1981
   2603318400     21    # 1 Jul 1982
   2634854400     22    # 1 Jul 1983
   2698012800     23    # 1 Jul 1985
   2776982400     24    # 1 Jan 1988
   2840140800     25    # 1 Jan 1990
   2871676800     26    # 1 Jan 1991
   2918937600     27    # 1 Jul 1992
   2950473600     28    # 1 Jul 1993
   2982009600     29    # 1 Jul 1994
   3029443200     30    # 1 Jan 1996
   3076704000     31    # 1 Jul 1997
   3124137600     32    # 1 Jan 1999
   3345062400     33    # 1 Jan 2006
   3439756800     34    # 1 Jan 2009
   3550089600     35    # 1 Jul 2012
   3644697600     36    # 1 Jul 2015
   3692217600     37    # 1 Jan 2017
   
   #h  e65754d4 8f39962b aa854a61 661ef546 d2af0bfa

   Copy to Clipboard Toggle word wrap

Procedure

1. Create the following PtpConfig CR, and then save the YAML in the boundary-clock-ptp-config.yaml file.

   Example PTP boundary clock configuration

   apiVersion: ptp.openshift.io/v1
   kind: PtpConfig
   metadata:
     name: boundary-clock
     namespace: openshift-ptp
     annotations: {}
   spec:
     profile:
       - name: boundary-clock
         ptp4lOpts: "-2"
         phc2sysOpts: "-a -r -n 24"
         ptpSchedulingPolicy: SCHED_FIFO
         ptpSchedulingPriority: 10
         ptpSettings:
           logReduce: "true"
         ptp4lConf: |
           # The interface name is hardware-specific
           [$iface_slave]
           masterOnly 0
           [$iface_master_1]
           masterOnly 1
           [$iface_master_2]
           masterOnly 1
           [$iface_master_3]
           masterOnly 1
           [global]
           #
           # Default Data Set
           #
           twoStepFlag 1
           slaveOnly 0
           priority1 128
           priority2 128
           domainNumber 24
           #utc_offset 37
           clockClass 248
           clockAccuracy 0xFE
           offsetScaledLogVariance 0xFFFF
           free_running 0
           freq_est_interval 1
           dscp_event 0
           dscp_general 0
           dataset_comparison G.8275.x
           G.8275.defaultDS.localPriority 128
           #
           # Port Data Set
           #
           logAnnounceInterval -3
           logSyncInterval -4
           logMinDelayReqInterval -4
           logMinPdelayReqInterval -4
           announceReceiptTimeout 3
           syncReceiptTimeout 0
           delayAsymmetry 0
           fault_reset_interval -4
           neighborPropDelayThresh 20000000
           masterOnly 0
           G.8275.portDS.localPriority 128
           #
           # Run time options
           #
           assume_two_step 0
           logging_level 6
           path_trace_enabled 0
           follow_up_info 0
           hybrid_e2e 0
           inhibit_multicast_service 0
           net_sync_monitor 0
           tc_spanning_tree 0
           tx_timestamp_timeout 50
           unicast_listen 0
           unicast_master_table 0
           unicast_req_duration 3600
           use_syslog 1
           verbose 0
           summary_interval 0
           kernel_leap 1
           check_fup_sync 0
           clock_class_threshold 135
           #
           # Servo Options
           #
           pi_proportional_const 0.0
           pi_integral_const 0.0
           pi_proportional_scale 0.0
           pi_proportional_exponent -0.3
           pi_proportional_norm_max 0.7
           pi_integral_scale 0.0
           pi_integral_exponent 0.4
           pi_integral_norm_max 0.3
           step_threshold 2.0
           first_step_threshold 0.00002
           max_frequency 900000000
           clock_servo pi
           sanity_freq_limit 200000000
           ntpshm_segment 0
           #
           # Transport options
           #
           transportSpecific 0x0
           ptp_dst_mac 01:1B:19:00:00:00
           p2p_dst_mac 01:80:C2:00:00:0E
           udp_ttl 1
           udp6_scope 0x0E
           uds_address /var/run/ptp4l
           #
           # Default interface options
           #
           clock_type BC
           network_transport L2
           delay_mechanism E2E
           time_stamping hardware
           tsproc_mode filter
           delay_filter moving_median
           delay_filter_length 10
           egressLatency 0
           ingressLatency 0
           boundary_clock_jbod 0
           #
           # Clock description
           #
           productDescription ;;
           revisionData ;;
           manufacturerIdentity 00:00:00
           userDescription ;
           timeSource 0xA0
     recommend:
       - profile: boundary-clock
         priority: 4
         match:
           - nodeLabel: "node-role.kubernetes.io/$mcp"

   Copy to Clipboard Toggle word wrap

   Expand

   Table 5.7. PTP boundary clock CR configuration options

   CR field	Description
   name	The name of the PtpConfig CR.
   profile	Specify an array of one or more profile objects.
   name	Specify the name of a profile object which uniquely identifies a profile object.
   ptp4lOpts	Specify system config options for the ptp4l service. The options should not include the network interface name -i <interface> and service config file -f /etc/ptp4l.conf because the network interface name and the service config file are automatically appended.
   ptp4lConf	Specify the required configuration to start ptp4l as boundary clock. For example, ens1f0 synchronizes from a grandmaster clock and ens1f3 synchronizes connected devices.
   <interface_1>	The interface that receives the synchronization clock.
   <interface_2>	The interface that sends the synchronization clock.
   tx_timestamp_timeout	For Intel Columbiaville 800 Series NICs, set tx_timestamp_timeout to 50.
   boundary_clock_jbod	For Intel Columbiaville 800 Series NICs, ensure boundary_clock_jbod is set to 0. For Intel Fortville X710 Series NICs, ensure boundary_clock_jbod is set to 1.
   phc2sysOpts	Specify system config options for the phc2sys service. If this field is empty, the PTP Operator does not start the phc2sys service.
   ptpSchedulingPolicy	Scheduling policy for ptp4l and phc2sys processes. Default value is SCHED_OTHER. Use SCHED_FIFO on systems that support FIFO scheduling.
   ptpSchedulingPriority	Integer value from 1-65 used to set FIFO priority for ptp4l and phc2sys processes when ptpSchedulingPolicy is set to SCHED_FIFO. The ptpSchedulingPriority field is not used when ptpSchedulingPolicy is set to SCHED_OTHER.
   ptpClockThreshold	Optional. If ptpClockThreshold is not present, default values are used for the ptpClockThreshold fields. ptpClockThreshold configures how long after the PTP master clock is disconnected before PTP events are triggered. holdOverTimeout is the time value in seconds before the PTP clock event state changes to FREERUN when the PTP master clock is disconnected. The maxOffsetThreshold and minOffsetThreshold settings configure offset values in nanoseconds that compare against the values for CLOCK_REALTIME (phc2sys) or master offset (ptp4l). When the ptp4l or phc2sys offset value is outside this range, the PTP clock state is set to FREERUN. When the offset value is within this range, the PTP clock state is set to LOCKED.
   recommend	Specify an array of one or more recommend objects that define rules on how the profile should be applied to nodes.
   .recommend.profile	Specify the .recommend.profile object name defined in the profile section.
   .recommend.priority	Specify the priority with an integer value between 0 and 99. A larger number gets lower priority, so a priority of 99 is lower than a priority of 10. If a node can be matched with multiple profiles according to rules defined in the match field, the profile with the higher priority is applied to that node.
   .recommend.match	Specify .recommend.match rules with nodeLabel or nodeName values.
   .recommend.match.nodeLabel	Set nodeLabel with the key of the node.Labels field from the node object by using the oc get nodes --show-labels command. For example, node-role.kubernetes.io/worker.
   .recommend.match.nodeName	Set nodeName with the value of the node.Name field from the node object by using the oc get nodes command. For example, compute-1.example.com.

   Show more

2. Create the CR by running the following command:

   $ oc create -f boundary-clock-ptp-config.yaml

   Copy to Clipboard Toggle word wrap

Verification

1. Check that the PtpConfig profile is applied to the node.

   1. Get the list of pods in the openshift-ptp namespace by running the following command:

      $ oc get pods -n openshift-ptp -o wide

      Copy to Clipboard Toggle word wrap

      Example output

      NAME                            READY   STATUS    RESTARTS   AGE   IP               NODE
      linuxptp-daemon-4xkbb           1/1     Running   0          43m   10.1.196.24      compute-0.example.com
      linuxptp-daemon-tdspf           1/1     Running   0          43m   10.1.196.25      compute-1.example.com
      ptp-operator-657bbb64c8-2f8sj   1/1     Running   0          43m   10.129.0.61      control-plane-1.example.com

      Copy to Clipboard Toggle word wrap

   2. Check that the profile is correct. Examine the logs of the linuxptp daemon that corresponds to the node you specified in the PtpConfig profile. Run the following command:

      $ oc logs linuxptp-daemon-4xkbb -n openshift-ptp -c linuxptp-daemon-container

      Copy to Clipboard Toggle word wrap

      Example output

      I1115 09:41:17.117596 4143292 daemon.go:107] in applyNodePTPProfile
      I1115 09:41:17.117604 4143292 daemon.go:109] updating NodePTPProfile to:
      I1115 09:41:17.117607 4143292 daemon.go:110] ------------------------------------
      I1115 09:41:17.117612 4143292 daemon.go:102] Profile Name: profile1
      I1115 09:41:17.117616 4143292 daemon.go:102] Interface:
      I1115 09:41:17.117620 4143292 daemon.go:102] Ptp4lOpts: -2
      I1115 09:41:17.117623 4143292 daemon.go:102] Phc2sysOpts: -a -r -n 24
      I1115 09:41:17.117626 4143292 daemon.go:116] ------------------------------------

      Copy to Clipboard Toggle word wrap