6.4. 路由公告设置示例


作为集群管理员,您可以为集群配置以下示例路由公告设置。此配置旨在作为演示如何配置路由广告的示例。

6.4.1. 路由公告设置示例

作为集群管理员,您可以为集群启用边框网关协议(BGP)路由支持。此配置旨在作为演示如何配置路由广告的示例。配置使用路由反映而不是完整的网格设置。

注意

BGP 路由只在裸机基础架构上被支持。

先决条件

  • 已安装 OpenShift CLI(oc)。
  • 以具有 cluster-admin 权限的用户身份登录集群。
  • 集群安装在裸机基础架构上。
  • 您有一个裸机系统,可访问集群,在其中计划运行 FRR 守护进程容器。

流程

  1. 运行以下命令确认启用了 RouteAdvertisements 功能门:

    $ oc get featuregate -oyaml | grep -i routeadvertisement
    Copy to Clipboard Toggle word wrap

    输出示例

          - name: RouteAdvertisements
    Copy to Clipboard Toggle word wrap

  2. 运行以下命令配置 Cluster Network Operator (CNO):

    $ oc patch Network.operator.openshift.io cluster --type=merge \
      -p='
        {"spec":{
          "additionalRoutingCapabilities": {
            "providers": ["FRR"]},
            "defaultNetwork":{"ovnKubernetesConfig"{
              "routeAdvertisements":"Enabled"
              }}}}'
    Copy to Clipboard Toggle word wrap

    CNO 重启所有节点可能需要几分钟时间。

  3. 运行以下命令,获取节点的 IP 地址:

    $ oc get node -owide
    Copy to Clipboard Toggle word wrap

    输出示例

    NAME                                       STATUS   ROLES                  AGE   VERSION   INTERNAL-IP      EXTERNAL-IP   OS-IMAGE                                                KERNEL-VERSION                 CONTAINER-RUNTIME
    master-0                                   Ready    control-plane,master   27h   v1.31.3   192.168.111.20   <none>        Red Hat Enterprise Linux CoreOS 418.94.202501062026-0   5.14.0-427.50.1.el9_4.x86_64   cri-o://1.31.4-2.rhaos4.18.git33d7598.el9
    master-1                                   Ready    control-plane,master   27h   v1.31.3   192.168.111.21   <none>        Red Hat Enterprise Linux CoreOS 418.94.202501062026-0   5.14.0-427.50.1.el9_4.x86_64   cri-o://1.31.4-2.rhaos4.18.git33d7598.el9
    master-2                                   Ready    control-plane,master   27h   v1.31.3   192.168.111.22   <none>        Red Hat Enterprise Linux CoreOS 418.94.202501062026-0   5.14.0-427.50.1.el9_4.x86_64   cri-o://1.31.4-2.rhaos4.18.git33d7598.el9
    worker-0                                   Ready    worker                 27h   v1.31.3   192.168.111.23   <none>        Red Hat Enterprise Linux CoreOS 418.94.202501062026-0   5.14.0-427.50.1.el9_4.x86_64   cri-o://1.31.4-2.rhaos4.18.git33d7598.el9
    worker-1                                   Ready    worker                 27h   v1.31.3   192.168.111.24   <none>        Red Hat Enterprise Linux CoreOS 418.94.202501062026-0   5.14.0-427.50.1.el9_4.x86_64   cri-o://1.31.4-2.rhaos4.18.git33d7598.el9
    worker-2                                   Ready    worker                 27h   v1.31.3   192.168.111.25   <none>        Red Hat Enterprise Linux CoreOS 418.94.202501062026-0   5.14.0-427.50.1.el9_4.x86_64   cri-o://1.31.4-2.rhaos4.18.git33d7598.el9
    Copy to Clipboard Toggle word wrap

  4. 运行以下命令,获取每个节点的默认 pod 网络:

    $ oc get node <node_name> -o=jsonpath={.metadata.annotations.k8s\\.ovn\\.org/node-subnets}
    Copy to Clipboard Toggle word wrap

    输出示例

    {"default":["10.129.0.0/23"],"ns1.udn-network-primary-layer3":["10.150.6.0/24"]}
    Copy to Clipboard Toggle word wrap

  5. 在裸机虚拟机监控程序上,运行以下命令来获取要使用的外部 FRR 容器的 IP 地址:

    $ ip -j -d route get <a cluster node's IP> | jq -r '.[] | .dev' | xargs ip -d -j address show | jq -r '.[] | .addr_info[0].local'
    Copy to Clipboard Toggle word wrap
  6. 为 FRR 创建一个 frr.conf 文件,其中包含每个节点的 IP 地址,如下例所示:

    frr.conf 配置文件示例

    router bgp 64512
     no bgp default ipv4-unicast
     no bgp default ipv6-unicast
     no bgp network import-check
     neighbor 192.168.111.20 remote-as 64512
     neighbor 192.168.111.20 route-reflector-client
     neighbor 192.168.111.21 remote-as 64512
     neighbor 192.168.111.21 route-reflector-client
     neighbor 192.168.111.22 remote-as 64512
     neighbor 192.168.111.22 route-reflector-client
     neighbor 192.168.111.40 remote-as 64512
     neighbor 192.168.111.40 route-reflector-client
     neighbor 192.168.111.47 remote-as 64512
     neighbor 192.168.111.47 route-reflector-client
     neighbor 192.168.111.23 remote-as 64512
     neighbor 192.168.111.23 route-reflector-client
     neighbor 192.168.111.24 remote-as 64512
     neighbor 192.168.111.24 route-reflector-client
     neighbor 192.168.111.25 remote-as 64512
     neighbor 192.168.111.25 route-reflector-client
     address-family ipv4 unicast
      network 192.168.1.0/24
      network 192.169.1.1/32
     exit-address-family
     address-family ipv4 unicast
      neighbor 192.168.111.20 activate
      neighbor 192.168.111.20 next-hop-self
      neighbor 192.168.111.21 activate
      neighbor 192.168.111.21 next-hop-self
      neighbor 192.168.111.22 activate
      neighbor 192.168.111.22 next-hop-self
      neighbor 192.168.111.40 activate
      neighbor 192.168.111.40 next-hop-self
      neighbor 192.168.111.47 activate
      neighbor 192.168.111.47 next-hop-self
      neighbor 192.168.111.23 activate
      neighbor 192.168.111.23 next-hop-self
      neighbor 192.168.111.24 activate
      neighbor 192.168.111.24 next-hop-self
      neighbor 192.168.111.25 activate
      neighbor 192.168.111.25 next-hop-self
     exit-address-family
     neighbor  remote-as 64512
     neighbor  route-reflector-client
     address-family ipv6 unicast
      network 2001:db8::/128
     exit-address-family
     address-family ipv6 unicast
      neighbor  activate
      neighbor  next-hop-self
     exit-address-family
    Copy to Clipboard Toggle word wrap

  7. 创建名为 daemon 的文件,其中包含以下内容:

    守护进程 配置文件示例

    # This file tells the frr package which daemons to start.
    #
    # Sample configurations for these daemons can be found in
    # /usr/share/doc/frr/examples/.
    #
    # ATTENTION:
    #
    # When activating a daemon for the first time, a config file, even if it is
    # empty, has to be present *and* be owned by the user and group "frr", else
    # the daemon will not be started by /etc/init.d/frr. The permissions should
    # be u=rw,g=r,o=.
    # When using "vtysh" such a config file is also needed. It should be owned by
    # group "frrvty" and set to ug=rw,o= though. Check /etc/pam.d/frr, too.
    #
    # The watchfrr and zebra daemons are always started.
    #
    bgpd=yes
    ospfd=no
    ospf6d=no
    ripd=no
    ripngd=no
    isisd=no
    pimd=no
    ldpd=no
    nhrpd=no
    eigrpd=no
    babeld=no
    sharpd=no
    pbrd=no
    bfdd=yes
    fabricd=no
    vrrpd=no
    
    #
    # If this option is set the /etc/init.d/frr script automatically loads
    # the config via "vtysh -b" when the servers are started.
    # Check /etc/pam.d/frr if you intend to use "vtysh"!
    #
    vtysh_enable=yes
    zebra_options="  -A 127.0.0.1 -s 90000000"
    bgpd_options="   -A 127.0.0.1"
    ospfd_options="  -A 127.0.0.1"
    ospf6d_options=" -A ::1"
    ripd_options="   -A 127.0.0.1"
    ripngd_options=" -A ::1"
    isisd_options="  -A 127.0.0.1"
    pimd_options="   -A 127.0.0.1"
    ldpd_options="   -A 127.0.0.1"
    nhrpd_options="  -A 127.0.0.1"
    eigrpd_options=" -A 127.0.0.1"
    babeld_options=" -A 127.0.0.1"
    sharpd_options=" -A 127.0.0.1"
    pbrd_options="   -A 127.0.0.1"
    staticd_options="-A 127.0.0.1"
    bfdd_options="   -A 127.0.0.1"
    fabricd_options="-A 127.0.0.1"
    vrrpd_options="  -A 127.0.0.1"
    
    # configuration profile
    #
    #frr_profile="traditional"
    #frr_profile="datacenter"
    
    #
    # This is the maximum number of FD's that will be available.
    # Upon startup this is read by the control files and ulimit
    # is called. Uncomment and use a reasonable value for your
    # setup if you are expecting a large number of peers in
    # say BGP.
    #MAX_FDS=1024
    
    # The list of daemons to watch is automatically generated by the init script.
    #watchfrr_options=""
    
    # for debugging purposes, you can specify a "wrap" command to start instead
    # of starting the daemon directly, e.g. to use valgrind on ospfd:
    #   ospfd_wrap="/usr/bin/valgrind"
    # or you can use "all_wrap" for all daemons, e.g. to use perf record:
    #   all_wrap="/usr/bin/perf record --call-graph -"
    # the normal daemon command is added to this at the end.
    Copy to Clipboard Toggle word wrap

  8. frr.conf守护进程 文件保存在同一目录中,如 /tmp/frr
  9. 运行以下命令来创建外部 FRR 容器:

    $ sudo podman run -d --privileged --network host --rm --ulimit core=-1 --name frr --volume /tmp/frr:/etc/frr quay.io/frrouting/frr:9.1.0
    Copy to Clipboard Toggle word wrap
  10. 创建以下 FRRConfigurationRouteAdvertisements 配置:

    1. 创建一个包含以下内容的 receive_all.yaml 文件:

      receive_all.yaml 配置文件示例

      apiVersion: frrk8s.metallb.io/v1beta1
      kind: FRRConfiguration
      metadata:
        name: receive-all
        namespace: openshift-frr-k8s
      spec:
        bgp:
          routers:
          - asn: 64512
            neighbors:
            - address: 192.168.111.1
              asn: 64512
              toReceive:
                allowed:
                  mode: all
      Copy to Clipboard Toggle word wrap

    2. 创建一个包含以下内容的 ra.yaml 文件:

      ra.yaml 配置文件示例

      apiVersion: k8s.ovn.org/v1
      kind: RouteAdvertisements
      metadata:
        name: default
      spec:
        nodeSelector: {}
        frrConfigurationSelector: {}
        networkSelectors:
        - networkSelectionType: DefaultNetwork
        advertisements:
        - "PodNetwork"
        - "EgressIP"
      Copy to Clipboard Toggle word wrap

  11. 运行以下命令应用 receive_all.yamlra.yaml 文件:

    $ for f in receive_all.yaml ra.yaml; do oc apply -f $f; done
    Copy to Clipboard Toggle word wrap

验证

  1. 验证配置是否已应用:

    1. 运行以下命令验证 FRRConfiguration 配置是否已创建:

      $ oc get frrconfiguration -A
      Copy to Clipboard Toggle word wrap

      输出示例

      NAMESPACE           NAME                   AGE
      openshift-frr-k8s   ovnk-generated-6lmfb   4h47m
      openshift-frr-k8s   ovnk-generated-bhmnm   4h47m
      openshift-frr-k8s   ovnk-generated-d2rf5   4h47m
      openshift-frr-k8s   ovnk-generated-f958l   4h47m
      openshift-frr-k8s   ovnk-generated-gmsmw   4h47m
      openshift-frr-k8s   ovnk-generated-kmnqg   4h47m
      openshift-frr-k8s   ovnk-generated-wpvgb   4h47m
      openshift-frr-k8s   ovnk-generated-xq7v6   4h47m
      openshift-frr-k8s   receive-all            4h47m
      Copy to Clipboard Toggle word wrap

    2. 运行以下命令验证 RouteAdvertisements 配置是否已创建:

      $ oc get ra -A
      Copy to Clipboard Toggle word wrap

      输出示例

      NAME      STATUS
      default   Accepted
      Copy to Clipboard Toggle word wrap

  2. 运行以下命令来获取外部 FRR 容器 ID:

    $ sudo podman ps | grep frr
    Copy to Clipboard Toggle word wrap

    输出示例

    22cfc713890e  quay.io/frrouting/frr:9.1.0              /usr/lib/frr/dock...  5 hours ago   Up 5 hours ago               frr
    Copy to Clipboard Toggle word wrap

  3. 使用您在上一步中获取的容器 ID 检查外部 FRR 容器的 vtysh 会话中的 BGP 邻居和路由。运行以下命令:

    $ sudo podman exec -it <container_id> vtysh -c "show ip bgp"
    Copy to Clipboard Toggle word wrap

    输出示例

    BGP table version is 10, local router ID is 192.168.111.1, vrf id 0
    Default local pref 100, local AS 64512
    Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
                   i internal, r RIB-failure, S Stale, R Removed
    Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
    Origin codes:  i - IGP, e - EGP, ? - incomplete
    RPKI validation codes: V valid, I invalid, N Not found
    
        Network          Next Hop            Metric LocPrf Weight Path
     *>i10.128.0.0/23    192.168.111.22           0    100      0 i
     *>i10.128.2.0/23    192.168.111.23           0    100      0 i
     *>i10.129.0.0/23    192.168.111.20           0    100      0 i
     *>i10.129.2.0/23    192.168.111.24           0    100      0 i
     *>i10.130.0.0/23    192.168.111.21           0    100      0 i
     *>i10.130.2.0/23    192.168.111.40           0    100      0 i
     *>i10.131.0.0/23    192.168.111.25           0    100      0 i
     *>i10.131.2.0/23    192.168.111.47           0    100      0 i
     *> 192.168.1.0/24   0.0.0.0                  0         32768 i
     *> 192.169.1.1/32   0.0.0.0                  0         32768 i
    Copy to Clipboard Toggle word wrap

  4. 运行以下命令,为每个集群节点查找 frr-k8s pod:

    $ oc -n openshift-frr-k8s get pod -owide
    Copy to Clipboard Toggle word wrap

    输出示例

    NAME                                      READY   STATUS    RESTARTS   AGE   IP               NODE                                       NOMINATED NODE   READINESS GATES
    frr-k8s-86wmq                             6/6     Running   0          25h   192.168.111.20   master-0                                   <none>           <none>
    frr-k8s-h2wl6                             6/6     Running   0          25h   192.168.111.21   master-1                                   <none>           <none>
    frr-k8s-jlbgs                             6/6     Running   0          25h   192.168.111.40   node1.example.com   <none>           <none>
    frr-k8s-qc6l5                             6/6     Running   0          25h   192.168.111.25   worker-2                                   <none>           <none>
    frr-k8s-qtxdc                             6/6     Running   0          25h   192.168.111.47   node2.example.com   <none>           <none>
    frr-k8s-s5bxh                             6/6     Running   0          25h   192.168.111.24   worker-1                                   <none>           <none>
    frr-k8s-szgj9                             6/6     Running   0          25h   192.168.111.22   master-2                                   <none>           <none>
    frr-k8s-webhook-server-6cd8b8d769-kmctw   1/1     Running   0          25h   10.131.2.9       node3.example.com   <none>           <none>
    frr-k8s-zwmgh                             6/6     Running   0          25h   192.168.111.23   worker-0                                   <none>           <none>
    Copy to Clipboard Toggle word wrap

  5. 在 OpenShift Container Platform 集群中,运行以下命令来检查 FRR 容器中集群节点 frr-k8s pod 上的 BGP 路由:

    $ oc -n openshift-frr-k8s -c frr rsh frr-k8s-86wmq
    Copy to Clipboard Toggle word wrap
  6. 运行以下命令,检查集群节点的 IP 路由:

    sh-5.1# vtysh
    Copy to Clipboard Toggle word wrap

    输出示例

    Hello, this is FRRouting (version 8.5.3).
    Copyright 1996-2005 Kunihiro Ishiguro, et al.
    Copy to Clipboard Toggle word wrap

  7. 运行以下命令检查 IP 路由:

    worker-2# show ip bgp
    Copy to Clipboard Toggle word wrap

    输出示例

    BGP table version is 10, local router ID is 192.168.111.25, vrf id 0
    Default local pref 100, local AS 64512
    Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
                   i internal, r RIB-failure, S Stale, R Removed
    Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
    Origin codes:  i - IGP, e - EGP, ? - incomplete
    RPKI validation codes: V valid, I invalid, N Not found
    
        Network          Next Hop            Metric LocPrf Weight Path
     *>i10.128.0.0/23    192.168.111.22           0    100      0 i
     *>i10.128.2.0/23    192.168.111.23           0    100      0 i
     *>i10.129.0.0/23    192.168.111.20           0    100      0 i
     *>i10.129.2.0/23    192.168.111.24           0    100      0 i
     *>i10.130.0.0/23    192.168.111.21           0    100      0 i
     *>i10.130.2.0/23    192.168.111.40           0    100      0 i
     *> 10.131.0.0/23    0.0.0.0                  0         32768 i
     *>i10.131.2.0/23    192.168.111.47           0    100      0 i
     *>i192.168.1.0/24   192.168.111.1            0    100      0 i
     *>i192.169.1.1/32   192.168.111.1            0    100      0 i
    
    Displayed  10 routes and 10 total paths
    Copy to Clipboard Toggle word wrap

  8. 在 OpenShift Container Platform 集群中,运行以下命令来调试节点:

    $ oc debug node/<node_name>
    Copy to Clipboard Toggle word wrap

    输出示例

    Temporary namespace openshift-debug-lbtgh is created for debugging node...
    Starting pod/worker-2-debug-zrg4v ...
    To use host binaries, run `chroot /host`
    Pod IP: 192.168.111.25
    If you don't see a command prompt, try pressing enter.
    Copy to Clipboard Toggle word wrap

  9. 运行以下命令确认 BGP 路由正在公告:

    sh-5.1# ip route show | grep bgp
    Copy to Clipboard Toggle word wrap

    输出示例

    10.128.0.0/23 nhid 268 via 192.168.111.22 dev br-ex proto bgp metric 20
    10.128.2.0/23 nhid 259 via 192.168.111.23 dev br-ex proto bgp metric 20
    10.129.0.0/23 nhid 260 via 192.168.111.20 dev br-ex proto bgp metric 20
    10.129.2.0/23 nhid 261 via 192.168.111.24 dev br-ex proto bgp metric 20
    10.130.0.0/23 nhid 266 via 192.168.111.21 dev br-ex proto bgp metric 20
    10.130.2.0/23 nhid 262 via 192.168.111.40 dev br-ex proto bgp metric 20
    10.131.2.0/23 nhid 263 via 192.168.111.47 dev br-ex proto bgp metric 20
    192.168.1.0/24 nhid 264 via 192.168.111.1 dev br-ex proto bgp metric 20
    192.169.1.1 nhid 264 via 192.168.111.1 dev br-ex proto bgp metric 20
    Copy to Clipboard Toggle word wrap

返回顶部
Red Hat logoGithubredditYoutubeTwitter

学习

尝试、购买和销售

社区

关于红帽文档

通过我们的产品和服务,以及可以信赖的内容,帮助红帽用户创新并实现他们的目标。 了解我们当前的更新.

让开源更具包容性

红帽致力于替换我们的代码、文档和 Web 属性中存在问题的语言。欲了解更多详情,请参阅红帽博客.

關於紅帽

我们提供强化的解决方案,使企业能够更轻松地跨平台和环境(从核心数据中心到网络边缘)工作。

Theme

© 2025 Red Hat