6.4. 경로 광고 설정 예시
클러스터 관리자는 클러스터에 대한 다음 예제 경로 광고 설정을 구성할 수 있습니다. 이 구성은 경로 광고를 구성하는 방법을 보여주는 샘플입니다.
6.4.1. 샘플 경로 광고 설정 링크 복사링크가 클립보드에 복사되었습니다!
클러스터 관리자는 클러스터에 대한 BGP(Border Gateway Protocol) 라우팅 지원을 활성화할 수 있습니다. 이 구성은 경로 광고를 구성하는 방법을 보여주는 샘플입니다. 이 구성에서는 전체 메시 설정 대신 경로 반사를 사용합니다.
BGP 라우팅은 베어메탈 인프라에서만 지원됩니다.
사전 요구 사항
-
OpenShift CLI(
oc)를 설치합니다. -
cluster-admin권한이 있는 사용자로 클러스터에 로그인합니다. - 클러스터는 베어메탈 인프라에 설치됩니다.
- FRR 데몬 컨테이너를 실행할 클러스터에 액세스할 수 있는 베어 메탈 시스템이 있습니다.
프로세스
다음 명령을 실행하여
RouteAdvertisements기능 게이트가 활성화되었는지 확인하세요.$ oc get featuregate -oyaml | grep -i routeadvertisement출력 예
- name: RouteAdvertisements다음 명령을 실행하여 클러스터 네트워크 운영자(CNO)를 구성합니다.
$ oc patch Network.operator.openshift.io cluster --type=merge \ -p=' {"spec":{ "additionalRoutingCapabilities": { "providers": ["FRR"]}, "defaultNetwork":{"ovnKubernetesConfig"{ "routeAdvertisements":"Enabled" }}}}'CNO가 모든 노드를 다시 시작하는 데 몇 분이 걸릴 수 있습니다.
다음 명령을 실행하여 노드의 IP 주소를 가져옵니다.
$ oc get node -owide출력 예
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME master-0 Ready control-plane,master 27h v1.31.3 192.168.111.20 <none> Red Hat Enterprise Linux CoreOS 418.94.202501062026-0 5.14.0-427.50.1.el9_4.x86_64 cri-o://1.31.4-2.rhaos4.18.git33d7598.el9 master-1 Ready control-plane,master 27h v1.31.3 192.168.111.21 <none> Red Hat Enterprise Linux CoreOS 418.94.202501062026-0 5.14.0-427.50.1.el9_4.x86_64 cri-o://1.31.4-2.rhaos4.18.git33d7598.el9 master-2 Ready control-plane,master 27h v1.31.3 192.168.111.22 <none> Red Hat Enterprise Linux CoreOS 418.94.202501062026-0 5.14.0-427.50.1.el9_4.x86_64 cri-o://1.31.4-2.rhaos4.18.git33d7598.el9 worker-0 Ready worker 27h v1.31.3 192.168.111.23 <none> Red Hat Enterprise Linux CoreOS 418.94.202501062026-0 5.14.0-427.50.1.el9_4.x86_64 cri-o://1.31.4-2.rhaos4.18.git33d7598.el9 worker-1 Ready worker 27h v1.31.3 192.168.111.24 <none> Red Hat Enterprise Linux CoreOS 418.94.202501062026-0 5.14.0-427.50.1.el9_4.x86_64 cri-o://1.31.4-2.rhaos4.18.git33d7598.el9 worker-2 Ready worker 27h v1.31.3 192.168.111.25 <none> Red Hat Enterprise Linux CoreOS 418.94.202501062026-0 5.14.0-427.50.1.el9_4.x86_64 cri-o://1.31.4-2.rhaos4.18.git33d7598.el9다음 명령을 실행하여 각 노드의 기본 Pod 네트워크를 가져옵니다.
$ oc get node <node_name> -o=jsonpath={.metadata.annotations.k8s\\.ovn\\.org/node-subnets}출력 예
{"default":["10.129.0.0/23"],"ns1.udn-network-primary-layer3":["10.150.6.0/24"]}베어 메탈 하이퍼바이저에서 다음 명령을 실행하여 사용할 외부 FRR 컨테이너의 IP 주소를 가져옵니다.
$ ip -j -d route get <a cluster node's IP> | jq -r '.[] | .dev' | xargs ip -d -j address show | jq -r '.[] | .addr_info[0].local'다음 예와 같이 각 노드의 IP 주소를 포함하는 FRR에 대한
frr.conf파일을 만듭니다.frr.conf구성 파일 예시router bgp 64512 no bgp default ipv4-unicast no bgp default ipv6-unicast no bgp network import-check neighbor 192.168.111.20 remote-as 64512 neighbor 192.168.111.20 route-reflector-client neighbor 192.168.111.21 remote-as 64512 neighbor 192.168.111.21 route-reflector-client neighbor 192.168.111.22 remote-as 64512 neighbor 192.168.111.22 route-reflector-client neighbor 192.168.111.40 remote-as 64512 neighbor 192.168.111.40 route-reflector-client neighbor 192.168.111.47 remote-as 64512 neighbor 192.168.111.47 route-reflector-client neighbor 192.168.111.23 remote-as 64512 neighbor 192.168.111.23 route-reflector-client neighbor 192.168.111.24 remote-as 64512 neighbor 192.168.111.24 route-reflector-client neighbor 192.168.111.25 remote-as 64512 neighbor 192.168.111.25 route-reflector-client address-family ipv4 unicast network 192.168.1.0/24 network 192.169.1.1/32 exit-address-family address-family ipv4 unicast neighbor 192.168.111.20 activate neighbor 192.168.111.20 next-hop-self neighbor 192.168.111.21 activate neighbor 192.168.111.21 next-hop-self neighbor 192.168.111.22 activate neighbor 192.168.111.22 next-hop-self neighbor 192.168.111.40 activate neighbor 192.168.111.40 next-hop-self neighbor 192.168.111.47 activate neighbor 192.168.111.47 next-hop-self neighbor 192.168.111.23 activate neighbor 192.168.111.23 next-hop-self neighbor 192.168.111.24 activate neighbor 192.168.111.24 next-hop-self neighbor 192.168.111.25 activate neighbor 192.168.111.25 next-hop-self exit-address-family neighbor remote-as 64512 neighbor route-reflector-client address-family ipv6 unicast network 2001:db8::/128 exit-address-family address-family ipv6 unicast neighbor activate neighbor next-hop-self exit-address-family다음 내용을 포함하는
daemons라는 파일을 만듭니다.데몬구성 파일 예# This file tells the frr package which daemons to start. # # Sample configurations for these daemons can be found in # /usr/share/doc/frr/examples/. # # ATTENTION: # # When activating a daemon for the first time, a config file, even if it is # empty, has to be present *and* be owned by the user and group "frr", else # the daemon will not be started by /etc/init.d/frr. The permissions should # be u=rw,g=r,o=. # When using "vtysh" such a config file is also needed. It should be owned by # group "frrvty" and set to ug=rw,o= though. Check /etc/pam.d/frr, too. # # The watchfrr and zebra daemons are always started. # bgpd=yes ospfd=no ospf6d=no ripd=no ripngd=no isisd=no pimd=no ldpd=no nhrpd=no eigrpd=no babeld=no sharpd=no pbrd=no bfdd=yes fabricd=no vrrpd=no # # If this option is set the /etc/init.d/frr script automatically loads # the config via "vtysh -b" when the servers are started. # Check /etc/pam.d/frr if you intend to use "vtysh"! # vtysh_enable=yes zebra_options=" -A 127.0.0.1 -s 90000000" bgpd_options=" -A 127.0.0.1" ospfd_options=" -A 127.0.0.1" ospf6d_options=" -A ::1" ripd_options=" -A 127.0.0.1" ripngd_options=" -A ::1" isisd_options=" -A 127.0.0.1" pimd_options=" -A 127.0.0.1" ldpd_options=" -A 127.0.0.1" nhrpd_options=" -A 127.0.0.1" eigrpd_options=" -A 127.0.0.1" babeld_options=" -A 127.0.0.1" sharpd_options=" -A 127.0.0.1" pbrd_options=" -A 127.0.0.1" staticd_options="-A 127.0.0.1" bfdd_options=" -A 127.0.0.1" fabricd_options="-A 127.0.0.1" vrrpd_options=" -A 127.0.0.1" # configuration profile # #frr_profile="traditional" #frr_profile="datacenter" # # This is the maximum number of FD's that will be available. # Upon startup this is read by the control files and ulimit # is called. Uncomment and use a reasonable value for your # setup if you are expecting a large number of peers in # say BGP. #MAX_FDS=1024 # The list of daemons to watch is automatically generated by the init script. #watchfrr_options="" # for debugging purposes, you can specify a "wrap" command to start instead # of starting the daemon directly, e.g. to use valgrind on ospfd: # ospfd_wrap="/usr/bin/valgrind" # or you can use "all_wrap" for all daemons, e.g. to use perf record: # all_wrap="/usr/bin/perf record --call-graph -" # the normal daemon command is added to this at the end.-
frr.conf와daemons파일을 모두 같은 디렉토리(예:/tmp/frr)에 저장합니다. 다음 명령을 실행하여 외부 FRR 컨테이너를 만듭니다.
$ sudo podman run -d --privileged --network host --rm --ulimit core=-1 --name frr --volume /tmp/frr:/etc/frr quay.io/frrouting/frr:9.1.0다음
FRRConfiguration및RouteAdvertisements구성을 만듭니다.다음 내용을 포함하는
receive_all.yaml파일을 만듭니다.receive_all.yaml구성 파일 예시apiVersion: frrk8s.metallb.io/v1beta1 kind: FRRConfiguration metadata: name: receive-all namespace: openshift-frr-k8s spec: bgp: routers: - asn: 64512 neighbors: - address: 192.168.111.1 asn: 64512 toReceive: allowed: mode: all다음 내용을 포함하는
ra.yaml파일을 만듭니다.ra.yaml구성 파일 예시apiVersion: k8s.ovn.org/v1 kind: RouteAdvertisements metadata: name: default spec: nodeSelector: {} frrConfigurationSelector: {} networkSelectors: - networkSelectionType: DefaultNetwork advertisements: - "PodNetwork" - "EgressIP"
다음 명령을 실행하여
receive_all.yaml및ra.yaml파일을 적용합니다.$ for f in receive_all.yaml ra.yaml; do oc apply -f $f; done
검증
구성이 적용되었는지 확인하세요.
다음 명령을 실행하여
FRRConfiguration구성이 생성되었는지 확인하세요.$ oc get frrconfiguration -A출력 예
NAMESPACE NAME AGE openshift-frr-k8s ovnk-generated-6lmfb 4h47m openshift-frr-k8s ovnk-generated-bhmnm 4h47m openshift-frr-k8s ovnk-generated-d2rf5 4h47m openshift-frr-k8s ovnk-generated-f958l 4h47m openshift-frr-k8s ovnk-generated-gmsmw 4h47m openshift-frr-k8s ovnk-generated-kmnqg 4h47m openshift-frr-k8s ovnk-generated-wpvgb 4h47m openshift-frr-k8s ovnk-generated-xq7v6 4h47m openshift-frr-k8s receive-all 4h47m다음 명령을 실행하여
RouteAdvertisements구성이 생성되었는지 확인하세요.$ oc get ra -A출력 예
NAME STATUS default Accepted
다음 명령을 실행하여 외부 FRR 컨테이너 ID를 가져옵니다.
$ sudo podman ps | grep frr출력 예
22cfc713890e quay.io/frrouting/frr:9.1.0 /usr/lib/frr/dock... 5 hours ago Up 5 hours ago frr이전 단계에서 얻은 컨테이너 ID를 사용하여 외부 FRR 컨테이너의
vtysh세션에서 BGP 이웃과 경로를 확인합니다. 다음 명령을 실행합니다.$ sudo podman exec -it <container_id> vtysh -c "show ip bgp"출력 예
BGP table version is 10, local router ID is 192.168.111.1, vrf id 0 Default local pref 100, local AS 64512 Status codes: s suppressed, d damped, h history, * valid, > best, = multipath, i internal, r RIB-failure, S Stale, R Removed Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self Origin codes: i - IGP, e - EGP, ? - incomplete RPKI validation codes: V valid, I invalid, N Not found Network Next Hop Metric LocPrf Weight Path *>i10.128.0.0/23 192.168.111.22 0 100 0 i *>i10.128.2.0/23 192.168.111.23 0 100 0 i *>i10.129.0.0/23 192.168.111.20 0 100 0 i *>i10.129.2.0/23 192.168.111.24 0 100 0 i *>i10.130.0.0/23 192.168.111.21 0 100 0 i *>i10.130.2.0/23 192.168.111.40 0 100 0 i *>i10.131.0.0/23 192.168.111.25 0 100 0 i *>i10.131.2.0/23 192.168.111.47 0 100 0 i *> 192.168.1.0/24 0.0.0.0 0 32768 i *> 192.169.1.1/32 0.0.0.0 0 32768 i다음 명령을 실행하여 각 클러스터 노드의
frr-k8sPod를 찾으세요.$ oc -n openshift-frr-k8s get pod -owide출력 예
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES frr-k8s-86wmq 6/6 Running 0 25h 192.168.111.20 master-0 <none> <none> frr-k8s-h2wl6 6/6 Running 0 25h 192.168.111.21 master-1 <none> <none> frr-k8s-jlbgs 6/6 Running 0 25h 192.168.111.40 node1.example.com <none> <none> frr-k8s-qc6l5 6/6 Running 0 25h 192.168.111.25 worker-2 <none> <none> frr-k8s-qtxdc 6/6 Running 0 25h 192.168.111.47 node2.example.com <none> <none> frr-k8s-s5bxh 6/6 Running 0 25h 192.168.111.24 worker-1 <none> <none> frr-k8s-szgj9 6/6 Running 0 25h 192.168.111.22 master-2 <none> <none> frr-k8s-webhook-server-6cd8b8d769-kmctw 1/1 Running 0 25h 10.131.2.9 node3.example.com <none> <none> frr-k8s-zwmgh 6/6 Running 0 25h 192.168.111.23 worker-0 <none> <none>OpenShift Container Platform 클러스터에서 다음 명령을 실행하여 FRR 컨테이너의 클러스터 노드
frr-k8sPod에서 BGP 경로를 확인합니다.$ oc -n openshift-frr-k8s -c frr rsh frr-k8s-86wmq다음 명령을 실행하여 클러스터 노드의 IP 경로를 확인하세요.
sh-5.1# vtysh출력 예
Hello, this is FRRouting (version 8.5.3). Copyright 1996-2005 Kunihiro Ishiguro, et al.다음 명령을 실행하여 IP 경로를 확인하세요.
worker-2# show ip bgp출력 예
BGP table version is 10, local router ID is 192.168.111.25, vrf id 0 Default local pref 100, local AS 64512 Status codes: s suppressed, d damped, h history, * valid, > best, = multipath, i internal, r RIB-failure, S Stale, R Removed Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self Origin codes: i - IGP, e - EGP, ? - incomplete RPKI validation codes: V valid, I invalid, N Not found Network Next Hop Metric LocPrf Weight Path *>i10.128.0.0/23 192.168.111.22 0 100 0 i *>i10.128.2.0/23 192.168.111.23 0 100 0 i *>i10.129.0.0/23 192.168.111.20 0 100 0 i *>i10.129.2.0/23 192.168.111.24 0 100 0 i *>i10.130.0.0/23 192.168.111.21 0 100 0 i *>i10.130.2.0/23 192.168.111.40 0 100 0 i *> 10.131.0.0/23 0.0.0.0 0 32768 i *>i10.131.2.0/23 192.168.111.47 0 100 0 i *>i192.168.1.0/24 192.168.111.1 0 100 0 i *>i192.169.1.1/32 192.168.111.1 0 100 0 i Displayed 10 routes and 10 total pathsOpenShift Container Platform 클러스터에서 다음 명령을 실행하여 노드를 디버깅합니다.
$ oc debug node/<node_name>출력 예
Temporary namespace openshift-debug-lbtgh is created for debugging node... Starting pod/worker-2-debug-zrg4v ... To use host binaries, run `chroot /host` Pod IP: 192.168.111.25 If you don't see a command prompt, try pressing enter.다음 명령을 실행하여 BGP 경로가 광고되는지 확인하세요.
sh-5.1# ip route show | grep bgp출력 예
10.128.0.0/23 nhid 268 via 192.168.111.22 dev br-ex proto bgp metric 20 10.128.2.0/23 nhid 259 via 192.168.111.23 dev br-ex proto bgp metric 20 10.129.0.0/23 nhid 260 via 192.168.111.20 dev br-ex proto bgp metric 20 10.129.2.0/23 nhid 261 via 192.168.111.24 dev br-ex proto bgp metric 20 10.130.0.0/23 nhid 266 via 192.168.111.21 dev br-ex proto bgp metric 20 10.130.2.0/23 nhid 262 via 192.168.111.40 dev br-ex proto bgp metric 20 10.131.2.0/23 nhid 263 via 192.168.111.47 dev br-ex proto bgp metric 20 192.168.1.0/24 nhid 264 via 192.168.111.1 dev br-ex proto bgp metric 20 192.169.1.1 nhid 264 via 192.168.111.1 dev br-ex proto bgp metric 20