L3VPN GR Configuration
Using BGP graceful restart, the data-forwarding plane of a router can continue to process and forward packets even if the control plane - which is responsible for determining best paths - fails. Graceful restart also reduces routing flaps, which stabilizes the network and reduces the consumption of control-plane resources.
When the initial BGP connection is established then both the restarting router and its peers indicate their understanding of the BGP graceful restart mechanism by exchanging a new BGP capability (BGP capability code 64) in the initial BGP open messages that establish the session. The restarting router also provides to its peers a list of supported address-families (VPNv4, IPv4, IPV6) for which it has the capability to maintain forwarding state across a BGP restart.
When the router restarts its BGP process, the TCP connection to the peer router might be cleared. Under normal circumstances, this would cause the peer router to clear all routes associated with the restarting router. This does not occur with BGP graceful restart, however. Instead, the peer router marks all routes as "stale," but continues to use them to forward packets based on the expectation that the restarting router will re-establish the BGP session shortly. Likewise, the restarting router also continues forwarding packets in the interim.
When the restarting router opens the new BGP session, it will again send BGP capability 64 to its peers. But this time, flags will be set in the graceful restart capabilities exchange to let the peer router know that the BGP process has restarted.
BGP graceful restart was developed to minimize the duration and reach of an outage associated with a failed BGP process. To do so, the software extensions must be deployed on the router restarting the BGP process and on that router's BGP peers. The peers help the BGP process regain lost forwarding information and also help isolate failures from the rest of the network.
While continuing to forward packets, the peer router will refresh the restarting router with any relevant BGP routing information base (RIB) updates. The peer signals that it has finished sending the updates with an "End-of-RIB" (EOR) marker - an "empty" BGP update message. EOR markers help speed convergence because once the restarting router has received them from all peers; it knows it can begin best-path selection again using the new routing information. Similarly, the restarting router then sends any updates to its peer routers and uses the EOR marker to indicate the completion of the process.
As part of this feature, we will be extending the feature for VPNv4 AF.
Topology
In the below example shows to configure bgp vpnv4 neighborship between PE1 and PE2.
L3VPN GR Topology
L3VPN GR Configuration
Configuration
Below are the configurations and validations of L3VPN GR with OSPF as IGP. We can also configure ISIS as IGP and LDP/RSVP as transport.
PE1
#configure terminal | Enter configuration mode. |
(config)#interface lo | Specify the loopback (lo) interface to be configured. |
(config-if)#ip address 1.1.1.1/32 secondary | Set the IP address of the loopback interface to 1.1.1.1/32 |
(config-if)#exit | Exit interface mode. |
(config)#ip vrf l3vpn | Ip vrf l3vpn |
(config-vrf)#rd 1:300 | Enter RD value |
(config-vrf)#route-target both 300:400 | Enter RT value |
(config-vrf)#exit | Exiting from vrf mode |
(config)#router ldp | Enter router mode for LDP. |
(config-router)#router-id 1.1.1.1 | Set the router ID to IP address 1.1.1.1 |
(config-router)#exit | Exit router mode |
(config)#interface eth1 | Enter interface mode. |
(config-if)#ip address 10.10.10.1/24 | Configure IPv4 address for eth1. |
(config-if)#label-switching | Enable label switching on interface eth1. |
(config-if)#enable-ldp ipv4 | Enable LDP for IPv4 on eth1. |
(config-if)#exit | Exit interface mode |
(config)#router ospf 1 | Configure the routing process and specify the Process ID 100. The Process ID should be a unique positive integer identifying the routing process. |
(config)#ospf router-id 1.1.1.1 | Configure OSPF router-ID same as loopback interface IP address |
(config-router) #network 1.1.1.1/32 area 0 | Define the interface on which OSPF runs and associate the area ID (0) with the interface |
(config-router) #network 10.10.10.0/24 area 0 | Define the interface on which OSPF runs and associate the area ID (0) with the interface |
(config-if)#exit | Exit interface mode. |
(config)#router bgp 100 | Enter router bgp mode |
(config-router)#bgp router-id 1.1.1.1 | Configuring the bgp router id 1.1.1.1 |
(config-router)# bgp graceful-restart restart-time 100 | Enable BGP GR with restart timer 100 |
(config-router)# neighbor 5.5.5.5 remote-as 100 | Configure neighbor 5.5.5.5 |
(config-router)#neighbor 5.5.5.5 update-source lo | Update source lo for neighbor 5.5.5.5 |
(config-router)#address-family vpnv4 unicast | Entering Address family vpnv4 unicast |
(config-router-af)# neighbor 5.5.5.5 activate | Activate the neighbor 5.5.5.5 |
(config-router-af)# neighbor 5.5.5.5 capability graceful-restart | Activate capability graceful restart for neighbor 5.5.5.5 |
(config-router-af)# exit-address-family | Exit address family |
(config-router)# address-family ipv4 vrf l3vpn | Entering address family |
(config-router-af)# redistribute connected | Redistribute connected |
(config-router-af)#commit | Commit all the transactions |
P1
#configure terminal | Enter configuration mode. |
(config)#interface lo | Specify the loopback (lo) interface to be configured. |
(config-if)#ip address 2.2.2.2/32 secondary | Set the IP address of the loopback interface to 2.2.2.2/32 |
(config-if)#exit | Exit interface mode. |
(config)#router ldp | Enter router mode for LDP. |
(config-router)#router-id 2.2.2.2 | Set the router ID to IP address 2.2.2.2 |
(config-router)#transport-address ipv4 2.2.2.2 0 | Configure the transport address for IPV4 (for IPV6 use ipv6) to be used for a TCP session over which LDP will run. Note: It is preferable to use the loopback address as the transport address. |
(config-router)#exit | Exit-targeted-peer-mode |
(config-if)#exit | Exit router mode |
(config)#interface eth1 | Enter interface mode. |
(config-if)#ip address 10.10.10.2/24 | Configure IPv4 address for eth1. |
(config-if)#label-switching | Enable label switching on interface eth1. |
(config-if)#enable-ldp ipv4 | Enable LDP for IPv4 on eth1. |
(config-if)#exit | Exit interface mode |
(config)#interface eth2 | Enter interface mode. |
(config-if)#ip address 40.40.40.1/24 | Configure IPv4 address for eth2 |
(config-if)#label-switching | Enable label switching on interface eth2. |
(config-if)#enable-ldp ipv4 | Enable LDP for IPv4 on eth2. |
(config-if)#exit | Exit interface mode |
(config)#router ospf 1 | Configure the routing process and specify the Process ID • The Process ID should be a unique positive integer identifying the routing process. |
(config)#ospf router-id 2.2.2.2 | Configure OSPF router-ID same as loopback interface IP address |
(config-router) #network 2.2.2.2/32 area 0 | Define the interface on which OSPF runs and associate the area ID (0) with the interface |
(config-router) #network 10.10.10.0/24 area 0 | Define the interface on which OSPF runs and associate the area ID (0) with the interface |
(config-router)#network 40.40.40.0/24 area 0 | Define the interface on which OSPF runs and associate the area ID (0) with the interface |
(config-router)#bfd all-interfaces | Enable the OSPF enabled interfaces with bfd |
(config-if)#exit | Exit interface mode. |
PE-2
#configure terminal | Enter configuration mode. |
(config)#interface lo | Specify the loopback (lo) interface to be configured. |
(config)#ip vrf l3vpn | Ip vrf l3vpn |
(config-vrf)#rd 1:300 | Enter RD value |
(config-vrf)#route-target both 300:400 | Enter RT value |
(config-vrf)#exit | Exiting from vrf mode |
(config-if)#ip address 5.5.5.5/32 secondary | Set the IP address of the loopback interface to 5.5.5.5/32 |
(config-if)#exit | Exit interface mode. |
(config)#router ldp | Enter router mode for LDP. |
(config-router)#router-id 5.5.5.5 | Set the router ID to IP address 5.5.5.5 |
(config-router)#exit | Exit router mode |
(config)#interface eth1 | Enter interface mode. |
(config-if)#ip address 40.40.40.2/24 | Configure IPv4 address for eth1. |
(config-if)#label-switching | Enable label switching on interface eth1. |
(config-if)#enable-ldp ipv4 | Enable LDP for IPv4 on eth1. |
(config-if)#exit | Exit interface mode |
(config-if)#exit | Exit interface mode |
(config)#router ospf 1 | Configure the routing process and specify the Process ID • The Process ID should be a unique positive integer identifying the routing process. |
(config)#ospf router-id 5.5.5.5 | Configure OSPF router-ID same as loopback interface IP address |
(config-router) #network 5.5.5.5/32 area 0 | Define the interface on which OSPF runs and associate the area ID (0) with the interface. |
(config-router) #network 40.40.40.0/24 area 0 | Define the interface on which OSPF runs and associate the area ID (0) with the interface. |
(config-if)#exit | Exit interface mode. |
(config)#router bgp 100 | Enter router bgp mode |
(config-router)#bgp router-id 5.5.5.5 | Configuring the bgp router id 1.1.1.1 |
(config-router)# bgp graceful-restart restart-time 100 | Enable BGP GR with restart timer 100 |
(config-router)# neighbor 1.1.1.1 remote-as 100 | Configure neighbor 1.1.1.1 |
(config-router)#neighbor 1.1.1.1 update-source lo | Update source lo for neighbor 1.1.1.1 |
(config-router)#address-family vpnv4 unicast | Entering Address family vpnv4 unicast |
(config-router-af)# neighbor 1.1.1.1 activate | Activate the neighbor 1.1.1.1 |
(config-router-af)# neighbor 1.1.1.1 capability graceful-restart | Activate capability graceful restart for neighbor 1.1.1.1 |
(config-router-af)# exit-address-family | Exit address family |
(config-router)# address-family ipv4 vrf l3vpn | Entering address family |
(config-router-af)# redistribute connected | Redistribute connected |
(config-router-af)#commit | Commit all the transactions |
Validation
Restart bgp gracefully:
PE1:
PE1#restart bgp graceful
%Warning : BGP process will stop and needs to restart manually,
You may lose bgp configuration,if not saved
Proceed for graceful restart? (y/n):y
%% Managed module is down or crashed
R1#sh mpls ilm-table
Codes: > - installed ILM, * - selected ILM, p - stale ILM
K - CLI ILM, T - MPLS-TP, s - Stitched ILM
S - SNMP, L - LDP, R - RSVP, C - CRLDP
B - BGP , K - CLI , V - LDP_VC, I - IGP_SHORTCUT
O - OSPF/OSPF6 SR, i - ISIS SR, k - SR CLI
P - SR Policy, U - unknown
Code FEC/VRF/L2CKT ILM-ID In-Label Out-Label In-Intf Out-Intf/VRF Nexthop LSP-Type
LSP_DEFAULT
B> p 77.77.80.0/24 7 24323 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> p 77.77.78.0/24 5 24321 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> p 77.77.77.0/24 4 24320 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> p 77.77.79.0/24 6 24322 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> p 77.77.81.0/24 8 24324 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> p 172.168.25.0/24 9 24325 Nolabel N/A l3vpn N/A LSP_DEFAULT
V l2ckt:900 1 24960 Nolabel po1 xe1 N/A LSP_DEFAULT
PE1#sh mpls vrf-forwarding-table
Codes: > - installed FTN, * - selected FTN, p - stale FTN, B - BGP FTN
(m) - Service mapped over multipath transport
Code FEC FTN-ID Tunnel-id Pri LSP-Type Out-Label Out-Intf Nexthop
B> p 88.88.88.0/24 1 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B>p 88.88.89.0/24 2 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B> p 88.88.90.0/24 3 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B >p 88.88.91.0/24 4 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B >p 88.88.92.0/24 5 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B> p 172.168.26.0/24 6 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
PE1#sh nsm forwarding-timer
Protocol-Name GR-State Time Remaining (sec) Disconnected-time
BGP ACTIVE 74 2022/01/13 16:33:43
PE# sh run bgp
!
PE1#sh ip bgp vpnv4 all
PE2:
PE2#sh ip bgp vpnv4 all
Status codes: s suppressed, d damped, h history, a add-path, * valid, > best, i - internal, l - labeled
S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 1:300 (Default for VRF l3vpn)
*>i 77.77.77.0/24 1.1.1.1 0 100 0 600 i
*>i 77.77.78.0/24 1.1.1.1 0 100 0 600 i
*>i 77.77.79.0/24 1.1.1.1 0 100 0 600 i
*>i 77.77.80.0/24 1.1.1.1 0 100 0 600 i
*>i 77.77.81.0/24 1.1.1.1 0 100 0 600 i
*> l 88.88.88.0/24 172.168.26.1 0 100 0 700 i
*> l 88.88.89.0/24 172.168.26.1 0 100 0 700 i
*> l 88.88.90.0/24 172.168.26.1 0 100 0 700 i
*> l 88.88.91.0/24 172.168.26.1 0 100 0 700 i
*> l 88.88.92.0/24 172.168.26.1 0 100 0 700 i
*>i 172.168.25.0/24 1.1.1.1 0 100 0 ?
*> l 172.168.26.0/24 0.0.0.0 0 100 32768 ?
Announced routes count = 6
Accepted routes count = 6
Route Distinguisher: 1:300
S>i 77.77.77.0/24 1.1.1.1 0 100 0 600 i
S>i 77.77.78.0/24 1.1.1.1 0 100 0 600 i
S>i 77.77.79.0/24 1.1.1.1 0 100 0 600 i
S>i 77.77.80.0/24 1.1.1.1 0 100 0 600 i
S>i 77.77.81.0/24 1.1.1.1 0 100 0 600 i
S>i 172.168.25.0/24 1.1.1.1 0 100 0 ?
Announced routes count = 0
After restarting the bgp manually:
PE1:
PE1#start-shell
bash-5.0$ su
Password:
root@PE1:/home/ocnos# cd /usr/local/sbin/
root@PE1:/usr/local/sbin# ./bgpd -d
PE1#sh mpls ilm-table
Codes: > - installed ILM, * - selected ILM, p - stale ILM
K - CLI ILM, T - MPLS-TP, s - Stitched ILM
S - SNMP, L - LDP, R - RSVP, C - CRLDP
B - BGP , K - CLI , V - LDP_VC, I - IGP_SHORTCUT
O - OSPF/OSPF6 SR, i - ISIS SR, k - SR CLI
P - SR Policy, U - unknown
Code FEC/VRF/L2CKT ILM-ID In-Label Out-Label In-Intf Out-Intf/VRF Nexthop LSP-Type
B> 77.77.80.0/24 7 24323 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> 77.77.78.0/24 5 24321 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> 77.77.77.0/24 4 24320 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> 77.77.79.0/24 6 24322 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> 77.77.81.0/24 8 24324 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> 172.168.25.0/24 9 24325 Nolabel N/A l3vpn N/A LSP_DEFAULT
V l2ckt:900 1 24960 Nolabel po1 xe1 N/A LSP_DEFAULT
PE1#sh mpls vrf-forwarding-table
Codes: > - installed FTN, * - selected FTN, p - stale FTN, B - BGP FTN
(m) - Service mapped over multipath transport
Code FEC FTN-ID Tunnel-id Pri LSP-Type Out-Label Out-Intf Nexthop
B>88.88.88.0/24 1 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B>88.88.89.0/24 2 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B>88.88.90.0/24 3 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B>88.88.91.0/24 4 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B>88.88.92.0/24 5 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B> 172.168.26.0/24 6 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
PE2:
PE2#sh ip bgp vpnv4 all
Status codes: s suppressed, d damped, h history, a add-path, * valid, > best, i - internal, l - labeled
S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 1:300 (Default for VRF l3vpn)
*>i 77.77.77.0/24 1.1.1.1 0 100 0 600 i
*>i 77.77.78.0/24 1.1.1.1 0 100 0 600 i
*>i 77.77.79.0/24 1.1.1.1 0 100 0 600 i
*>i 77.77.80.0/24 1.1.1.1 0 100 0 600 i
*>i 77.77.81.0/24 1.1.1.1 0 100 0 600 i
*> l 88.88.88.0/24 172.168.26.1 0 100 0 700 i
*> l 88.88.89.0/24 172.168.26.1 0 100 0 700 i
*> l 88.88.90.0/24 172.168.26.1 0 100 0 700 i
*> l 88.88.91.0/24 172.168.26.1 0 100 0 700 i
*> l 88.88.92.0/24 172.168.26.1 0 100 0 700 i
*>i 172.168.25.0/24 1.1.1.1 0 100 0 ?
*> l 172.168.26.0/24 0.0.0.0 0 100 32768 ?
Announced routes count = 6
Accepted routes count = 6
Route Distinguisher: 1:300
>i 77.77.77.0/24 1.1.1.1 0 100 0 600 i
>i 77.77.78.0/24 1.1.1.1 0 100 0 600 i
>i 77.77.79.0/24 1.1.1.1 0 100 0 600 i
>i 77.77.80.0/24 1.1.1.1 0 100 0 600 i
>i 77.77.81.0/24 1.1.1.1 0 100 0 600 i
>i 172.168.25.0/24 1.1.1.1 0 100 0 ?
Announced routes count = 0