L3VPN GR Configuration
Using BGP graceful restart, the data-forwarding plane of a router can continue to process and forward packets even if the control plane - which is responsible for determining best paths - fails. Graceful restart also reduces routing flaps, stabilizing the network and reducing control-plane resource consumption.
By exchanging a new BGP capability (BGP capability code 64) in the initial BGP open messages that establish the session, the restarting router and its peers show that they are aware of the BGP graceful restart mechanism when the initial BGP connection is established. In addition, the restarting router provides its peers with a list of supported address-families (VPNv4, IPv4, and IPV6) for which it can maintain a forwarding state across a BGP restart.
The peer router's TCP connection might be cleared, when the router's BGP process is restarted. Under normal circumstances, this would cause the peer router to clear all routes associated with the restarting router. But with a BGP graceful restart, this doesn't happen. Instead, in expectation of the restarting router shortly re-establishing the BGP session, the peer router marks all routes as “stale” yet continues to use them to forward packets. Likewise, the restarting router also continues forwarding packets in the interim.
When the restarting router opens the new BGP session, it will again send BGP capability 64 to its peers. But this time, flags will be set in the graceful restart capabilities exchange to let the peer router know that the BGP process has restarted.
The goal of the BGP graceful restart was to minimize the duration and reach of an outage associated with a failed BGP process. To do this, the software extensions must be deployed on both the router restarting the BGP process and the BGP peers of that router. The peers help the BGP process regain lost forwarding information and also help isolate failures from the rest of the network.
While forwarding packets, the peer router will refresh the restarting router with any relevant BGP routing information base (RIB) updates. The peer signals that it has finished sending the updates with an "End-of-RIB" (EOR) marker - an “empty” BGP update message. EOR markers help speed convergence because once the restarting router has received them from all peers, it can begin best-path selection again using the new routing information. Similarly, the restarting router then sends any updates to its peer routers and uses the EOR marker to indicate the completion of the process.
As part of this feature, we will be extending the feature for VPNv4 AF.
Topology
In the below example shows to configure BGP VPNv4 neighborship between PE1 and PE2.
L3VPN GR Topology
L3VPN GR Configuration
Configuration
Below are the configurations and validations of L3VPN GR with OSPF as IGP. We can also configure ISIS as IGP and LDP/RSVP as transport.
PE1
#configure terminal | Enter configuration mode. |
(config)#interface lo | Specify the loopback (lo) interface to be configured. |
(config-if)#ip address 1.1.1.1/32 secondary | Set the IP address of the loopback interface to 1.1.1.1/32 |
(config-if)#exit | Exit interface mode. |
(config)#ip-vrf l3vpn | Configure IP VRF L3VPN. |
(config-vrf)#rd 1:300 | Enter RD value. |
(config-vrf)#route-target both 300:400 | Enter RT value. |
(config-vrf)#exit | Exiting from VRF mode. |
(config)#router ldp | Enter router mode for LDP. |
(config-router)#router-id 1.1.1.1 | Set the router ID to IP address 1.1.1.1. |
(config-router)#targeted-peer ipv4 2.2.2.2 | Configure targeted peer. |
(config-router-targeted-peer)#exit-targeted- peer-mode | Exit-targeted-peer-mode |
(config-router)#targeted-peer ipv4 5.5.5.5 | Configure targeted peer. |
(config-router-targeted-peer)#exit-targeted- peer-mode | Exit-targeted-peer-mode |
(config-router)#exit | Exit router mode |
(config)#interface xe1 | Enter interface mode. |
(config-if)#ip address 10.10.10.1/24 | Configure IPv4 address for xe1. |
(config-if)#label-switching | Enable label switching on interface xe1. |
(config-if)#enable-ldp ipv4 | Enable LDP for IPv4 on xe1. |
(config-if)#exit | Exit interface mode |
(config)#interface xe2.1 | Configure access-port . |
(config-if)#description MPLS-L3VPN | Giving Interface Description |
(config-if)#encapsulation dot1q 4 | Setting Encapsulation to dot1q with VLAN ID 4 |
(config-if)#load-interval 30 | Load interval setting |
(config-if)#ip vrf forwarding l3vpn | Bind the interface connected to the CE router with VRF l3vpn |
(config-if)#ip address 172.168.25.2/24 | Assign the IPv4 address. |
(config)#router ospf 1 | Configure the routing process and specify the Process ID 100. The Process ID should be a unique positive integer identifying the routing process. |
(config)#ospf router-id 1.1.1.1 | Configure OSPF router-ID same as loopback interface IP address |
(config-router) #network 1.1.1.1/32 area 0 | Define the interface on which OSPF runs and associate the area ID (0) with the interface. |
(config-router) #network 10.10.10.0/24 area 0 | Define the interface on which OSPF runs and associate the area ID (0) with the interface. |
(config-if)#exit | Exit interface mode. |
(config)#router bgp 100 | Enter router BGP mode |
(config-router)#bgp router-id 1.1.1.1 | Configuring the BGP router id 1.1.1.1. |
(config-router)#bgp graceful-restart restart-time 100 | Enable BGP GR with restart timer 100. |
(config-router)#neighbor 5.5.5.5 remote-as 100 | Configure neighbor 5.5.5.5. |
(config-router)#neighbor 5.5.5.5 update-source lo | Update source lo for neighbor 5.5.5.5. |
(config-router)#address-family ipv4 unicast | Enter address-family IPv4 unicast. |
(config-router-af)#redistribute connected | Redistribute connected. |
(config-router)#neighbor 5.5.5.5 activate | Activate neighbor. |
(config-router)#address-family vpnv4 unicast | Entering Address family VPNv4 unicast. |
(config-router-af)#neighbor 5.5.5.5 activate | Activate the neighbor 5.5.5.5. |
(config-router-af)#neighbor 5.5.5.5 capability graceful-restart | Activate capability graceful restart for neighbor 5.5.5.5. |
(config-router-af)#exit-address-family | Exit address family. |
(config-router)#address-family ipv4 vrf l3vpn | Entering address family. |
(config-router-af)#redistribute connected | Redistribute connected. |
(config-router)#neighbor 172.168.25.1 remote-as 600 | Configure neighbor 172.168.25.1. |
(config-router)#neighbor 172.168.25.1 activate | Activate neighbor. |
(config-router-af)#commit | Commit all the transactions. |
P1
#configure terminal | Enter configuration mode. |
(config)#interface lo | Specify the loopback (lo) interface to be configured. |
(config-if)#ip address 2.2.2.2/32 secondary | Set the IP address of the loopback interface to 2.2.2.2/32. |
(config-if)#exit | Exit interface mode. |
(config)#router ldp | Enter router mode for LDP. |
(config-router)#router-id 2.2.2.2 | Set the router ID to IP address 2.2.2.2. |
(config-router)#transport-address ipv4 2.2.2.2 0 | Configure the transport address for IPV4 (for IPV6, use ipv6) to be used for a TCP session over which LDP will run. |
Note: | It is preferable to use the loopback address as the transport address. |
(config-router)#targeted-peer ipv4 1.1.1.1 | Configure targeted peer. |
(config-router-targeted-peer)#exit-targeted- peer-mode | Exit-targeted-peer-mode. |
(config-router)#targeted-peer ipv4 5.5.5.5 | Configure targeted peer. |
(config-router-targeted-peer)#exit-targeted- peer-mode | Exit-targeted-peer-mode. |
(config-router)#exit | Exit-targeted-peer-mode. |
(config-if)#exit | Exit router mode. |
(config)#interface xe1 | Enter interface mode. |
(config-if)#ip address 10.10.10.2/24 | Configure IPv4 address for xe1. |
(config-if)#label-switching | Enable label switching on interface xe1. |
(config-if)#enable-ldp ipv4 | Enable LDP for IPv4 on xe1. |
(config-if)#exit | Exit interface mode. |
(config)#interface xe2 | Enter interface mode. |
(config-if)#ip address 40.40.40.1/24 | Configure IPv4 address for xe2. |
(config-if)#label-switching | Enable label switching on interface xe2. |
(config-if)#enable-ldp ipv4 | Enable LDP for IPv4 on xe2. |
(config-if)#exit | Exit interface mode. |
(config)#router ospf 1 | Configure the routing process and specify the Process ID. The Process ID should be a unique positive integer identifying the routing process. |
(config)#ospf router-id 2.2.2.2 | Configure OSPF router-ID same as loopback interface IP address. |
(config-router) #network 2.2.2.2/32 area 0 | Define the interface on which OSPF runs and associate the area ID (0) with the interface. |
(config-router) #network 10.10.10.0/24 area 0 | Define the interface on which OSPF runs and associate the area ID (0) with the interface. |
(config-router)#network 40.40.40.0/24 area 0 | Define the interface on which OSPF runs and associate the area ID (0) with the interface. |
(config-router)#bfd all-interfaces | Enable the OSPF enabled interfaces with BFD. |
(config-if)#exit | Exit interface mode. |
PE-2
#configure terminal | Enter configuration mode. |
(config)#interface lo | Specify the loopback (lo) interface to be configured. |
(config-if)#ip address 5.5.5.5/32 secondary | Set the IP address of the loopback interface to 5.5.5.5/32. |
(config-if)#exit | Exit interface mode. |
(config)#router ldp | Enter router mode for LDP. |
(config-router)#router-id 5.5.5.5 | Set the router ID to IP address 5.5.5.5. |
(config-router)#targeted-peer ipv4 1.1.1.1 | Configure targeted peer. |
(config-router-targeted-peer)#exit-targeted- peer-mode | Exit-targeted-peer-mode. |
(config-router)#targeted-peer ipv4 2.2.2.2 | Configure targeted peer. |
(config-router-targeted-peer)#exit-targeted-peer-mode | Exit-targeted-peer-mode. |
(config-router)#exit | Exit router mode. |
(config)#interface xe1 | Enter interface mode. |
(config-if)#ip address 40.40.40.2/24 | Configure IPv4 address for xe1. |
(config-if)#label-switching | Enable label switching on interface xe1. |
(config-if)#enable-ldp ipv4 | Enable LDP for IPv4 on xe1. |
(config-if)#exit | Exit interface mode. |
(config-if)#exit | Exit interface mode. |
(config)#interface xe2.1 | Enter interface mode. |
(config-if)#description MPLS-L3VPN | Giving Interface Description. |
(config-if)#encapsulation dot1q 4 | Setting Encapsulation to dot1q with VLAN ID 4. |
(config-if)#load-interval 30 | Load interval setting. |
(config-if)#ip vrf forwarding l3vpn | Bind the interface connected to the CE router with VRF L3VPN. |
(config-if)#ip address 172.168.25.2/24 | Assign the IPv4 address. |
(config)#router ospf 1 | The Process ID should be a unique positive integer identifying the routing process. |
(config)#ospf router-id 5.5.5.5 | Configure OSPF router-ID same as loopback interface IP address. |
(config-router) #network 5.5.5.5/32 area 0 | Define the interface on which OSPF runs and associate the area ID (0) with the interface. |
(config-router) #network 40.40.40.0/24 area 0 | Define the interface on which OSPF runs and associate the area ID (0) with the interface. |
(config-if)#exit | Exit interface mode. |
(config)#router bgp 100 | Enter router BGP mode |
(config-router)#bgp router-id 5.5.5.5 | Configuring the bgp router id 1.1.1.1. |
(config-router)#bgp graceful-restart restart-time 100 | Enable BGP GR with restart timer 100. |
(config-router)#neighbor 1.1.1.1 remote-as 100 | Configure neighbor 1.1.1.1. |
(config-router)#neighbor 1.1.1.1 update-source lo | Update source lo for neighbor 1.1.1.1. |
(config-router)#address-family ipv4 unicast | Enter address-family ipv4 unicast. |
(config-router-af)#redistribute connected | Redistribute connected. |
(config-router)#neighbor 1.1.1.1 activate | Activate neighbor. |
(config-router)#address-family vpnv4 unicast | Entering Address family VPNv4 unicast. |
(config-router-af)#neighbor 1.1.1.1 activate | Activate the neighbor 1.1.1.1. |
(config-router-af)#neighbor 1.1.1.1 capability graceful-restart | Activate capability graceful restart for neighbor 1.1.1.1. |
(config-router-af)#exit-address-family | Exit address family. |
(config-router)#address-family ipv4 vrf l3vpn | Entering address family. |
(config-router-af)#redistribute connected | Redistribute connected. |
(config-router)#neighbor 172.168.26.1 remote-as 700 | Configure neighbor 172.168.26.1. |
(config-router)#neighbor 172.168.26.1 activate | Activate neighbor. |
(config-router-af)#commit | Commit all the transactions. |
Validation
Restart BGP Gracefully
PE1:
PE1#restart bgp graceful
%Warning : BGP process will stop and needs to restart manually,
You may lose bgp configuration,if not saved
Proceed for graceful restart? (y/n):y
%% Managed module is down or crashed
R1#show mpls ilm-table
Codes: > - installed ILM, * - selected ILM, p - stale ILM
K - CLI ILM, T - MPLS-TP, s - Stitched ILM
S - SNMP, L - LDP, R - RSVP, C - CRLDP
B - BGP , K - CLI , V - LDP_VC, I - IGP_SHORTCUT
O - OSPF/OSPF6 SR, i - ISIS SR, k - SR CLI
P - SR Policy, U - unknown
Code FEC/VRF/L2CKT ILM-ID In-Label Out-Label In-Intf Out-Intf/VRF Nexthop LSP-Type
LSP_DEFAULT
B> p 77.77.80.0/24 7 24323 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> p 77.77.78.0/24 5 24321 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> p 77.77.77.0/24 4 24320 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> p 77.77.79.0/24 6 24322 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> p 77.77.81.0/24 8 24324 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> p 172.168.25.0/24 9 24325 Nolabel N/A l3vpn N/A LSP_DEFAULT
V l2ckt:900 1 24960 Nolabel po1 xe1 N/A LSP_DEFAULT
PE1#show mpls vrf-forwarding-table
Codes: > - installed FTN, * - selected FTN, p - stale FTN, B - BGP FTN
(m) - Service mapped over multipath transport
Code FEC FTN-ID Tunnel-id Pri LSP-Type Out-Label Out-Intf Nexthop
B> p 88.88.88.0/24 1 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B>p 88.88.89.0/24 2 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B> p 88.88.90.0/24 3 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B >p 88.88.91.0/24 4 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B >p 88.88.92.0/24 5 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B> p 172.168.26.0/24 6 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
PE1#show nsm forwarding-timer
Protocol-Name GR-State Time Remaining (sec) Disconnected-time
BGP ACTIVE 74 2022/01/13 16:33:43
PE#show run bgp
!
PE1#show ip bgp vpnv4 all
PE2:
PE2#show ip bgp vpnv4 all
Status codes: s suppressed, d damped, h history, a add-path, * valid, > best, i - internal, l - labeled
S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 1:300 (Default for VRF l3vpn)
*>i 77.77.77.0/24 1.1.1.1 0 100 0 600 i
*>i 77.77.78.0/24 1.1.1.1 0 100 0 600 i
*>i 77.77.79.0/24 1.1.1.1 0 100 0 600 i
*>i 77.77.80.0/24 1.1.1.1 0 100 0 600 i
*>i 77.77.81.0/24 1.1.1.1 0 100 0 600 i
*> l 88.88.88.0/24 172.168.26.1 0 100 0 700 i
*> l 88.88.89.0/24 172.168.26.1 0 100 0 700 i
*> l 88.88.90.0/24 172.168.26.1 0 100 0 700 i
*> l 88.88.91.0/24 172.168.26.1 0 100 0 700 i
*> l 88.88.92.0/24 172.168.26.1 0 100 0 700 i
*>i 172.168.25.0/24 1.1.1.1 0 100 0 ?
*> l 172.168.26.0/24 0.0.0.0 0 100 32768 ?
Announced routes count = 6
Accepted routes count = 6
Route Distinguisher: 1:300
S>i 77.77.77.0/24 1.1.1.1 0 100 0 600 i
S>i 77.77.78.0/24 1.1.1.1 0 100 0 600 i
S>i 77.77.79.0/24 1.1.1.1 0 100 0 600 i
S>i 77.77.80.0/24 1.1.1.1 0 100 0 600 i
S>i 77.77.81.0/24 1.1.1.1 0 100 0 600 i
S>i 172.168.25.0/24 1.1.1.1 0 100 0 ?
Announced routes count = 0
After Restarting the BGP Manually
PE1:
PE1#start-shell
bash-5.0$ su
Password:
root@PE1:/home/ocnos# cd /usr/local/sbin/
root@PE1:/usr/local/sbin# ./bgpd -d
PE1#show mpls ilm-table
Codes: > - installed ILM, * - selected ILM, p - stale ILM
K - CLI ILM, T - MPLS-TP, s - Stitched ILM
S - SNMP, L - LDP, R - RSVP, C - CRLDP
B - BGP , K - CLI , V - LDP_VC, I - IGP_SHORTCUT
O - OSPF/OSPF6 SR, i - ISIS SR, k - SR CLI
P - SR Policy, U - unknown
Code FEC/VRF/L2CKT ILM-ID In-Label Out-Label In-Intf Out-Intf/VRF Nexthop LSP-Type
B> 77.77.80.0/24 7 24323 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> 77.77.78.0/24 5 24321 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> 77.77.77.0/24 4 24320 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> 77.77.79.0/24 6 24322 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> 77.77.81.0/24 8 24324 Nolabel N/A l3vpn N/A LSP_DEFAULT
B> 172.168.25.0/24 9 24325 Nolabel N/A l3vpn N/A LSP_DEFAULT
V l2ckt:900 1 24960 Nolabel po1 xe1 N/A LSP_DEFAULT
PE1#show mpls vrf-forwarding-table
Codes: > - installed FTN, * - selected FTN, p - stale FTN, B - BGP FTN
(m) - Service mapped over multipath transport
Code FEC FTN-ID Tunnel-id Pri LSP-Type Out-Label Out-Intf Nexthop
B>88.88.88.0/24 1 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B>88.88.89.0/24 2 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B>88.88.90.0/24 3 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B>88.88.91.0/24 4 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B>88.88.92.0/24 5 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
B> 172.168.26.0/24 6 0 Yes LSP_DEFAULT 24321 - 5.5.5.5
PE2:
PE2#show ip bgp vpnv4 all
Status codes: s suppressed, d damped, h history, a add-path, * valid, > best, i - internal, l - labeled
S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 1:300 (Default for VRF l3vpn)
*>i 77.77.77.0/24 1.1.1.1 0 100 0 600 i
*>i 77.77.78.0/24 1.1.1.1 0 100 0 600 i
*>i 77.77.79.0/24 1.1.1.1 0 100 0 600 i
*>i 77.77.80.0/24 1.1.1.1 0 100 0 600 i
*>i 77.77.81.0/24 1.1.1.1 0 100 0 600 i
*> l 88.88.88.0/24 172.168.26.1 0 100 0 700 i
*> l 88.88.89.0/24 172.168.26.1 0 100 0 700 i
*> l 88.88.90.0/24 172.168.26.1 0 100 0 700 i
*> l 88.88.91.0/24 172.168.26.1 0 100 0 700 i
*> l 88.88.92.0/24 172.168.26.1 0 100 0 700 i
*>i 172.168.25.0/24 1.1.1.1 0 100 0 ?
*> l 172.168.26.0/24 0.0.0.0 0 100 32768 ?
Announced routes count = 6
Accepted routes count = 6
Route Distinguisher: 1:300
>i 77.77.77.0/24 1.1.1.1 0 100 0 600 i
>i 77.77.78.0/24 1.1.1.1 0 100 0 600 i
>i 77.77.79.0/24 1.1.1.1 0 100 0 600 i
>i 77.77.80.0/24 1.1.1.1 0 100 0 600 i
>i 77.77.81.0/24 1.1.1.1 0 100 0 600 i
>i 172.168.25.0/24 1.1.1.1 0 100 0 ?
Announced routes count = 0