BGP Graceful Restart Configuration
During a BGP restart, all BGP peers detect that a session had gone down and come back up. OcNOS invalidates the associated portion of the IP forwarding cache, does a BGP route re-computation, and generates BGP routing updates. The forwarding tables become corrupted and unstable.
Graceful restart helps minimize these negative effects on routing caused by a BGP restart by allowing the restarting BGP router to temporarily retain routing information and continue forwarding packets while BGP restarts. In this way, even while a router rebuilds routing and forwarding tables, the router continues to operate across the TCP connection.
Graceful restart allows a restarting router, and its neighbors, to continue forwarding packets, without disrupting network performance. Because neighboring routers assist in the restart, the restarting router can quickly resume full operation.
The graceful restart capability extends to the case when a configuration change forces a peer reset.
Graceful reset is a refinement of graceful restart to help ensure smooth restarts when a configuration change forces BGP peer reset.
Topology
Device topology for BGP in VR/VRF
RTR1
#configure terminal | Enter Configuration mode. |
(config)#interface lo | Enter interface mode for loopback. |
(config-if)#ip address 1.1.1.1/32 secondary | Configure ip address on loopback. |
(config-if)#exit | Exit interface mode. |
(config)#interface xe4 | Enter interface mode for eth2. |
(config-if)#ip address 10.10.10.1/24 | Configure ip address on eth2. |
(config-if)#exit | Exit interface mode for eth2. |
(config)# router bgp 100 | Enter router bgp mode. |
(config-router)# bgp router-id 1.1.1.11.1.1.1 | Configure bgp router-id same as loopback ip address. |
(config-router)# bgp graceful-restart | Configure Graceful Restart for BGP. |
(config-router)# neighbor 10.10.10.2 remote-as 400 | Configure Neighbor for AS-400. |
(config-router)# address-family ipv4 unicast | Enter address-family ipv4 unicast mode |
(config-router-af)# neighbor 10.10.10.2 activate | Activate neighborship |
(config-router-af)#redistribute connected | Redistributing connected Routes inside BGP. |
(config-router-af)# neighbor 10.10.10.2 capability graceful-restart | Configure GR capability inside router bgp. |
(config-router-af)#end | Exit router BGP mode. |
RTR2
#configure terminal | Enter Configuration mode. |
(config)#interface lo | Enter interface mode for loopback. |
(config-if)# ip address 2.2.2.2/32 secondary | Configure ip address on loopback. |
(config-if)#exit | Exit interface mode. |
(config)#interface xe4 | Enter interface mode for eth1. |
(config-if)#ip address 10.10.10.2/24 | Configure ip address on eth1. |
(config-if)#exit | Exit interface mode for eth1. |
(config)#interface xe8 | Enter interface mode for eth2. |
(config-if)#ip address 20.20.20.1/24 | Configure ip address on eth2. |
(config-if)#exit | Exit interface mode for eth2. |
(config)#router bgp 400 | Enter router BGP mode. |
(config-router)# bgp router-id 2.2.2.22.2.2.2 | Configure bgp router-id same as loopback ip address. |
(config-router)# bgp graceful-restart | Configure Graceful Restart for BGP. |
(config-router)# neighbor 10.10.10.1 remote-as 100 | Configure Neighbor for AS-100. |
(config-router)# neighbor 20.20.20.2 remote-as 300 | Configure Neighbor for AS-300. |
(config-router)#address-family ipv4 unicast | Enter address-family ipv4 unicast mode |
(config-router-af)#redistribute connected | Redistributing connected Routes inside BGP. |
(config-router-af)# neighbor 10.10.10.1 activate | Activate neighbor |
(config-router-af)# neighbor 20.20.20.2 activate | Activate neighbor |
(config-router-af)# neighbor 10.10.10.1 capability graceful-restart | Configure GR capability inside router bgp. |
(config-router-af)#end | Exit router ospf mode. |
RTR3
#configure terminal | Enter Configuration mode. |
(config)#interface lo | Enter interface mode for loopback. |
(config-if)#ip address 3.3.3.3/32 secondary | Configure ip address on loopback. |
(config-if)#exit | Exit interface mode. |
(config)#interface xe8 | Enter interface mode for eth2. |
(config-if)#ip address 20.20.20.2/24 | Configure ip address on eth2. |
(config-if)#exit | Exit interface mode for eth2. |
(config)# router bgp 300 | Enter router BGP mode. |
(config-router)# bgp router-id 3.3.3.33.3.3.3 | Configure bgp router-id same as loopback ip address. |
(config-router)# bgp graceful-restart | Configure Graceful Restart for BGP. |
(config-router)# neighbor 20.20.20.1 remote-as 400 | Configure Neighbor for AS-400. |
(config-router)# address-family ipv4 unicast | Enter address-family ipv4 unicast mode |
(config-router-af)#redistribute connected | Redistributing connected Routes inside BGP. |
(config-router-af)# neighbor 20.20.20.1 activate | Activate the neighbor. |
(config-router-af)# neighbor 20.20.20.1 capability graceful-restart | Configure GR capability inside router bgp. |
(config-router-af)#end | Exit router BGP mode. |
Validation
RTR1
RTR1#show bgp neighbors
BGP neighbor is 10.10.10.2, remote AS 400, local AS 100, external link
BGP version 4, local router ID 1.1.1.1, remote router ID 2.2.2.2
BGP state = Established, up for 00:03:31
Last read 00:00:15, hold time is 90, keepalive interval is 30 seconds
Neighbor capabilities:
Route refresh: advertised and received (old and new)
Address family IPv4 Unicast: advertised and received
Received 13 messages, 1 notifications, 0 in queue
Sent 13 messages, 0 notifications, 0 in queue
Route refresh request: received 0, sent 0
Minimum time between advertisement runs is 30 seconds
For address family: IPv4 Unicast
BGP table version 2, neighbor version 2
Index 1, Offset 0, Mask 0x2
AF-dependant capabilities:
Graceful restart: advertised, received
Forwarding states are being preserved
Community attribute sent to this neighbor (both)
3 accepted prefixes
2 announced prefixes
Connections established 2; dropped 1
Graceful-restart Status:
Remote restart-time is 90 sec
Local host: 10.10.10.1, Local port: 179
Foreign host: 10.10.10.2, Foreign port: 60024
Nexthop: 10.10.10.1
Nexthop global: ::
Nexthop local: ::
BGP connection: non shared network
Last Reset: 00:03:36, due to BGP Notification received
Notification Error Message: (Cease/Other Configuration Change.)
#show ip route databaseCodes: K - kernel, C - connected, S - static, R - RIP, B - BGP O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2,
ia - IS-IS inter area, E - EVPN, v - vrf leaked
> - selected route, * - FIB route, p - stale info
IP Route Table for VRF "default"
C *> 1.1.1.11.1.1.1/32 is directly connected, lo, 00:10:23
B *> 2.2.2.22.2.2.2/32 [20/0] via 10.10.10.2, xe4, 00:03:56
B *> 3.3.3.33.3.3.3/32 [20/0] via 10.10.10.2, xe4, 00:00:56
C *> 10.10.10.0/24 is directly connected, xe4, 00:09:37
B *> 20.20.20.0/24 [20/0] via 10.10.10.2, xe4, 00:03:56
C *> 127.0.0.0/8 is directly connected, lo, 00:28:58
Gateway of last resort is not set
RTR2
#show ip route database
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2,
ia - IS-IS inter area, E - EVPN, v - vrf leaked
> - selected route, * - FIB route, p - stale info
IP Route Table for VRF "default"
B *> 1.1.1.11.1.1.1/32 [20/0] via 10.10.10.1, xe4, 00:03:52
C *> 2.2.2.22.2.2.2/32 is directly connected, lo, 00:07:36
B *> 3.3.3.33.3.3.3/32 [20/0] via 20.20.20.2, xe8, 00:00:57
C *> 10.10.10.0/24 is directly connected, xe4, 00:07:12 C *> 20.20.20.0/24 is directly connected, xe8, 00:06:31
C *> 127.0.0.0/8 is directly connected, lo, 00:25:32
Gateway of last resort is not set
RTR2#show bgp neighbors
BGP neighbor is 10.10.10.1, remote AS 100, local AS 400, external link
BGP version 4, local router ID 2.2.2.2, remote router ID 1.1.1.1
BGP state = Established, up for 00:04:28
Last read 00:00:10, hold time is 90, keepalive interval is 30 seconds
Neighbor capabilities:
Route refresh: advertised and received (old and new)
Address family IPv4 Unicast: advertised and received
Received 43 messages, 1 notifications, 0 in queue
Sent 41 messages, 2 notifications, 0 in queue
Route refresh request: received 0, sent 0
Minimum time between advertisement runs is 30 seconds
For address family: IPv4 Unicast
BGP table version 2, neighbor version 2
Index 1, Offset 0, Mask 0x2
AF-dependant capabilities:
Graceful restart: advertised, received
Forwarding states are being preserved
Community attribute sent to this neighbor (both)
2 accepted prefixes
3 announced prefixes
Connections established 4; dropped 3
Graceful-restart Status:
Remote restart-time is 90 sec
Local host: 10.10.10.2, Local port: 60050
Foreign host: 10.10.10.1, Foreign port: 179
Nexthop: 10.10.10.2
Nexthop global: ::
Nexthop local: ::
BGP connection: non shared network
Last Reset: 00:04:33, due to Administratively Reset (Cease Notification sent)
Notification Error Message: (Cease/Administratively Reset.)
BGP neighbor is 20.20.20.2, remote AS 300, local AS 400, external link
BGP version 4, local router ID 2.2.2.2, remote router ID 3.3.3.3
BGP state = Established, up for 00:04:22
Last read 00:00:03, hold time is 90, keepalive interval is 30 seconds
Neighbor capabilities:
Route refresh: advertised and received (old and new)
Address family IPv4 Unicast: advertised and received
Received 31 messages, 2 notifications, 0 in queue
Sent 40 messages, 3 notifications, 0 in queue
Route refresh request: received 0, sent 0
Minimum time between advertisement runs is 30 seconds
For address family: IPv4 Unicast
BGP table version 2, neighbor version 2
Index 2, Offset 0, Mask 0x4
AF-dependant capabilities:
Graceful restart: advertised, received
Forwarding states are being preserved
Community attribute sent to this neighbor (both)
0 accepted prefixes
4 announced prefixes
Connections established 4; dropped 3
Graceful-restart Status:
Remote restart-time is 90 sec
Local host: 20.20.20.1, Local port: 179
Foreign host: 20.20.20.2, Foreign port: 56342
Nexthop: 20.20.20.1
Nexthop global: ::
Nexthop local: ::
BGP connection: non shared network
Last Reset: 00:04:22, due to Administratively Reset (Cease Notification sent)
Notification Error Message: (Cease/Administratively Reset.)
RTR2#
RTR3
#show ip route database
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
> - selected route, * - FIB route, p - stale info
IP Route Table for VRF "default"
B *> 1.1.1.11.1.1.1/32 [20/0] via 20.20.20.1, xe8, 00:01:15
B *> 2.2.2.22.2.2.2/32 [20/0] via 20.20.20.1, xe8, 00:01:15
C *> 3.3.3.3/32 is directly connected, lo
B *> 10.10.10.0/24 [20/0] via 20.20.20.1, xe8, 00:01:15
C *> 20.20.20.0/24 is directly connected, xe8
C *> 127.0.0.0/8 is directly connected, lo
Gateway of last resort is not set
RTR3#show bgp neighbors
BGP neighbor is 20.20.20.1, remote AS 400, local AS 300, external link
BGP version 4, local router ID 3.3.3.3, remote router ID 2.2.2.2
BGP state = Established, up for 00:06:47
Last read 00:00:13, hold time is 90, keepalive interval is 30 seconds
Neighbor capabilities:
Route refresh: advertised and received (old and new)
Address family IPv4 Unicast: advertised and received
Received 45 messages, 1 notifications, 0 in queue
Sent 38 messages, 2 notifications, 0 in queue
Route refresh request: received 0, sent 0
Minimum time between advertisement runs is 30 seconds
For address family: IPv4 Unicast
BGP table version 2, neighbor version 2
Index 1, Offset 0, Mask 0x2
AF-dependant capabilities:
Graceful restart: advertised, received
Forwarding states are being preserved
Community attribute sent to this neighbor (both)
4 accepted prefixes
0 announced prefixes
Connections established 4; dropped 3
Graceful-restart Status:
Remote restart-time is 90 sec
Local host: 20.20.20.2, Local port: 56342
Foreign host: 20.20.20.1, Foreign port: 179
Nexthop: 20.20.20.2
Nexthop global: ::
Nexthop local: ::
BGP connection: non shared network
Last Reset: 00:06:52, due to Administratively Reset (Cease Notification sent)
Notification Error Message: (Cease/Administratively Reset.)
RTR3#
Validation After BGP Graceful Restart
RTR2
#write
Building configuration... [OK]
#restart bgp graceful
2003 Sep 19 07:20:00.947 : NOS : BGP : CRITI : [BGP_OPR_NEIGH_STATE_DOWN_2]: Neighbour
[10.10.10.1] Session down as GR configured/unconfigured
2003 Sep 19 07:20:00.947 : NOS : BGP : CRITI : [BGP_OPR_NEIGH_STATE_DOWN_2]: Neighbour
[10.10.10.1] Session down due to config deletion
2003 Sep 19 07:20:00.947 : NOS : BGP : CRITI : [BGP_OPR_NEIGH_STATE_DOWN_2]: Neighbour
[20.20.20.2] Session down as GR configured/unconfigured
2003 Sep 19 07:20:00.947 : NOS : BGP : CRITI : [BGP_OPR_NEIGH_STATE_DOWN_2]: Neighbour
[20.20.20.2] Session down due to config deletion
#show ip route database
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2,
ia - IS-IS inter area, E - EVPN, v - vrf leaked
> - selected route, * - FIB route, p - stale info
IP Route Table for VRF "default"
B *>p 1.1.1.1/32 [20/0] via 10.10.10.1, xe4, 00:19:31
C *> 2.2.2.2/32 is directly connected, lo, 00:50:45
B *>p 3.3.3.3/32 [20/0] via 20.20.20.2, xe8, 00:19:32
C *> 10.10.10.0/24 is directly connected, xe4, 00:50:21 C *> 20.20.20.0/24 is directly connected, xe8, 00:49:40
C *> 127.0.0.0/8 is directly connected, lo, 01:08:41 Gateway of last resort is not set
#show rib forwarding-timer
Protocol-Name GR-State Time Remaining (sec) Disconnected-time BGP ACTIVE 57 2001/06/07 19:50:38
RTR1
#show ip bgp
BGP table version is 8, local router ID is 1.1.1.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, l - Labeled, S Stale
Origin codes: i - IGP, e - EGP,? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 1.1.1.1/32
0.0.0.0 0 100 32768 ?
S> 2.2.2.2/32
10.10.10.2 0 100 0 400 ?
S> 3.3.3.3/32
10.10.10.2 0 100 0 400 300 ?
*> 10.10.10.0/24 0.0.0.0 0 100 32768 ?
S 10.10.10.2 0 100 0 400 ?
S> 20.20.20.0/24 10.10.10.2 0 100 0 400 ?
Total number of prefixes 5
RTR3
#sh ip bgp
BGP table version is 14, local router ID is 3.3.3.3
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, l - labeled, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
S> 1.1.1.1/32
20.20.20.1 0 100 0 400 100 ?
S> 2.2.2.2/32
20.20.20.1 0 100 0 400 ?
*> 3.3.3.3/32
0.0.0.0 0 100 32768 ?
S> 10.10.10.0/24 20.20.20.1 0 100 0 400 ?
*> 20.20.20.0/24 0.0.0.0 0 100 32768 ?
S 20.20.20.1 0 100 0 400 ?
Total number of prefixes 5