VXLAN Quality of Service Configuration
This chapter contains the configurations for VXLAN Quality of Service (QoS) .
Overview
VXLAN enables multiple tenants to operate in a data center. Each tenant is assigned a priority group to prioritize their traffic. Cloud carriers want to use quality of service to differentiate different applications.
Data center networks are being increasingly used by telecommunications operators as well as by enterprises. Currently these networks are organized as one large Layer 2 network in a single building. In some cases such a network is extended geographically using Virtual Local Area Network (VLAN) technologies as an even larger Layer 2 network  connecting the virtual machines (VM), each with its own MAC address.
Multiple tenants might want their own isolated network domain. In a data center hosting multiple tenants, each tenant may independently assign MAC addresses and VLAN IDs and this might lead to duplication.
Cloud carriers wish to categorize the traffic based on the application such as voice, video, etc. Based on the type of the application different traffic classes may be identified and different  priority levels can be assigned to each. To do so, quality of service marking is needed in VXLAN.
This chapter shows how to mark packet headers with the VXLAN tunnel end point (VTEP) when the frames are introduced by the virtual machines. The (re)marking /setting of QoS field DSCP/TOS in the VXLAN IP header is done with the two modes which are set globally.
Topology
VXLAN QoS
COS-DSCP
RTR1/VTEP1
 
| #configure terminal | Enter Configure mode. | 
| (config)#mac vrf vrf1 | Create mac routing/forwarding instance with vrf1 name and enter into vrf mode | 
| (config-vrf)#rd 1.1.1.1:11 | Assign RD value | 
| (config-vrf)#route-target both 10.10.10.10:100 | Assign route-target value for import/export | 
| (config-vrf)#exit | Exit from vrf mode | 
| (config)#qos enable | Enable qos | 
| (config)#qos profile cos-to-queue COS-QUE | Create qos profile for mapping traffic towards tunnel from access-if. | 
| (config-ingress-cos-map)#cos 2 queue 3 | Configure particular COS value to the queue value for con-figured profile. | 
| (config-ingress-cos-map)#exit | Exit from qos profile config mode | 
| (config)#qos profile queue-color-to-dscp QUE-DSCP | Create qos profile for attaching in vxlan tunnel egress.  | 
| (config-egress-dscp-map)#queue 3 dscp 16 | Configure particular queue value to the dscp value for configured profile. | 
| (config-egress-dscp-encap-map)#exit | Exit from qos profile config mode | 
| (config)#interface po2 | Create a port channel po2 | 
| (config-if)#switchport | Configure port as switchport | 
| (config-if)#load-interval 30 | Set load-interval | 
| (config-if)#interface po24 | Create a port channel po24 | 
| (config-if)#load-interval 30 | Configure port as switchport | 
| (config-if)#ip address 24.1.1.1/30 | Set load-interval | 
| (config-if)#interface lo | Enter in to loopback interface | 
| (config-if)#ip address 1.1.1.1/32 secondary | Configure ip address | 
| (config-if)#interface xe1 | Enter in to interface mode | 
| (config-if)#channel-group 2 mode active | Map to channel-group | 
| (config-if)#interface xe2 | Enter in to interface mode | 
| (config-if)#channel-group 2 mode active | Map to channel-group | 
| (config-if)#interface xe14 | Enter in to interface mode | 
| (config-if)#channel-group 24 mode ac-tive | Map to channel-group | 
| (config-if)#interface xe15 | Enter in to interface mode | 
| (config-if)#channel-group 24 mode ac-tive | Map to channel-group | 
| (config-if)#router ospf 1 | Create ospf instance | 
| (config-router)#ospf router-id 1.1.1.1 | Configure ospf router-id | 
| (config-router)#network 1.1.1.1/32 area 0.0.0.0 | Configure loopback network address in to ospf | 
| (config-router)#network 24.1.1.0/30 area 0.0.0.0 | Configure network address in to ospf | 
| (config-router)#router bgp 100 | Enter into Router BGP mode | 
| (config-router)#neighbor 6.6.6.6 re-mote-as 100 | Specify a neighbor router with peer ip address and remote-as defined | 
| (config-router)#neighbor 6.6.6.6 up-date-source lo | Specify the neighbor to use loopback address as source | 
| (config-router)#address-family l2vpn evpn | Enter into l2vpn evpn address-family | 
| (config-router-af)#neighbor 6.6.6.6 activate |  Activate the neighbor to address-family | 
| (config-router)#nvo vxlan vtep-ip-global 1.1.1.1 | Configure Source vtep-ip-global configuration | 
| (config)#nvo vxlan tunnel qos-map-mode cos-dscp egress QUE-DSCP | Configure the mapping qos profile in to vxlan tunnel egress | 
| (config)#nvo vxlan id 1 in-gress-replication inner-vid-disabled | Create vnid 1 and disable inner-vid | 
| (config-nvo)#vxlan host-reachability-protocol evpn-bgp vrf1 | Assign vrf for evpn-bgp to carry EVPN route | 
| (config-nvo)#nvo vxlan access-if port-vlan po2 1001 | Create vxlan access-if with vlan 1001 | 
| (config-nvo-acc-if)#no shutdown | No shut the vxlan access-if | 
| (config-nvo-acc-if)#map vnid 1 | Map vnid to the vxlan access-if | 
| (config-nvo-acc-if)#map qos-profile cos-to-queue COS-QUE | Map qos profile for vxlan access-if ingress traffic from CE | 
| (config-nvo-acc-if)#exit | Exit from VxLAN access-interface mode and enter into configuration mode. | 
| (config)#commit | Commit the candidate configuration to the running configuration | 
R2
 
| #configure terminal | Enter Configure mode. | 
| (config)#interface po24 | Create port channel | 
| (config-if)#load-interval 30 | Set load-interval | 
| (config-if)#ip address 24.1.1.2/30 | Assign ip address | 
| (config-if)#interface po46 | Create port channel | 
| (config-if)#load-interval 30 | Set load-interval | 
| (config-if)#ip address 46.1.1.1/30 | Assign ip address | 
| (config-if)#interface lo | Enter in to loopback interface | 
| (config-if)#ip address 4.4.4.4/32 sec-ondary | Assign secondary ip address | 
| (config-if)#interface xe4 | Enter into interface mode  | 
| (config-if)#channel-group 46 mode active | Map port channel to the interface | 
| (config-if)#interface xe5 | Enter into interface mode  | 
| (config-if)#channel-group 46 mode active | Map port channel to the interface | 
| (config-if)#interface xe14 | Enter into interface mode  | 
| (config-if)#channel-group 24 mode active | Map port channel to the interface | 
| (config-if)#interface xe15 | Enter into interface mode  | 
| (config-if)#channel-group 24 mode active | Map port channel to the interface | 
| (config-if)#router ospf 1 | Create ospf instance  | 
| (config-router)#ospf router-id 4.4.4.4 | Configure ospf router-id | 
| (config-router)#network 4.4.4.4/32 area 0.0.0.0 | Configure ospf network address with respective area  | 
| (config-router)#network 24.1.1.0/30 area 0.0.0.0 | Configure ospf network address with respective area | 
| (config-router)#network 46.1.1.0/30 area 0.0.0.0 | Configure ospf network address with respective area | 
| (config-router)#exit | Exit from router mode. | 
| (config)#commit | Commit the candidate configuration to the running configuration | 
RTR3/VTEP2
 
| #configure terminal |  Enter Configuration mode | 
| (config)#mac vrf vrf1 | Create mac routing/forwarding instance with vrf1 name and enter into vrf mode | 
| (config-vrf)#rd 6.6.6.6:11 | Assign RD value | 
| (config-vrf)#route-target both 10.10.10.10:100 | Assign route-target value for import/export | 
| (config-vrf)#exit | Exit from vrf mode | 
| (config)#qos enable | Enable QOS | 
| (config)#qos statistics | Enable QOS statistics | 
| (config)#qos profile queue-color-to-cos QUE-COS | Create qos profile for mapping incoming traffic from tunnel to access-if. | 
| (config-egress-cos-map)#queue 4 cos 5 | Configure particular queue value to the cos value for con-figured profile. | 
| (config-egress-cos-map)#qos profile dscp-to-queue DSCP-QUE | Create qos profile for attaching in vxlan tunnel ingress. | 
| (config-ingress-dscp-map)#dscp 16 queue 4 | Configure particular dscp value to the queue value for con-figured profile. | 
| (config-egress-dscp-map)#interface po46 | Create port channel | 
| (config-if)#load-interval 30 | Set load interval | 
| (config-if)#ip address 46.1.1.2/30 |  Assign ip address | 
| (config-if)#interface lo | Enter into loopback interface | 
| (config-if)#ip address 6.6.6.6/32 secondary |  Assign secondary ip address | 
| (config-if)#interface xe4 | Enter into interface mode | 
| (config-if)#channel-group 46 mode active | Map channel group into the interface | 
| (config-if)#interface xe5 | Enter into interface mode | 
| (config-if)#channel-group 46 mode active | Map channel group into the interface | 
| (config-if)#interface xe15 | Enter into interface mode | 
| (config-if)#switchport | Make interface as L2 port | 
| (config-if)#load-interval 30 | Set load interval | 
| (config-if)#router ospf 1 | Create ospf instance | 
| (config-router)#ospf router-id 6.6.6.6 | Configure ospf router-id | 
| (config-router)#network 6.6.6.6/32 area 0.0.0.0 | Configure ospf network address with respective area | 
| (config-router)#network 46.1.1.0/30 area 0.0.0.0 | Configure ospf network address with respective area | 
| (config-router)#router bgp 100 | Enter into Router BGP mode | 
| (config-router)#neighbor 1.1.1.1 re-mote-as 100 | Specify a neighbor router with peer ip address and remote-as defined | 
| (config-router)#neighbor 1.1.1.1 up-date-source lo | Specify the neighbor to use loopback address as source | 
| (config-router)#address-family l2vpn evpn | Enter into l2vpn evpn address-family | 
| (config-router-af)#neighbor 1.1.1.1 activate | Activate the neighbor to address-family | 
| (config)#nvo vxlan vtep-ip-global 6.6.6.6 | Configure Source vtep-ip-global configuration | 
| (config)#nvo vxlan tunnel qos-map-mode cos-dscp ingress DSCP-QUE | Configure the mapping qos profile in to vxlan tunnel ingress | 
| (config)#nvo vxlan id 1 in-gress-replication inner-vid-disabled | Create vnid 1 and disable inner-vid | 
| (config-nvo)#vxlan host-reachability-protocol evpn-bgp vrf1 | Assign vrf for evpn-bgp to carry EVPN route | 
| (config-nvo)#nvo vxlan access-if port-vlan xe15 1000 | Create vxlan access-if with vlan 1000 | 
| (config-nvo-acc-if)#no shutdown | No shut the vxlan access-if | 
| (config-nvo-acc-if)#map vnid 1 | Map vnid to the vxlan access-if | 
| (config-nvo-acc-if)#map qos-profile queue-color-to-cos QUE-COS | Map qos profile for vxlan access-if egress traffic to CE | 
| (config-nvo-acc-if)#exit | Exit from VxLAN access-interface mode and enter into configuration mode | 
| (config)#commit | Commit the candidate configuration to the running configuration | 
Validation
As per the QoS configuration, when L2 traffic with cos value 2 sent to VTEP1 access-if, the packets forwarded to queue 3 and packets in queue 3 are mapped with overlay dscp value 16 while egress out of tunnel. At VTEP2, when packets with overlay dscp value 16 ingresses at tunnel, it is forwarded to queue 4 and packets of queue 4 are remarked with cos value 5.
RTR1/VTEP1
VTEP1#sh run nvo vxlan
!
nvo vxlan enable
!
nvo vxlan vtep-ip-global 1.1.1.1
!
nvo vxlan tunnel qos-map-mode cos-dscp egress QUE-DSCP
!
nvo vxlan id 1 ingress-replication inner-vid-disabled
 vxlan host-reachability-protocol evpn-bgp vrf1
!
nvo vxlan access-if port-vlan po2 1001
 map vnid 1
 map qos-profile cos-to-queue COS-QUE
!
VTEP1#show run qos
qos enable
!
qos profile cos-to-queue COS-QUE
 cos 2 dei all queue 3
!
qos profile queue-color-to-dscp QUE-DSCP
 queue 3 color all dscp 16
!
 
VTEP1#show interface xe14 count queue-stats
E - Egress, I - Ingress, Q-Size is in bytes
+--------------------+--------+-----------------+-------------------+-----------------+-------------------+
|   Queue/Class-map  | Q-Size |     Tx pkts     |     Tx bytes      |  Dropped pkts   |   Dropped bytes   |
+--------------------+--------+-----------------+-------------------+-----------------+-------------------+
q0                (E) 12517376 0                 0                   0                 0
q1                (E) 12517376 0                 0                   0                 0
q2                (E) 12517376 0                 0                   0                 0
q3                (E) 12517376 205284588         188040683524        0                 0
q4                (E) 12517376 0                 0                   0                 0
q5                (E) 12517376 0                 0                   0                 0
q6                (E) 12517376 0                 0                   0                 0
q7                (E) 12517376 7518              1007412             0                 0
 
VTEP1#show interface xe15 count queue-stats
E - Egress, I - Ingress, Q-Size is in bytes
+--------------------+--------+-----------------+-------------------+-----------------+-------------------+
|   Queue/Class-map  | Q-Size |     Tx pkts     |     Tx bytes      |  Dropped pkts   |   Dropped bytes   |
+--------------------+--------+-----------------+-------------------+-----------------+-------------------+
q0                (E) 12517376 0                 0                   0                 0
q1                (E) 12517376 0                 0                   0                 0
q2                (E) 12517376 0                 0                   0                 0
q3                (E) 12517376 205624494         188352040168        0                 0
q4                (E) 12517376 0                 0                   0                 0
q5                (E) 12517376 0                 0                   0                 0
q6                (E) 12517376 0                 0                   0                 0
q7                (E) 12517376 9006              1136741             0                 0
 
VTEP1#show nvo vxlan
VXLAN Information
=================
   Codes: NW - Network Port
          AC - Access Port
         (u) - Untagged
 
VNID     VNI-Name     VNI-Type Type Interface ESI                           VLAN DF-Status Src-Addr         Dst-Addr
_______________________________________________________________________________________________________________________________
1        ----         L2       NW   ----      ----                          ---- ----      1.1.1.1          6.6.6.6
1        ----         AC   po2       --- Single Homed port ---    1001 ----      ----             ----
 
Total number of entries are 3
 
VTEP1#show nvo vxlan mac-table
=====================================================================================================================================================
                                                         VXLAN MAC Entries
=====================================================================================================================================================
VNID      Interface  VlanId Inner-VlanId Mac-Addr       VTEP-Ip/ESI                    Type                 Status              AccessPortDesc
_____________________________________________________________________________________________________________________________________________________
 
1         po2        1001   ----         0000.2000.9991 1.1.1.1                        Dynamic Local        -------             -------
 
Total number of entries are : 1
 
VTEP1#show nvo vxlan tunnel
VXLAN Network tunnel Entries
 Source           Destination      Status          Up/Down        Update
====================================================================================
1.1.1.1          6.6.6.6          Installed        00:11:29        00:11:29
 
Total number of entries are 2
 
VTEP1#show qos-profile COS-QUE
profile name: COS-QUE
profile type: cos-to-queue
profile attached to 1 instances
configured mapping:
 cos 2 dei all queue 3
Detailed mapping:
 ---------------+----------------- | ---------------+-----------------
      INPUT     |     OUTPUT       |      INPUT     |     OUTPUT
 ---------------+----------------- | ---------------+-----------------
   COS  |  DEI  | Queue |  Color   |   COS  |  DEI  | Queue |  Color
 -------+-------+-------+--------- | -------+-------+-------+---------
  0       0       0       green    |  0       1       0       yellow
  1       0       1       green    |  1       1       1       yellow
  2       0       3       green    |  2       1       3       yellow
  3       0       3       green    |  3       1       3       yellow
  4       0       4       green    |  4       1       4       yellow
  5       0       5       green    |  5       1       5       yellow
  6       0       6       green    |  6       1       6       yellow
  7       0       7       green    |  7       1       7       yellow
 
VTEP1#show qos-profile QUE-DSCP
profile name: QUE-DSCP
profile type: queue-color-to-dscp
profile attached to 1 instances
configured mapping:
 queue 3 color all dscp 16
Detailed mapping:
 ----------------+-------- | ----------------+-------- | ----------------+--------
      INPUT      | OUTPUT  |      INPUT      | OUTPUT  |      INPUT      | OUTPUT
 ----------------+-------- | ----------------+-------- | ----------------+--------
  Queue | Color  | DSCP    |  Queue | Color  | DSCP    |  Queue | Color  | DSCP
 -------+--------+-------- | -------+--------+-------- | -------+--------+--------
  0       green    0       |  0       yellow   0       |  0       red      0
  1       green    10      |  1       yellow   12      |  1       red      14
  2       green    18      |  2       yellow   20      |  2       red      22
  3       green    16      |  3       yellow   16      |  3       red      16
  4       green    34      |  4       yellow   36      |  4       red      38
  5       green    40      |  5       yellow   40      |  5       red      40
  6       green    48      |  6       yellow   48      |  6       red      48
  7       green    56      |  7       yellow   56      |  7       red      56
 
RTR2/VTEP2
 
VTEP2#show run nvo vxlan
!
nvo vxlan enable
!
nvo vxlan vtep-ip-global 6.6.6.6
!
nvo vxlan tunnel qos-map-mode cos-dscp ingress DSCP-QUE
!
nvo vxlan id 1 ingress-replication inner-vid-disabled
 vxlan host-reachability-protocol evpn-bgp vrf1
!
nvo vxlan access-if port-vlan xe15 1000
 map vnid 1
 map qos-profile queue-color-to-cos QUE-COS
!
!
VTEP2#show run qos
qos enable
qos statistics
!
qos profile queue-color-to-cos QUE-COS
 queue 4 color all cos 5
!
qos profile dscp-to-queue DSCP-QUE
 dscp 16 queue 4
!
VTEP2#show nvo vxlan mac-table
 
=====================================================================================================================================================
                                                         VXLAN MAC Entries
=====================================================================================================================================================
VNID      Interface  VlanId Inner-VlanId Mac-Addr       VTEP-Ip/ESI                    Type                 Status              AccessPortDesc
_____________________________________________________________________________________________________________________________________________________
 
1         ----       ----   ----         0000.2000.9991 1.1.1.1                        Dynamic Remote       -------             -------
 
Total number of entries are : 1
 
VTEP2#show nvo vxlan tunnel
VXLAN Network tunnel Entries
 Source           Destination      Status          Up/Down        Update
====================================================================================
6.6.6.6          1.1.1.1          Installed        00:09:39        00:09:39
 
Total number of entries are 2
VTEP2#show nvo vxlan
VXLAN Information
=================
   Codes: NW - Network Port
          AC - Access Port
         (u) - Untagged
 
VNID     VNI-Name     VNI-Type Type Interface ESI                           VLAN DF-Status Src-Addr         Dst-Addr
_______________________________________________________________________________________________________________________________
1        ----         L2       NW   ----      ----                          ---- ----      6.6.6.6          1.1.1.1
1        ----         AC   xe15      --- Single Homed port ---    1000 ----      ----             ----
 
Total number of entries are 3
 
VTEP2#show interface xe15 count queue-stats
E - Egress, I - Ingress, Q-Size is in bytes
+--------------------+--------+-----------------+-------------------+-----------------+-------------------+
|   Queue/Class-map  | Q-Size |     Tx pkts     |     Tx bytes      |  Dropped pkts   |   Dropped bytes   |
+--------------------+--------+-----------------+-------------------+-----------------+-------------------+
q0                (E) 12517376 0                 0                   0                 0
q1                (E) 12517376 0                 0                   0                 0
q2                (E) 12517376 0                 0                   0                 0
q3                (E) 12517376 0                 0                   0                 0
q4                (E) 12517376 37895872          36455829826         0                 0
q5                (E) 12517376 0                 0                   0                 0
q6                (E) 12517376 0                 0                   0                 0
q7                (E) 12517376 0                 0                   0                 0
 
VTEP2#show qos-profile QUE-COS
profile name: QUE-COS
profile type: queue-color-to-cos
profile attached to 1 instances
configured mapping:
 queue 4 color all cos 5
Detailed mapping:
 ----------------+-------- | ----------------+-------- | ----------------+--------
      INPUT      | OUTPUT  |      INPUT      | OUTPUT  |      INPUT      | OUTPUT
 ----------------+-------- | ----------------+-------- | ----------------+--------
  Queue | Color  | COS     |  Queue | Color  | COS     |  Queue | Color  | COS
 -------+--------+-------- | -------+--------+-------- | -------+--------+--------
  0       green    0       |  0       yellow   0       |  0       red      0
  1       green    1       |  1       yellow   1       |  1       red      1
  2       green    2       |  2       yellow   2       |  2       red      2
  3       green    3       |  3       yellow   3       |  3       red      3
  4       green    5       |  4       yellow   5       |  4       red      5
  5       green    5       |  5       yellow   5       |  5       red      5
  6       green    6       |  6       yellow   6       |  6       red      6
  7       green    7       |  7       yellow   7       |  7       red      7
 
VTEP2#show qos-profile DSCP-QUE
profile name: DSCP-QUE
profile type: dscp-to-queue
profile attached to 1 instances
configured mapping:
 dscp 16 queue 4
Detailed mapping:
 Ingress map for dscp to queue-color
 -------+----------------- | -------+----------------- | -------+----------------- | -------+-----------------
  INPUT |     OUTPUT       |  INPUT |     OUTPUT       |  INPUT |     OUTPUT       |  INPUT |     OUTPUT
 -------+----------------- | -------+----------------- | -------+----------------- | -------+-----------------
  DSCP  | Queue | Color    |  DSCP  | Queue | Color    |  DSCP  | Queue | Color    |  DSCP  | Queue | Color
 -------+-------+--------- | -------+-------+--------- | -------+-------+--------- | -------+-------+---------
  0       0       green    |  16      4       green    |  32      4       green    |  48      6       green
  1       0       green    |  17      2       green    |  33      4       green    |  49      6       green
  2       0       green    |  18      2       green    |  34      4       green    |  50      6       green
  3       0       green    |  19      2       green    |  35      4       green    |  51      6       green
  4       0       green    |  20      2       yellow   |  36      4       yellow   |  52      6       green
  5       0       green    |  21      2       green    |  37      4       green    |  53      6       green
  6       0       green    |  22      2       red      |  38      4       red      |  54      6       green
  7       0       green    |  23      2       green    |  39      4       green    |  55      6       green
  8       1       green    |  24      3       green    |  40      5       green    |  56      7       green
  9       1       green    |  25      3       green    |  41      5       green    |  57      7       green
  10      1       green    |  26      3       green    |  42      5       green    |  58      7       green
  11      1       green    |  27      3       green    |  43      5       green    |  59      7       green
  12      1       yellow   |  28      3       yellow   |  44      5       green    |  60      7       green
  13      1       green    |  29      3       green    |  45      5       green    |  61      7       green
  14      1       red      |  30      3       red      |  46      5       green    |  62      7       green
  15      1       green    |  31      3       green    |  47      5       green    |  63      7       green