Friday, April 13, 2018

MPLS Layer 2 VPN with L2VPN Context and Service Instances

Layer 2 VPN is a large topic domain, as such, its important to cover the basics first. There are older posts breaking down some of the behind the scenes, but we are focusing on the configuration, the mapping and proving L2VPN is actually working. The goal is to basically "simulate" 2 Customer sites or 2 sites period being able to see each other over the WAN as directly connected. Historically the "xconnect" or cross connect configured under the interface facing the customer was used to bridge traffic over the SP core. 

We're taking that same concept, and using newer syntax that is used to scale the L2VPN design overall. No more xconnects under the interface, rather several individual pieces tied together to extend the broadcast domain over the SP core.

There are three components used here:
1. The service instance
2. The Pseudowire interface
3. The l2vpn xconnect context

The SI or service instance creates the EFP or Ethernet Flow Point that allows highly scalable L2 services to be configured at the same time under the interface facing the customer. Instead of just bridging, EFP gives us several options to match on for enhanced flexibility. SI 1 could be used for P2P connectivity while SI 2 could be used for VPLS, while SI 3 could be used for H-VPLS, each matching on different L2 traffic. 

The Pseudowire or PW as I'll reference it, is used to create a tLDP or Targeted LDP peering between the PE devices in this case. There are several options that are available under the PW construct, too many to list here. The goal is to have a dedicated construct that can be used for signaling and other options between the PEs. Remote AC or Attachment Circuit failure detection on the local PE is useful and enabled with the use of the CW or Control Word.

The L2VPN xconnect context is what connect or maps the SI to the PW to allow the PEs to communicate with each other. Like the other 2 constructs, there are many options available. The context references the SI and PW as members of the construct which pretty much just binds them together. That's a simplification of course.

Between the three components, this makes L2VPN much more scalable than just the "xconnect". So much so, that when I kickoff the CCIE SPv4 video series later this year, L2VPN will have a dedicated section. One thing to note is that L2VPN is not limited to 1 PW or context but rather several of each, and not just one AS either, Inter AS and CSC designs are supported and will be covered later, after I test them out of course.
The goal here, enable R18 and R16 to communicate with each other over AS 50693 with each router having an interface in the common subnet, 10.1.1.0/24.

R15
interface GigabitEthernet2
 no ip address
 negotiation auto
 no keepalive
 service instance 1 ethernet
  encapsulation default
!
interface pseudowire1
 encapsulation mpls
 neighbor 192.0.2.17 1
 control-word include
!
l2vpn xconnect context P2P
 member pseudowire1
 member GigabitEthernet2 service-instance 1 


R17
interface GigabitEthernet2
 no ip address
 negotiation auto
 no keepalive
 service instance 1 ethernet
  encapsulation default
!
interface pseudowire1
 encapsulation mpls
 neighbor 192.0.2.15 1
 control-word include
!
l2vpn xconnect context P2P
 member GigabitEthernet2 service-instance 1 
 member pseudowire1


Now that we have the PEs configured, let's verify some basic connectivity.

R17#show l2vpn atom vc 

                                       Service
Interface Peer ID         VC ID      Type   Name                     Status
--------- --------------- ---------- ------ ------------------------ ----------
pw1       192.0.2.15      1          p2p    P2P                      UP        

This output simply prove that there is an active pseudowire between R15 and R17.

R17#  show mpls ldp neighbor 192.0.2.15
    Peer LDP Ident: 192.0.2.15:0; Local LDP Ident 192.0.2.17:0
        TCP connection: 192.0.2.15.646 - 192.0.2.17.27620
        State: Oper; Msgs sent/rcvd: 37/36; Downstream
        Up time: 00:04:19
        LDP discovery sources:
          Targeted Hello 192.0.2.17 -> 192.0.2.15, active, passive
        Addresses bound to peer LDP Ident:
          100.64.157.15   100.64.158.15   192.0.2.15      100.64.151.15   

This output shows that there is a targeted LDP peering between R15 and R17. Targeted meaning that R15 and R17 are not directly connected. 

Let's test connectivity between R16 and R18.

R16
interface GigabitEthernet2
 ip address 10.1.1.16 255.255.255.0

R18
interface GigabitEthernet2
 ip address 10.1.1.18 255.255.255.0

The verficiation:

R16#ping 10.1.1.18
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.1.1.18, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 7/21/65 ms

R15#show mpls forwarding-table labels 44
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
44         No Label   l2ckt(1)         647           Gi2        point2point 

We'll send more pings to verify it again.

R15#show mpls forwarding-table labels 44
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
44         No Label   l2ckt(1)         1217          Gi2        point2point 

As you can see, there are packets being labeled switched.

Thursday, April 12, 2018

MPLS Inter AS Option 1 / Option A Back to Back VRF Exchange

In this post we will be taking a look at the "easiest" Inter AS VPN technique, at least in my opinion. It takes advantage of what is already understood with VRF connectivity, instead of connecting to a customer device, we peer with the remote provider in a VRF. This is done per customer, so for every customer we're trying to extend between the SPs, a VRF must be created. This also means that a routing protocol adjacency/peering needs to be configured as well. This is where the scalability of Option A becomes an issue, 1 for 1 VRFs and BGP/IGP peerings will quickly tie up resources on the ASBRs.
I added 2 new customers, each with a VRF, VPNA and VPNB. Having just a single customer on all the PE and ASBRs doesn't show the pros/cons of Option A. The cool thing about this design is that if a PE has a VRF configured with the appropriate RT import/export policy setup, it will only receive traffic that matches the RT policies. The drawback about this design is that an ASBR with multiple VRFs configured will have to form 1 to 1 VRF to IGP/BGP peerings with the remote ASBR.

The VRFs laid out below will be needed to learn and then propagate BGP routes between the providers.

R5 and R6
vrf definition CSC
 rd 1:1
 route-target export 1:1
 route-target import 1:1
 !
 address-family ipv4
 exit-address-family
 !
 address-family ipv6
 exit-address-family
!
vrf definition VPNA
 rd 2:2
 route-target export 2:2
 route-target import 2:2
 !
 address-family ipv4
 exit-address-family
 !
 address-family ipv6
 exit-address-family
!
vrf definition VPNB
 rd 3:3
 route-target export 3:3
 route-target import 3:3
 !
 address-family ipv4
 exit-address-family
 !
 address-family ipv6
 exit-address-family



XR6
vrf TEST
 address-family ipv4 unicast
  import route-target
   1:1
  !
  export route-target
   1:1
  !
 !
 address-family ipv6 unicast
  import route-target
   1:1
  !
  export route-target
   1:1
  !
vrf VPNA
 address-family ipv4 unicast
  import route-target
   2:2
  !
  export route-target
   2:2
  !
 !
 address-family ipv6 unicast
  import route-target
   2:2
  !
  export route-target
   2:2
  !
 !
!
vrf VPNB
 address-family ipv4 unicast
  import route-target
   3:3
  !
  export route-target
   3:3
  !
 !
 address-family ipv6 unicast
  import route-target
   3:3
  !
  export route-target
   3:3
  !
 !
!


Now we'll have to apply the VRFs to interfaces.

R5#sh vrf
  Name                             Default RD            Protocols   Interfaces
  CSC                              1:1                   ipv4,ipv6   Gi1.56
  VPNA                             2:2                   ipv4,ipv6   Gi1.22
  VPNB                             3:3                   ipv4,ipv6   Gi1.33

R6#sh vrf
  Name                             Default RD            Protocols   Interfaces
  CSC                              1:1                   ipv4,ipv6   Gi1.56
                                                                     Gi1.166
  VPNA                             2:2                   ipv4,ipv6   Gi1.22
                                                                     Gi1.122
  VPNB                             3:3                   ipv4,ipv6   Gi1.33
                                                                     Gi1.133

RP/0/0/CPU0:XR6#sh ip int br | ex default
Thu Apr 12 15:01:34.989 UTC

Interface                      IP-Address      Status          Protocol Vrf-Name
GigabitEthernet0/0/0/0.122     100.64.122.16   Up              Up       VPNA 
GigabitEthernet0/0/0/0.133     100.64.133.16   Up              Up       VPNB 
GigabitEthernet0/0/0/0.166     100.64.166.16   Up              Up       TEST 


Now that the interfaces are in the correct VRFs, we can go ahead and setup the BGP configuration.

R5
router bgp 50693
 !
 address-family ipv4 vrf CSC
  neighbor 100.64.56.6 remote-as 2
  neighbor 100.64.56.6 activate
 exit-address-family
 !
 address-family ipv4 vrf VPNA
  neighbor 100.64.22.6 remote-as 2
  neighbor 100.64.22.6 activate
 exit-address-family
 !
 address-family ipv4 vrf VPNB
  neighbor 100.64.33.6 remote-as 2
  neighbor 100.64.33.6 activate
 exit-address-family


R6
router bgp 2
 address-family ipv4 vrf CSC
  neighbor 100.64.56.5 remote-as 50693
  neighbor 100.64.56.5 activate
  neighbor 100.64.166.16 remote-as 50693
  neighbor 100.64.166.16 activate
 exit-address-family
 !     
 address-family ipv4 vrf VPNA
  neighbor 100.64.22.5 remote-as 50693
  neighbor 100.64.22.5 activate
  neighbor 100.64.122.16 remote-as 50693
  neighbor 100.64.122.16 activate
 exit-address-family
 !
 address-family ipv4 vrf VPNB
  neighbor 100.64.33.5 remote-as 50693
  neighbor 100.64.33.5 activate
  neighbor 100.64.133.16 remote-as 50693
  neighbor 100.64.133.16 activate
 exit-address-family


XR6
router bgp 50693
  vrf TEST
  rd 1:1
  address-family ipv4 unicast
  !
  neighbor 100.64.166.6
   remote-as 2
   address-family ipv4 unicast
    route-policy RPL_EBGP_PEERINGS in
    route-policy RPL_EBGP_PEERINGS out
   !
  !
 !
 vrf VPNA
  rd 2:2
  address-family ipv4 unicast
  !
  neighbor 100.64.122.6
   remote-as 2
   address-family ipv4 unicast
    route-policy RPL_EBGP_PEERINGS in
    route-policy RPL_EBGP_PEERINGS out
   !
  !
 !
 vrf VPNB
  rd 3:3
  address-family ipv4 unicast
  !
  neighbor 100.64.133.6
   remote-as 2
   address-family ipv4 unicast
    route-policy RPL_EBGP_PEERINGS in
    route-policy RPL_EBGP_PEERINGS out


The next thing for us to do is verify the VRF/BGP configuration.

RP/0/0/CPU0:XR6#sh bgp vrf all summary | i "Neighbor|100.64."
Thu Apr 12 15:06:51.637 UTC
Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd
100.64.166.6      0     2     371     334      110    0    0 05:28:33          6
Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd
100.64.122.6      0     2      16      13      110    0    0 00:09:58          4
Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd
100.64.133.6      0     2      16      13      110    0    0 00:09:57          4

R5#            sh bgp vpnv4 unicast all summary | b Neighbor
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
100.64.22.6     4            2     250     247       57    0    0 03:39:37        4
100.64.33.6     4            2     244     244       57    0    0 03:36:02        4
100.64.56.6     4            2     400     401       57    0    0 05:58:40        6
192.0.2.8       4        50693    1379    1343       57    0    0 20:08:40       12

R6# sh bgp vpnv4 unicast all summary | b Neighbor
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
100.64.22.5     4        50693     248     250       65    0    0 03:40:04        4
100.64.33.5     4        50693     245     244       65    0    0 03:36:29        4
100.64.56.5     4        50693     402     401       65    0    0 05:59:07        4
100.64.122.16   4        50693      18      21       65    0    0 00:14:20        2
100.64.133.16   4        50693      18      21       65    0    0 00:14:19        2
100.64.166.16   4        50693     338     375       65    0    0 05:32:56        4

Now that we have all the verification complete. We need to test.

R2#sh bgp ipv4 unicast | b Network
     Network          Next Hop            Metric LocPrf Weight Path
 r>  100.64.21.0/24   100.64.21.1              0             0 50693 i
 *>  100.64.114.0/24  100.64.21.1                            0 50693 2 65004 i
 *>  100.64.144.0/24  100.64.21.1                            0 50693 2 65014 i
 *>  100.64.165.0/24  100.64.21.1                            0 50693 65016 i
 *>  100.64.222.0/24  100.64.21.1                            0 50693 2 65014 i
 *>  100.64.233.0/24  100.64.21.1                            0 50693 2 65014 i
 *>  192.0.2.2/32     0.0.0.0                  0         32768 i
 *>  192.0.2.4/32     100.64.21.1                            0 50693 2 65004 i
 *>  192.0.2.14/32    100.64.21.1                            0 50693 2 65014 i
 *>  192.0.2.16/32    100.64.21.1                            0 50693 65016 i

We see that we've learned several loopbacks, R4, R14 and R16. We trace to R14 from our loopback.

R2#traceroute 192.0.2.14 source lo0 num
Type escape sequence to abort.
Tracing the route to 192.0.2.14
VRF info: (vrf in name/id, vrf out name/id)
  1 100.64.21.1 [AS 50693] 5 msec 3 msec 3 msec
  2 100.64.18.8 [MPLS: Labels 20/44 Exp 0] 6 msec 6 msec 7 msec
  3 100.64.56.5 [MPLS: Label 44 Exp 0] 14 msec 16 msec 15 msec
  4 100.64.56.6 19 msec 11 msec 10 msec
  5 100.64.106.10 [MPLS: Labels 22/24010 Exp 0] 16 msec 25 msec 17 msec
  6 100.64.103.13 [MPLS: Labels 24008/24010 Exp 0] 20 msec 19 msec 21 msec
  7 100.64.134.14 [MPLS: Label 24010 Exp 0] 18 msec 19 msec 20 msec
  8 100.64.144.14 [AS 65014] 21 msec *  11 msec

We reach it taking the R5-R6 path to get there. You'll also notice that the traceroute is 2 LSPs and an IP path. LSP1 is R1 to R5, the IP path is R5 to R6 and LSP2 is R6 to XR4. This is expected with option A since it is back to back VRF exchange. There is no label exchange or allocation.

R12#sh bgp ipv4 unicast | b Network
     Network          Next Hop            Metric LocPrf Weight Path
 *>  100.64.121.0/24  0.0.0.0                  0         32768 i
 *>  100.64.144.0/24  100.64.121.11                          0 50693 2 65014 i
 *>  100.64.178.0/24  100.64.121.11                          0 50693 65018 i
 *>  100.64.222.0/24  100.64.121.11                          0 50693 2 65014 i
 *>  100.64.233.0/24  100.64.121.11                          0 50693 2 65014 i
 *>  192.0.2.12/32    0.0.0.0                  0         32768 i
 *>  192.0.2.14/32    100.64.121.11                          0 50693 2 65014 i
 *>  192.0.2.18/32    100.64.121.11                          0 50693 65018 i

We check R12 as well, which is a different customer. We see that R18 and R14 loopbacks are learned. R14 is peered with XR4 in all VRFs, so it has visibility in all VPNs. This can be done by peering the PE to the CE in all VRFs or simply importing/exporting the right RT values in the VRF confguration.

R12#traceroute 192.0.2.14 so lo0 num
Type escape sequence to abort.
Tracing the route to 192.0.2.14
VRF info: (vrf in name/id, vrf out name/id)
  1 100.64.121.11 2 msec 2 msec 1 msec
  2 100.64.151.15 [MPLS: Labels 20/49 Exp 0] 8 msec 9 msec 7 msec
  3 100.64.158.8 [MPLS: Labels 20/49 Exp 0] 28 msec 31 msec 31 msec
  4 100.64.33.5 [MPLS: Label 49 Exp 0] 21 msec 21 msec 20 msec
  5 100.64.33.6 20 msec 13 msec 12 msec
  6 100.64.106.10 [MPLS: Labels 22/24014 Exp 0] 15 msec 20 msec 20 msec
  7 100.64.103.13 [MPLS: Labels 24008/24014 Exp 0] 23 msec 19 msec 23 msec
  8 100.64.134.14 [MPLS: Label 24014 Exp 0] 25 msec 19 msec 22 msec
  9 100.64.233.14 [AS 65014] 22 msec *  14 msec

We trace over the Inter AS path, again with 2 LSPs and 1 IP path. We also have Intra AS reachability with R18 but the focus was Inter AS here.

Wednesday, April 11, 2018

MPLS Inter AS Option 2 / Option B VPNv4 BGP Exchange

In this post we'll take a look at Option B which is a VPNv4 eBGP peering in the global table to connect the 2 MPLS SPs together. The purpose of the VPNv4 AFI BGP peering is to easily exchange VPNv4 traffic with each other. Just like in Option C, we'll lay out the steps that are needed to get the configuration working.

What needs to be implemented ahead of time is the Intra AS VPN setup, IGP/LDP, MP-BGP, VRFs/PE-CE routing. 
1. VPNv4 eBGP peering between the ASBRs.
2. Allow VPNv4 routes. We'll be using the IOS "no bgp default route-target filter" and XR "retain route-target all". This will just allow whatever route targets to be learned.
Option B is a pretty straightforward configuration overall. Not as configuration intensive as Option A or C. The Intra AS configuration is in place already. We have to configure the eBGP VPNv4 peering and disable the default route-target filter and enable retain route-target all under BGP.

With that said, unlike Option C and CSC, we'll have 3 separate LSPs in our setup. I will break that down once the traceroute is complete.



R5
router bgp 50693
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 no bgp default route-target filter
 neighbor 100.64.56.6 remote-as 2
 neighbor 192.0.2.8 remote-as 50693
 neighbor 192.0.2.8 update-source Loopback0
 !
 address-family ipv4
 exit-address-family
 !
 address-family vpnv4
  neighbor 100.64.56.6 activate
  neighbor 100.64.56.6 send-community extended
  neighbor 192.0.2.8 activate
  neighbor 192.0.2.8 send-community extended
  neighbor 192.0.2.8 next-hop-self
 exit-address-family




R6
router bgp 2
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 no bgp default route-target filter
 neighbor 100.64.56.5 remote-as 50693
 neighbor 100.64.166.16 remote-as 50693
 neighbor 192.0.2.23 remote-as 2
 neighbor 192.0.2.23 update-source Loopback0
 !
 address-family ipv4
 exit-address-family
 !
 address-family vpnv4
  neighbor 100.64.56.5 activate
  neighbor 100.64.56.5 send-community extended
  neighbor 100.64.166.16 activate
  neighbor 100.64.166.16 send-community extended
  neighbor 192.0.2.23 activate
  neighbor 192.0.2.23 send-community extended
  neighbor 192.0.2.23 next-hop-self
 exit-address-family




XR6
router bgp 50693
 address-family vpnv4 unicast
  retain route-target all
 !
 neighbor 100.64.166.6
  remote-as 2
  address-family ipv4 labeled-unicast
   route-policy RPL_LOCAL_PREF in
   route-policy RPL_EBGP_PEERINGS out
  !
  address-family vpnv4 unicast
   route-policy RPL_EBGP_PEERINGS in
   route-policy RPL_EBGP_PEERINGS out

With the configuration in place now, we'll verify.

R5#show bgp vpnv4 unicast all summary | b Neighbor
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
100.64.56.6     4            2     141     143       17    0    0 02:03:22        2
192.0.2.8       4        50693     152     150       17    0    0 02:10:42        4

R5#show bgp vpnv4 unicast all | b Network
     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1 (default for vrf CSC)
 *>i 100.64.21.0/24   192.0.2.1                0    100      0 i
 *>  100.64.114.0/24  100.64.56.6                            0 2 65004 i
 *>i 100.64.165.0/24  192.0.2.15               0    100      0 65016 i
 *>i 192.0.2.2/32     192.0.2.1                0    100      0 65002 i
 *>  192.0.2.4/32     100.64.56.6                            0 2 65004 i
 *>i 192.0.2.16/32    192.0.2.15               0    100      0 65016 i


R6#sh bgp vpnv4 unicast all summary | b Neighbor
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
100.64.56.5     4        50693     145     142       15    0    0 02:04:46        4
100.64.166.16   4        50693     127     140       15    0    0 02:01:12        4
192.0.2.23      4            2     135     150       15    0    0 02:08:25        2

R6#sh bgp vpnv4 unicast all | b Network
     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1 (default for vrf CSC)
 *   100.64.21.0/24   100.64.166.16                          0 50693 i
 *>                   100.64.56.5                            0 50693 i
 *>i 100.64.114.0/24  192.0.2.11               0    100      0 65004 i
 *   100.64.165.0/24  100.64.166.16                          0 50693 65016 i
 *>                   100.64.56.5                            0 50693 65016 i
 *   192.0.2.2/32     100.64.166.16                          0 50693 65002 i
 *>                   100.64.56.5                            0 50693 65002 i
 *>i 192.0.2.4/32     192.0.2.11               0    100      0 65004 i
 *   192.0.2.16/32    100.64.166.16                          0 50693 65016 i
 *>                   100.64.56.5                            0 50693 65016 i



RP/0/0/CPU0:XR6#show bgp vpnv4 unicast summary | b Neighbor
Wed Apr 11 21:16:57.455 UTC
Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd
100.64.166.6      0     2     186     169       17    0    0 02:03:57          2

RP/0/0/CPU0:XR6#show bgp vpnv4 unicast | b Network
Wed Apr 11 21:17:21.413 UTC
   Network            Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1
*>i100.64.21.0/24     192.0.2.1                0    100      0 i
*> 100.64.114.0/24    100.64.166.6                           0 2 65004 i
* i                   192.0.2.5                0    100      0 2 65004 i
*>i100.64.165.0/24    192.0.2.15               0    100      0 65016 i
*>i192.0.2.2/32       192.0.2.1                0    100      0 65002 i
*> 192.0.2.4/32       100.64.166.6                           0 2 65004 i
* i                   192.0.2.5                0    100      0 2 65004 i
*>i192.0.2.16/32      192.0.2.15               0    100      0 65016 i

Processed 6 prefixes, 8 paths



Now that we have seen that the routes have been propagated and learned. We'll focus on the end to end connectivity the 3 LSPs.

R2#traceroute 192.0.2.4 source lo0 num
Type escape sequence to abort.
Tracing the route to 192.0.2.4
VRF info: (vrf in name/id, vrf out name/id)
  1 100.64.21.1 [AS 50693] 5 msec 3 msec 4 msec
  2 100.64.17.7 [MPLS: Labels 26/44 Exp 0] 9 msec 8 msec 10 msec
  3 100.64.75.5 [MPLS: Label 44 Exp 0] 31 msec 31 msec 31 msec
  4 100.64.56.6 [MPLS: Label 28 Exp 0] 30 msec 31 msec 32 msec
  5 100.64.106.10 [MPLS: Labels 17/57 Exp 0] 21 msec 32 msec 31 msec
  6 100.64.114.11 [AS 65004] [MPLS: Label 57 Exp 0] 19 msec 19 msec 72 msec
  7 100.64.114.4 [AS 65004] 49 msec *  8 msec



R2#traceroute 192.0.2.4 source lo0 num
LSP1
  1 100.64.21.1 [AS 50693] 5 msec 3 msec 4 msec
  2 100.64.17.7 [MPLS: Labels 26/44 Exp 0] 9 msec 8 msec 10 msec
  3 100.64.75.5 [MPLS: Label 44 Exp 0] 31 msec 31 msec 31 msec

LSP2
  4 100.64.56.6 [MPLS: Label 28 Exp 0] 30 msec 31 msec 32 msec

LSP3
  5 100.64.106.10 [MPLS: Labels 17/57 Exp 0] 21 msec 32 msec 31 msec
  6 100.64.114.11 [AS 65004] [MPLS: Label 57 Exp 0] 19 msec 19 msec 72 msec
  7 100.64.114.4 [AS 65004] 49 msec *  8 msec

The reason for the 3 LSPs, three separate next hops.
LSP1 - ingress PE to egress PE/ASBR
LSP2 - egress PE/ASBR to ingress PE/ASBR
LSP3 - ingress PE/ASBR to egress PE

Tuesday, April 10, 2018

MPLS Inter AS Option 3 / Option C - Multihop VPNv4 BGP Exchange

In this post we will take a look at the third option for Inter AS VPN connectivity with Option C. This option is among my favorite SP topics as it gives quite a bit of scale while not being as configuration intensive as Option A or back to back VRFs. We're not looking at all the little variations or options we could used to get end to end reachability. We are looking at the more "common" ways to roll this out. Like the CSC post, we are forming VPNv4 peerings between the RRs, but these are 2 different SPs, so route reflection isn't supported here. We will also advertise local BGP RR and PE loopbacks on the ASBRs and redistribute the learned routes into IGP via a route map that is allowing only /32 prefixes into IGP. This will allow for LDP label allocation of the remote RR and PEs. 

The process that is needed to get this configuration up and running is pretty straightforward, there are slight differences/options as we go along but that is for the RIT workbooks and VoDs.

1. eBGP IPv4 unicast peerings with label allocation between the ASBRs in each AS, there maybe several ASBRs.
2. While activating the peering and enabling labeled unicast, advertise the local RR and PE loopback IPs to the remote AS.
3. Redistribute the eBGP learned routes into IGP, filter if desired, or use prefix-list to match /32 routes called from a route-map during redistribution.
4. On IOS XR ASBRs, configure a static /32 route to the connected neighbor pointing out the connected interface. Needed for LDP to allocate a label for the next hop IP. IOS does this by default with the "mpls bgp forwarding" command.
5. From the BGP route reflectors/PE routers, ping and trace route from the local device to the remote RR/PE, if a ping/labeled path is good, the next step is ready.
6. On the RR, PEs can also be configured if needed, configure a VPNv4 eBGP multihop peering between RRs. The VPNv4 peering should form and routes should be propagated between PEs.
7. When traces from CE to CE are done, 2 LSPs should be seen, 1 from ingress PE to the remote ASN RR and then down to the remote PE. This will cause issues for things like QoS.
8. On the RRs, under the VPNv4 AFI, set the "next-hop-unchagned" command to remove the RR from the traffic path.

Now that a process has been laid out, I use a process or order of operations to build with, verify against and troubleshoot from to make sure I didn't miss anything.

To cut down on the amount of configuration here, IGP (IS-IS and OSPF) and LDP are running their respective SP cores. I won't be adding the IGP/LDP/MP-BGP/VRF/PE-CE Routing configuration, there would be a lot. I will add in the associated configuration from the above mentioned bullet points. The traceroute outputs are to prove that IGP/LDP are working. This also assumes that the individual SPs MPLS Cores are ready to go, IGP/LDP/MP-BGP VPNv4 Route Reflected Peerings/VRFs/PE-CE Routing.

IS-IS/BGP ASN 50693
R1#traceroute 192.0.2.15 source lo0 num
Type escape sequence to abort.
Tracing the route to 192.0.2.15
VRF info: (vrf in name/id, vrf out name/id)
  1 100.64.18.8 [MPLS: Label 30 Exp 0] 4 msec 3 msec 3 msec
  2 100.64.158.15 3 msec *  4 msec

R1#traceroute 192.0.2.21 source lo0 num
Type escape sequence to abort.
Tracing the route to 192.0.2.21
VRF info: (vrf in name/id, vrf out name/id)
  1 100.64.18.8 [MPLS: Label 18 Exp 0] 10 msec 6 msec 6 msec
  2 100.64.158.15 [MPLS: Label 35 Exp 0] 16 msec 20 msec 20 msec
  3 100.64.151.11 20 msec *  6 msec

OSPF/BGP ASN 2
R11#traceroute 192.0.2.24 so lo0 num
Type escape sequence to abort.
Tracing the route to 192.0.2.24
VRF info: (vrf in name/id, vrf out name/id)
  1 100.64.110.10 [MPLS: Label 22 Exp 0] 7 msec 6 msec 6 msec
  2 100.64.103.13 [MPLS: Label 24008 Exp 0] 15 msec 20 msec 20 msec
  3 100.64.134.14 20 msec *  7 msec

The eBGP IPv4 labeled unicast peerings on the ASBRs.

R5 (ASBR)
router bgp 50693
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 100.64.56.6 remote-as 2
 !
 address-family ipv4
  network 192.0.2.1 mask 255.255.255.255
  network 192.0.2.8 mask 255.255.255.255
  network 192.0.2.15 mask 255.255.255.255
  network 192.0.2.17 mask 255.255.255.255
  network 192.0.2.21 mask 255.255.255.255
  network 192.0.2.25 mask 255.255.255.255
  neighbor 100.64.56.6 activate
  neighbor 100.64.56.6 send-label

R6 (ASBR)
router bgp 2
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 100.64.56.5 remote-as 50693
 neighbor 100.64.166.16 remote-as 50693
 !
 address-family ipv4
  network 192.0.2.11 mask 255.255.255.255
  network 192.0.2.22 mask 255.255.255.255
  network 192.0.2.23 mask 255.255.255.255
  network 192.0.2.24 mask 255.255.255.255
  neighbor 100.64.56.5 activate
  neighbor 100.64.56.5 send-label
  neighbor 100.64.166.16 activate
  neighbor 100.64.166.16 send-label

XR6 (ASBR)
router bgp 50693
 address-family ipv4 unicast
  network 192.0.2.1/32
  network 192.0.2.3/32
  network 192.0.2.8/32
  network 192.0.2.15/32
  network 192.0.2.17/32
  network 192.0.2.21/32
  network 192.0.2.25/32
  allocate-label all
 !
 neighbor 100.64.166.6
  remote-as 2
  address-family ipv4 labeled-unicast
   route-policy RPL_EBGP_PEERINGS in
   route-policy RPL_EBGP_PEERINGS out
!
router static
 address-family ipv4 unicast
  100.64.166.6/32 GigabitEthernet0/0/0/0.166


Now that the configuration is in place, let's see what we've learned.

R5#sh bgp ipv4 unicast summary | b Neighbor
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
100.64.56.6     4            2    1019    1024      166    0    0 15:22:01        6

R5#sh bgp ipv4 unicast | b Network
     Network          Next Hop            Metric LocPrf Weight Path
 *>  192.0.2.1/32     100.64.75.7             30         32768 i
 *>  192.0.2.3/32     100.64.56.6                            0 2 50693 i
 *>  192.0.2.8/32     100.64.85.8             20         32768 i
 *>  192.0.2.11/32    100.64.56.6              3             0 2 i
 *>  192.0.2.15/32    100.64.85.8             30         32768 i
 *>  192.0.2.17/32    100.64.175.17           20         32768 i
 *   192.0.2.21/32    100.64.56.6                            0 2 50693 i
 *>                   100.64.85.8             40         32768 i
 *>  192.0.2.22/32    100.64.56.6              4             0 2 i
 *>  192.0.2.23/32    100.64.56.6              3             0 2 i
 *>  192.0.2.24/32    100.64.56.6              4             0 2 i
 *>  192.0.2.25/32    100.64.75.7             40         32768 i



R6#sh bgp ipv4 unicast summary | b Neighbor
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
100.64.56.5     4        50693    1027    1022       17    0    0 15:25:19        6
100.64.166.16   4        50693     261     288       17    0    0 04:14:49        7


R6#sh bgp ipv4 unicast | b Network
     Network          Next Hop            Metric LocPrf Weight Path
 *   192.0.2.1/32     100.64.166.16           40             0 50693 i
 *>                   100.64.56.5             30             0 50693 i
 *>  192.0.2.3/32     100.64.166.16           20             0 50693 i
 *   192.0.2.8/32     100.64.166.16           30             0 50693 i
 *>                   100.64.56.5             20             0 50693 i
 *>  192.0.2.11/32    100.64.106.10            3         32768 i
 *   192.0.2.15/32    100.64.166.16           30             0 50693 i
 *>                   100.64.56.5             30             0 50693 i
 *   192.0.2.17/32    100.64.166.16           50             0 50693 i
 *>                   100.64.56.5             20             0 50693 i
 *>  192.0.2.21/32    100.64.166.16           20             0 50693 i
 *                    100.64.56.5             40             0 50693 i
 *>  192.0.2.22/32    100.64.106.10            4         32768 i
 *>  192.0.2.23/32    100.64.106.10            3         32768 i
 *>  192.0.2.24/32    100.64.106.10            4         32768 i
 *   192.0.2.25/32    100.64.166.16           50             0 50693 i
 *>                   100.64.56.5             40             0 50693 i



RP/0/0/CPU0:XR6#sh bgp ipv4 labeled-unicast summary | b Neighbor
Tue Apr 10 12:00:10.744 UTC
Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd
100.64.166.6      0     2     289     262       19    0    0 04:15:49          4

RP/0/0/CPU0:XR6#sh bgp ipv4 labeled-unicast | b Network
Tue Apr 10 12:00:29.733 UTC
   Network            Next Hop            Metric LocPrf Weight Path
*> 192.0.2.1/32       100.64.163.3            40         32768 i
*> 192.0.2.3/32       100.64.163.3            20         32768 i
*> 192.0.2.8/32       100.64.163.3            30         32768 i
*> 192.0.2.11/32      100.64.166.6             3             0 2 i
*> 192.0.2.15/32      100.64.116.11           30         32768 i
*> 192.0.2.17/32      100.64.163.3            50         32768 i
*> 192.0.2.21/32      100.64.116.11           20         32768 i
*> 192.0.2.22/32      100.64.166.6             4             0 2 i
*> 192.0.2.23/32      100.64.166.6             3             0 2 i
*> 192.0.2.24/32      100.64.166.6             4             0 2 i
*> 192.0.2.25/32      100.64.163.3            50         32768 i

Processed 11 prefixes, 11 paths

Now that we have learned these routes, we'll configure the BGP to IGP redistribution.


XR6
prefix-set PS_SP_LOOPBACKS
  0.0.0.0/0 ge 32
end-set
!
route-policy RPL_SP_LOOPBACKS
  if destination in PS_SP_LOOPBACKS then
    pass
  endif
end-policy
!
router isis 50693
 address-family ipv4 unicast
  redistribute bgp 50693 level-2 route-policy RPL_SP_LOOPBACKS



R5
router isis 50693
 redistribute bgp 50693 route-map RM_BGP_ROUTES
!
ip prefix-list PL_BGP_ROUTES seq 5 permit 0.0.0.0/0 ge 32
!
route-map RM_BGP_ROUTES permit 10
 match ip address prefix-list PL_BGP_ROUTES



R6
router ospf 50693
 redistribute bgp 2 subnets route-map RM_BGP_ROUTES
!
ip prefix-list PL_BGP_ROUTES seq 5 permit 0.0.0.0/0 ge 32
!
route-map RM_BGP_ROUTES permit 10
 match ip address prefix-list PL_BGP_ROUTES



Now that we have this configuration in place, we need to make sure that R8 can talk to XR3.


R8#sh ip route 192.0.2.23
Routing entry for 192.0.2.23/32
  Known via "isis", distance 115, metric 10, type level-2
  Redistributing via isis 50693
  Last update from 100.64.85.5 on GigabitEthernet1.85, 15:27:37 ago
  Routing Descriptor Blocks:
  * 100.64.85.5, from 192.0.2.5, 15:27:37 ago, via GigabitEthernet1.85
      Route metric is 10, traffic share count is 1

We have a route in the RIB, which means we should also have an LDP allocation.

R8#sho mpls forwarding-table 192.0.2.23
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
38         39         192.0.2.23/32    1188          Gi1.85     100.64.85.5 

We have a label allocation.

R8#traceroute 192.0.2.23 so lo0 num
Type escape sequence to abort.
Tracing the route to 192.0.2.23
VRF info: (vrf in name/id, vrf out name/id)
  1 100.64.85.5 [MPLS: Label 39 Exp 0] 8 msec 7 msec 7 msec
  2 100.64.56.6 [MPLS: Label 22 Exp 0] 25 msec 30 msec 30 msec
  3 100.64.106.10 [MPLS: Label 18 Exp 0] 20 msec 20 msec 20 msec
  4 100.64.103.13 59 msec *  6 msec

We can traceroute to the the remote end via R5.

With the current configuration, we'll have 2 LSPs, 1 from the ingress PE to the remote RR and another from the remote RR to the egress PE.

R1#sh bgp vpnv4 unicast all 192.0.2.4/32
BGP routing table entry for 1:1:192.0.2.4/32, version 173
Paths: (1 available, best #1, table CSC)
  Advertised to update-groups:
     13        
  Refresh Epoch 3
  2 65004
    192.0.2.23 (metric 20) (via default) from 192.0.2.8 (192.0.2.8)
      Origin IGP, metric 0, localpref 100, valid, internal, best
      Extended Community: RT:1:1
      mpls labels in/out nolabel/24015
      rx pathid: 0, tx pathid: 0x0

The route update was reflected to us from R8 and R8's next hop is XR3. That means that another LSP from XR3 down to the remote PE is needed.

R2#traceroute 192.0.2.4 source lo0 num
Type escape sequence to abort.
Tracing the route to 192.0.2.4
VRF info: (vrf in name/id, vrf out name/id)
  1 100.64.21.1 [AS 50693] 4 msec 4 msec 4 msec
  2 100.64.18.8 [MPLS: Labels 38/24015 Exp 0] 11 msec 10 msec 50 msec
  3 100.64.85.5 [MPLS: Labels 39/24015 Exp 0] 47 msec 47 msec 47 msec
  4 100.64.56.6 [MPLS: Labels 22/24015 Exp 0] 47 msec 47 msec 47 msec
  5 100.64.106.10 [MPLS: Labels 18/24015 Exp 0] 47 msec 77 msec 24 msec
  6 100.64.103.13 [MPLS: Label 24015 Exp 0] 28 msec 36 msec 35 msec
  7 100.64.103.10 [MPLS: Labels 17/57 Exp 0] 36 msec 36 msec 36 msec
  8 100.64.114.11 [AS 65004] [MPLS: Label 57 Exp 0] 31 msec 77 msec 145 msec
  9 100.64.114.4 [AS 65004] 20 msec *  11 msec

As you can see, traffic flows down to the RR and back to the P router that connects to the PE. This is not the desired result, we can change this with the "next-hop-unchanged" command under the VPNv4 AFI. After applying it, it will take a few moments for the BGP propagation to occur.

R1#sh bgp vpnv4 unicast all 192.0.2.4/32
BGP routing table entry for 1:1:192.0.2.4/32, version 182
Paths: (1 available, best #1, table CSC)
  Advertised to update-groups:
     13        
  Refresh Epoch 3
  2 65004
    192.0.2.11 (metric 20) (via default) from 192.0.2.8 (192.0.2.8)
      Origin IGP, metric 0, localpref 100, valid, internal, best
      Extended Community: RT:1:1
      mpls labels in/out nolabel/57
      rx pathid: 0, tx pathid: 0x0

By telling VPNv4 to not change the next hop, we see that R11 advertised the route to XR3 which then advertised it to R8. R1 receives the update from R8, R8 shows us that the route was learned from R11, which is removing the XR3 advertisement thus passing us directly to the remote PE.

R2#traceroute 192.0.2.4 source lo0 num
Type escape sequence to abort.
Tracing the route to 192.0.2.4
VRF info: (vrf in name/id, vrf out name/id)
  1 100.64.21.1 [AS 50693] 5 msec 4 msec 4 msec
  2 100.64.18.8 [MPLS: Labels 43/57 Exp 0] 9 msec 9 msec 10 msec
  3 100.64.85.5 [MPLS: Labels 40/57 Exp 0] 31 msec 31 msec 31 msec
  4 100.64.56.6 [MPLS: Labels 21/57 Exp 0] 32 msec 31 msec 31 msec
  5 100.64.106.10 [MPLS: Labels 17/57 Exp 0] 31 msec 76 msec 21 msec
  6 100.64.114.11 [AS 65004] [MPLS: Label 57 Exp 0] 20 msec 19 msec 19 msec
  7 100.64.114.4 [AS 65004] 20 msec 

Our traceroute bypasses the RR now and goes directly to the remote PE.

You may have noticed that we were going through R5-R6 and bypassing XR6. I shutdown the eBGP peering on R6s side and that forced the traffic to go through XR6.

R8#traceroute 192.0.2.23 so lo0 num
Type escape sequence to abort.
Tracing the route to 192.0.2.23
VRF info: (vrf in name/id, vrf out name/id)
  1 100.64.38.3 [MPLS: Label 42 Exp 0] 10 msec 7 msec 8 msec
  2 100.64.163.16 [MPLS: Label 24024 Exp 0] 24 msec 35 msec 30 msec
  3 100.64.166.6 [MPLS: Label 22 Exp 0] 31 msec 30 msec 30 msec
  4 100.64.106.10 [MPLS: Label 18 Exp 0] 16 msec 19 msec 21 msec
  5 100.64.103.13 20 msec *  8 msec

We trace between the RRs and take XR6 on hops 2 and 3.

R2#traceroute 192.0.2.4 source lo0 num
Type escape sequence to abort.
Tracing the route to 192.0.2.4
VRF info: (vrf in name/id, vrf out name/id)
  1 100.64.21.1 [AS 50693] 5 msec 3 msec 3 msec
  2 100.64.18.8 [MPLS: Labels 43/57 Exp 0] 11 msec 11 msec 11 msec
  3 100.64.38.3 [MPLS: Labels 43/57 Exp 0] 22 msec 30 msec 35 msec
  4 100.64.163.16 [MPLS: Labels 24025/57 Exp 0] 29 msec 32 msec 32 msec
  5 100.64.166.6 [MPLS: Labels 21/57 Exp 0] 198 msec 24 msec 21 msec
  6 100.64.106.10 [MPLS: Labels 17/57 Exp 0] 21 msec 21 msec 20 msec
  7 100.64.114.11 [AS 65004] [MPLS: Label 57 Exp 0] 19 msec 21 msec 20 msec
  8 100.64.114.4 [AS 65004] 19 msec *  10 msec

We confirm that our routing between the CEs is still good as we flow through XR6 with an XR assigned label.