Sunday, April 8, 2018

Carrier Support Carrier MPLS Inter AS L3 VPN IOS and XR Part 2

In this second part we'll take a look at the breakdown of the setup and a packet walk.

The goal is to get R2 to learn R4's loopabck address and vice versa. 

We can verify R1 is learning routes from both R2 and R4 which means that routing is working correctly.

R1#sh bgp vpnv4 unicast all | b Network
     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1 (default for vrf CSC)
 *>  100.64.21.0/24   0.0.0.0                  0         32768 i
 *>i 100.64.34.0/24   192.0.2.3                0    100      0 i
 *>  192.0.2.2/32     100.64.21.2              0             0 65002 i
 *>i 192.0.2.4/32     192.0.2.3                0    100      0 65004 i

The 192.0.2.4/32 routes shows that it was learned from 192.0.2.3 but that is the other side of the VPN. We'll need to prove the routes were learned the right way.

R1#sh bgp vpnv4 unicast all  192.0.2.4/32      
BGP routing table entry for 1:1:192.0.2.4/32, version 48
Paths: (1 available, best #1, table CSC)
  Advertised to update-groups:
     6         
  Refresh Epoch 2
  65004
    192.0.2.3 (metric 20) (via default) from 192.0.2.8 (192.0.2.8)
      Origin IGP, metric 0, localpref 100, valid, internal, best
      Extended Community: RT:1:1
      Originator: 192.0.2.3, Cluster list: 192.0.2.8, 192.0.2.26
      mpls labels in/out nolabel/17
      rx pathid: 0, tx pathid: 0x0

When we expand the route we see that it was learned from 192.0.2.8 which is also present from the cluster list with the same IP address, which happens to be the VPNv4 route reflector. We'll jump to R8 and see what it sees.

R8#sh bgp vpnv4 unicast all 192.0.2.4/32
BGP routing table entry for 1:1:192.0.2.4/32, version 33
Paths: (1 available, best #1, no table)
  Advertised to update-groups:
     1         
  Refresh Epoch 1
  65004
    192.0.2.3 (metric 10) (via default) from 192.0.2.26 (192.0.2.26)
      Origin IGP, metric 0, localpref 100, valid, internal, best
      Extended Community: RT:1:1
      Originator: 192.0.2.3, Cluster list: 192.0.2.26
      mpls labels in/out nolabel/17
      rx pathid: 0, tx pathid: 0x0

After expanding that same route on R8, the next hop is 192.0.2.26 which is also the other side of the VPN. What this means is that there is a VPNv4 peering between R8 and XR6.

R8#sh bgp vpnv4 unicast all summary | b Neighbor
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.0.2.1       4        50693     290     289       33    0    0 04:14:45        2
192.0.2.5       4        50693     305     328       33    0    0 04:33:07        0
192.0.2.15      4        50693     647     693       33    0    0 09:42:54        0
192.0.2.21      4        50693     583     685       33    0    0 09:40:06        0
192.0.2.26      4        50693     261     291       33    0    0 04:17:02        2

The reason we need to form an iBGP peering with either the remote route reflector or with the PE router that connects to the customer. This enables customer routes to be propagated between the regional POPs. In order for the routes to get propagated, one of many possible solutions needs to be enabled. Basic redistribution into BGP from IGP and BGP into IGP is one way. In our configuration we went to the ASBR where the PE/RR router loopbacks would be learned via IGP are manually configured as BGP network statements to be advertised into BGP. The opposite needs to happen, BGP learned routes need to be learned by IGP so that the RR/PE routers would learn them.

R5
router isis 50693
 redistribute bgp 50693 route-map RM_BGP_ROUTES
 mpls ldp autoconfig
!
router bgp 50693
 address-family ipv4
  network 192.0.2.1 mask 255.255.255.255
  network 192.0.2.8 mask 255.255.255.255
  network 192.0.2.15 mask 255.255.255.255
!
 ip prefix-list PL_BGP_ROUTES seq 5 permit 0.0.0.0/0 ge 32
!
route-map RM_BGP_ROUTES permit 10
 match ip address prefix-list PL_BGP_ROUTES

This allows for specific advertisement and learning. This is an important point in the deployment, propagating routes between the Regional SPs is the one thing needed to allow customer routes to be learned.

In order for the routes to be learned between regional SPs, there needs to be a way for those routes to have labels associated between ASNs. To enable this, R5 and R6, R12 and R11 and XR4 and R12 need to exchange labels in their BGP peering capabilities. In IOS, this is with the "send-label" command under the IPv4 unicast peering and XR with the "labeled-unicast" address family.

R5#sh bgp ipv4 unicast labels 
   Network          Next Hop      In label/Out label
   192.0.2.1/32     100.64.75.7     20/nolabel
   192.0.2.3/32     100.64.56.6     nolabel/32
   192.0.2.8/32     100.64.85.8     24/nolabel
   192.0.2.13/32    100.64.56.6     nolabel/36
   192.0.2.15/32    100.64.75.7     22/nolabel
   192.0.2.26/32    100.64.56.6     nolabel/27

R5#sh bgp ipv4 unicast neighbors 100.64.56.6 | b Neighbor
    ipv4 MPLS Label capability: advertised and received
!
RP/0/0/CPU0:XR4#sh bgp vrf CSC  ipv4 labeled-unicast summary | b Neighbor
Sun Apr  8 00:25:32.908 UTC
Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd
100.64.142.12     0 50693     560     508      282    0    0 07:56:15          3
100.64.145.15     0 50693     520     556      282    0    0 08:04:49          0

Without this capability, customer routes would not be able to use label switching. On the ASBRs, under the interface facing the core provider, the command "mpls bgp forwarding" is enabled on IOS by default. This helps propagate labels.

IOS XR is a bit unique in this case, manual intervention is required where a static route needs to be applied.

XR4
 router static
 vrf CSC
  address-family ipv4 unicast
   100.64.142.12/32 GigabitEthernet0/0/0/0.142
   100.64.145.15/32 GigabitEthernet0/0/0/0.145

RP/0/0/CPU0:XR4#show mpls forwarding vrf CSC 
Sun Apr  8 00:30:15.909 UTC
Local  Outgoing    Prefix             Outgoing     Next Hop        Bytes       
Label  Label       or ID              Interface                    Switched    
------ ----------- ------------------ ------------ --------------- ------------
24011  Pop         100.64.145.15/32[V]   \
                                      Gi0/0/0/0.145 100.64.145.15   57589       
24026  28          192.0.2.1/32[V]                 192.0.2.6       0           
24027  20          192.0.2.3/32[V]    Gi0/0/0/0.142 100.64.142.12   266919      
24029  40          192.0.2.8/32[V]                 192.0.2.6       0           
24031  26          192.0.2.13/32[V]   Gi0/0/0/0.142 100.64.142.12   0           
24032  29          192.0.2.15/32[V]                192.0.2.6       0           
24035  24          192.0.2.26/32[V]   Gi0/0/0/0.142 100.64.142.12   113630      
24036  Pop         100.64.142.12/32[V]   \
                                      Gi0/0/0/0.142 100.64.142.12   48129900 

Without the static routes, labels would not be allocated for the next hop of the customer carrier.

RP/0/0/CPU0:XR4#show route vrf CSC connected 
Sun Apr  8 00:32:08.381 UTC

C    100.64.142.0/24 is directly connected, 08:05:22, GigabitEthernet0/0/0/0.142
C    100.64.145.0/24 is directly connected, 08:43:36, GigabitEthernet0/0/0/0.145

LDP won't allocate a label for a non /32 route, so a static /32 host route is needed. IOS does this automatically with the "mpls bgp forwarding" command.

R5
interface GigabitEthernet1.56
 mpls bgp forwarding

With all of this configuration and verification, the propagation of routes should now be complete.

R1#sh bgp vpnv4 unicast all | b Network
     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1 (default for vrf CSC)
 *>  100.64.21.0/24   0.0.0.0                  0         32768 i
 *>i 100.64.34.0/24   192.0.2.3                0    100      0 i
 *>  192.0.2.2/32     100.64.21.2              0             0 65002 i
 *>i 192.0.2.4/32     192.0.2.3                0    100      0 65004 i

This output proves that R3 has advertised routes to R1 and R1 learned them.

R2#sh ip bgp | b Network
     Network          Next Hop            Metric LocPrf Weight Path
 r>  100.64.21.0/24   100.64.21.1              0             0 50693 i
 *>  100.64.34.0/24   100.64.21.1                            0 50693 i
 *>  192.0.2.2/32     0.0.0.0                  0         32768 i
 *>  192.0.2.4/32     100.64.21.1                            0 50693 65004 i

R2 shows in the BGP table.

R2#sh ip route bgp | b Gateway
Gateway of last resort is not set

      100.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
B        100.64.34.0/24 [20/0] via 100.64.21.1, 05:19:41
      192.0.2.0/32 is subnetted, 2 subnets
B        192.0.2.4 [20/0] via 100.64.21.1, 05:19:41

Those BGP learned routes are then injected into the RIB.

Now we can do a trace and see the results.

R2#traceroute 192.0.2.4 so lo0 num
Type escape sequence to abort.
Tracing the route to 192.0.2.4
VRF info: (vrf in name/id, vrf out name/id)
  1 100.64.21.1 [AS 50693] 5 msec 3 msec 4 msec
  2 100.64.17.7 [MPLS: Labels 38/17 Exp 0] 14 msec 13 msec 13 msec
  3 100.64.75.5 [MPLS: Labels 28/17 Exp 0] 25 msec 36 msec 31 msec
  4 100.64.56.6 [MPLS: Labels 32/17 Exp 0] 31 msec 32 msec 31 msec
  5 100.64.106.10 [MPLS: Labels 22/24027/17 Exp 0] 34 msec 31 msec 32 msec
  6 100.64.103.13 [MPLS: Labels 24008/24027/17 Exp 0] 147 msec 37 msec 85 msec
  7 100.64.134.14 [MPLS: Labels 24027/17 Exp 0] 26 msec 19 msec 20 msec
  8 100.64.142.12 [MPLS: Labels 20/17 Exp 0] 20 msec 20 msec 31 msec
  9 100.64.129.9 [MPLS: Labels 18/17 Exp 0] 31 msec 32 msec 32 msec
 10 100.64.34.3 [AS 50693] [MPLS: Label 17 Exp 0] 20 msec 19 msec 22 msec
 11 100.64.34.4 [AS 50693] 20 msec *  14 msec

R2 has reachability to R4's loopback, which means that the routes propagated.

Let's walk or hop from R1 to R3 and see how the labels that we see getting allocated are allocated and why.

R1#sh ip route vrf CSC 192.0.2.4

Routing Table: CSC
Routing entry for 192.0.2.4/32
  Known via "bgp 50693", distance 200, metric 0
  Tag 65004, type internal
  Last update from 192.0.2.3 03:33:33 ago
  Routing Descriptor Blocks:
  * 192.0.2.3 (default), from 192.0.2.8, 03:33:33 ago
      Route metric is 0, traffic share count is 1
      AS Hops 1
      Route tag 65004
      MPLS label: 17
      MPLS Flags: MPLS Required

We see that MPLS label gets allocated and that 192.0.2.3 is our next hop. This indicates that R3 learned a route from R4 in the VRF and VPNv4 allocated a label for that learned route. In this case label 17 is the VPN label or the label allocated for routes advertised from the customer. You'll notice that label 17 doesn't change in the above traceroute at all, pointing us to the VPN label.

Next we'll see how we get to the next hop of 192.0.2.3, we'll always do an L3 lookup first, which will tell us how to reach the outgoing interface which will point the CEF table to begin forwarding traffic.

R1#sh ip route 192.0.2.3
Routing entry for 192.0.2.3/32
  Known via "isis", distance 115, metric 20, type level-2
  Redistributing via isis 50693
  Last update from 100.64.18.8 on GigabitEthernet1.18, 03:38:20 ago
  Routing Descriptor Blocks:
    100.64.18.8, from 192.0.2.5, 03:38:20 ago, via GigabitEthernet1.18
      Route metric is 20, traffic share count is 1
  * 100.64.17.7, from 192.0.2.5, 03:38:20 ago, via GigabitEthernet1.17
      Route metric is 20, traffic share count is 1

We have 2 different ways to get to the next hop, but no label is specified, but we know the outgoing interface.

R1#show ip cef 192.0.2.3
192.0.2.3/32
  nexthop 100.64.17.7 GigabitEthernet1.17 label 38
  nexthop 100.64.18.8 GigabitEthernet1.18 label 25

The labels are 25 and 38. Our second hop shows an outgoing label of 38 which points us to R7. Label 38 is the transport label. Now we'll jump to R7 and repeat the process we did on R1, this happens on a per hop basis until the LSP has been completely traced out.

R7#sh ip route 192.0.2.3
Routing entry for 192.0.2.3/32
  Known via "isis", distance 115, metric 10, type level-2
  Redistributing via isis 50693
  Last update from 100.64.75.5 on GigabitEthernet1.75, 03:47:25 ago
  Routing Descriptor Blocks:
  * 100.64.75.5, from 192.0.2.5, 03:47:25 ago, via GigabitEthernet1.75
      Route metric is 10, traffic share count is 1

R7#sh ip cef 192.0.2.3  
192.0.2.3/32
  nexthop 100.64.75.5 GigabitEthernet1.75 label 28

Since verification in the customer carrier core is pretty much the same until the ASBR, we'll jump to R5 and continue.

R5#sh bgp vpnv4 unicast all

Since R5 isn't a PE router, there is no VPNv4 learned routes, however, there should be IPv4 learned routes as that is how we are peering with R6.

R5#sh bgp ipv4 unicast 
BGP table version is 120, local router ID is 192.0.2.5
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
 *>  192.0.2.1/32     100.64.75.7             30         32768 i
 *>  192.0.2.3/32     100.64.56.6                            0 2 50693 i
 *>  192.0.2.8/32     100.64.85.8             20         32768 i
 *>  192.0.2.13/32    100.64.56.6                            0 2 50693 i
 *>  192.0.2.15/32    100.64.75.7             30         32768 i
 *>  192.0.2.26/32    100.64.56.6                            0 2 50693 i

We see routes learned both internally and externally, the next thing we need to see is the labels associated to then learned routes.

R5#sh bgp ipv4 unicast labels 
   Network          Next Hop      In label/Out label
   192.0.2.1/32     100.64.75.7     20/nolabel
   192.0.2.3/32     100.64.56.6     nolabel/32
   192.0.2.8/32     100.64.85.8     24/nolabel
   192.0.2.13/32    100.64.56.6     nolabel/36
   192.0.2.15/32    100.64.75.7     22/nolabel
   192.0.2.26/32    100.64.56.6     nolabel/27

In our trace from R2 to R4, the third hop uses label 32, which points us to R3, pointing us to R6. We'll jump to there as we're simply tracing the transport labels since we already know the VPN label.

R6#sh mpls forwarding-table labels 32
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
32         24027      192.0.2.3/32[V]  414722        Gi1.106    100.64.106.10

On R6 we receive traffic from R5 with label 32, which is swapped to label 24027 towards R10. But here's the thing, that is to R3, however, we are now in the core carrier. We need to find out the outgoing label for core carrier transport label towards the egress PE router. That router is XR4 or 192.0.2.24.

To find out that we need to use the same recursion process as before.

R6#sh ip route 192.0.2.24
Routing entry for 192.0.2.24/32
  Known via "ospf 50693", distance 110, metric 4, type intra area
  Last update from 100.64.106.10 on GigabitEthernet1.106, 1d02h ago
  Routing Descriptor Blocks:
  * 100.64.106.10, from 192.0.2.24, 1d02h ago, via GigabitEthernet1.106
      Route metric is 4, traffic share count is 1

We know the destination, and the outgoing interface, we need the label value now.

R6#sh ip cef 192.0.2.24  
192.0.2.24/32
  nexthop 100.64.106.10 GigabitEthernet1.106 label 22

Now we have the label value, now we need the label stack.

R6#sh mpls forwarding-table labels 26
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
26         22         192.0.2.24/32    0             Gi1.106    100.64.106.10

We have the label stack, it's a repeat of the process again.

The core carrier transport label here is 22, 24027 now becomes the core carrier "VPN" label, but it still remains the transport label position and label 17 is the VPN label.

At this point, its repetitive process, show ip route, show ip cef and show mpls forwarding are going to give the info needed to get to the next hop, in this case the label.

The thing to notice, is that when the trace goes over the core carrier, there is a three label stack. In the customer carrier, it's a 2 label stack.

1 comment:

  1. With the Popularity of iOS Devices comes the threat of hacking. FastestVPN is the Best iOS VPN that encodes all your personal and financial data securing it from intruders.

    ReplyDelete