CCIE SPv4 - MPLS Traffic Engineering - TE Introduction, Construction and Signaling

Software versions:
IOS XE 15.5
IOS XR 5.3

The topology for this demo:

In this post we will begin with understanding the need for MPLS Traffic Engineering or TE, why we would use it, how ti works, how it is built and how the signaling works to get things operational in the MPLS core. 

For OSPF, we'll focus on R1, R3, R4, XR1, XR4 and XR5.

For IS-IS , we'll focus on R2, R5, R6, XR2, XR3 and XR6.

The links between R1 and XR2; XR5 and XR6; R4 and R5 are shutdown to provide separation in the network. 

So for those of you not familiar with MPLS TE, I may refer to it as "TE" as we progress, normally when an MPLS core is built using LDP, LDP works because there is an IGP of some type that has flooded topology information around in the network, best path selection has been ran to find the "shortest path" and those best routes get installed into the RIB. LDP leverages this "loop free" topology to find and form adjacencies with other MPLS core routers, both P and PEs. The adjacencies formed allow labels to be allocated to routes learned in from the customer. LDP uses the IGP shortest path to figure out how to get to the egress PE that is used to reach the destination network. The problem with that mindset is that the "shortest path" doesn't mean the "best path". IGP can't take into consideration real time network traffic and utilization. 

The solution to this paradigm is MPLS TE. TE allows us to take into consideration what is going on in the network, not just what the IGP sees, but how much a link is being used, how much bandwidth is left, what links may not be used due to predefined rules. This is accomplished but using techniques like explicit paths, bandwidth reservations, link affinity, bandwidth auto scaling, QoS based preferred bandwidth reservations and other capabilities that will be discussed later. 

There are 2 main steps used to build TE tunnels, the tunnel creation itself and attributes tied to tunnel creation and LSP or label switch paths attributes that can be used, like tunnel attributes, to determine TE tunnel creation. TE must first be globally configured on every router that will participate in the TE topology, OSPF and IS-IS will be used to demonstrate the IGP portion. TE must be enabled under the IGP process and tied to either the area or the level it will work inside of. The interface that tunnel will ride over has to be configured to allow TE tunnel creation. That same interface also has to have RSVP, discussed shortly, enabled to allow the signaling that enables TE tunnels to determine if a path meets the "constraints" specified by the TE tunnel. 

The TE tunnel is created on a router, this router is referred to as the TE "Headend" which is the router that initiates the CSPF path calculation and RSVP signaling. The tunnel itself is configured as a TE tunnel and the destination is called the "Tailend" where the tunnel will end. TE Tunnels are unidirectional in nature. Configured identically to GRE tunnels, interface tunnel X, is given a source and destination address, and configured to be for TE, additional configuration, like LSP construction specifics are configured under the TE interface and how traffic will be mapped onto the TE tunnel itself. We'll start with enabling TE globally, under the IGP process and on the interfaces, for both IOS and XR. 

IOS

mpls traffic-eng tunnels

!

router ospf 1

 mpls traffic-eng router-id Loopback0

 mpls traffic-eng area 0

!

router isis CCIE

 net 49.0000.0000.0000.000X.00

 is-type level-2-only

 metric-style wide

 !

 address-family ipv6

  multi-topology

 exit-address-family

 mpls ldp autoconfig

 mpls traffic-eng router-id Loopback0

 mpls traffic-eng level-2

!

interface type/number

 mpls traffic-eng tunnels

 ip rsvp bandwidth

XR

rsvp

 interface type/number

 !

 interface type/number

 !

!

mpls traffic-eng

 interface type/number

 !

 interface type/number

!

router ospf 1

 area 0

   mpls traffic-eng

!

router isis CCIE

 is-type level-2-only

 net 49.0000.0000.0000.0012.00

 address-family ipv4 unicast

  metric-style wide

  mpls traffic-eng level-2-only

  mpls ldp auto-config

 !

 address-family ipv6 unicast

  metric-style wide

 !

 interface Loopback0

  address-family ipv4 unicast

  !

 !

 interface GigabitEthernet0/0/0/0.1213

  address-family ipv4 unicast

  !

 !

 interface GigabitEthernet0/0/0/0.1216

  address-family ipv4 unicast

The configuration above is the base minimum to enable TE in the network. We haven't done anything with it other than turn on the capability. IOS by default will auto reserve 75% of the interface bandwidth, IOS XR will not reserve any bandwidth. We won't actually need to configure a TE tunnel yet, there is plenty of verification that needs to be done before we move on.

The output below shows everything that R1 can see from a TE topology perspective. In order for TE to be effective, each router in the topology must know what all the other routers is doing, how much traffic is flowing, what is available, on what links etc. I selectively narrowed the output, the IGP ID points to the TE RID and the DR is specific to OSPF. 

R1#sh mpls traffic-eng topology area 0 brief | in IGP Id|DR

IGP Id: 192.168.1.1, MPLS TE Id:192.168.1.1 Router Node  (ospf 1  area 0)

      link[0]: Broadcast, DR: 10.1.15.15, nbr_node_id:2, gen:15

      link[1]: Broadcast, DR: 10.1.11.11, nbr_node_id:1, gen:15

IGP Id: 192.168.1.3, MPLS TE Id:192.168.1.3 Router Node  (ospf 1  area 0)

      link[0]: Broadcast, DR: 10.14.3.14, nbr_node_id:5, gen:17

      link[1]: Broadcast, DR: 10.3.4.4, nbr_node_id:3, gen:17

IGP Id: 192.168.1.4, MPLS TE Id:192.168.1.4 Router Node  (ospf 1  area 0)

      link[0]: Broadcast, DR: 10.3.4.4, nbr_node_id:3, gen:19

      link[1]: Broadcast, DR: 10.15.4.15, nbr_node_id:7, gen:19

IGP Id: 192.168.1.14, MPLS TE Id:192.168.1.14 Router Node  (ospf 1  area 0)

      link[0]: Broadcast, DR: 10.11.14.11, nbr_node_id:4, gen:24

      link[1]: Broadcast, DR: 10.14.15.14, nbr_node_id:6, gen:24

      link[2]: Broadcast, DR: 10.14.3.14, nbr_node_id:5, gen:24

IGP Id: 15.15.15.15, MPLS TE Id:192.168.1.15 Router Node  (ospf 1  area 0)

      link[0]: Broadcast, DR: 10.1.15.15, nbr_node_id:2, gen:27

      link[1]: Broadcast, DR: 10.14.15.14, nbr_node_id:6, gen:27

      link[2]: Broadcast, DR: 10.15.4.15, nbr_node_id:7, gen:27

IGP Id: 111.111.111.111, MPLS TE Id:192.168.1.11 Router Node  (ospf 1  area 0)

      link[0]: Broadcast, DR: 10.1.11.11, nbr_node_id:1, gen:21

      link[1]: Broadcast, DR: 10.11.14.11, nbr_node_id:4, gen:21

IGP Id: 10.1.11.11, Network Node  (ospf 1  area 0)

      link[0]: Broadcast, Nbr IGP Id: 111.111.111.111, nbr_node_id:11, gen:1

      link[1]: Broadcast, Nbr IGP Id: 192.168.1.1, nbr_node_id:8, gen:1

IGP Id: 10.1.15.15, Network Node  (ospf 1  area 0)

      link[0]: Broadcast, Nbr IGP Id: 15.15.15.15, nbr_node_id:13, gen:2

      link[1]: Broadcast, Nbr IGP Id: 192.168.1.1, nbr_node_id:8, gen:2

IGP Id: 10.3.4.4, Network Node  (ospf 1  area 0)

      link[0]: Broadcast, Nbr IGP Id: 192.168.1.4, nbr_node_id:10, gen:3

      link[1]: Broadcast, Nbr IGP Id: 192.168.1.3, nbr_node_id:9, gen:3

IGP Id: 10.11.14.11, Network Node  (ospf 1  area 0)

      link[0]: Broadcast, Nbr IGP Id: 111.111.111.111, nbr_node_id:11, gen:4

      link[1]: Broadcast, Nbr IGP Id: 192.168.1.14, nbr_node_id:12, gen:4

IGP Id: 10.14.3.14, Network Node  (ospf 1  area 0)

      link[0]: Broadcast, Nbr IGP Id: 192.168.1.3, nbr_node_id:9, gen:5

      link[1]: Broadcast, Nbr IGP Id: 192.168.1.14, nbr_node_id:12, gen:5

IGP Id: 10.14.15.14, Network Node  (ospf 1  area 0)

      link[0]: Broadcast, Nbr IGP Id: 15.15.15.15, nbr_node_id:13, gen:6

      link[1]: Broadcast, Nbr IGP Id: 192.168.1.14, nbr_node_id:12, gen:6

IGP Id: 10.15.4.15, Network Node  (ospf 1  area 0)

      link[0]: Broadcast, Nbr IGP Id: 15.15.15.15, nbr_node_id:13, gen:7

      link[1]: Broadcast, Nbr IGP Id: 192.168.1.4, nbr_node_id:10, gen:7

R2#sh mpls traffic-eng topology level-2 brief | in TE Id|DR

IGP Id: 0000.0000.0002.00, MPLS TE Id:192.168.1.2 Router Node  (isis  level-2)

      link[0]: Broadcast, DR: 0000.0000.0013.03, nbr_node_id:10, gen:4

      link[1]: Broadcast, DR: 0000.0000.0006.02, nbr_node_id:5, gen:4

IGP Id: 0000.0000.0005.00, MPLS TE Id:192.168.1.5 Router Node  (isis  level-2)

      link[0]: Broadcast, DR: 0000.0000.0006.01, nbr_node_id:8, gen:12

IGP Id: 0000.0000.0006.00, MPLS TE Id:192.168.1.6 Router Node  (isis  level-2)

      link[0]: Broadcast, DR: 0000.0000.0006.01, nbr_node_id:8, gen:6

      link[1]: Broadcast, DR: 0000.0000.0006.02, nbr_node_id:5, gen:6

IGP Id: 0000.0000.0012.00, MPLS TE Id:192.168.1.12 Router Node  (isis  level-2)

      link[0]: Broadcast, DR: 0000.0000.0012.03, nbr_node_id:12, gen:10

IGP Id: 0000.0000.0013.00, MPLS TE Id:192.168.1.13 Router Node  (isis  level-2)

      link[0]: Broadcast, DR: 0000.0000.0013.03, nbr_node_id:10, gen:8

      link[1]: Broadcast, DR: 0000.0000.0012.03, nbr_node_id:12, gen:8

This output is something we will see a lot of moving forward, no tunnels have been created and no TE attribute signaling has taken place yet, so not TE tunnels are up and operational. 

R1#sh mpls traffic-eng tunnels brief

Signalling Summary:

    LSP Tunnels Process:            running

    Passive LSP Listener:           running

    RSVP Process:                   running

    Forwarding:                     enabled

    Periodic reoptimization:        every 3600 seconds, next in 68 seconds

    Periodic FRR Promotion:         Not Running

    Periodic auto-bw collection:    every 300 seconds, next in 68 seconds

P2P TUNNELS/LSPs:

Displayed 0 (of 0) heads, 0 (of 0) midpoints, 0 (of 0) tails

P2MP TUNNELS:

Displayed 0 (of 0) P2MP heads

P2MP SUB-LSPS:

Displayed 0 P2MP sub-LSPs:

          0 (of 0) heads, 0 (of 0) midpoints, 0 (of 0) tails

This output below is unique to R4, showing the IGP and MPLS TE ID both being 192.168.1.4. Link [0] and [1], the OSPF network type (broadcast) and the DR for that segment, the interface address, it's TE metric, IGP metric and any attribute flags. Lastly the SRLG or shared risk link group, combining multiple services on a single link. 

R1#sh mpls traffic-eng topology igp-id ospf 192.168.1.4 brief

IGP Id: 192.168.1.4, MPLS TE Id:192.168.1.4 Router Node  (ospf 1  area 0)

      link[0]: Broadcast, DR: 10.3.4.4, nbr_node_id:3, gen:19

      frag_id: 7, Intf Address: 10.3.4.4

      TE metric: 1, IGP metric: 1, attribute flags: 0x0

      SRLGs: None

      link[1]: Broadcast, DR: 10.15.4.15, nbr_node_id:7, gen:19

      frag_id: 9, Intf Address: 10.15.4.4

      TE metric: 1, IGP metric: 1, attribute flags: 0x0

      SRLGs: None

We'll take a look at the OSPF database. Below is an output summary of the database, the type of LSAs in the DB and that type total. Opaque Area LSAs are present, 20 of them, Type 10 LSA or opaque LSAs are used to advertise MPLS TE topology information.

R1#sh ip ospf 1 database database-summary

            OSPF Router with ID (192.168.1.1) (Process ID 1)

Area 0 database summary

  LSA Type      Count    Delete   Maxage

  Router        6        0        0

  Network       7        0        0

  Summary Net   0        0        0

  Summary ASBR  0        0        0

  Type-7 Ext    0        0        0

    Prefixes redistributed in Type-7  0

  Opaque Link   0        0        0

  Opaque Area   20       0        0

  Subtotal      33       0        0

R2#sh isis database level-2

Tag CCIE:
IS-IS Level-2 Link State Database:
LSPID                 LSP Seq Num  LSP Checksum  LSP Holdtime      ATT/P/OL
R2.00-00            * 0x00000083   0xEB84        959               0/0/0
R5.00-00              0x0000007C   0xEC34        973               0/0/0
R6.00-00              0x0000007A   0x6A50        920               0/0/0
R6.01-00              0x00000071   0xF15B        1125              0/0/0
R6.02-00              0x00000072   0xACA1        1030              0/0/0
XR2.00-00             0x0000007C   0xFF3E        755               0/0/0
XR2.03-00             0x00000070   0x0B1B        911               0/0/0
XR3.00-00             0x0000007C   0xCDC2        785               0/0/0

XR3.03-00             0x00000070   0xB67E        1166              0/0/0

The output below is of the OSPF database. This is the perspective from R1 on R4, on the other end of the OSPF network. I'll breakdown each section.

R1#sh ip ospf 1 database opaque-area adv-router 192.168.1.4

            OSPF Router with ID (192.168.1.1) (Process ID 1)

                Type-10 Opaque Link Area Link States (Area 0)

This output is unique to the Opaque LSA, we see the link state ID is 1.0.0.0, the process id of the OSPF process is 1, the advertising router, etc.

  LS age: 1475

  Options: (No TOS-capability, DC)

  LS Type: Opaque Area Link

  Link State ID: 1.0.0.0

  Opaque Type: 1

  Opaque ID: 0

  Advertising Router: 192.168.1.4

  LS Seq Number: 80000004

  Checksum: 0xDE73

  Length: 28

  Fragment number : 0

    MPLS TE router ID : 192.168.1.4

    Number of Links : 0

The below output is unique to R4. We can see the attributes that are currently in use. 

  LS age: 980

  Options: (No TOS-capability, DC)

  LS Type: Opaque Area Link

  Link State ID: 1.0.0.7

  Opaque Type: 1

  Opaque ID: 7

  Advertising Router: 192.168.1.4

  LS Seq Number: 80000005

  Checksum: 0xD0A0

  Length: 124

  Fragment number : 7

    Link connected to Broadcast network

      Link ID : 10.3.4.4

      Interface Address : 10.3.4.4

      Admin Metric : 1

      Maximum bandwidth : 125000000

      Maximum reservable bandwidth : 93750000

      Number of Priority : 8

      Priority 0 : 93750000     Priority 1 : 93750000

      Priority 2 : 93750000     Priority 3 : 93750000

      Priority 4 : 93750000     Priority 5 : 93750000

      Priority 6 : 93750000     Priority 7 : 93750000

      Affinity Bit : 0x0

      IGP Metric : 1

    Number of Links : 1

R2#sh isis database level-2 R5.00-00 detail

Tag CCIE:

IS-IS Level-2 LSP R5.00-00
LSPID                 LSP Seq Num  LSP Checksum  LSP Holdtime      ATT/P/OL
R5.00-00              0x0000007C   0xEC34        918               0/0/0
  Area Address: 49.0000
  NLPID:        0xCC 0x8E
  Topology:     IPv4 (0x0)
                IPv6 (0x2)
  Router ID:    192.168.1.5
  Hostname: R5
  Metric: 10         IS-Extended R6.01
  IP Address:   192.168.1.5
  Metric: 10         IP 10.5.6.0/24
  Metric: 10         IP 10.16.5.0/24

  Metric: 10         IP 192.168.1.5/32

Let's take a look at some more TE specific show outputs

R1#sh mpls traffic-eng link-management summary  g1.111

System Information::

    Links Count:          2

    Flooding System:      enabled

IGP Area ID::  ospf 1  area 0

    Flooding Protocol:    OSPF

    Flooding Status:      data flooded

    Periodic Flooding:    enabled (every 60 seconds, next in 54 seconds)

    Flooded Links:        2

    IGP System ID:        192.168.1.1

    MPLS TE Router ID:    192.168.1.1

    Neighbors:            2

We can see that we have 2 neighbors and we will flood out the links to those neighbors, flooding occurs every 60 seconds.

Link ID::  Gi1.111 (10.1.11.1)

    Local Intfc ID:         7

    Link Status:

      SRLGs:                None

      Intfc Switching Capability Descriptors:

         Default:           Intfc Switching Cap psc1, Encoding ethernet

      Link Label Type:      Packet

      Physical Bandwidth:   1000000 kbits/sec

      Max Res Global BW:    750000 kbits/sec (reserved: 0% in, 0% out)

      Max Res Sub BW:       0 kbits/sec (reserved: 100% in, 100% out)

      MPLS TE Link State:   MPLS TE on, RSVP on, admin-up, flooded

      Inbound Admission:    reject-huge

      Outbound Admission:   allow-if-room

      Link MTU:             IP 1500, MPLS 1500

      Admin. Weight:        1 (IGP)

      IGP Neighbor Count:   1

We can see that the bandwidth on the interface is 1Gbps, and we are currently reserving 75% of that bandwidth or 750 Mbps, we can allocate preferred traffic bandwidth using the Sub-pool bandwidth if needed. TE is on, RSVP is up and flooding has occured. The MTU fo IP and MPLS are both 1500 butes, the Administrative weight is 1 and is using IGP for that metric, OSPF cost of a 1 Gbps link. 

RSVP or Resource Reservation Protocol is the signaling protocol used to carry the stated constraints in PATH messages from the headend router to the tailend router, at each node in the network to see if that device can meet the constraints. Once the Tailend has been reached, a RESV message is passed back in the reverse direction to the headend, at each node, if the constraint can met, then a label is allocated for that hop, if not, a PATHERR message is sent indicating an issue. If the headend router has multiple path options configured, then the next path option in line is used to signal the tunnel, if all fail, the tunnel does not come up. 

Let's hit the necessary terms and define them

PATH - message sent by the headend router from its source to its destination, from the headend to the tail end along the path that is dictated by the "path option". Each hop in the path option list is queried to determine it's capabilities. The PATH message carries the following objects.

Label Request Object or LRO - requests TE labels along the path to the tailend, it does not carry labels back to the headend, that is the job of the RESV message.

Explicit Route Object or ERO - The results from the PCALC or path calculation which tells the PATH message which hops it must traverse to reach the tail. RSVP is the signaling protocol in TE, the PATH message hits each node in the path to the tail, RSVP has no visibility into the TE Database. 

Record Route Object or RRO - Records the route taken by the PATH message, since a path option can be explicit or rely on LDP/IGP forwarding, uses this mechanism to prevent loops. 

Session Attribute Object - SAO - Information about the session that is carried to each hop, mode of operation, node and/or bandwidth info and fast reroute.

Sender Tspec - Carries bandwidth reservation information as an average rate.

RESV -  message sent hop by hop from the tail back to the head. The path taken from the head to the tail is the path the tail must take back to the head. There is a NHOP or next hop value carried inside the NHOP which gives the upstream node the "next hop IP" so they know the proper path of the LSP, the NHOP can be a link IP or an IP the current node knows how to reach.The idea is the path can be explicit or loose, depending on the constraints. 

Label Object or LO - carries the label value of the TE tunnel with the NHOP information, this information is used to build the path from the head to the tail in the forwarding plane.

Record Route Object or RRO - identical to the PATH RRO, just the tail to the head in this case. 

PATHERR - an issue was found somewhere between the head and the tail. When the headend receives this message, the next path option inline is used to form the TE path, if no other path exists, the tunnel fails to be calculated.