Rob Riker/ Simplified Networking: QinQ

Showing posts with label QinQ. Show all posts

Wednesday, December 14, 2016

CCIE SPv4 - MPLS L2VPN P2P EPL AToM VPWS with the L2VPN Protocol CLI - VLAN and QinQ

Software versions:
IOS XE 15.5
IOS XR 5.3

The topology for this demo:

In this post we will be taking the previous "Port" based configuration and leveraging the service instance for both "encapsulation dot1q VLAN" "VLAN" and "encapsulation dot1q VLAN second-dot1q VLAN" "QinQ" variations. Just like in the "Port" based configuration, the service instance is used to multiplex the port to support several configuration options that the EFP or Ethernet Flow Point capability is design to do. As discussed in previous posts, the EFP allows the SP to connect several customers to a single interface and, depending on the services needed, to selectively map the "service instance" to a "context", decoupling the interface from the service it is providing access to. This greatly increases the scale, and is supported for both P2P (EPL/EVPL), MP2MP (VPLS/H-VPLS) and P2MP (E-TREE or Rooted VPLS).

R1 and R6 are the PEs providing access into the SP core, R10 connecting to R6 and R13 to R1. Ihave OSPFv2 running on R13 and R10 to proove that there is dataplane connectivity between the CEs. We'll start out with the VLAN configuration variant to highlight the fact that if there are customers that require at least 1 VLAN, we'll be able to meet that need. The QinQ option will be visited a little later, very similar configuration, but with 2 VLAN tags.

To level set between the 2 variations, the only configuration difference is the service instance at the interface level, the VLAN variation has only a single VLAN encapsulated, where the QinQ variation has 2. Beyond that, the rest of the configuration is identical, I'll include all the necessary configuration to make sure you see the difference.

The VLAN Variant

interface GigabitEthernet2

service instance 16 ethernet

encapsulation dot1q 1013

interface pseudowire16

encapsulation mpls

neighbor 192.168.1.6 16

l2vpn xconnect context VLAN

member pseudowire16

member GigabitEthernet2 service-instance 16

The interface level configuration is the same as Inter-VLAN routing without an IP address configured at the interface level.

The pseudowire configuration set the encapsulation to MPLS and the neighbor to R6, mapping to the VC-ID of 16.

The xconnect context has 2 members, pseudowire 16 and G2 service instance 16, providing the mapping from interface to bridge. As traffic from R13 comes into R1 on G2 SI 16, it will be immediately encap'd into MPLS and tLDP to R6 which will place the traffic onto G2 SI 16 out to R10, thus completely the circuit.

interface GigabitEthernet2

service instance 16 ethernet

encapsulation dot1q 1013

interface pseudowire16

encapsulation mpls

neighbor 192.168.1.1 16

l2vpn xconnect context VLAN

member pseudowire16

member GigabitEthernet2 service-instance 16

I won't bore you with another breakdown, R1's config breakdown works on R6 😉.

So now we have completed the configuration between R1 and R6. Since we are dealing with P2P connections ro "xconnects", there are several debugs that show output of the AC or attachment circuit, PW or pseudowire and other details.

debug l2vpn xconnect event

debug l2vpn xconnect initialization

A couple things to point out, [192.168.1.1:16] indicates the PW; [Gi2:1013] indicates the SI.

If you leave the debugs running while you are enabling/disabling capabilities, be patient, there is a lot of update/sync type output.

XC VPWS[BF000001:192.168.1.1:16]: Update, i/f 4002

XC VPWS[BF000001:192.168.1.1:16]: Circuit attributes from client:

XC VPWS[BF000001:192.168.1.1:16]: Payload encap: Ethernet

XC VPWS[BF000001:192.168.1.1:16]: VLAN ID: 0

XC VPWS[BF000001:192.168.1.1:16]: Loadbalance mode: None

XC VPWS[BF000001:192.168.1.1:16]: Loadbalance Flow class: none

XC VPWS[BF000001:Gi2:1013]: Send active circuit attributes

XC VPWS[BF000001:Gi2:1013]: Circuit attributes to client:

XC VPWS[BF000001:Gi2:1013]: Payload encap: Ethernet

XC VPWS[BF000001:Gi2:1013]: VLAN ID: 0

XC VPWS[BF000001:Gi2:1013]: Loadbalance mode: None

XC VPWS[BF000001:Gi2:1013]: Loadbalance Flow class: none

XC VPWS[BF000001:192.168.1.1:16]: Update, i/f 4002

XC VPWS[BF000001:192.168.1.1:16]: Circuit attributes from client:

XC VPWS[BF000001:192.168.1.1:16]: Interface handle: 0x4002

XC VPWS[BF000001:192.168.1.1:16]: Status: UP (0x1)

XC VPWS[BF000001:192.168.1.1:16]: Payload encap: Ethernet

XC VPWS[BF000001:192.168.1.1:16]: Circuit Encap: Ethernet

XC VPWS[BF000001:192.168.1.1:16]: VCCV CC: 0x7

XC VPWS[BF000001:192.168.1.1:16]: Segment type: AToM

XC VPWS[BF000001:192.168.1.1:16]: Control Word: yes

XC VPWS[BF000001:192.168.1.1:16]: Fast Switchover: no

XC VPWS[BF000001:192.168.1.1:16]: MTU: 1500

XC VPWS[BF000001:192.168.1.1:16]: I/F Str: pw16

XC VPWS[BF000001:192.168.1.1:16]: Circuit string: 192.168.1.1:16

XC VPWS[BF000001:192.168.1.1:16]: Received status change, status is UP

XC VPWS[BF000001:Gi2:1013]: Circuit attributes to client:

XC VPWS[BF000001:Gi2:1013]: Interface handle: 0x4002

XC VPWS[BF000001:Gi2:1013]: Status: UP (0x1)

XC VPWS[BF000001:Gi2:1013]: Payload encap: Ethernet

XC VPWS[BF000001:Gi2:1013]: Circuit Encap: Ethernet

XC VPWS[BF000001:Gi2:1013]: VCCV CC: 0x7

XC VPWS[BF000001:Gi2:1013]: Segment type: AToM

XC VPWS[BF000001:Gi2:1013]: Control Word: yes

XC VPWS[BF000001:Gi2:1013]: Fast Switchover: no

XC VPWS[BF000001:Gi2:1013]: MTU: 1500

XC VPWS[BF000001:Gi2:1013]: I/F Str: pw16

XC VPWS[BF000001:Gi2:1013]: Circuit string: 192.168.1.1:16

XC: MPLS peer 192.168.1.1 vcid 16, VC state UP

So as you can see, the VC is up. Now it's time to look at some of the specific outputs, like the VC details, MPLS forwarding table, knowing which labels equal to forwarding and others to local mappings, how to correlate the outgoing label to the PW. Let's take a look at the basics first and then take a walk into the weeds.

R1#sh l2vpn atom vc

Service

Interface Peer ID VC ID Type Name Status

--------- --------------- ---------- ------ ------------------------ ----------

pw16 192.168.1.6 16 p2p VLAN UP

So we see that PW16 is up and operational.

R1#show mpls l2transport vc

Local intf Local circuit Dest address VC ID Status

------------- -------------------------- --------------- ---------- ----------

Gi2 Eth VLAN 1013 192.168.1.6 16 UP

Pretty much the same output, slightly different though.

R1#show xconnect all

Legend: XC ST=Xconnect State S1=Segment1 State S2=Segment2 State

UP=Up DN=Down AD=Admin Down IA=Inactive

SB=Standby HS=Hot Standby RV=Recovering NH=No Hardware

XC ST Segment 1 S1 Segment 2 S2

------+---------------------------------+--+---------------------------------+--

UP pri mpls 192.168.1.6:16 UP ac Gi2:1013(Eth VLAN) UP

This output gives us a bit more info, like the PW and the AC info.

Let's take a deeper look at the detailed output of the atom vc output.

R1#show l2vpn atom vc pseudowire 16 detail

pseudowire16 is up, VC status is up PW type: Ethernet

Create time: 1d01h, last status change time: 00:09:09

Last label FSM state change time: 00:09:09

Destination address: 192.168.1.6 VC ID: 16

Output interface: Gi1.112, imposed label stack {24207 30}

Preferred path: not configured

Default path: active

Next hop: 10.1.12.12

Member of xconnect service VLAN

Associated member Gi2 is up, status is up

Interworking type is Ethernet

Service id: 0x31000001

Signaling protocol: LDP, peer 192.168.1.6:0 up

Targeted Hello: 192.168.1.1(LDP Id) -> 192.168.1.6, LDP is UP

Graceful restart: not configured and not enabled

Non stop routing: not configured and not enabled

PWid FEC (128), VC ID: 16

Status TLV support (local/remote) : enabled/supported

LDP route watch : enabled

Label/status state machine : established, LruRru

Local dataplane status received : No fault

BFD dataplane status received : Not sent

BFD peer monitor status received : No fault

Status received from access circuit : No fault

Status sent to access circuit : No fault

Status received from pseudowire i/f : No fault

Status sent to network peer : No fault

Status received from network peer : No fault

Adjacency status of remote peer : No fault

Sequencing: receive disabled, send disabled

Bindings

Parameter Local Remote

------------ ------------------------------ ------------------------------

Label 32 30

Group ID 0 0

Interface

MTU 1500 1500

Control word on (configured: autosense) on

PW type Ethernet Ethernet

VCCV CV type 0x02 0x02

LSPV [2] LSPV [2]

VCCV CC type 0x07 0x07

CW [1], RA [2], TTL [3] CW [1], RA [2], TTL [3]

Status TLV enabled supported

SSO Descriptor: 192.168.1.6/16, local label: 32

Dataplane:

SSM segment/switch IDs: 4121/4096 (used), PWID: 1

Rx Counters

9773 input transit packets, 960362 bytes

0 drops, 0 seq err

Tx Counters

9778 output transit packets, 1255320 bytes

0 drops

There are 2 different sets of labels to pay attention to here, the "{24207 30}" and the "32 30".

The first set, 24207 is the outgoing label on R1 to XR2 to get to R6; 30 is the label that R6 will use to get to R1 via the P2P connection.

R1#show mpls forwarding-table 192.168.1.6

Local Outgoing Prefix Bytes Label Outgoing Next Hop

Label Label or Tunnel Id Switched interface

22 24207 192.168.1.6/32 0 Gi1.112 10.1.12.12

R6#show mpls forwarding-table labels 30

Local Outgoing Prefix Bytes Label Outgoing Next Hop

Label Label or Tunnel Id Switched interface

30 No Label l2ckt(1) 9598 Gi2 point2point

The second set is specific to the PW labels. Where R1 will use label 32 to reach R6 and will use label 30 to reach R1.

R1#show mpls forwarding-table labels 32

Local Outgoing Prefix Bytes Label Outgoing Next Hop

Label Label or Tunnel Id Switched interface

32 No Label l2ckt(1) 12134 Gi2 point2point

R6#show mpls forwarding-table labels 30

Local Outgoing Prefix Bytes Label Outgoing Next Hop

Label Label or Tunnel Id Switched interface

30 No Label l2ckt(1) 9598 Gi2 point2point

R6#show mpls forwarding-table 192.168.1.1

Local Outgoing Prefix Bytes Label Outgoing Next Hop

Label Label or Tunnel Id Switched interface

40 26 192.168.1.1/32 0 Gi1.26 10.2.6.2

526 192.168.1.1/32 0 Gi1.56 10.5.6.5

Being able to map the label to the service is important, the service being the P2P PW between the PEs, in this case, R1 to R6 is label 32 and R6 to R1 is label 30. The other label values, used to label switch to the other PE, 24207 on R1 and 26 on R6.

Taking a look at the "targeted LDP" outputs.

R6#show mpls ldp discovery

Targeted Hellos:

192.168.1.6 -> 192.168.1.1 (ldp): active/passive, xmit/recv

LDP Id: 192.168.1.1:0

We an see we have a P2P non-directly connected session up, transmitting and receiving. The label space is platform wide the "0" behind the LDP Id: 192.168.1.1:0.

More specifically...

R6#show mpls ldp discovery detail

Targeted Hellos:

192.168.1.6 -> 192.168.1.1 (ldp): active/passive, xmit/recv

Enabled by: MPLS AToM Circuit,

Hello interval: 10000 ms; Transport IP addr: 192.168.1.6

LDP Id: 192.168.1.1:0

Src IP addr: 192.168.1.1; Transport IP addr: 192.168.1.1

Hold time: 90 sec; Proposed local/peer: 90/90 sec

Reachable via 192.168.1.1/32

Password: not required, none, in use

It tells us this is an MPLS AToM circuit and the specifics of the peering.

We can see that R1 has advertised a number of label values.

R6#show mpls ldp bindings neighbor 192.168.1.1 detail

Local label filtering spec: host routes

lib entry: 10.1.11.0/24, rev 57

lib entry: 10.1.12.0/24, rev 58

lib entry: 10.1.15.0/24, rev 59

Not really applicable here, but it's important to know that under the hood, LDP is still the active label distribution protocol here and needs to be verified.

Let's take a look at the CEs, just to make sure that the connectivity is up and operational.

R10#sh ip ospf int br | in Gi1.1013

Gi1.1013 1 0 172.10.13.10/24 1 BDR 1/1

R10#show ip ospf neighbor | in GigabitEthernet1.1013

13.13.13.13 1 FULL/DR 00:00:33 172.10.13.13 GigabitEthernet1.1013

Awesome, good sign!. We can see that R10 has a peering with R13 and all is well.

The QinQ Variant

Now we'll look at the QinQ variant. This is where we will have 2 Dot1Q tags, one for the Service provider, the other for the customer. We've covered QinQ in detail in earlier posts, so I won't go into those details now.

interface GigabitEthernet2

service instance 61 ethernet

encapsulation dot1q 100 second-dot1q 200

interface pseudowire61

encapsulation mpls

neighbor 192.168.1.6 61

l2vpn xconnect context QinQ

member GigabitEthernet2 service-instance 61

member pseudowire61

interface GigabitEthernet2

service instance 61 ethernet

encapsulation dot1q 100 second-dot1q 200

interface pseudowire61

encapsulation mpls

neighbor 192.168.1.1 61

l2vpn xconnect context QinQ

member GigabitEthernet2 service-instance 61

member pseudowire61

So basically all we are doing is adding an additional VLAN tag to the forwarding process. The outer tag, the service provider tag, is stripped off as it enters the SP core and is label switched through the core. The inner tag, the customer tag, is not stripped off by default and is added encapsulation through the core. There are ways to "rewrite" that information which we will be taking a look at later on.

Let's take a look at the verification for this now, very similar to the VLAN variant, the main difference is simply a new set of labels used to switch the PW data between PEs.

R1#show l2vpn atom vc pseudowire 61 detail

pseudowire61 is up, VC status is up PW type: Ethernet

Create time: 4d00h, last status change time: 3d23h

Last label FSM state change time: 3d23h

Destination address: 192.168.1.6 VC ID: 61

Output interface: Gi1.115, imposed label stack {24508 28}

Preferred path: not configured

Default path: active

Next hop: 10.1.15.15

Member of xconnect service QinQ

Associated member Gi2 is up, status is up

Interworking type is Ethernet

Service id: 0x22000002

Signaling protocol: LDP, peer 192.168.1.6:0 up

Targeted Hello: 192.168.1.1(LDP Id) -> 192.168.1.6, LDP is UP

Graceful restart: not configured and not enabled

Non stop routing: not configured and not enabled

PWid FEC (128), VC ID: 61

Status TLV support (local/remote) : enabled/supported

LDP route watch : enabled

Label/status state machine : established, LruRru

Local dataplane status received : No fault

BFD dataplane status received : Not sent

BFD peer monitor status received : No fault

Status received from access circuit : No fault

Status sent to access circuit : No fault

Status received from pseudowire i/f : No fault

Status sent to network peer : No fault

Status received from network peer : No fault

Adjacency status of remote peer : No fault

Sequencing: receive disabled, send disabled

Bindings

Parameter Local Remote

------------ ------------------------------ ------------------------------

Label 39 28

Group ID 0 0

Interface

MTU 1500 1500

Control word on (configured: autosense) on

PW type Ethernet Ethernet

VCCV CV type 0x02 0x02

LSPV [2] LSPV [2]

VCCV CC type 0x07 0x07

CW [1], RA [2], TTL [3] CW [1], RA [2], TTL [3]

Status TLV enabled supported

SSO Descriptor: 192.168.1.6/61, local label: 39

Dataplane:

SSM segment/switch IDs: 20496/8198 (used), PWID: 2

Rx Counters

36973 input transit packets, 3781234 bytes

0 drops, 0 seq err

Tx Counters

36987 output transit packets, 4897012 bytes

0 drops

The 24508 label is used to reach R6's loopback. Label 28 is the label that R6 allocated to forward data over the PW.

R1#sh mpls forwarding-table labels 22

Local Outgoing Prefix Bytes Label Outgoing Next Hop

Label Label or Tunnel Id Switched interface

22 24207 192.168.1.6/32 0 Gi1.112 10.1.12.12

24508 192.168.1.6/32 0 Gi1.115 10.1.15.15

R6#show mpls forwarding-table labels 28

Local Outgoing Prefix Bytes Label Outgoing Next Hop

Label Label or Tunnel Id Switched interface

28 No Label l2ckt(2) 3783524 Gi2 point2point

Let's verify that R10 has an active OSPFv2 adjacency.

interface GigabitEthernet1.1310

encapsulation dot1Q 100 second-dot1q 200

ip address 192.168.10.10 255.255.255.0

end

R10#sh ip ospf int br | in Gi1.1310

Interface PID Area IP Address/Mask Cost State Nbrs F/C

Gi1.1310 1 0 192.168.10.10/24 1 DR 1/1

R10#sh ip route 13.14.0.0

Routing entry for 13.14.0.0/24

Known via "ospf 1", distance 110, metric 2, type intra area

Last update from 192.168.10.13 on GigabitEthernet1.1310, 3d08h ago

Routing Descriptor Blocks:

* 192.168.10.13, from 13.13.13.13, 4d00h ago, via GigabitEthernet1.1310

Route metric is 2, traffic share count is 1

All is well. So we know that the configuration up to this point is operational.

Saturday, November 26, 2016

CCIE SPv4 - MPLS L2VPN - VPLS BGP-Auto-Discovery using LDP with VLAN and QinQ

Software versions:
IOS XE 15.5
IOS XR 5.3

The topology for this demo:

In this post we will breaking down VPLS leveraging BGP for Auto Discovery using LDP. We are using the previous post (VLAN and QinQ) configuration to test this feature out. The nice thing is the configuration is already in place from the previous post so we don't have to setup up VPLS, it's already up. We will be looking at this from the E-LAN perspective. What we will be doing is making it more automated instead of the static configurations we've seen up until now, BGP AD using LDP or BGP is the next step. I have enabled debugs on R3 for both "L2 VPN VFI" and BGP updates using L2VPN VPLS AFI.

LDP is commonly a signalling mechanism for signalling BGP discovered PWs in a VPLS environment. Basically what ends up happening is the VPLS PE devices are configured to run BGP, depending on the rollout, either full mesh or RR (our use case, full mesh later on) configuration. Once the BGP Peerings are established, the PEs are learned by other PEs like in L3 VPN and then PW sessions are formed between them. We'll call on RT or Route Targets to be used for importing and exporting information, like L3 VPN. Note that this configuration is completely exclusive, has nothing to do with, the EFP or BD configuration, This is strictly under the VFI and BGP configuration modes.

debug bgp l2vpn vpls updates

debug l2vpn vfi event

Once the configuration is in place under the VFI configuration and BGP, The BGP debug output isn't terribly overwhelming, it's the L2VPN VFI output that you may want to send to the logging buffer. Once we see the debug output and the peerings have formed as they need to. There are quite a few verification and show commands we will look at to understand what is going on.

The first thing that needs to be done is all the PEs need to turn on autodiscovery on under the VFI configuration. I used the newer syntax everywhere but I'll toss in the older syntax in case you are using it. Make sure you have your debugs on, not much happens at first but there will be an avalanche of output later on.

IOS

New syntax

l2vpn vfi context VPLS

vpn id 1356

autodiscovery bgp signaling ldp

Older syntax

l2 vfi VPLS autodiscovery

vpn id 1356

bridge-domain 1356

IOS XR

XR1

l2vpn

bridge group VPLS

bridge-domain VPLS

vfi VPLS

autodiscovery bgp

rd auto

route-target import 1:1356

route-target export 1:1356

signaling-protocol ldp

Now that is it for the L2VPN VFI configuration on the PEs. Since we are using a RR, which is XR5, XR5 will need to be configured to all the PEs, just like VPNv4 or IPv4. I did not use any type of af/neighbor or session group, it's all laid out line by line for you. A couple of key things to point out, first is the AFI is "l2vpn vpls-vpws" since this is "not IP" BGP we are using, we are leveraging the IP connectivity between the routers that are bridging CE traffic over the MPLS core with. The second thing to notice is that with XR, you have to explicitly state the signaling protocol used, in this case, BGP has been disabled to let LDP work.

XR5 - Route Reflector

router bgp 1

address-family l2vpn vpls-vpws

neighbor 192.168.1.3

remote-as 1

update-source Loopback0

address-family l2vpn vpls-vpws

route-reflector-client

Signalling bgp disable

neighbor 192.168.1.5

remote-as 1

update-source Loopback0

address-family l2vpn vpls-vpws

route-reflector-client

Signalling bgp disable

neighbor 192.168.1.6

remote-as 1

update-source Loopback0

address-family l2vpn vpls-vpws

route-reflector-client

Signalling bgp disable

neighbor 192.168.1.11

remote-as 1

update-source Loopback0

address-family l2vpn vpls-vpws

route-reflector-client

Signalling bgp disable

Now onto the PE configuration, since it is the same everywhere with the exception of IOS and IOS XR, I'll show you R3 and XR1s configuration.

router bgp 1

bgp log-neighbor-changes

no bgp default ipv4-unicast

neighbor 192.168.1.15 remote-as 1

neighbor 192.168.1.15 transport connection-mode active

neighbor 192.168.1.15 update-source Loopback0

address-family ipv4

exit-address-family

address-family l2vpn vpls

neighbor 192.168.1.15 activate

neighbor 192.168.1.15 prefix-length-size 2

exit-address-family

XR1

router bgp 1

address-family l2vpn vpls-vpws

neighbor 192.168.1.15

remote-as 1

update-source Loopback0

session-open-mode active-only

address-family l2vpn vpls-vpws

Signalling bgp disable

As you can see very straightforward. One HUGE thing to point out is on the IOS boxes, you HAVE to leverage the "prefix-length-size 2" configuration. I have linked you to some documentation to fix the issue.

L2VPN LDP SIgnalling FIX

Prefixes are usually sent with a length of 1 byte. Cisco IOS software implemented the draft 'draft-ietf-l2vpn-signaling-08,' which later became RFC 6074. A length field of 1 byte was chosen at the time, indicating the length in bits.

RFC 6074 Provisioning, Auto-Discovery, and Signaling in Layer 2 Virtual Private Networks (L2VPN) specifies that the NLRI encoding for BGP auto-discovery should be a length of 2 bytes. The 2 bytes indicate how many bytes of prefix follow in variable length prefix.

If the neighbor x.x.x.x prefix-length-size 2 command is not present on the RRs, the BGP neighbor does not come up, and the RR interprets the length field as 1 byte only.

XR5 error output

RP/0/0/CPU0:Nov 24 22:38:17.931 : bgp[1046]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 192.168.1.3 (VRF: default) - message length 88 bytes, error flags 0x00000008, action taken "Reset". Error details: "Error 0x00000008, Field "MP_REACH-NLRI", Attribute 14 (Flags 0x00, Length 0), Data [60]". NLRIs: [IPv4 Unicast]

It's not obvious as to what the problem is.

So at this point all the PEs should be peered to the RR on XR5, so let's check that out.

RP/0/0/CPU0:XR5#sh bgp l2vpn vpls summary

Process RcvTblVer bRIB/RIB LabelVer ImportVer SendTblVer StandbyVer

Speaker 10 10 10 10 10 0

Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd

192.168.1.3 0 1 211 200 10 0 0 00:39:03 1

192.168.1.5 0 1 185 175 10 0 0 02:06:34 1

192.168.1.6 0 1 189 174 10 0 0 02:06:35 1

192.168.1.11 0 1 130 143 10 0 0 02:07:57 0

So we have neighbors that are peered, however you'll see that XR1 under the PfxRcd field, we have a big old "0". Since XR1 doesn't participate in the dataplane, that's all we'll see.

So back to R3 as R5 and R6 will give us the same outputs as R3.

R3#sh bgp l2vpn vpls all summary

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd

192.168.1.15 4 1 47 50 4 0 0 00:41:18 2

R3#sh bgp l2vpn vpls all

BGP table version is 4, local router ID is 192.168.1.3

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,

r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,

x best-external, a additional-path, c RIB-compressed,

Origin codes: i - IGP, e - EGP, ? - incomplete

RPKI validation codes: V valid, I invalid, N Not found

Network Next Hop Metric LocPrf Weight Path

Route Distinguisher: 1:1356

*> 1:1356:192.168.1.3/96

0.0.0.0 32768 ?

*>i 1:1356:192.168.1.5/96

192.168.1.5 0 100 0 ?

*>i 1:1356:192.168.1.6/96

192.168.1.6 0 100 0 ?

R3 debug output

BGP(9): nettable_walker called for 1:1356:192.168.1.3/96

BGP(9): nettable_walker 1:1356:192.168.1.3/96 route sourced locally

BGP(9): nettable_walker called for 1:1356:192.168.1.5/96

BGP(9): best path[0] 1:1356:192.168.1.5/96 source 192.168.1.3 nh 192.168.1.5 vpls-id: L2VPN AGI:1:1356

BGP(9): add XC RIB route 1:1356:192.168.1.5/96 masklen 96 L2VPN AGI:1:1356 pathcount: 1 [0] LDP source:192.168.1.3 nexthop:192.168.1.5 RT:1:1356

BGP(9): nettable_walker called for 1:1356:192.168.1.6/96

BGP(9): best path[0] 1:1356:192.168.1.6/96 source 192.168.1.3 nh 192.168.1.6 vpls-id: L2VPN AGI:1:1356

BGP(9): add XC RIB route 1:1356:192.168.1.6/96 masklen 96 L2VPN AGI:1:1356 pathcount: 1 [0] LDP source:192.168.1.3 nexthop:192.168.1.6 RT:1:1356

neighbor 192.168.1.15 prefix-length-size 2

BGP Specific debug output

%BGP-5-NBR_RESET: Neighbor 192.168.1.15 reset (Prefix Storage Size changed)

%BGP-5-ADJCHANGE: neighbor 192.168.1.15 Down Prefix Storage Size changed

%BGP_SESSION-5-ADJCHANGE: neighbor 192.168.1.15 L2VPN Vpls topology base removed from session Prefix Storage Size changed

%BGP-5-ADJCHANGE: neighbor 192.168.1.15 Up

BGP(9): 192.168.1.15 rcvd UPDATE w/ attr: nexthop 192.168.1.5, origin ?, localpref 100, metric 0, originator 192.168.1.5, clusterlist 192.168.1.15, extended community RT:1:1356 L2VPN AGI:1:1356

BGP(9): 192.168.1.15 rcvd 1:1356:192.168.1.5/96

BGP(9): bump net 1:1356:192.168.1.5/96, non bpath added

BGP(9): 192.168.1.15 rcvd UPDATE w/ attr: nexthop 192.168.1.6, origin ?, localpref 100, metric 0, originator 192.168.1.6, clusterlist 192.168.1.15, extended community RT:1:1356 L2VPN AGI:1:1356

BGP(9): 192.168.1.15 rcvd 1:1356:192.168.1.6/96

BGP(9): bump net 1:1356:192.168.1.6/96, non bpath added

BGP(9): nettable_walker called for 1:1356:192.168.1.3/96

BGP(9): nettable_walker 1:1356:192.168.1.3/96 route sourced locally

BGP(9): nettable_walker called for 1:1356:192.168.1.5/96

BGP(9): best path[0] 1:1356:192.168.1.5/96 source 192.168.1.3 nh 192.168.1.5 vpls-id: L2VPN AGI:1:1356

BGP(9): add XC RIB route 1:1356:192.168.1.5/96 masklen 96 L2VPN AGI:1:1356 pathcount: 1 [0] LDP source:192.168.1.3 nexthop:192.168.1.5 RT:1:1356

BGP(9): nettable_walker called for 1:1356:192.168.1.6/96

BGP(9): best path[0] 1:1356:192.168.1.6/96 source 192.168.1.3 nh 192.168.1.6 vpls-id: L2VPN AGI:1:1356

BGP(9): add XC RIB route 1:1356:192.168.1.6/96 masklen 96 L2VPN AGI:1:1356 pathcount: 1 [0] LDP source:192.168.1.3 nexthop:192.168.1.6 RT:1:1356

VPLS Specific output per PW

XC VFI CFG[VPLS]: Add AD taii 192.168.1.5, vpn_id 1356, src 192.168.1.3, agi A0001 54C, next-hop 192.168.1.5

Out to R5

XC VFI VC[VPLS/pw100021]: Adding a PW; total count: 4

XC VFI VC[VPLS/pw100021]: Sending provision message to client

XC VFI VC[VPLS/pw100021]: Circuit attributes to client:

XC VFI VC[VPLS/pw100021]: i/f: 4005

XC VFI VC[VPLS/pw100021]: Status: DOWN(hard-down, not-ready) (0xC0)

XC VFI VC[VPLS/pw100021]: Circuit directive: Go Active

XC VFI VC[VPLS/pw100021]: Payload encap: Ethernet

XC VFI VC[VPLS/pw100021]: Circuit Encap: VFI

XC VFI VC[VPLS/pw100021]: Interworking type: none

XC VFI VC[VPLS/pw100021]: Segment type: VFI

XC VFI VC[VPLS/pw100021]: Switch handle: 0

XC VFI VC[VPLS/pw100021]: MTU: 1500

XC VFI VC[VPLS/pw100021]: I/F Str: pw100017

XC VFI VC[VPLS/pw100021]: Circuit string: vfi

XC VFI VC[VPLS/pw100021]: Bridge-domain port add

Local provisioning of the PW from R3 to R5

XC VFI VC[VPLS/pw100021]: Processing provision of vfi-pw (192.168.1.5, 1356)

XC VFI VC[VPLS/pw100021]: Setting up circuit

XC VFI VC[VPLS/pw100021]: Provision vfi-pw switch

XC VFI VC[VPLS/pw100021]: Sending update message to client

XC VFI VC[VPLS/pw100021]: Circuit attributes to client:

XC VFI VC[VPLS/pw100021]: i/f: 4005

XC VFI VC[VPLS/pw100021]: Status: UP (0x1)

XC VFI VC[VPLS/pw100021]: Circuit directive: Go Active

XC VFI VC[VPLS/pw100021]: Payload encap: Ethernet

XC VFI VC[VPLS/pw100021]: Circuit Encap: VFI

XC VFI VC[VPLS/pw100021]: Interworking type: none

XC VFI VC[VPLS/pw100021]: Segment type: VFI

XC VFI VC[VPLS/pw100021]: Switch handle: 32876

XC VFI VC[VPLS/pw100021]: MTU: 1500

XC VFI VC[VPLS/pw100021]: I/F Str: pw100017

XC VFI VC[VPLS/pw100021]: Circuit string: vfi

In from R5 to R3

XC VFI VC[VPLS/pw100021]: Processing client Update message

XC VFI VC[VPLS/pw100021]: Circuit attributes from client:

XC VFI VC[VPLS/pw100021]: i/f: 400C

XC VFI VC[VPLS/pw100021]: Status: DOWN(not-forwarding) (0x2)

XC VFI VC[VPLS/pw100021]: Segment type: AToM

XC VFI VC[VPLS/pw100021]: Fast Switchover: no

XC VFI VC[VPLS/pw100021]: I/F Str: pw100021

XC VFI VC[VPLS/pw100021]: Circuit string: 192.168.1.5:1356

PW for R5 is up

XC VFI VC[VPLS/pw100021]: Processing client Update message

XC VFI VC[VPLS/pw100021]: Circuit attributes from client:

XC VFI VC[VPLS/pw100021]: Payload encap: Ethernet

XC VFI VC[VPLS/pw100021]: Update VC type not needed

XC VFI VC[VPLS/pw100021]: Processing client Update message

XC VFI VC[VPLS/pw100021]: Circuit attributes from client:

XC VFI VC[VPLS/pw100021]: Payload encap: Ethernet

XC VFI VC[VPLS/pw100021]: VLAN ID: 0

XC VFI VC[VPLS/pw100021]: Loadbalance mode: None

XC VFI VC[VPLS/pw100021]: Loadbalance Flow class: none

XC VFI VC[VPLS/pw100021]: Update VC type not needed

XC VFI VC[VPLS/pw100021]: Processing client Update message

XC VFI VC[VPLS/pw100021]: Circuit attributes from client:

XC VFI VC[VPLS/pw100021]: i/f: 400C

XC VFI VC[VPLS/pw100021]: Status: UP (0x1)

XC VFI VC[VPLS/pw100021]: Payload encap: Ethernet

XC VFI VC[VPLS/pw100021]: Circuit Encap: Ethernet

XC VFI VC[VPLS/pw100021]: VCCV CC: 0x6

XC VFI VC[VPLS/pw100021]: Segment type: AToM

XC VFI VC[VPLS/pw100021]: Control Word: yes

XC VFI VC[VPLS/pw100021]: Fast Switchover: no

XC VFI VC[VPLS/pw100021]: MTU: 1500

XC VFI VC[VPLS/pw100021]: I/F Str: pw100021

XC VFI VC[VPLS/pw100021]: Circuit string: 192.168.1.5:1356

The process repeats identically for R6. However you won't see any traffic for XR1 since there is no data plane communication.

R3#sh l2vpn atom vc pseudowire 100021

Service

Interface Peer ID VC ID Type Name Status

--------- --------------- ---------- ------ ------------------------ ----------

pw100021 192.168.1.5 1356 vfi VPLS UP

R3#sh l2vpn atom vc pseudowire 100022

Service

Interface Peer ID VC ID Type Name Status

--------- --------------- ---------- ------ ------------------------ ----------

pw100022 192.168.1.6 1356 vfi VPLS UP

And that about does it for that part.

Now lets' break down the BGP portion!

R3#sh bgp l2vpn vpls rd 1:1356 192.168.1.5

BGP routing table entry for 1:1356:192.168.1.5/96, version 3

Paths: (1 available, best #1, table L2VPN-VPLS-BGP-Table)

Not advertised to any peer

Refresh Epoch 1

Local

192.168.1.5 (metric 3) from 192.168.1.15 (192.168.1.15)

Origin incomplete, metric 0, localpref 100, valid, internal, best, AGI version(2130706439)

Extended Community: RT:1:1356 L2VPN AGI:1:1356

Originator: 192.168.1.5, Cluster list: 192.168.1.15

mpls labels in/out 16777215/16777215

rx pathid: 0, tx pathid: 0x0

A few fields that we need to go over:

The RT:1:1356 is auto generated from the XE feature "auto-route-target" that is enabled by default. This can be selectively disabled if needed. The reason it is used actually makes a lot of sense, XE figures that all VPLS endpoints with the same VPN-ID intend on communicating with each other. This works very well in an E-LAN environment like VPLS but isn't very useful in an E-TREE environment. In later posts we will disable the "auto-route-target" feature when we look at the E-TREE capability. IOS XR needs the RT import/export policies to be manually configured.

The AGI or Attachment Group Identifier is used to group PW connections together for signalling purposes. It defines the endpoints that can be connected via PWs, it essentially defines the VPN. You'll notice that the AGI value is 1:1356 or the ASN:VPN and is used to determine the VC number for the PWs that are dynamically created, they have the same build as an RD or RT. It is vitally important that the VPN ID (AGI) match in order for the VPN to form, this is regardless of the RT import/export policies. The AGI is an "extended community" just like an RT and will not be imported unless the AGI matches. If any part of the AGI changes, the ASN or VPN portion, it will be denied from being imported.

Let's take a closer look at one of the PWs for a deeper breakdown, there are lots of details to cover. We have to pretty specific in our verification command as you can see below.

R3#show l2vpn atom vc destination 192.168.1.5 vcid 1356 detail

pseudowire100021 is up, VC status is up PW type: Ethernet

Create time: 1d00h, last status change time: 1d00h

Last label FSM state change time: 1d00h

Destination address: 192.168.1.5 VC ID: 1356

Output interface: Gi1.34, imposed label stack {403 502}

Preferred path: not configured

Default path: active

Next hop: 10.3.4.4

Member of vfi service VPLS

Bridge-Domain id: 1356

Service id: 0xe6000005

Signaling protocol: LDP, peer 192.168.1.5:0 up

Targeted Hello: 192.168.1.3(LDP Id) -> 192.168.1.5, LDP is UP

Graceful restart: not configured and not enabled

Non stop routing: not configured and not enabled

Generalized PWid FEC (129)

AGI: type 1, len 8, 000A 0001 0000 054C

Local AII: type 1, len 4, C0A8 0103 (192.168.1.3)

Remote AII: type 1, len 4, C0A8 0105 (192.168.1.5)

Status TLV support (local/remote) : enabled/supported

LDP route watch : enabled

Label/status state machine : established, LruRru

Local dataplane status received : No fault

BFD dataplane status received : Not sent

BFD peer monitor status received : No fault

Status received from access circuit : No fault

Status sent to access circuit : No fault

Status received from pseudowire i/f : No fault

Status sent to network peer : No fault

Status received from network peer : No fault

Adjacency status of remote peer : No fault

Sequencing: receive disabled, send disabled

Bindings

Parameter Local Remote

------------ ------------------------------ ------------------------------

Label 313 502

Group ID n/a n/a

Interface

MTU 1500 1500

Control word on (configured: autosense) on

PW type Ethernet Ethernet

VCCV CV type 0x02 0x02

LSPV [2] LSPV [2]

VCCV CC type 0x06 0x06

RA [2], TTL [3] RA [2], TTL [3]

Status TLV enabled supported

SSO Descriptor: 192.168.1.5/1356, local label: 313

Dataplane:

SSM segment/switch IDs: 73841/32876 (used), PWID: 25

Rx Counters

10206 input transit packets, 1055504 bytes

49480 drops, 0 seq err

Tx Counters

10169 output transit packets, 1358048 bytes

0 drops

Let's focus on all the areas in RED.

The first one is the VC ID of 1356, the outgoing interface and the label stack, all pretty standard. Why call it out, we use label forwarding to get from the inbound PE to the outbound PE, so if there is an issue with label forwarding, being able to ensure a label value is being properly allocated is important.

The next is that the signaling protocol is LDP and the peer is 192.168.1.5 and best of all, the PW is up.

The AGI and the AII. The AGI we've already covered, the AII or Attachment Individual Identifier, is meant to identify separate components within an AGI.

We have the Local AII with Type 1 and length of 4 same as the Remote AII, but you'll notice that the next portion is in hex followed by the IP in parenthesis. Fist the Type 1 and the Len 4. So we have to look at the RFC 6073 for specifics which is in accordance with RFC 5085, Cisco documentation is rather slim on this. The PW switching point PE sub-TLV types, 1 in this case or 0x01 has a fixed length of 4 bits or a nibble. The RFC references MS-PW and VCCV, MS-PW hasn't been covered by me yet, but VCCV is coming. Basically what is happening is the PE transmitting the VCCV packet must set the TTL to the appropriate value to reach the destination S-PE or switching PE, if the packet is destined for the T-PE or terminating PE, set the TTL to any value sufficient to reach the T-PE. The output via debug, wireshark nor show command on the PE give us any indication specifically what's going on, however you can infer based on the IP at the end of the string that we are likely pointing to the T-PE.

Since the PWs were dynamically created, there isn't much that can be done to them unless you create a pseudowire template, configure an arguement and then apply the template to the autodiscovery configuration under the l2vpn vfi context.

template type pseudowire LOAD_BALANCE

encapsulation mpls

load-balance flow

l2vpn vfi context VPLS

autodiscovery bgp signaling ldp template LOAD_BALANCE

R3#show l2vpn atom vc vcid 1356 detail | in Load|Destination

Destination address: 192.168.1.5 VC ID: 1356

Load Balance: ECMP

Destination address: 192.168.1.6 VC ID: 1356

Load Balance: ECMP

There is a ton of debug output, but it is basically what I have already shown you.

The VPLS ID is also carried BGP updates. If configured manually to be a value other than the AGI value, the VPN will not form.

l2vpn vfi context VPLS

autodiscovery bgp signaling ldp template LOAD_BALANCE

vpls-id 1:100

BGP(9): 192.168.1.15 rcv UPDATE w/ attr: nexthop 192.168.1.3, origin ?, localpref 100, metric 0, originator 192.168.1.3, clusterlist 192.168.1.15, merged path , AS_PATH , community , extended community RT:1:1356 L2VPN AGI:1:100, SSA attribute

BGP(9): 192.168.1.15 rcv UPDATE about 1:1356:192.168.1.3/96 -- DENIED due to: ORIGINATOR is us; MP_REACH NEXTHOP is our own address;

Undo the configuration, put a "no" in front of the vpls id and that fixes the issue. Another flood of debug output and you'll be back in business.

That about wraps this introduction to VPLS BGP AD with LDP signalling.

Thanks for stopping by!

Rob Riker, CCIE #50693