IOS XE 15.5
IOS XR 5.3
The topology for this demo:
In this post we'll take a look at having multiple pseudowires configured between R1, R5 and R6. We won't be leveraging XRv for the simple reason that P2P PWs don't appear to be supported in Dot1Q, SI Dot1Q or SI QinQ modes, only in port mode. We have several different variant to look at in this post so I will keep the intro short. We will look at the following variants, SI and Dot1Q, QinQ is very similar, just adding a VLAN ID. Port mode simply grabs all VLAN IDs. So we'll focus more on the connectivity scenarios rather than the encapsulation that the customer connects with, since we have already seen that.
We will be looking at having a minimum of 2 PWs configured for redundancy purposes between any 2 PEs. We will have 4 separate designs, we will use 100/101, 200/201, 300/301, 400/401, the even PW is the primary and the odd is the backup, for the Dot1Q/SI value, PW, the xconnect context name will change based on the scenario. Any caveats will be identified as we go through each scenario. After we configured scenario 100, we'll use just service instances moving forward. I would recommend being able to flip back and forth and knowing the caveats.
1 PE and 1 CE, 1 AC - 100/101
Done between R1 and R13
1 PE and 1 CE, 2 AC - 200/201
Done between R1 and R13, R6 and R10
1 PE and 2 CE, 2 AC - 300/301
Done between R5, R9 and R10
2 PE and 2 CE, 2 AC - 400/401
Done between R5, R6, R9 and R10
1 PE and 1 CE, 1 AC - 100/101
R1
interface GigabitEthernet2.100
encapsulation dot1Q 100
!
interface pseudowire100
encapsulation mpls
neighbor 192.168.1.6 100
interface pseudowire101
encapsulation mpls
neighbor 192.168.1.6 101
!
l2vpn xconnect context 1PE_1CE_1AC
member GigabitEthernet2.100
member pseudowire100 group RED priority 10
member pseudowire101 group RED priority 11
R6
interface GigabitEthernet2.100
encapsulation dot1Q 100
!
interface pseudowire100
encapsulation mpls
neighbor 192.168.1.1 100
interface pseudowire101
encapsulation mpls
neighbor 192.168.1.1 101
!
l2vpn xconnect context 1PE_1CE_1AC
member GigabitEthernet2.100
member pseudowire100 group RED priority 10
member pseudowire101 group RED priority 11
The interface level configuration for the G2.100 and the PW are pretty straightforward. It's the xconnect context that changes, specifically the "group" is used to bundle 2 pseudowires together for failover capabilities. The group is coupled with a priority to make sure that the appropriate PW is used for forwarding. A lower priority is preferred. One caveat, no more that 4 members to any one context is permitted, so we could attach 3 PWs for 3 separate LSP to the remote PE. From a design perspective, PWs 1 and 2 will likely have a TE tunnels attached to them, each taking separate paths in the network and the third will likely use the LDP path. We'll take a look at that in the TE section.
The interface level configuration for the G2.100 and the PW are pretty straightforward. It's the xconnect context that changes, specifically the "group" is used to bundle 2 pseudowires together for failover capabilities. The group is coupled with a priority to make sure that the appropriate PW is used for forwarding. A lower priority is preferred. One caveat, no more that 4 members to any one context is permitted, so we could attach 3 PWs for 3 separate LSP to the remote PE. From a design perspective, PWs 1 and 2 will likely have a TE tunnels attached to them, each taking separate paths in the network and the third will likely use the LDP path. We'll take a look at that in the TE section.
Verification
R6#show mpls l2transport vc detail | in label
Output interface: Gi1.26, imposed label stack {26 27}
Last label FSM state change time: 00:04:41
MPLS VC labels: local 26, remote 27
SSO Descriptor: 192.168.1.1/100, local label: 26
Output interface: Gi1.26, imposed label stack {26 26}
Last label FSM state change time: 00:04:41
MPLS VC labels: local 25, remote 26
SSO Descriptor: 192.168.1.1/101, local label: 25
R6#show l2vpn atom vc
Service
Interface Peer ID VC ID Type Name Status
--------- --------------- ---------- ------ ------------------------ ----------
pw100 192.168.1.1 100 p2p 1PE_1CE_1AC UP
pw101 192.168.1.1 101 p2p 1PE_1CE_1AC STANDBY
So as you can see, we have 2 pseudowires attached to the context and pw100 is UP and primary and pw101 is STANDBY and backup.
Another caveat is with xconnect switchover, on the CSRs, this is not supported when you try to switchover from pw100 to pw101 on the same peer, R1 for instance. If you are peered to 1 PW to R5 and 1 PW to R6, switchover is supported.
R6#xconnect backup force-switchover peer 192.168.1.1 101
%%Xconnect: manual switchover failed
CE Verification
R10#sh ip eigrp nei
EIGRP-IPv4 Neighbors for AS(1)
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
0 192.168.10.13 Gi1.100 14 00:46:28 42 252 0 117
1 PE and 1 CE, 2 AC - 200/201
So R1 and R6 both have 2 AC to reach their connected CEs. This provides HA in case one AC goes down for some reason. Here is some important information, only 1 AC can be mapped to 1 xconnect context. If you map 2 or more, the context will prevent the pair from establishing, this is to prevent loops in the SP core. I won't demo a bad config, I will post the good config so you can see it.
R1
Another caveat is with xconnect switchover, on the CSRs, this is not supported when you try to switchover from pw100 to pw101 on the same peer, R1 for instance. If you are peered to 1 PW to R5 and 1 PW to R6, switchover is supported.
R6#xconnect backup force-switchover peer 192.168.1.1 101
%%Xconnect: manual switchover failed
CE Verification
R10#sh ip eigrp nei
EIGRP-IPv4 Neighbors for AS(1)
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
0 192.168.10.13 Gi1.100 14 00:46:28 42 252 0 117
1 PE and 1 CE, 2 AC - 200/201
So R1 and R6 both have 2 AC to reach their connected CEs. This provides HA in case one AC goes down for some reason. Here is some important information, only 1 AC can be mapped to 1 xconnect context. If you map 2 or more, the context will prevent the pair from establishing, this is to prevent loops in the SP core. I won't demo a bad config, I will post the good config so you can see it.
R1
interface GigabitEthernet2
service instance 200 ethernet
encapsulation dot1q 200
!
service instance 201 ethernet
encapsulation dot1q 201
!
interface pseudowire2001
encapsulation mpls
neighbor 192.168.1.6 2001
interface pseudowire2002
encapsulation mpls
neighbor 192.168.1.6 2002
interface pseudowire2011
encapsulation mpls
neighbor 192.168.1.6 2011
interface pseudowire2012
encapsulation mpls
neighbor 192.168.1.6 2012
service instance 200 ethernet
encapsulation dot1q 200
!
service instance 201 ethernet
encapsulation dot1q 201
!
interface pseudowire2001
encapsulation mpls
neighbor 192.168.1.6 2001
interface pseudowire2002
encapsulation mpls
neighbor 192.168.1.6 2002
interface pseudowire2011
encapsulation mpls
neighbor 192.168.1.6 2011
interface pseudowire2012
encapsulation mpls
neighbor 192.168.1.6 2012
!
l2vpn xconnect context 1PE_1CE_2AC_200
member GigabitEthernet2 service-instance 200
member pseudowire2001 group 2AC priority 1
member pseudowire2002 group 2AC priority 2
l2vpn xconnect context 1PE_1CE_2AC_201
member GigabitEthernet2 service-instance 201
member pseudowire2011 group 2AC priority 1
member pseudowire2012 group 2AC priority 2
PW 2001 and 2002 are for service instance 200, PW 2011 and 2012 are for service instance 201. I appended 1 and 2 respectively for primary (1) and secondary (2). For context 200, I mapped SI 200 to PW 2001 and 2002, both grouped together and 2001 getting priority 1 (better). R6 gets an identical configuration, the only difference is that the PW neighbor points to R1.
Verification:
R1#sh l2vpn atom vc
Service
Interface Peer ID VC ID Type Name Status
--------- --------------- ---------- ------ ------------------------ ----------
pw2001 192.168.1.6 2001 p2p 1PE_1CE_2AC_200 UP
pw2002 192.168.1.6 2002 p2p 1PE_1CE_2AC_200 STANDBY
pw2011 192.168.1.6 2011 p2p 1PE_1CE_2AC_201 UP
pw2012 192.168.1.6 2012 p2p 1PE_1CE_2AC_201 STANDBY
1 PE and 2 CE, 2 AC - 300/301
This scenario is basically where 1 PE terminates 2 different ACs to the same customer but to different CE routers. It could be used in a multihoming scenario, but in this case this is a dual home to the same PE. The configuration is very similar to the previous one, with the exception of the 2 different CEs.
R1
interface GigabitEthernet2
service instance 300 ethernet
encapsulation dot1q 300
!
service instance 301 ethernet
encapsulation dot1q 301
!
interface pseudowire3001
encapsulation mpls
neighbor 192.168.1.5 3001
interface pseudowire3002
encapsulation mpls
neighbor 192.168.1.5 3002
interface pseudowire3011
encapsulation mpls
neighbor 192.168.1.5 3011
interface pseudowire3012
encapsulation mpls
neighbor 192.168.1.5 3012
!
l2vpn xconnect context 1PE_2CE_2AC_300
member GigabitEthernet2 service-instance 300
member pseudowire3001 group 2AC priority 1
member pseudowire3002 group 2AC priority 2
l2vpn xconnect context 1PE_2CE_2AC_301
member GigabitEthernet2 service-instance 301
member pseudowire3011 group 2AC priority 1
member pseudowire3012 group 2AC priority 2
R5
interface GigabitEthernet2
service instance 300 ethernet
encapsulation dot1q 300
!
PE Verification
R1# sh l2vpn atom vc
Service
Interface Peer ID VC ID Type Name Status
--------- --------------- ---------- ------ ------------------------ ----------
pw3001 192.168.1.5 3001 p2p 1PE_2CE_2AC_300 UP
pw3002 192.168.1.5 3002 p2p 1PE_2CE_2AC_300 STANDBY
pw3011 192.168.1.5 3011 p2p 1PE_2CE_2AC_301 UP
pw3012 192.168.1.5 3012 p2p 1PE_2CE_2AC_301 STANDBY
As you can see, the 2 contexts are up and running, the pseudowires that received the priority of "1" are UP.
R1# sh l2vpn atom vc pseudowire 3001 detail | in label
Last label FSM state change time: 01:22:47
Output interface: Gi1.115, imposed label stack {24501 534}
SSO Descriptor: 192.168.1.5/3001, local label: 33
We can see that label 24501 is used to reach R5, label 33 is for the tLDP session between R1 and R5. Label 534 is used by R5 for the tLDP session back to R1.
R1#sh mpls forwarding-table labels 23
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
23 24202 192.168.1.5/32 0 Gi1.112 10.1.12.12
24501 192.168.1.5/32 0 Gi1.115 10.1.15.15
Service
Interface Peer ID VC ID Type Name Status
--------- --------------- ---------- ------ ------------------------ ----------
pw2001 192.168.1.6 2001 p2p 1PE_1CE_2AC_200 UP
pw2002 192.168.1.6 2002 p2p 1PE_1CE_2AC_200 STANDBY
pw2011 192.168.1.6 2011 p2p 1PE_1CE_2AC_201 UP
pw2012 192.168.1.6 2012 p2p 1PE_1CE_2AC_201 STANDBY
As you can see, SI 200 has 2 different PWs mapped to it, 2001 is UP, same with SI 201, PW 2011 is up.
R1#sh l2vpn atom vc pseudowire 2001 detail | in label
Last label FSM state change time: 00:24:50
Output interface: Gi1.115, imposed label stack {24508 28}
SSO Descriptor: 192.168.1.6/2001, local label: 28
We can see that there are 2 label values associated here, 24508 is used by R1 to reach R6. Label 28 is used by R1 to forward data over the P2P LSP established by the tLDP session initiated by the "xconnect"
R1#sh mpls forwarding-table labels 22
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
22 24207 192.168.1.6/32 0 Gi1.112 10.1.12.12
24508 192.168.1.6/32 0 Gi1.115 10.1.15.15
R1#sh mpls forwarding-table labels 28
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
28 No Label l2ckt(20) 42464 Gi2 point2point
CE verification
R10#sh ip eigrp nei
EIGRP-IPv4 Neighbors for AS(1)
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
2 192.168.21.13 Gi1.201 11 00:04:31 1600 5000 0 118
1 192.168.20.13 Gi1.200 13 00:39:07 36 216 0 116
1 PE and 2 CE, 2 AC - 300/301
This scenario is basically where 1 PE terminates 2 different ACs to the same customer but to different CE routers. It could be used in a multihoming scenario, but in this case this is a dual home to the same PE. The configuration is very similar to the previous one, with the exception of the 2 different CEs.
R1
interface GigabitEthernet2
service instance 300 ethernet
encapsulation dot1q 300
!
service instance 301 ethernet
encapsulation dot1q 301
!
interface pseudowire3001
encapsulation mpls
neighbor 192.168.1.5 3001
interface pseudowire3002
encapsulation mpls
neighbor 192.168.1.5 3002
interface pseudowire3011
encapsulation mpls
neighbor 192.168.1.5 3011
interface pseudowire3012
encapsulation mpls
neighbor 192.168.1.5 3012
!
l2vpn xconnect context 1PE_2CE_2AC_300
member GigabitEthernet2 service-instance 300
member pseudowire3001 group 2AC priority 1
member pseudowire3002 group 2AC priority 2
l2vpn xconnect context 1PE_2CE_2AC_301
member GigabitEthernet2 service-instance 301
member pseudowire3011 group 2AC priority 1
member pseudowire3012 group 2AC priority 2
R5
interface GigabitEthernet2
service instance 300 ethernet
encapsulation dot1q 300
!
interface GigabitEthernet3
service instance 301 ethernet
encapsulation dot1q 301
!
interface pseudowire3001
encapsulation mpls
neighbor 192.168.1.1 3001
interface pseudowire3002
encapsulation mpls
neighbor 192.168.1.1 3002
interface pseudowire3011
encapsulation mpls
neighbor 192.168.1.1 3011
interface pseudowire3012
encapsulation mpls
neighbor 192.168.1.1 3012
!
l2vpn xconnect context 1PE_2CE_2AC_300
member GigabitEthernet2 service-instance 300
member pseudowire3001 group 2AC priority 1
member pseudowire3002 group 2AC priority 2
l2vpn xconnect context 1PE_2CE_2AC_301
member pseudowire3011 group 2AC priority 1
member pseudowire3012 group 2AC priority 2
member GigabitEthernet3 service-instance 301
PE Verification
R1# sh l2vpn atom vc
Service
Interface Peer ID VC ID Type Name Status
--------- --------------- ---------- ------ ------------------------ ----------
pw3001 192.168.1.5 3001 p2p 1PE_2CE_2AC_300 UP
pw3002 192.168.1.5 3002 p2p 1PE_2CE_2AC_300 STANDBY
pw3011 192.168.1.5 3011 p2p 1PE_2CE_2AC_301 UP
pw3012 192.168.1.5 3012 p2p 1PE_2CE_2AC_301 STANDBY
As you can see, the 2 contexts are up and running, the pseudowires that received the priority of "1" are UP.
R1# sh l2vpn atom vc pseudowire 3001 detail | in label
Last label FSM state change time: 01:22:47
Output interface: Gi1.115, imposed label stack {24501 534}
SSO Descriptor: 192.168.1.5/3001, local label: 33
We can see that label 24501 is used to reach R5, label 33 is for the tLDP session between R1 and R5. Label 534 is used by R5 for the tLDP session back to R1.
R1#sh mpls forwarding-table labels 23
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
23 24202 192.168.1.5/32 0 Gi1.112 10.1.12.12
24501 192.168.1.5/32 0 Gi1.115 10.1.15.15
R1#sh mpls forwarding-table labels 33
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
33 No Label l2ckt(24) 81531 Gi2 point2point
CE Verification
R13(config-subif)#do sh ip eigrp nei
EIGRP-IPv4 Neighbors for AS(1)
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
4 192.168.31.10 Gi2.301 13 01:49:11 37 222 0 150
3 192.168.30.9 Gi2.300 13 01:49:40 692 4152 0 15
R9#sh ip eigrp nei
EIGRP-IPv4 Neighbors for AS(1)
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
0 192.168.30.13 Gi1.300 14 01:51:52 12 150 0 160
R10(config-subif)#do sh ip eigrp nei
EIGRP-IPv4 Neighbors for AS(1)
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
3 192.168.31.13 Gi2.301 11 01:51:46 69 414 0 162
So as you can see, R13 is peered to both R9 and R10. So if R9 or R10, individually failed, not together, separately, our design would be ok. This gives our HA that we would expect. However, no one that does this in real life would ever terminate 2 ACs to the same PE. If R5 ever died, the customer would be down as well. So this segways us into the 2 PE and 2 CE design next.
2 PE and 2 CE, 2 AC - 400/401
This design gives much more flexibility, high availability and scale. It requires more routers and more configuration on the SP side, but it gives path diversity that the customer is more than likely demand. So we'll setup R5 and R6 to terminate ACs from R9 and R10. Now if R9 and R10 have a backdoor link, this would be a routed connected to the provider, unless it's a L2 P2P circuit, it would be terminated on a L3 interface and no worry of L2 traffic crossing broadcast domains.
R1
interface GigabitEthernet2
service instance 400 ethernet
encapsulation dot1q 400
!
service instance 401 ethernet
encapsulation dot1q 401
!
interface pseudowire4001
encapsulation mpls
neighbor 192.168.1.5 4001
interface pseudowire4002
encapsulation mpls
neighbor 192.168.1.5 4002
interface pseudowire4011
encapsulation mpls
neighbor 192.168.1.6 4011
interface pseudowire4012
encapsulation mpls
neighbor 192.168.1.6 4012
!
l2vpn xconnect context 2PE_2CE_2AC_400
member GigabitEthernet2 service-instance 400
member pseudowire4001 group 2AC priority 1
member pseudowire4002 group 2AC priority 2
l2vpn xconnect context 2PE_2CE_2AC_401
member GigabitEthernet2 service-instance 401
member pseudowire4011 group 2AC priority 1
member pseudowire4012 group 2AC priority 2
Since we have PWs to different PEs, we have twice the verification. You can see the PWs are up and operational, that is the first thing we need to highlight.
R1#sh l2vpn atom vc | in 40
pw4001 192.168.1.5 4001 p2p 2PE_2CE_2AC_400 UP
pw4002 192.168.1.5 4002 p2p 2PE_2CE_2AC_400 STANDBY
pw4011 192.168.1.6 4011 p2p 2PE_2CE_2AC_401 UP
pw4012 192.168.1.6 4012 p2p 2PE_2CE_2AC_401 STANDBY
Next we have to look at the labels allocated, 34 is used for the PW and 24501 is used to reach R5. Label 530 is used by R5 for the PW.
R1#sh l2vpn atom vc pseudowire 4001 detail | in label
Last label FSM state change time: 00:34:08
Output interface: Gi1.115, imposed label stack {24501 530}
SSO Descriptor: 192.168.1.5/4001, local label: 34
R1#show mpls forwarding-table labels 34
R1#show mpls forwarding-table labels 23
R1#sh l2vpn atom vc pseudowire 4011 detail | in label
Last label FSM state change time: 00:31:24
Output interface: Gi1.112, imposed label stack {24207 33}
SSO Descriptor: 192.168.1.6/4011, local label: 39
This design gives much more flexibility, high availability and scale. It requires more routers and more configuration on the SP side, but it gives path diversity that the customer is more than likely demand. So we'll setup R5 and R6 to terminate ACs from R9 and R10. Now if R9 and R10 have a backdoor link, this would be a routed connected to the provider, unless it's a L2 P2P circuit, it would be terminated on a L3 interface and no worry of L2 traffic crossing broadcast domains.
R1
interface GigabitEthernet2
service instance 400 ethernet
encapsulation dot1q 400
!
service instance 401 ethernet
encapsulation dot1q 401
!
interface pseudowire4001
encapsulation mpls
neighbor 192.168.1.5 4001
interface pseudowire4002
encapsulation mpls
neighbor 192.168.1.5 4002
interface pseudowire4011
encapsulation mpls
neighbor 192.168.1.6 4011
interface pseudowire4012
encapsulation mpls
neighbor 192.168.1.6 4012
!
l2vpn xconnect context 2PE_2CE_2AC_400
member GigabitEthernet2 service-instance 400
member pseudowire4001 group 2AC priority 1
member pseudowire4002 group 2AC priority 2
l2vpn xconnect context 2PE_2CE_2AC_401
member GigabitEthernet2 service-instance 401
member pseudowire4011 group 2AC priority 1
member pseudowire4012 group 2AC priority 2
Since we have PWs to different PEs, we have twice the verification. You can see the PWs are up and operational, that is the first thing we need to highlight.
R1#sh l2vpn atom vc | in 40
pw4001 192.168.1.5 4001 p2p 2PE_2CE_2AC_400 UP
pw4002 192.168.1.5 4002 p2p 2PE_2CE_2AC_400 STANDBY
pw4011 192.168.1.6 4011 p2p 2PE_2CE_2AC_401 UP
pw4012 192.168.1.6 4012 p2p 2PE_2CE_2AC_401 STANDBY
Next we have to look at the labels allocated, 34 is used for the PW and 24501 is used to reach R5. Label 530 is used by R5 for the PW.
R1#sh l2vpn atom vc pseudowire 4001 detail | in label
Last label FSM state change time: 00:34:08
Output interface: Gi1.115, imposed label stack {24501 530}
SSO Descriptor: 192.168.1.5/4001, local label: 34
R1#show mpls forwarding-table labels 34
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
34 No Label l2ckt(26) 59980 Gi2 point2point
R1#show mpls forwarding-table labels 23
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
23 24202 192.168.1.5/32 0 Gi1.112 10.1.12.12
24501 192.168.1.5/32 0 Gi1.115 10.1.15.15
Same as above, label 39 is used for the PW, label 24207 is used to reach R6 and label 33 is used by R6 for the PW to reach R1. It is very important to know how to identify what labels are used for what purpose.
R1#sh l2vpn atom vc pseudowire 4011 detail | in label
Last label FSM state change time: 00:31:24
Output interface: Gi1.112, imposed label stack {24207 33}
SSO Descriptor: 192.168.1.6/4011, local label: 39
R1#show mpls forwarding-table labels 39
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
39 No Label l2ckt(29) 25468 Gi2 point2point
R1#show mpls forwarding-table labels 22
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
22 24207 192.168.1.6/32 0 Gi1.112 10.1.12.12
24508 192.168.1.6/32 0 Gi1.115 10.1.15.15
CE Verification
R13#sh ip eigrp nei
EIGRP-IPv4 Neighbors for AS(1)
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
6 192.168.41.10 Gi2.401 14 00:21:25 45 270 0 155
5 192.168.40.9 Gi2.400 14 00:42:21 1070 5000 0 14
So we have connectivity on R13 to R9 and R10 respectively. "Q" counts are 0 which means that updates are being exchanged.
Thanks for stopping by!
Rob Riker, CCIE #50693
