# Virtual Private LAN Service – VPLS Basic Info – 1 #

In this post I tried to summarize what a Pseudowire is in the context of Ethernet Emulation over an MPLS core. To understand VPLS is necessary to have an idea of what is a Pseudowire because the same basic element is used to deploy VPLS service too.

Virtuale Private LAN Services (VPLS in short) is a mechanism that uses Pseudowires to establish a technology able to offer Multipoint and Broadcast capabilities to customer, we can say that customers connected to a VPLS service perceive this service as if they were connected to a Common Emulated LAN Segment (we can call it a VPLS domain) realized over an MPLS core, where different remote sites are connected and able to communicate as if they were connected to a virtualized big switch offering them traditional L2 forwarding mechanism (MACs learning and aging, forwarding of unicast and BUM (Broadcast-Unknown Unicast-Multicast) traffic). The following figure should give an idea of the concept:

Before going deeper about VPLS concepts, let’s see how to configure PEs that will offer VPLS service. Next figure shows the topology I used for these first tests:

Here I have 3 CSR1000v working as PEs, PE1 Lo0 10.1.1.1, PE2 Lo0 10.2.2.2, PE3 10.3.3.3. Prerequisites to have working VPLS, as for other MPLS application, is to have a configured and working MPLS core (here made of P1,P2,P3,P5 working as P routers) and MPLS LSP Paths established between the PEs Loopbacks, here the trace confirming we have working MPLS Paths, note that I have constrained the MPLS label space on each Ps and PEs routers for easy reading of the imposed labels:

P1,P2,P3,P5 ==> Label Space [1000-1999,2000-2999,3000-3999,5000-5999]
PE1,PE2,PE3 ==> 100000-100999,200000-200999,300000-300999

PE1#traceroute 10.2.2.2 source 10.1.1.1
Type escape sequence to abort.
Tracing the route to 10.2.2.2
VRF info: (vrf in name/id, vrf out name/id)
1 10.0.0.6 [MPLS: Label 1003 Exp 0] 57 msec 6 msec 7 msec
2 10.0.0.22 [MPLS: Label 5000 Exp 0] 5 msec 6 msec 5 msec
3 10.0.0.26 [MPLS: Label 3000 Exp 0] 3 msec 5 msec 6 msec
4 10.0.0.18 7 msec *  5 msec

PE1#traceroute 10.3.3.3 source 10.1.1.1
Type escape sequence to abort.
Tracing the route to 10.3.3.3
VRF info: (vrf in name/id, vrf out name/id)
1 10.0.0.6 [MPLS: Label 1009 Exp 0] 5 msec 5 msec 5 msec
2 10.0.0.22 [MPLS: Label 5009 Exp 0] 4 msec 5 msec 5 msec
3 10.0.0.38 6 msec *  5 msec

PE3#traceroute 10.2.2.2 source 10.3.3.3
Type escape sequence to abort.
Tracing the route to 10.2.2.2
VRF info: (vrf in name/id, vrf out name/id)
1 10.0.0.37 [MPLS: Label 5000 Exp 0] 4 msec 4 msec 5 msec
2 10.0.0.26 [MPLS: Label 3000 Exp 0] 5 msec 4 msec 3 msec
3 10.0.0.18 11 msec *  4 msec

To understand VPLS configuration it’s better to understand how a PE can be represented in a VPLS scenario, the following figure shows a representation of a PE extracted from the book “L2 VPN Architectures” a worth reading book IMHO)

Looking at the picture we can see that the PE can be represented as a Virtual Switch in which a Bridge Module sees the Customer Network via an Attachment Circuit and a Virtual Forwarding Instance (VFI) sees the MPLS core via Pseudowires.

Bridge Module and VFI have an equivalent function that is the one of a classical ethernet switch: learning mac address and forwarding ethernet frames based on destination MACs and then building a L2 forwarding table, the difference is that Bridge Module does this for attachment circuit and VFI operates instead on Pseudowires. We can see the L2 forwarding table of the PE as a single logical entity built by both Bridge Module and VFI.

Having an idea about functional operations of VPLS PE helps us in configuring it. I will start from PE1. My PEs are 3 CSR1000v and VPLS can be configured using two different CLI syntax, I will use the configuration that follows this set of commands ==> L2VPN Protocol-Based CLIs feature (available after 3.7S, see this link for more detail)

Let’s start from the Attachment Circuit side where the CEs are connected to the PEs, on this physical interface the PE router can receive from customer:

– untagged frames
– tagged frames
– double tagged frames

First thing we need is a mechanism to match the incoming type of frame with the Ethernet Virtual Circuit (EVC) we want to use to forward the frame to the remote PE over the MPLS core.

Here it’s worth, for me, spending some time to recap some concepts about the EVC framework introduced by Cisco to be compliant with MEF (Metro Ethernet Frame) terms in some way.

EVC framework uses 4 building blocks:

1] ETHERNET FLOW POINT (EFP) – SERVICE INSTANCE: looking at Cisco Documentation on the web

LINK1
LINK2

You can find this definitions:

EFP is a “Transport-agnostic abstraction of an Ethernet service on an interface” Cryptic? Maybe. “An Ethernet flow point (EFP) SERVICE INSTANCE is a logical interface that connects a bridge domain to a physical port or to an EtherChannel. Configuring a service instance on a Layer 2 port creates a pseudoport or EFP on which you configure EVC features.”

Then, to me, EFP and Service Instance are more or less the same thing.

2] ETHERNET VIRTUAL CIRCUIT (EVC): “Device local object (container) for network-wide service parameters” Cryptic? Maybe. Here we have a meta definition, EVC framework defined by the same acronym… confusing? Maybe

3] BRIDGE DOMAIN (BD): ETHERNET BROADCAST DOMAIN LOCAL TO A DEVICE

4] BRIDGE DOMAIN INTERFACE (BDI): Logical Layer 3 interface associated with a BD to perform integrated routing and bridging

Let’s try to put all the pieces together in a practical way. Our PE1 can receives different type of frames on its physical ports, to make an efficient use of these ports, we divide the physical ports in Logical Entities called Service Instances that do the task of matching different type of frames that could enter the ports.

Here what we could do for example on PE1:

PE1(config)#int G1

PE1(config-if)#service instance ?
<1-8000>  Service Instance Identifier

PE1(config-if)#service instance 100 ?
ethernet  Configure an Ethernet Instance

PE1(config-if)#service instance 100 ethernet ?
WORD  Attach an EVC to the service instance
<cr>

PE1(config-if)#service instance 100 ethernet

PE1#show run int G1 | b interface
interface GigabitEthernet1
no ip address
negotiation auto
service instance 100 ethernet
!

Adding more service instance ==>

PE1(config)#int G1
PE1(config-if)#service instance 200 ethernet
PE1(config-if-srv)#exit
PE1(config-if)#service instance 300 ethernet

PE1#show run int G1 | b interface
interface GigabitEthernet1
no ip address
negotiation auto
service instance 100 ethernet
service instance 200 ethernet
service instance 300 ethernet
!

At this level I have only divided interface G1 in three logical but empty entities. Let’s see what we can define under each service instance:

PE1(config)#int G1
PE1(config-if)#service instance 100 ethernet
PE1(config-if-srv)#?
Ethernet EFP configuration commands:
……………
ENCAPSULATION   Configure ethernet frame match criteria
……………

PE1(config-if-srv)#encapsulation ?
default          catch-all unconfigured encapsulation
  dot1ad           802.1ad – Provider Bridges
  dot1q            IEEE 802.1Q Virtual LAN or S-VLAN
  priority-tagged  Priority tagged EFP
  untagged         Untagged encapsulation

Based on the configured type of encapsulation each service instance can match different types of ethernet frames, we find QinQ (for double tagged frame), single tagged frames, untagged frames, interestingly we see a default keyword that catch all things that are not matched by any other service instances.

To match frames arriving on G1 interfaces we must associate an encapsulation type to each service instance, for example I could set:

PE1#sh run int G1 | b interface
interface GigabitEthernet1
no ip address
negotiation auto
service instance 100 ethernet
  encapsulation untagged
service instance 200 ethernet
  encapsulation dot1q 12
 service instance 300 ethernet
  encapsulation dot1q 3000 second-dot1q 13
!

Here I created 3 service instance that match different type of ethernet frames, untagged, 802.1q tagged with vlan-id 12 and double tagged with outer vlan tag 3000 and inner vlan tag 13

The second piece of information we need to establish an Ethernet Virtual Circuit is a Bridge Domain, we know from its definition that is an ethernet broadcast domain local to a device (PE1 in this example), in this broadcast domain I can put, for example, all the entities of a customer that need to share this L2 broadcast domain; Bridge-Domain will be the glue that links Customer Side (Service Instance on interfaces facing customers) and Core Side (the VFI to which PWs facing the core are attached), the Bridge-Domain will receive MACs info from these two entities and from there PE will build its L2 forwarding table. Here the steps to configure the Bridge Domain.

PE1(config)#bridge-domain ?
<1-8192>       bridge-domain ID
c-mac          Configure c-mac global commands
otv            Configure overlay bridge domains
parameterized  bridge-domain is derived from parameters

Don’t consider the last three options, we need to set an id number for the brdige domain, let’s choose 1000:

PE1(config)#bridge-domain 1000
PE1(config-bdomain)#?
Bridge-domain mode commands:
default   Set a command to its defaults
exit      Leave the bridge-domain configuration submode
ip        IP related commands
mac       MAC Address related commands
member    bridge-domain member
no        Negate a command or set its defaults
shutdown  Take the bridge domain admin down

We can set some parameters under the defined BD, for example aging time and limit for learnt MACs:

PE1(config-bdomain)#mac ?
aging-time  Set age time of mac addresses per bridge domain
  learning    Disable mac learning on bridge domain
  limit       Limit the MAC Addresses learnt

Then we need to configure who are the members of this BD, you can put under the bridge domain all usable interfaces, in this example interface is G1 and its service instance:

PE1(config-bdomain)#member GigabitEthernet 1 ?
service-instance  ethernet serivce instance

PE1(config-bdomain)#member GigabitEthernet 1 service-instance 100 ==> this defines Service Instance 100 of G1 belonging to BD 1000

With the configuration made so far I know that if an untagged ethernet frame hits interface G1 of PE1 it will be matched by service instance 100 and will be attached to BD 1000.

To let the frame exit PE1 toward its destination through the MPLS core, we miss a final piece, gluing the BD 1000, to which service instance 100 of G1 belongs to, with a Virtual Forwarding Instance (VFI) that links a Pseudowire that will transport the frame to the egress PE. Let’s see how to define this VFI:

PE1(config)#l2vpn vfi context ?
WORD  Virtual Forwarding Instance (VFI) name

PE1(config)#l2vpn vfi context CUSTOMER1
PE1(config-vfi)#?
L2VPN vfi configuration commands:
default  Set a command to its defaults
exit     Exit from L2VPN vfi configuration mode
no       Negate a command or set its defaults
vpn      VPN configuration

PE1(config-vfi)#vpn ?
id  VPN id

PE1(config-vfi)#vpn id ?
<1-4294967295>  VPN id value

PE1(config-vfi)#vpn id 1

Then we need to define to which pseudowires this VFI is attached to:

PE1(config-vfi)#member 10.2.2.2 ? ==> 10.2.2.2 is PE2
<1-4294967295>  Enter VC ID value
encapsulation   Data encapsulation method
template        Template to use for encapsulation and protocol configuration

PE1(config-vfi)#member 10.2.2.2 12 ? ==> I set 12 for VC ID between PE1 and PE2 for this VFI.
encapsulation  Data encapsulation method
template       Template to use for encapsulation and protocol configuration

PE1(config-vfi)#member 10.2.2.2 12 encapsulation ?
mpls  Use MPLS encapsulation

PE1(config-vfi)#member 10.2.2.2 12 encapsulation mpls ==> mpls encapsulation for the pseudowire.

Last step is to tell router that this VFI is linked to BD 1000:

PE1(config)#bridge-domain 1000
PE1(config-bdomain)#member vfi CUSTOMER1

*Apr 12 13:57:34.753: %VFI-6-VFI_STATUS_CHANGED: Status of VFI CUSTOMER1 changed from DOWN to UP

At this point, we had defined this mapping:

Now that we should have a basic idea about the functional elements of a PE router in the context of EVC framework, we can use them to configure a basic VPLS service on CSR1Kv. A VPLS domain can be considered, in some way, a specific case within the EVC framework.

Let’s start from the first case where we must connect at L2 level three customer sites that are sending Untagged frames to the PEs where they are connected, the idea is shown  in the figure at the beginning of this post, you can reference it here

Let’s complete the configuration on PE1, I need to add under Bridge-Domain 1000 another member peer (PE3), I will name PW13 the Pseudowire that connect PE1 and PE3

PE1#show run | b vfi
l2vpn vfi context CUSTOMER1
vpn id 1
member 10.3.3.3 13 encapsulation mpls
member 10.2.2.2 12 encapsulation mpls

Let’s summarize all the relevant configuration on PE1:

I have not yet completed configuration for PE2 and PE3, let’s see some shows on PE1 before doing that.

PE1#show ethernet service instance id 100 interface G1 detail
Service Instance ID: 100
Service Instance Type: Static
Associated Interface: GigabitEthernet1
Associated EVC:
L2protocol drop
CE-Vlans:
Encapsulation: untagged
Interface Dot1q Tunnel Ethertype: 0x8100
State: Up
EFP Statistics:
Pkts In   Bytes In   Pkts Out  Bytes Out
436      47667          0          0
EFP Microblocks:
****************
Microblock type: Bridge-domain
Bridge-domain: 1000

Microblock type: L2Mcast
L2 Multicast GID: 1

Microblock type: dhcp_snoop
L2 Multicast GID: 1

The command above is useful to check:
– packets exchange over the Service Instance,
– the State of SI (Up in this case)
– the matching encapsulation (untagged)
– action performed on Layer2 protocol (drop)

Next command gives me info about the BD:

PE1#show bridge-domain 1000
Bridge-domain 1000 (3 ports in all)
State: UP                    Mac learning: Enabled
Aging-Timer: 300 second(s)
    GigabitEthernet1 service instance 100
    vfi CUSTOMER1 neighbor 10.2.2.2 12
    vfi CUSTOMER1 neighbor 10.3.3.3 13
   AED MAC address    Policy  Tag       Age  Pseudoport
   0   AABB.CC00.0700 forward dynamic   294  GigabitEthernet1.EFP100

Looking at this command it’s esy to see how BD is the glue that connect SI and VFI (Attachment Circuits (ACs) and Pseudowires using other words), this command tells me:

– The state of the bridge domain (UP)
– If MAC learning is enabled or not (Enabled)
– Which is the aging timer for the MACs learnt over the BD (300 seconds)
– Which is (are) the Service Instance attached to the BD (SI 100 of G1)
– Which is the VFI attached to the BD and which are the routers member of the VPN (PE2 10.2.2.2 using PW(VC)12 and PE3 10.3.3.3 using (PW(VC)13)
– Which are the MACs learnt over the BD, which is the action configured (forward)

The MAC reported in the command is the one belonging to e0/0 interface of CE1-1 connected to G1 and matched by SI 100 (untagged traffic):

CE1-1#show ip arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  192.168.123.1           –   aabb.cc00.0700  ARPA   Ethernet0/0

– Which is the Pseudoport created when the SI is configured ==> you see that IOS-XE defines it as Ethernet Flow Point 100 ==> GigabitEthernet1.EFP100

Next commands gives some info about VFI and configured PWs:

PE1#show l2vpn vfi
Legend: RT=Route-target, S=Split-horizon, Y=Yes, N=No
VFI name: CUSTOMER1, state: up, type: multipoint, signaling: LDP
VPN ID: 1
Bridge-Domain 1000 attachment circuits:
Pseudo-port interface: pseudowire100001
Interface          Peer Address     VC ID        S
  pseudowire100003   10.3.3.3         13           Y
  pseudowire100002   10.2.2.2         12           Y

Using this command we can verify the State(UP), the type (Multipoint), the signaling (LDP) and the status of L2 Split-Horizon on each Pseudowire (enabled by default).

NOTE: L2 SPLIT-HORIZON is a mechanism to prevent loop in a Layer 2 Network made of fully meshed Pseudowires. More on this later.

It’s interesting to capture the LDP signaling packets. Peering PEs will discover each other sending targeted Hello to the Loopback of remote PE and will find an agreement about Pseudowires that must be established using Label Mapping Messages, the principles are the same of the Point-to-Point pseudowires case (if you want to give a detailed look at how label mapping messages are exchanged, read this other article I wrote.

Let’s complete the configuration on PE2 and PE3:

To track LDP signaling it’s enough that there is IP/MPLS connectivity between PEs (to send/receive the targeted Hello) and that VFIs are active (to signal the pseudowires).

I will put first all BD and VFI in shut and then I will re-enable them one by one.

PE1#sh run bridge-domain 1000 | b Cur
Current bridge-domain configuration:
1000
shutdown
member GigabitEthernet1 service-instance 100
member vfi CUSTOMER1

PE1#show run | b vfi
l2vpn vfi context CUSTOMER1
shutdown
vpn id 1
member 10.2.2.2 12 encapsulation mpls
member 10.3.3.3 13 encapsulation mpls

Same shutdown status on PE2 and PE3.

NOTE: When BD is in shutdown VFI attached to it remains down, then to activate the VFI, first activate the BD.

I will restart LDP activating interface of the PEs toward the core one by one and activating VFIs, here what happen when I activate interface of PE1 and PE2 and VFI only on PE1:

PE1(config)#int G2
PE1(config-if)#no shut

PE2(config)#int G2
PE2(config-if)#no shut

PE1(config)#l2vpn vfi context CUSTOMER1
PE1(config-vfi)#no shut

PE1(config)#bridge-domain 1000
PE1(config-bdomain)#no shut

*Apr 14 13:41:47 ITALY: %VFI-6-VFI_STATUS_CHANGED: Status of VFI CUSTOMER1 changed from ADMINDOWN to DOWN
*Apr 14 13:41:47 ITALY: %VFI-6-VFI_STATUS_CHANGED: Status of VFI CUSTOMER1 changed from DOWN to UP
*Apr 14 13:41:47 ITALY: ldp: Sent label mapping msg to 10.2.2.2:0 (pp 0x7F90249F19A8)

Below you can see the captured Label Mapping Message sent by PE1 to PE2

The label mapping message is exactly the same we can see in the Point-to-Point Pseudowire case (see again this link for more details)

Activating BD and VFI also on PE2:

PE2(config)#bridge-domain 1000
PE2(config-bdomain)#no shut

PE2(config)#l2vpn vfi context CUSTOMER1
PE2(config-vfi)#no shut

PE2#
*Apr 14 14:01:44 ITALY: %VFI-6-VFI_STATUS_CHANGED: Status of VFI CUSTOMER1 changed from ADMINDOWN to DOWN
*Apr 14 14:01:44 ITALY: %VFI-6-VFI_STATUS_CHANGED: Status of VFI CUSTOMER1 changed from DOWN to UP

PE1#
*Apr 14 14:01:44 ITALY: ldp: Rcvd label mapping msg from 10.2.2.2:0 (pp 0x7F90249F19A8)

Here the label mapping messages exchanged by PE1 and PE2:

I can see that PE1 assigns label 100014 to PW12 and PE2 assigns 200010

After enabling BD and VFI on last PE3, I’ll have a fully meshed set of 3 Pseudowires (12,13 and 23) that realize the Virtual Swithcing Emulation offered by VPLS service.

PE3(config)#int G2
PE3(config-if)#no shut

PE3(config)#bridge-domain 1000
PE3(config-bdomain)#no shut

PE3(config)#l2vpn vfi context CUSTOMER1
PE3(config-vfi)#no shut

Next figure summarize Labels, Pseudowires and BD,VFI,SI relationship:

Last step to verify if our emulated switch works is enabling Service Instance on all three PEs and check how MACs are learnt over the MPLS backbone.

I reconfigured MAC addresses of interfaces e0/0 on all three CEs for easy reading:

CE1-1#show int e0/0 | i Hard
Hardware is AmdP2, address is 0001.1111.1111 (bia aabb.cc00.0700)

CE1-2#sh int e0/0 | i Hard
Hardware is AmdP2, address is 0002.2222.2222 (bia aabb.cc00.0600)

CE1-3#show int e0/0 | i Hard
Hardware is AmdP2, address is 0003.3333.3333 (bia aabb.cc00.0500)

Activating the 3 SIs on each PEs and keeping some traffic flowing between CEs, I have:

PE1#sh bridge-domain 1000
Bridge-domain 1000 (3 ports in all)
State: UP                    Mac learning: Enabled
Aging-Timer: 300 second(s)
GigabitEthernet1 service instance 100
vfi CUSTOMER1 neighbor 10.2.2.2 12
vfi CUSTOMER1 neighbor 10.3.3.3 13
AED MAC address    Policy  Tag       Age  Pseudoport
0   0002.2222.2222 forward dynamic   299  CUSTOMER1.1004013
   0   0003.3333.3333 forward dynamic   298  CUSTOMER1.1004014
   0   0001.1111.1111 forward dynamic   300  GigabitEthernet1.EFP100

PE2#show bridge-domain 1000
Bridge-domain 1000 (3 ports in all)
State: UP                    Mac learning: Enabled
Aging-Timer: 300 second(s)
GigabitEthernet1 service instance 100
vfi CUSTOMER1 neighbor 10.3.3.3 23
vfi CUSTOMER1 neighbor 10.1.1.1 12
AED MAC address    Policy  Tag       Age  Pseudoport
0   0002.2222.2222 forward dynamic   298  GigabitEthernet1.EFP100
   0   0003.3333.3333 forward dynamic   228  CUSTOMER1.1004011
   0   0001.1111.1111 forward dynamic   298  CUSTOMER1.1004012

PE3#sh bridge-domain 1000
Bridge-domain 1000 (3 ports in all)
State: UP                    Mac learning: Enabled
Aging-Timer: 300 second(s)
GigabitEthernet1 service instance 100
vfi CUSTOMER1 neighbor 10.2.2.2 23
vfi CUSTOMER1 neighbor 10.1.1.1 13
AED MAC address    Policy  Tag       Age  Pseudoport
0   0002.2222.2222 forward dynamic   116  CUSTOMER1.1004011
   0   0003.3333.3333 forward dynamic   300  GigabitEthernet1.EFP100
   0   0001.1111.1111 forward dynamic   299  CUSTOMER1.1004012

CE1-1#sh ip arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  192.168.123.1           –   0001.1111.1111  ARPA   Ethernet0/0
Internet  192.168.123.2          10   0002.2222.2222  ARPA   Ethernet0/0
Internet  192.168.123.3          10   0003.3333.3333  ARPA   Ethernet0/0

CE1-2#sh ip arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  192.168.123.1          12   0001.1111.1111  ARPA   Ethernet0/0
Internet  192.168.123.2           –   0002.2222.2222  ARPA   Ethernet0/0
Internet  192.168.123.3          10   0003.3333.3333  ARPA   Ethernet0/0

CE1-3#sh ip arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  192.168.123.1          12   0001.1111.1111  ARPA   Ethernet0/0
Internet  192.168.123.2          10   0002.2222.2222  ARPA   Ethernet0/0
Internet  192.168.123.3           –   0003.3333.3333  ARPA   Ethernet0/0

I can see that 3 CEs work as if they were directly connected and that each PE learns the MACs of remote CEs via the MPLS core, under BD 1000 each MAC is learnt against a pseudoport that represents the pseudowire over which the PE sees the remote MAC.

In the following figure you can see two ping packets (one toward CE1-2 and one toward CE1-3) sent by CE1-1:

PE1#show mpls l2 binding 12
Destination Address: 10.2.2.2,VC ID: 12
    Local Label:  100000
Cbit: 1,    VC Type: Ethernet,    GroupID: n/a
MTU: 1500,   Interface Desc: n/a
VCCV: CC Type: RA [2], TTL [3]
CV Type: LSPV [2]
Remote Label: 200001
Cbit: 1,    VC Type: Ethernet,    GroupID: 0
MTU: 1500,   Interface Desc: n/a
VCCV: CC Type: RA [2], TTL [3]
CV Type: LSPV [2]

PE1#show mpls l2 binding 13
Destination Address: 10.3.3.3,VC ID: 13
    Local Label:  100001
Cbit: 1,    VC Type: Ethernet,    GroupID: n/a
MTU: 1500,   Interface Desc: n/a
VCCV: CC Type: RA [2], TTL [3]
CV Type: LSPV [2]
Remote Label: 300001
Cbit: 1,    VC Type: Ethernet,    GroupID: 0
MTU: 1500,   Interface Desc: n/a
VCCV: CC Type: RA [2], TTL [3]
CV Type: LSPV [2]

Looking at the picture it’s possible to see how the packets in the data plane are MPLS pseudowire encapsulated packets with the same structure of a Point-to-Point L2 service.

NOTE: You see different labels comparing them with the first pictures/shows because I restarted routers more than one time, grasp the concept.

Now that we have a basic idea of what is a VPLS service, let’s see which are the L2 Split-Horizon rules. The basic VPLS deployment model requires that PEs are fully meshed, since VPLS emulates common bridge domain over a MPLS core, we need to adopt some measure to avoid loop as in traditional L2 switching. We know that when there are multiple L2 paths closed in a ring, broadcast traffic can represent a problem (aka count to infinity), historically this problem is solved in a L2 traditional network using Spanning-Tree protocol. In case of VPLS a similar concept must be adopted to break the ring.

L2 split horizon rules says: (reference “L2 VPN architecture”)

“When a packet with an unknown destination mac address arrives on an attachment circuit, it is flooded to all other attachment circuits and all pseudowires that are bound to the virtual switch.”

“When layer 2 split horizon is enabled on a pseudowire, packets that arrive on this pseudowire are flooded to all attachment circuits, but not to a pseudowire. When Layer 2 split horizon is disabled, packets are flooded to all other pseudowires and all attachment circuits that are bound to the virtual switch.”

To see this behavior, let’s modify a bit our CEs network.

Now, interfaces of PEs looking at customer side are connected to layer 2 switches, to demonstrate the flexibility of the EVC framework I configured routers on each site in different VLANs, (100,101 on Site1, 102,103 on Site3 and 104,105 on Site2) but belonging to the same IP subnet 192.168.123.0/24. PEs will be configured in such a way that local communication within a site will be granted by the Service Instance of the PEs and communication between remote sites will be granted by the VPLS service; the following picture shows the relevant configurations on each device.

The commands “rewrite ingress tag pop 1 symmetric” under Service Instance of G1 on each PE, tells the router what action to implement when it receives frames under the specific SI, in this case I configured PEs to remove the first dot1q tag that could be present on the received frame and to use a symmetric option when it needs to send frame out the SI, symmetric means that it will use the same tag as configured under the encapsulation dot1q that is configured under the SI.

Now, let’s try to understand what happens when R1-Site1 wants to speak with R1-Site2.

STEP 1] At the beginning no device sent traffic, ARP tables are:

R1-Site1#sh ip arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  192.168.123.11          –   0001.1111.1111  ARPA   Ethernet0/0

R2-Site1#show ip arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  192.168.123.21          –   0001.2222.2222  ARPA   Ethernet0/1

R1-Site2#sh ip arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  192.168.123.12          –   0002.1111.1111  ARPA   Ethernet0/0

R2-Site2#show ip arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  192.168.123.22          –   0002.2222.2222  ARPA   Ethernet0/1

STEP 2] R1-Site1 (192.168.123.11) wants to send five pings to R1-Site2 (192.168.123.12)

R1-Site1 sees R1-Site2 as connected to its same subnet, then it broadcasts an ARP requests to learn the MAC of e0/0 interface of R1-Site2.

Broadcast frame reaches interface e0/0 on switch SW1-1, SW1-1 learns source MAC 0001.1111.1111 against VLAN100 on port e0/0 and forwards the broadcast frame on its VLAN 100.

The only one port that belongs to VLAN100, different from the port where it received the frame, is trunk port e1/0, being a trunk SW1-1 tag the frame with dot1q for VLAN100. See below this packet:

Frame 2832: 64 bytes on wire (512 bits), 64 bytes captured (512 bits) on interface 0
Ethernet II, Src: 00:01:11:11:11:11, Dst: ff:ff:ff:ff:ff:ff
802.1Q Virtual LAN, PRI: 0, CFI: 0, ID: 100
Address Resolution Protocol (request)
Hardware type: Ethernet (1)
Protocol type: IPv4 (0x0800)
Hardware size: 6
Protocol size: 4
Opcode: request (1)
Sender MAC address: 00:01:11:11:11:11
    Sender IP address: 192.168.123.11
    Target MAC address: 00:00:00:00:00:00
    Target IP address: 192.168.123.12

STEP 3.1] PE1 receives the tagged broadcast frame on its G1 interface, the frame being tagged with VLAN100 is matched by Service Instance 100, PE1 removes TAG100 and at this point it must decide where to send this broadcast frame. Based on L2 Split-Horizon Rule:

– it will sends the frame to all Attachment Circuits that are part of the same Bridge-Domain (BD 1000) to which belongs the Attachment Circuit (aka SI 100) where it received the frame

– it will forward the frame to all Pseudowires configured under the VFI that is part of the same Bridge-Domain 1000

From the configuraton above we see that under BD 1000 we have SI 101 and PW12 and PW13, PE1 then sends the frame in 3 different directions:

– Out of Service Instance 101 ==> since it is configured for rewriting in a symmetric way it takes the frame (the broadcast frame without any tag) and tags it with TAG101:

Frame 2833: 64 bytes on wire (512 bits), 64 bytes captured (512 bits) on interface 0
Ethernet II, Src: 00:01:11:11:11:11, Dst: ff:ff:ff:ff:ff:ff
802.1Q Virtual LAN, PRI: 0, CFI: 0, ID: 101
Address Resolution Protocol (request)
Hardware type: Ethernet (1)
Protocol type: IPv4 (0x0800)
Hardware size: 6
Protocol size: 4
Opcode: request (1)
Sender MAC address: 00:01:11:11:11:11
    Sender IP address: 192.168.123.11
    Target MAC address: 00:00:00:00:00:00
    Target IP address: 192.168.123.12

This frame reaches e1/0 interface on SW1-1. SW1-1 will learn source MAC 0001.1111.1111 against VLAN101 on port e1/0, SW1-1 will forward the frame on all its ports that belongs to VLAN101 (except the one where it received the frame (e1/0)) then frame exits port e0/1 and then it will be dropped by R2-Site1.

– Out of PW12 using MPLS Label 200001 as inner label of the encapsulated packet.

PE1#show mpls l2 binding 12 | i Local|Remote
Local Label:  100000
Remote Label: 200001

Frame 4617: 86 bytes on wire (688 bits), 86 bytes captured (688 bits) on interface 0
Ethernet II, Src: 00:fa:e8:d5:92:01, Dst: aa:bb:cc:00:03:10
MultiProtocol Label Switching Header, Label: 1008, Exp: 0, S: 0, TTL: 255
MultiProtocol Label Switching Header, Label: 200001, Exp: 0, S: 1, TTL: 255
PW Ethernet Control Word
Ethernet II, Src: 00:01:11:11:11:11, Dst: ff:ff:ff:ff:ff:ff
Address Resolution Protocol (request)
Hardware type: Ethernet (1)
Protocol type: IPv4 (0x0800)
Hardware size: 6
Protocol size: 4
Opcode: request (1)
  Sender MAC address: 00:01:11:11:11:11
    Sender IP address: 192.168.123.11
    Target MAC address: 00:00:00:00:00:00
    Target IP address: 192.168.123.12

 – Out of PW13 using MPLS Label 300012 as inner label of the encapsulated packet.

PE1#show mpls l2 binding 13 | i Local|Remote
Local Label:  100001
Remote Label: 300012

STEP 3.2] the MPLS encapsulated broadcast frame reaches PE2 and PE3, based on L2 Split-Horizon Rules, PE2 and PE3 will forward the frame in this way:

– They will send the frame on all Attachment Circuits configured under the same BD1000 where is configured the VFI over which they received the frame.

– They will not send the frame over any other configured PWs ==> this is the rule that prevents Forwarding Loops over the MPLS Backbone.

The ARP broadcast will then exit both service instances under interface G1 on PE2 and PE3, the frame will be tagged based on the symmetric option and SW1-2 and SW1-3 will learn two times the frame under different VLANs (104,105 on SW1-2 and 102,103 on SW1-3)

STEP 4] the frame reaches its intended destination, R1-Site2 will then reply with its MAC and a similar process starts in the oppposite directions.

Following picture should give an idea of the trip that ARP Request generated by R1-Site1 about MAC of R1-Site2 will do on the network:

It’s interesting to see how tables are populated when continous traffic is sent between all routers of all sites (I configured sla probes on each routers pinging the other 5)

We can see how the Service Instances configured on PEs decouples the Broadcast Domain concept from the VLANs, broadcast frames are sent to all the circuits that belongs to the same Bridge-Domain, L2 SPLIT-HORIZON prevents L2 loops, blocking frames received on a PW from going over other PWs. We can see also that, based on the configured service instances and PWs that are present under the VFI that connects to the same bridge-domain, every bridge domain in each PEs has a number of Pseudo-port and Ehternet Flow Points, in this example every PE has 2 Pseudoports that correspond to the configured PWs and 2 Ethernet Flow Points created by the configured service instances, the brdige-domain is then the unifying element that lets L2 frames to be switched between sites.

With the specific configuration applied in this example, we can identify two types of L2 switching:

– Local Switching within the same PE

– Remote Switching between remote PEs

Local Switching happens when frames are contained on the same PEs and switched between its service instance, as an example here I put remote CEs routers, that need to be IP connected,on different access VLANs of a switch that trunks with the PE, this is only an example, with this configuration routers installed on the same site cannot talk directly bypassing the PE because they are in different VLANs but they can talk to each other using the PE as sort of L2 gateway. This is not a specific configuration of VPLS, I reported here this example to show the flexibility of the Service Instance concepts.

Remote Switching happens when frames cross the MPLS core, in this case PEs uses their configured bridge-domain to forward frames between their Virtual Pseudo Ports and Ethernet flow points.

The VPLS service is, raw speaking, offered by the Virtual Switching Systems in which the PEs can be subdivided for every bridge-domain, to recap the elements are:

– Service Instances <==> Ethernet Flow POINTs
– VFI where PWs <==> Pseudoports are attached
– Bridge-Domain that glues Ethernet Flow Points and Pseudoports
– L2 SPLIT-HORIZON rules that prevents frames to be L2 looped between PEs.

Now if you want, you can proceed to the second part of this post where I did some tests about L2protocol forwarding over VPLS.