Here I want to review VRF Lite feature. I will lab it configuring this network topology:
The VRF-Lite feature extends VRF capability to a CE router, this means that at CE we can separate routing informations among different customers. The typical scenario is that one single customer has a bigger site where many routers are connected to the same CE and this CE is serving different divisions that don’t need to share routing information. On CE we have different VRFs one per router/division. The term LITE refers to the fact that we don’t need any other feature we usually find on a PE like LDP, Labels, MP-BGP and so on. Only thing we configure on CE are separated routing domain (VRFs). For every VRF configured on the MULTI-VRF CE we must have a VRF configured on the PE. The connections between PE and Multi-VRF-CE can be multi-physical interfaces or we can have only one physical interfaces connecting PE and Multi-VRF-CE, in these latter case we must define a subinterface for every defined VRF on the CE. Here I will test this case.
Scenario is very basic, think at one Customer named EuroGame connected to our MPLS/VPN core, smaller sites serve one different division (Sales or Operations or Finance) then we have a bigger site connected to PE4 where we find all the three divisions, customer asked us to keep separated traffic of each division into the network. For remote sites things are simple, it’s enough configuring the VRF as three different VRFs with different export/import rules. At site 4 instead, we have only one CE and only one link connected the CE and PE4. The best and probably most expensive scenario would be having one CE for every division like:
Other possible solutions are configuring the one available CE as a new PE, but this involve managing it by the ISP, configuring LDP, MP-BGP and other stuff, or connecting directly and logically the 3 end routers (Finance, Sales, operations) to the PE with some sort of Tunneling technique. All this solutions are not so efficient. Let’s look at how VRF-Lite help us.
Main elements of a VRF-Lite solutions are:
– a Multi-VRF-CE: is a router VRF capable that we can consider a logical extension of the PEs in terms of separation in the routing domains. At one side it is connected to the PE, and at opposite side it looks at two or more routers that must keep their traffic isolated.
– some VRFs: the separation in routing domains is realized by the CE in the same way as a PE does: using a VRF for each traffic source to isolate.
– special VRFs: VRFs defined on CE don’t need Route-Target, because MULTI-VRF-CE is not a PE, so we don’t need to worry about importing/exporting from MP-BGP, redistribution in BGP, MPLS LDP Labelling; the VRF are used only to create the Virtual Routing Table –> different Routing Table –> Traffic not shared.
– Links assigned to VRFs: a Multi-VRF-CE will have both upstream and downstream links belonging to the VRF, while a PE normally have VRF assigned only to links looking at CE.
– PE: the PE, which Multi-VRF-CE is connected to, is completely unaware that its CE is a Multi-VRF-CE. So in MPLS/core nothing change from a configuration point of view.
Here I used a Multilayer Switch working as Multi-VRF-CE:
Let’s look how things are configured before powering-on the whole MPLS/VPN core. The three divisions’ routers have nothing special, these are end-user devices totally unaware of any vrf, here their relevant configurations:
I used OSPF as the routing protocol between End-Routers <-> Multi-VRF-CE <-> PE4. Now let’s check Multi-VRF-CE configuration:
I created 6 Point-to-Point (/30) Vlans and each pair of Vlans [4(Up),104(Dwn)]-[5(Up),105(Dwn)]-[6(Up),106(Dwn)] belong to one of the three VRFs EG-Sales, EG-Operations, EG-Finance. Each one of these VRF has one link downstream (to End-Router) and one in upstream (to PE). The physical link to PE is only one (interface e0/3) and I configured this link as dot1q trunk carrying the three upstream Point-to-Point Vlans (4,5,6) to PE:
EG-Multi-VRF-CE#show run int e0/3 | b interface
interface Ethernet0/3
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1,4-6
switchport mode trunk
I can verifiy the ospf neighborship between Multi-VRF-CE/End-Routers and Multi-VRF-CE/PE4:
EG-Multi-VRF-CE#show ip ospf neighbor
Neighbor ID Pri State Dead Time Address Interface
101.0.106.254 1 FULL/DR 00:00:35 10.40.101.9 Vlan6
101.106.106.106 1 FULL/DR 00:00:32 192.168.0.10 Vlan106
101.0.105.254 1 FULL/DR 00:00:35 10.40.101.5 Vlan5
101.105.105.105 1 FULL/DR 00:00:35 192.168.0.6 Vlan105
101.0.104.254 1 FULL/DR 00:00:39 10.40.101.1 Vlan4
101.104.104.104 1 FULL/DR 00:00:32 192.168.0.2 Vlan104
Inside each one of the three VRFs Multi-VRF-CE is learning about the right networks:
EG-Multi-VRF-CE#show ip route vrf EG-Sales ospf | b Gate
Gateway of last resort is not set
O 192.168.104.0/24 [110/11] via 192.168.0.2, 01:15:19, Vlan104
EG-Multi-VRF-CE#show ip route vrf EG-Operations ospf | b Gate
Gateway of last resort is not set
O 192.168.105.0/24 [110/11] via 192.168.0.6, 01:15:31, Vlan105
EG-Multi-VRF-CE#show ip route vrf EG-Finance ospf | b Gate
Gateway of last resort is not set
O 192.168.106.0/24 [110/11] via 192.168.0.10, 01:15:44, Vlan106
Routing info are separated because belonging to three different VRFs. Inter-Vlan routing is not possible because each Vlan 104,105,106 is part of a different VRF, connectivity inside the VRF instead is ok:
EG-Multi-VRF-CE#ping vrf EG-Sales 192.168.104.104 source 10.40.101.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.104.104, timeout is 2 seconds:
Packet sent with a source address of 10.40.101.2
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 7/7/9 ms
EG-Multi-VRF-CE#ping vrf EG-Operations 192.168.105.105 source 10.40.101.6
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.105.105, timeout is 2 seconds:
Packet sent with a source address of 10.40.101.6
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 5/8/11 ms
EG-Multi-VRF-CE#ping vrf EG-Finance 192.168.106.106 source 10.40.101.10
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.106.106, timeout is 2 seconds:
Packet sent with a source address of 10.40.101.10
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 6/8/12 ms
Now, let’s check PE4 configurations:
We can see that the PE is the router still defining route-target import/export rules. Also to notice is that PE and Multi-VRF-CE are talking to each other on links configured with different VRF names (and different configurations inside these VRFs). For example Upstream link int Vlan4 on Multi-VRF-CE is configured for VRF EG-Sales
EG-Multi-VRF-CE#show run int Vlan4 | b interface
interface Vlan4
ip vrf forwarding EG-Sales
ip address 10.40.101.2 255.255.255.252
While downstream link int e0/3.104 on PE4 is configured for VRF EuroGame-Sales:
PE4#show run int e0/3.104 | b interface
interface Ethernet0/3.104
encapsulation dot1Q 4
ip vrf forwarding EuroGame-Sales
ip address 10.40.101.1 255.255.255.252
Of course here I can choose to use the same name, but important thing to understand is that would be only a name, when PE4 sends traffic to Multi-VRF-CE on its link e0/3.104 is unaware that on the other side VRF are configured, most important the same is true when it receives traffic on this link, for PE4 traffic received on this link will belong to ITS VRF EuroGame-Sales, it knows nothing about VRF EG-Sales configured on EG-Multi-VRF-CE.
The same concept applies to EG-Multi-VRF-CE when it sends/receives traffic to/from PE4. When EG-Multi-VRF-CE sends traffic on int Vlan4 it knows nothing about VRFs configured on PE4, it’s routing traffic on this link because routing inside ITS VRF EG-Sales says to send traffic on that next-hop. When it receives traffic from PE4 it consider this traffic as pure IP traffic (as indeed it is) and since is entering its int Vlan4 it considers it as belonging to ITS VRF EG-Sales,EG-Multi-VRF-CE doesn’t know that the traffic is coming from a link of PE4 belonging to some VRF of PE4.
VRF is a local concept, here we have twice the same thing that happen normally between a PE anc CE, CE knows nothing about VRFs of PE, PE knows nothing about VRFs of a Multi-VRF-CE.
NOTE: Of course, upstream link and downstream link on Multi-VRF-CE must belong to the same table/VRF, otherwise router will not be able to route traffic downstream-to-upstream or upstream-to-downstream
Now I’m going to complete BGP configuration on PE4 so routes learned from EG-Multi-VRF-CE will be redistributed into MP-BGP by PE4:
PE4#show run | s r b
router bgp 1000
bgp log-neighbor-changes
no bgp default ipv4-unicast
neighbor Other-PE peer-group
neighbor Other-PE remote-as 1000
neighbor Other-PE update-source Loopback40
neighbor 10.10.10.10 peer-group Other-PE
neighbor 20.20.20.20 peer-group Other-PE
neighbor 30.30.30.30 peer-group Other-PE
!
address-family ipv4
bgp scan-time 5 –> don’t use on production router
exit-address-family
!
address-family vpnv4
bgp scan-time 5 –> don’t use on production router
neighbor Other-PE send-community both
neighbor 10.10.10.10 activate
neighbor 20.20.20.20 activate
neighbor 30.30.30.30 activate
exit-address-family
!
address-family ipv4 vrf EuroGame-Finance
redistribute ospf 106 metric 1000 match internal external 1 external 2
exit-address-family
!
address-family ipv4 vrf EuroGame-Operations
redistribute ospf 105 metric 1000 match internal external 1 external 2
exit-address-family
!
address-family ipv4 vrf EuroGame-Sales
redistribute ospf 104 metric 1000 match internal external 1 external 2
exit-address-family
PE4 is correctly injecting the three remote LANs 192.168.(104,105,106).0/24
PE4#show ip bgp vpnv4 all 192.168.104.0/24
BGP routing table entry for 101:104:192.168.104.0/24, version 4
Paths: (1 available, best #1, table EuroGame-Sales)
Not advertised to any peer
Refresh Epoch 1
Local
10.40.101.2 from 0.0.0.0 (101.0.4.254)
Origin incomplete, metric 1000, localpref 100, weight 32768, valid, sourced, best
Extended Community: RT:101:104 OSPF DOMAIN ID:0x0005:0x000000680200
OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:101.0.104.254:0
mpls labels in/out 4002/nolabel
rx pathid: 0, tx pathid: 0x0
PE4#show ip bgp vpnv4 all 192.168.105.0/24
BGP routing table entry for 101:105:192.168.105.0/24, version 7
Paths: (1 available, best #1, table EuroGame-Operations)
Not advertised to any peer
Refresh Epoch 1
Local
10.40.101.6 from 0.0.0.0 (101.0.4.254)
Origin incomplete, metric 1000, localpref 100, weight 32768, valid, sourced, best
Extended Community: RT:101:105 OSPF DOMAIN ID:0x0005:0x000000690200
OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:101.0.105.254:0
mpls labels in/out 4005/nolabel
rx pathid: 0, tx pathid: 0x0
PE4#show ip bgp vpnv4 all 192.168.106.0/24
BGP routing table entry for 101:106:192.168.106.0/24, version 10
Paths: (1 available, best #1, table EuroGame-Finance)
Not advertised to any peer
Refresh Epoch 1
Local
10.40.101.10 from 0.0.0.0 (101.0.4.254)
Origin incomplete, metric 1000, localpref 100, weight 32768, valid, sourced, best
Extended Community: RT:101:106 OSPF DOMAIN ID:0x0005:0x0000006A0200
OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:101.0.106.254:0
mpls labels in/out 4008/nolabel
rx pathid: 0, tx pathid: 0x0
Now, let’s redistribute BGP learned paths back into OSPF process of every VRFs:
PE4(config)#router ospf 104 vrf EuroGame-Sales
PE4(config-router)#redistribute bgp 1000 subnets metric 1000
PE4(config)#router ospf 105 vrf EuroGame-Operations
PE4(config-router)#redistribute bgp 1000 subnets metric 1000
PE4(config)#router ospf 106 vrf EuroGame-Finance
PE4(config-router)#redistribute bgp 1000 subnets metric 1000
Here the bgp table of PE4 after mutual redistribution between BGP and OSPF is configured:
In bgp table of every VRF I see LAN network from remote site and LAN network from local site and network of links connecting PE4 to EG-Multi-VRF-CE and those one connecting EG-Multi-VRF-CE to the End-Routers
– 192.168.101.0/24,192.168.106.0/24 + 192.168.0.8/30,10.40.101.8/30,10.10.101.0/24 –> vrf EuroGame-Finance
– 192.168.103.0/24,192.168.105.0/24 + 192.168.0.4/30,10.40.101.4/30,10.30.101.0/24 –> vrf EuroGame-Operations
– 192.168.102.0/24,192.168.104.0/24 + 192.168.0.0/30,10.40.101.0/30,10.20.101.0/24 –> vrf EuroGame-Sales
From ouptut above we can see that from PE4 point of view the presence of different VRFs on EG-Multi-VRF-CE is transparent and it would be like if each VRF on PE4 extended itself to the End-Router, this is what a VRF-Lite really does.
Last step it’s to verify if remote LAN routing info are reaching the End-Routers:
EG-Sales#show ip route ospf | b Gate
Gateway of last resort is not set
10.0.0.0/30 is subnetted, 1 subnets
O IA 10.40.101.0 [110/11] via 192.168.0.1, 00:24:42, Ethernet0/0
EG-operations#show ip route ospf | b Gate
Gateway of last resort is not set
10.0.0.0/30 is subnetted, 1 subnets
O IA 10.40.101.4 [110/11] via 192.168.0.5, 00:25:15, Ethernet1/2
EG-Finance#show ip route ospf | b Gate
Gateway of last resort is not set
10.0.0.0/30 is subnetted, 1 subnets
O IA 10.40.101.8 [110/11] via 192.168.0.9, 00:25:40, Ethernet1/3
I’m missing network info about remote LAN, so something is going wrong:
EG-Sales#show ip route 192.168.102.0
% Network not in table
EG-operations#show ip route 192.168.103.0
% Network not in table
EG-Finance#show ip route 192.168.101.0
% Network not in table
I have just verified that these networks are received into OSPF processes by EG-Multi-VRF-CE, let’s check the database:
So EG-Multi-VRF-CE is getting the right LSA in Area 0 from PE4 but is not forwarding these LSA down to the End-Routers in Area 4:
EG-Multi-VRF-CE#show ip ospf 104 database | b Router Link States \(Area 4\)
Router Link States (Area 4)
Link ID ADV Router Age Seq# Checksum Link count
101.0.104.104 101.0.104.104 313 0x80000004 0x0031BB 1
101.104.104.104 101.104.104.104 294 0x80000003 0x009196 2
Net Link States (Area 4)
Link ID ADV Router Age Seq# Checksum
192.168.0.2 101.104.104.104 294 0x80000002 0x00CB91
Summary Net Link States (Area 4)
Link ID ADV Router Age Seq# Checksum
10.40.101.0 101.0.104.104 313 0x80000002 0x00E18E
Summary ASB Link States (Area 4)
Link ID ADV Router Age Seq# Checksum
101.0.104.254 101.0.104.104 313 0x80000002 0x000D2A
EG-Multi-VRF-CE#show ip ospf 105 database | b Router Link States \(Area 4\)
Router Link States (Area 4)
Link ID ADV Router Age Seq# Checksum Link count
101.0.105.105 101.0.105.105 325 0x80000003 0x007170 1
101.105.105.105 101.105.105.105 298 0x80000003 0x0025F3 2
Net Link States (Area 4)
Link ID ADV Router Age Seq# Checksum
192.168.0.6 101.105.105.105 298 0x80000002 0x00C48C
Summary Net Link States (Area 4)
Link ID ADV Router Age Seq# Checksum
10.40.101.4 101.0.105.105 325 0x80000002 0x00ACBD
Summary ASB Link States (Area 4)
Link ID ADV Router Age Seq# Checksum
101.0.105.254 101.0.105.105 325 0x80000002 0x00F43F
EG-Multi-VRF-CE#show ip ospf 106 database | b Router Link States \(Area 4\)
Router Link States (Area 4)
Link ID ADV Router Age Seq# Checksum Link count
101.0.106.106 101.0.106.106 390 0x80000004 0x00AD27 1
101.106.106.106 101.106.106.106 294 0x80000003 0x00B851 2
Net Link States (Area 4)
Link ID ADV Router Age Seq# Checksum
192.168.0.10 101.106.106.106 294 0x80000002 0x00BD87
Summary Net Link States (Area 4)
Link ID ADV Router Age Seq# Checksum
10.40.101.8 101.0.106.106 390 0x80000002 0x0077EC
Summary ASB Link States (Area 4)
Link ID ADV Router Age Seq# Checksum
101.0.106.254 101.0.106.106 390 0x80000002 0x00DC54
In none of the three database I see external summary info about remote LAN 192.168(101,102,103).0/24
The problem here is that EG-Multi-VRF-EG is working as PE, ok not a true PE but it’s managing its VRF, right? So EG-Multi-VRF-CE is preventing from advertising downward the info about remote LAN because it is looking at the DOWN BIT SET in summary LSA received by PE4.
PE4 set the DOWN bit when redistributing BGP learned paths into OSPF, a PE to prevent loops checks if the received info has the DOWN BIT SET in this case it discards the LSA. Same thing does the Multi-VRF-CE because is working with VRFs, so it checks for this DOWN bit too.
Let’s try to disable this loop prevention mechanism on EG-Multi-VRF-CE, the command is:
EG-Multi-VRF-CE(config)#router ospf 104 vrf EG-Sales
EG-Multi-VRF-CE(config-router)#capability ?
……………..
vrf-lite Do not perform PE specific checks
EG-Multi-VRF-CE(config-router)#capability vrf-lite ?
<cr>
EG-Multi-VRF-CE(config-router)#capability vrf-lite
EG-Multi-VRF-CE(config-router)#
*Aug 7 15:40:17.960: %OSPF-5-ADJCHG: Process 104, Nbr 101.0.104.254 on Vlan4 from FULL to DOWN, Neighbor Down: Interface down or detached
*Aug 7 15:40:18.007: %OSPF-5-ADJCHG: Process 104, Nbr 101.104.104.104 on Vlan104 from FULL to DOWN, Neighbor Down: Interface down or detached
*Aug 7 15:40:18.049: %OSPF-5-ADJCHG: Process 104, Nbr 101.0.104.254 on Vlan4 from LOADING to FULL, Loading Done
*Aug 7 15:40:18.049: %OSPF-5-ADJCHG: Process 104, Nbr 101.104.104.104 on Vlan104 from LOADING to FULL, Loading Done
EG-Multi-VRF-CE(config-router)#
When I tell router to disable specific PE-like checking the neighborship is reset then:
EG-Sales#show ip route 192.168.102.0
Routing entry for 192.168.102.0/24
Known via “ospf 104”, distance 110, metric 1011, type inter area
Last update from 192.168.0.1 on Ethernet0/0, 00:01:42 ago
Routing Descriptor Blocks:
* 192.168.0.1, from 101.0.104.104, 00:01:42 ago, via Ethernet0/0
Route metric is 1011, traffic share count is 1
Now I see the remote LAN, as a further check I can see summary entry in Area4 of process 104
EG-Multi-VRF-CE#show ip ospf 104 database | b Router Link States \(Area 4\)
Router Link States (Area 4)
Link ID ADV Router Age Seq# Checksum Link count
101.0.104.104 101.0.104.104 186 0x80000005 0x002FBC 1
101.104.104.104 101.104.104.104 1169 0x80000003 0x009196 2
Net Link States (Area 4)
Link ID ADV Router Age Seq# Checksum
192.168.0.2 101.104.104.104 1169 0x80000002 0x00CB91
Summary Net Link States (Area 4)
Link ID ADV Router Age Seq# Checksum
10.20.101.0 101.0.104.104 181 0x80000001 0x001B7B
10.40.101.0 101.0.104.104 181 0x80000001 0x00E38D
192.168.102.0 101.0.104.104 181 0x80000001 0x00D178
Summary ASB Link States (Area 4)
Link ID ADV Router Age Seq# Checksum
101.0.104.254 101.0.104.104 181 0x80000001 0x000F29
Of course, capability vrf-lite must be configured in every OSPF/VRF process:
EG-Multi-VRF-CE#show run | s r o
router ospf 104 vrf EG-Sales
router-id 101.0.104.104
capability vrf-lite
network 10.40.101.0 0.0.0.3 area 0
network 192.168.0.0 0.0.0.3 area 4
router ospf 105 vrf EG-Operations
router-id 101.0.105.105
capability vrf-lite
network 10.40.101.4 0.0.0.3 area 0
network 192.168.0.4 0.0.0.3 area 4
router ospf 106 vrf EG-Finance
router-id 101.0.106.106
capability vrf-lite
network 10.40.101.8 0.0.0.3 area 0
network 192.168.0.8 0.0.0.3 area 4
Here some connectivity checks inside each VRFs:
And here the complete configurations of all routers.