Thursday, February 2, 2012

MPLS TROUBLESHOOTING , IMPORT MAP, VPN LABEL

MPLS LAYER 3 VPN VERIFICATION & TROUBLESHOOTING
------------------------------------------------
sh bgp vpnv4 unicast vrf A 10.1.10.10
If we do this command  on the PE that is recieving the route from the CE you will see next
hop + label defined as the incoming Labels.
When this is then advertised to other PE this will be set as there outgoing label.
The label is the vpnv4 label not the transport label so an mpls packet would be like this


MPLS TRANSPORT LABEL----MPLS VPN LABEL---- Payload

The mpls transport label is the label that will be swapped through the core of the network
to get from one PE to another. The vpn label will be used by the recieving PE to identify
what customer or what vpn the payload is for.
You can view the labels with the command debug mpls packet

Another topic there is no limit on route-targe import and export for under vrf for example
ip vrf a
route-target 11:11 import
route-target 12:12 import
route-target 13:13 import
route-target 5:5 export

We also have the option when exporting or importing to use an export or import map
this gives us more granular control over which network get advertised to which vpnv4
peers

e.g
export map Test
route-map TEST
match ip address 1
set ext community route-target 11:11
access-list 1 permit 10.0.0.0 0.0.0.255

same id for import map

You can also filter what mpls labels you want to advertise
mpls ldp advertise-labels for "access-list 1"
acess-list 1 permit 10.0.0.0  
so it would only advertise for the 10.0.0.0 network
or we could do the opposite do the no form of the command and it would advertise for
everything but 10.0.0.0

When you are troubleshooting the core network it can be difficult to spot issue with
the mpls config whether mpls ip is enabled on interface. As the igp will have reachability
etc. Routes may be on both CEs as they are advertising via IBGP so the core mpls network
only comes into play when we are switching traffic accross it. The main way is to follow
the path of the traffic confirming. A quick way of confirming if you do
sh mpls forwarding-table - it should never say untagged for outgoing mpls interface.

Another example potential issue in MPLS core network. Is that certain ios version have
difficult dealing difference in subnetwork size. 
So a scenerion you are running ospf in the core mpls network and you advertising a loopback
your loopback is /25 but in ospf as it is the default type of loopback it is advertised as a
/32
There is then difference in the routes in the originating routers cef/lfib table and the
routes in the routers that was recieved via ospf. So the original router generates a label
/32
advertise it to  the others with /25 they do not have this router in there igp so they reject
they have in there cef/lfib /32 route they generate labels for this this can cause problems
with the mpls switching path for that route.
This is also the case when we summarise route this can cause problems in the mpls switching
path

The last thing that i looked is what makes up layer 3 vpn config

Prerequisite is you have your IGP and LDP MPLS configured correctly in the core so this is
checked first after we can look at troublshooting in this order

1)VRF - is RD configured, is  RT configured, Is correct import / export
2)VRF- is the vrf correctly asssigned to the right interface
3)Is the vrf routing proces correct??? this routing with CE like for BGP is neighbor
statement
  under the address-family ipv4 "vrf" for eigrp have specfied the autonomous system under the
  address-family ipv4 vrf etc
4)Are the vpnv4 peerings up. Have formed a specfic bgp relationships. Under the
address-family vpnv4
  have activate the neigbors and added send and recieve communities.
5) Have we redistributed correctly addding metric for relevant protocols redistributed under
the address
   family in the relevant protocols and the address family in BGP

LAB
----- 
OK we will have a quick look at the switching path mpls vpn



R6
---
r6#sh bgp vpnv4 unicast vrf B 11.1.1.1
BGP routing table entry for 111:11:11.1.1.1/32, version 5
Paths: (1 available, best #1, table B)
  Advertised to update-groups:
     1
  Local
    10.164.49.1 from 0.0.0.0 (6.6.6.6)
      Origin incomplete, metric 2297856, localpref 100, weight 32768, valid, sourced, best
      Extended Community: RT:111:11
        Cost:pre-bestpath:128:2297856 (default-2145185791) 0x8800:32768:0
        0x8801:10:640000 0x8802:65281:1657856 0x8803:65281:1500
      mpls labels in/out 27/nolabel
so r6 is sending the other PE's the route 11.1.1.1 with a vpn label of 27 so the in label

r4
---

r4#sh bgp vpnv4 unicast vrf B 11.1.1.1
BGP routing table entry for 111:11:11.1.1.1/32, version 27
Paths: (1 available, best #1, table B)
Flag: 0x820
  Not advertised to any peer
  Local
    6.6.6.6 (metric 4) from 6.6.6.6 (6.6.6.6)
      Origin incomplete, metric 2297856, localpref 100, valid, internal, best
      Extended Community: RT:111:11
        Cost:pre-bestpath:128:2297856 (default-2145185791) 0x8800:32768:0
        0x8801:10:640000 0x8802:65281:1657856 0x8803:65281:1500
      mpls labels in/out nolabel/27
r4#

so the outgoing label vpn label is 27 with a next hop of 6.6.6.6
so r4 will check for 6.6.6.6 to put it transport label on

r4#sh mpls ldp bindings
  tib entry: 1.1.1.1/32, rev 10
        local binding:  tag: 18
        remote binding: tsr: 2.2.2.2:0, tag: 17
  tib entry: 2.2.2.2/32, rev 12
        local binding:  tag: 19
        remote binding: tsr: 2.2.2.2:0, tag: imp-null
  tib entry: 3.3.3.3/32, rev 22
        local binding:  tag: 27
        remote binding: tsr: 2.2.2.2:0, tag: 22
  tib entry: 4.4.4.4/32, rev 4
        local binding:  tag: imp-null
        remote binding: tsr: 2.2.2.2:0, tag: 18
  tib entry: 5.5.5.5/32, rev 24
        local binding:  tag: 28
        remote binding: tsr: 2.2.2.2:0, tag: 23
  tib entry: 6.6.6.6/32, rev 14
        local binding:  tag: 20
        remote binding: tsr: 2.2.2.2:0, tag: 19
  tib entry: 155.0.2.0/24, rev 8
        local binding:  tag: 17
        remote binding: tsr: 2.2.2.2:0, tag: 16
  tib entry: 155.0.3.0/24, rev 6
        local binding:  tag: 16
        remote binding: tsr: 2.2.2.2:0, tag: imp-null
  tib entry: 155.0.4.0/24, rev 16
        local binding:  tag: 21
        remote binding: tsr: 2.2.2.2:0, tag: imp-null
  tib entry: 155.0.5.0/24, rev 18
        local binding:  tag: 22
        remote binding: tsr: 2.2.2.2:0, tag: 20
  tib entry: 155.0.6.0/24, rev 20
        local binding:  tag: 26
        remote binding: tsr: 2.2.2.2:0, tag: 21
  tib entry: 155.0.7.0/24, rev 2
        local binding:  tag: imp-null
        remote binding: tsr: 2.2.2.2:0, tag: imp-null

so r4 will put the vpn label of 27 and then it will put on the transport label of 19

r4#sh ip route 6.6.6.6
Routing entry for 6.6.6.6/32
  Known via "ospf 200", distance 110, metric 4, type intra area
  Last update from 155.0.7.2 on FastEthernet0/0.24, 00:06:11 ago
  Routing Descriptor Blocks:
  * 155.0.7.2, from 6.6.6.6, 00:06:11 ago, via FastEthernet0/0.24
      Route metric is 4, traffic share count is 1
r4#

r4 will then send it out fa0/0.24 to r2

r2
--
so r2 recieves the packet checks the label of 19

r2#sh mpls forwa
Local  Outgoing    Prefix            Bytes tag  Outgoing   Next Hop
tag    tag or VC   or Tunnel Id      switched   interface
16     Pop tag     155.0.2.0/24      0          Fa0/0.12   155.0.3.1
17     Pop tag     1.1.1.1/32        0          Fa0/0.12   155.0.3.1
18     Pop tag     4.4.4.4/32        21042      Fa0/0.24   155.0.7.4
19     19          6.6.6.6/32        1595       Fa0/0.12   155.0.3.1
20     Pop tag     155.0.5.0/24      0          Fa0/0.12   155.0.3.1
       Pop tag     155.0.5.0/24      0          Fa0/0.23   155.0.4.3
21     Pop tag     155.0.6.0/24      0          Fa0/0.23   155.0.4.3
22     Pop tag     3.3.3.3/32        0          Fa0/0.23   155.0.4.3
23     16          5.5.5.5/32        13783      Fa0/0.23   155.0.4.3
r2#

so an incoming label of 19 will get an outgoing label of 19 and will be sent out fa0/0.12
to R1 it does not go near the vpn label of 27 it just swaps the transport label

R1
---
r1>en
*Mar  1 01:31:11.743: %OSPF-5-ADJCHG: Process 200, Nbr 3.3.3.3 on FastEthernet0/0.13 from
LOADING to FULL, Loading Done
Password:
r1#sh mpls forward
Local  Outgoing    Prefix            Bytes tag  Outgoing   Next Hop
tag    tag or VC   or Tunnel Id      switched   interface
16     Pop tag     155.0.7.0/24      0          Fa0/0.12   155.0.3.2
17     Pop tag     2.2.2.2/32        0          Fa0/0.12   155.0.3.2
18     18          4.4.4.4/32        13748      Fa0/0.12   155.0.3.2
19     Pop tag     6.6.6.6/32        2602       Fa0/0.16   155.0.2.6
20     Pop tag     155.0.4.0/24      0          Fa0/0.12   155.0.3.2
r1#

so comes in with a label 19 and php (pops) the label and sends it out r6

SO r6 has the packet coming with vpn label of 27 this is the vpn label which identfies that
is part of VRF B

Another thing we can import routes like from rip into eigrp
OK so on r6
r6
---
we will import the rip routes to vrf B
r6(config)#ip vrf B
r6(config-vrf)#route-target import 110:10
r6(config-vrf)#

r6#sh run | begin ip vrf B
ip vrf B
 rd 111:11
 route-target export 111:11
 route-target import 111:11
 route-target import 110:10

 r6#sh ip route vrf B
Routing Table: B
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route
Gateway of last resort is not set
     2.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
B       2.3.3.0/24 [200/156160] via 4.4.4.4, 00:18:50
B       2.4.4.4/32 [200/1] via 4.4.4.4, 00:01:00
     10.0.0.0/24 is subnetted, 4 subnets
B       10.229.254.0 [200/0] via 4.4.4.4, 00:18:50
B       10.164.50.0 [200/0] via 5.5.5.5, 00:01:00
B       10.164.48.0 [200/0] via 4.4.4.4, 00:01:00
C       10.164.49.0 is directly connected, Serial0/0/0
     11.0.0.0/32 is subnetted, 2 subnets
B       11.3.3.3 [200/1] via 5.5.5.5, 00:01:00
D       11.1.1.1 [90/2297856] via 10.164.49.1, 00:28:11, Serial0/0/0
r6#

so we are getting the rip routes also like 2.4.4.4 for example

let say we want to import every apart from 2.4.4.4 and we only have access to r6
r6(config)#access-list 1 deny 2.4.4.4 0.0.0.0
r6(config)#access-list 1 permit any

r6(config)#route-map IMPORT
r6(config-route-map)#match ip address 1
r6(config-route-map)#exit
r6(config)#ip vrf B
r6(config-vrf)#import map IMPORT

r6#sh ip route vrf B
Routing Table: B
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route
Gateway of last resort is not set
     2.0.0.0/24 is subnetted, 1 subnets
B       2.3.3.0 [200/156160] via 4.4.4.4, 00:00:59
     10.0.0.0/24 is subnetted, 4 subnets
B       10.229.254.0 [200/0] via 4.4.4.4, 00:00:59
B       10.164.50.0 [200/0] via 5.5.5.5, 00:00:59
B       10.164.48.0 [200/0] via 4.4.4.4, 00:00:59
C       10.164.49.0 is directly connected, Serial0/0/0
     11.0.0.0/32 is subnetted, 2 subnets
B       11.3.3.3 [200/1] via 5.5.5.5, 00:00:59
D       11.1.1.1 [90/2297856] via 10.164.49.1, 00:00:59, Serial0/0/0
r6#

so we have filtered the route

Ok i will look at an issue of mpls being down on int

we will take  mpls of the link to r2 on r1
r1
---

r1(config)#int fa0/0.12
r1(config-subif)#no mpls ip
r1(config-subif)#exit
*Mar  1 01:51:30.243: %LDP-5-NBRCHG: LDP Neighbor 2.2.2.2:0 is DOWN (LDP disabled on
interface)

r6
---
bb1#sh ip route
Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route
Gateway of last resort is not set
     2.0.0.0/24 is subnetted, 1 subnets
D       2.3.3.0 [90/2300416] via 10.164.49.6, 00:34:47, Serial0
     10.0.0.0/24 is subnetted, 4 subnets
D       10.229.254.0 [90/2172416] via 10.164.49.6, 00:34:47, Serial0
D EX    10.164.50.0 [170/2560512256] via 10.164.49.6, 00:06:10, Serial0
D EX    10.164.48.0 [170/2560512256] via 10.164.49.6, 00:06:10, Serial0
C       10.164.49.0 is directly connected, Serial0
     11.0.0.0/32 is subnetted, 2 subnets
D EX    11.3.3.3 [170/2560512256] via 10.164.49.6, 00:06:11, Serial0
C       11.1.1.1 is directly connected, Loopback0
bb1#

notice the routes are all still coming in even though the transport has an issue

notice also D EX for the rip external routes

all though we have the routes the pings are failing
bb1#ping 2.3.3.3
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 2.3.3.3, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
bb1#ping 10.229.254.3
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.229.254.3, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
bb1#

a quick way to troubleshoot an issue like this is to look at the mpls table
r1#sh mpls forw
Local  Outgoing    Prefix            Bytes tag  Outgoing   Next Hop
tag    tag or VC   or Tunnel Id      switched   interface
16     Untagged    155.0.7.0/24      0          Fa0/0.12   155.0.3.2
17     Untagged    2.2.2.2/32        0          Fa0/0.12   155.0.3.2
18     Untagged    4.4.4.4/32        385        Fa0/0.12   155.0.3.2
19     Pop tag     6.6.6.6/32        8023       Fa0/0.16   155.0.2.6
20     Pop tag     155.0.4.0/24      0          Fa0/0.13   155.0.5.3
21     Pop tag     155.0.6.0/24      0          Fa0/0.13   155.0.5.3
22     Pop tag     3.3.3.3/32        0          Fa0/0.13   155.0.5.3
23     16          5.5.5.5/32        3874       Fa0/0.13   155.0.5.3
r1#

nothing should be going out mpls interface as untagged it can pop or a label but not
untagged.so we know there is an issue

r1#sh mpls ldp neig
    Peer LDP Ident: 6.6.6.6:0; Local LDP Ident 1.1.1.1:0
        TCP connection: 6.6.6.6.58139 - 1.1.1.1.646
        State: Oper; Msgs sent/rcvd: 60/60; Downstream
        Up time: 00:39:53
        LDP discovery sources:
          FastEthernet0/0.16, Src IP addr: 155.0.2.6
        Addresses bound to peer LDP Ident:
          6.6.6.6         155.0.2.6
    Peer LDP Ident: 3.3.3.3:0; Local LDP Ident 1.1.1.1:0
        TCP connection: 3.3.3.3.24547 - 1.1.1.1.646
        State: Oper; Msgs sent/rcvd: 45/45; Downstream
        Up time: 00:26:52
        LDP discovery sources:
          FastEthernet0/0.13, Src IP addr: 155.0.5.3
        Addresses bound to peer LDP Ident:
          3.3.3.3         155.0.5.3       155.0.4.3       155.0.6.3
r1#

notice we are missing our mpls ldp neighbor 2.2.2.2

lets enable mpls ip back on

r1(config)#int fa0/0.12
r1(config-subif)#mpls ip
r1(config-subif)#exit
r1(config)#exit

r1#
*Mar  1 01:59:22.695: %SYS-5-CONFIG_I: Configured from console by console
*Mar  1 01:59:24.503: %LDP-5-NBRCHG: LDP Neighbor 2.2.2.2:0 is UP
r1#sh mpls forw
Local  Outgoing    Prefix            Bytes tag  Outgoing   Next Hop
tag    tag or VC   or Tunnel Id      switched   interface
16     Pop tag     155.0.7.0/24      0          Fa0/0.12   155.0.3.2
17     Pop tag     2.2.2.2/32        0          Fa0/0.12   155.0.3.2
18     18          4.4.4.4/32        0          Fa0/0.12   155.0.3.2
19     Pop tag     6.6.6.6/32        8312       Fa0/0.16   155.0.2.6
20     Pop tag     155.0.4.0/24      0          Fa0/0.12   155.0.3.2
       Pop tag     155.0.4.0/24      0          Fa0/0.13   155.0.5.3
21     Pop tag     155.0.6.0/24      0          Fa0/0.13   155.0.5.3
22     Pop tag     3.3.3.3/32        0          Fa0/0.13   155.0.5.3
23     16          5.5.5.5/32        4309       Fa0/0.13   155.0.5.3
r1#

BB1
---

bb1#ping 10.229.254.3
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.229.254.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 68/69/72 ms
bb1#ping 2.3.3.3
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 2.3.3.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 68/70/76 ms
bb1#




No comments:

Post a Comment