Cisconinja’s Blog

Policy Based Routing ‘set’ Commands

Posted by Andy on May 29, 2009

Here we will look at the various set commands that can be used in PBR and some of the differences between them.  The topology and configurations for this example are shown below:

PBR topology2

R1:
interface Loopback1
 ip address 1.1.1.1 255.255.255.255
!
interface Loopback2
 ip address 1.1.1.2 255.255.255.255
!
interface Serial0/0
 ip address 10.0.12.1 255.255.255.0
!
ip route 0.0.0.0 0.0.0.0 10.0.12.2

R2:
no ip cef
!
interface FastEthernet0/0
 ip address 10.0.234.2 255.255.255.0
 no ip route-cache
!
interface Serial0/0
 encapsulation frame-relay
 no ip route-cache
!
interface Serial0/0.1 point-to-point
 ip address 10.1.23.2 255.255.255.0
 no ip route-cache
 frame-relay interface-dlci 101
!
interface Serial0/0.2 point-to-point
 ip address 10.2.23.2 255.255.255.0
 no ip route-cache
 frame-relay interface-dlci 102
!
interface Serial0/0.3 point-to-point
 ip address 10.3.23.2 255.255.255.0
 no ip route-cache
 frame-relay interface-dlci 103
!
interface Serial0/0.4 point-to-point
 ip address 10.4.23.2 255.255.255.0
 no ip route-cache
 frame-relay interface-dlci 104
!
interface Serial0/0.5 point-to-point
 ip address 10.5.23.2 255.255.255.0
 no ip route-cache
 frame-relay interface-dlci 105
!
interface Serial0/0.6 point-to-point
ip address 10.6.23.2 255.255.255.0
no ip route-cache
frame-relay interface-dlci 106
!
interface Serial0/0.7 point-to-point
ip address 10.7.23.2 255.255.255.0
no ip route-cache
frame-relay interface-dlci 107
!
interface Serial0/1
 ip address 10.0.12.2 255.255.255.0
 no ip route-cache

!
ip route 1.1.1.0 255.255.255.0 10.0.12.1

R3:
interface FastEthernet0/0
 ip address 10.0.234.3 255.255.255.0
!
interface Serial0/0
 encapsulation frame-relay
!
interface Serial0/0.1 point-to-point
 ip address 10.1.23.3 255.255.255.0
 frame-relay interface-dlci 101
!
interface Serial0/0.2 point-to-point
 ip address 10.2.23.3 255.255.255.0
 frame-relay interface-dlci 102
!
interface Serial0/0.3 point-to-point
 ip address 10.3.23.3 255.255.255.0
 frame-relay interface-dlci 103
!
interface Serial0/0.4 point-to-point
 ip address 10.4.23.3 255.255.255.0
 frame-relay interface-dlci 104
!
interface Serial0/0.5 point-to-point
 ip address 10.5.23.3 255.255.255.0
 frame-relay interface-dlci 105
!
interface Serial0/0.6 point-to-point
ip address 10.6.23.3 255.255.255.0
frame-relay interface-dlci 106
!
interface Serial0/0.7 point-to-point
ip address 10.7.23.3 255.255.255.0
frame-relay interface-dlci 107
!
interface FastEthernet0/1
 ip address 10.0.34.3 255.255.255.0
!
ip route 0.0.0.0 0.0.0.0 10.0.234.2

R4:
interface FastEthernet0/0
 ip address 10.0.234.4 255.255.255.0
!
interface FastEthernet0/1
 ip address 10.0.34.4 255.255.255.0
!
ip route 0.0.0.0 0.0.0.0 10.0.234.2

R2 will policy route packets received from R1.  CEF and fast switching have been disabled on R2 so that all packets can be seen in debug output.  First we will configure PBR on R2 to route traffic sourced from R1 loopback#1 out F0/0 to R3 using set ip next-hop:

R2:
access-list 1 permit 1.1.1.1
!
route-map pbr permit 10
 match ip address 1
 set ip next-hop 10.0.234.3
!
interface Serial0/1
 ip policy route-map pbr

R1#ping 10.0.34.3 repeat 1 source loopback1

R2#debug ip policy
R2#debug ip packet

Mar 1 00:31:18.031: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 00:31:18.031: IP: route map pbr, item 10, permit
Mar 1 00:31:18.035: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), len 100, policy routed
Mar 1 00:31:18.035: IP: Serial0/1 to FastEthernet0/0 10.0.234.3
Mar 1 00:31:18.035: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), g=10.0.234.3, len 100, forward

The packet is forwarded as expected using the next-hop address in the route-map.  Next let’s configure R2 to route packets from R1 loopback#2 out F0/0 to R3 using set ip default next-hop:

R2:
access-list 2 permit 1.1.1.2
!
route-map pbr permit 20
 match ip address 2
 set ip default next-hop 10.0.234.3

R1#ping 10.0.34.3 repeat 1 source loopback2

R2#debug ip policy
R2#debug ip packet
Mar 1 00:36:03.479: IP: s=1.1.1.2 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 00:36:03.479: IP: route map pbr, item 20, permit
Mar 1 00:36:03.483: IP: s=1.1.1.2 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), len 100, policy routed
Mar 1 00:36:03.483: IP: Serial0/1 to FastEthernet0/0 10.0.234.3
Mar 1 00:36:03.483: IP: s=1.1.1.2 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), g=10.0.234.3, len 100, forward

So far the results of each look the same.  The difference between adding the default keyword is the order of operations that is used.  Without default, the router first attempts to route using PBR.  If there is no match or a match on a deny statement the router then uses the normal forwarding process.  With the default keyword, the router instead attempts to use the routing table first and if no route can be found then tries PBR.  Let’s add a static route to 10.0.34.0/24 to demonstrate this:

R2:
ip route 10.0.34.0 255.255.255.0 Serial0/0.1

R1#ping 10.0.34.3 repeat 1 source loopback1

R2#debug ip policy
R2#debug ip packet
Mar 1 00:48:56.915: IP: tableid=0, s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.1), routed via RIB
Mar 1 00:48:56.915: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 00:48:56.915: IP: route map pbr, item 10, permit
Mar 1 00:48:56.915: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), len 100, policy routed
Mar 1 00:48:56.919: IP: Serial0/1 to FastEthernet0/0 10.0.234.3
Mar 1 00:48:56.919: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), g=10.0.234.3, len 100, forward

R1#ping 10.0.34.3 repeat 1 source loopback2

R2#debug ip policy
R2#debug ip packet

Mar 1 00:49:08.239: IP: tableid=0, s=1.1.1.2 (Serial0/1), d=10.0.34.3 (Serial0/0.1), routed via RIB
Mar 1 00:49:08.239: IP: s=1.1.1.2 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 00:49:08.239: IP: route map pbr, item 20, permit
Mar 1 00:49:08.243: IP: s=1.1.1.2 (Serial0/1), d=10.0.34.3 (Serial0/0.1), len 100, policy rejected -- normal forwarding
Mar 1 00:49:08.243: IP: s=1.1.1.2 (Serial0/1), d=10.0.34.3 (Serial0/0.1), g=10.0.34.3, len 100, forward

R2 now forwards traffic from R1 loopback #2 out S0/0.1 as specified by the routing table instead of using PBR.  Traffic from R1 loopback #1 continues to be sent out F0/0.

 

Next we will change the route-map statements to use an interface rather than a next-hop address.  The default keyword can also be used when setting an interface, and it has the same effect that we just saw of changing the order of operation.  Traffic from R1 loopback #1 will use set interface and traffic from R1 loopback #2 will use set default interface.  We will use F0/0 as the exit interface to demonstrate the problems that can occur when using a multiaccess network in a set interface statement.  We will also remove the static route to 10.0.34.0/24 that we created previously:

R2:
route-map pbr permit 10
 no set ip next-hop 10.0.234.3
 set interface FastEthernet0/0
!
route-map pbr permit 20
 no set ip default next-hop 10.0.234.3
 set default interface FastEthernet0/0
!
no ip route 10.0.34.0 255.255.255.0 Serial0/0.1

R1#ping 10.0.34.3 repeat 1 source loopback1

R2#debug ip policy
R2#debug ip packet

Mar 1 00:56:06.603: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 00:56:06.603: IP: route map pbr, item 10, permit
Mar 1 00:56:06.603: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy rejected -- normal forwarding
Mar 1 00:56:06.607: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, unroutable

R1#ping 10.0.34.3 repeat 1 source loopback2

R2#debug ip policy
R2#debug ip packet

Mar 1 00:56:10.923: IP: s=1.1.1.2 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 00:56:10.923: IP: route map pbr, item 20, permit
Mar 1 00:56:10.923: IP: s=1.1.1.2 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), len 100, policy routed
Mar 1 00:56:10.927: IP: Serial0/1 to FastEthernet0/0 10.0.34.3
Mar 1 00:56:10.927: IP: s=1.1.1.2 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), g=10.0.34.3, len 100, forward
Mar 1 00:56:10.931: IP: s=1.1.1.2 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), len 100, encapsulation failed

Both pings fail.  Interestingly, each fails for a different reason and gives a different ping response output.  The ping from loopback #1 (using set interface) attempts to use PBR first.  Finding no L2 address to use for 10.0.34.3 on F0/0, PBR rejects the packet and attempts to use normal forwarding.  No matching route exists so the packet is dropped and an ICMP unreachable is sent to R1.  The ping from loopback #2 (using set default interface) attempts to use the route table first.  No route to the destination exists so R2 next tries PBR.  PBR does not know what L2 address to use for 10.0.34.3 on F0/0 so it generates an ‘encapsulation failed’ message and simply drops the packet with no notification to R1.  Let’s put a static route back on R2 using an outgoing interface of F0/0 in the static route:

R2:
ip route 10.0.34.0 255.255.255.0 FastEthernet0/0

R1#ping 10.0.34.3 repeat 1 source loopback1

R2#debug ip policy
R2#debug ip packet

Mar 1 01:14:02.179: IP: tableid=0, s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), routed via RIB
Mar 1 01:14:02.179: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 01:14:02.183: IP: route map pbr, item 10, permit
Mar 1 01:14:02.183: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), len 100, policy routed
Mar 1 01:14:02.183: IP: Serial0/1 to FastEthernet0/0 10.0.34.3
Mar 1 01:14:02.187: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), g=10.0.34.3, len 100, forward
Mar 1 01:14:02.191: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), len 100, encapsulation failed

R1#ping 10.0.34.3 repeat 1 source loopback1

R2#debug ip policy
R2#debug ip packet

Mar 1 01:14:21.887: IP: tableid=0, s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), routed via RIB
Mar 1 01:14:21.887: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 01:14:21.891: IP: route map pbr, item 10, permit
Mar 1 01:14:21.891: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), len 100, policy routed
Mar 1 01:14:21.891: IP: Serial0/1 to FastEthernet0/0 10.0.34.3
Mar 1 01:14:21.891: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), g=10.0.34.3, len 100, forward

The first ping from loopback #1 fails again.  This time, however, when the normal forwarding process finds a match for the static route with an outgoing interface of F0/0, it sends an ARP for 10.0.34.3 which is answered with proxy ARP by R3.  The next ping is policy routed using the ARP cache entry.  This combined with the behavior that we saw when no route existed in the routing table shows that PBR does not even attempt to perform address resolution by itself on a multiaccess network, but if the L2 address is known (either from normal forwarding or a static entry), PBR will make use of it and successfully route the packet.

 

Both set ip next-hop and set ip default next-hop require that the next-hop be found on a directly connected subnet (PBR will fail if it is not and the next set statement or normal forwarding will be used).  With set ip next-hop recursive, the next-hop address does not need to be directly connected.  Instead a recursive lookup is performed (if necessary) on the next-hop address, and matching traffic is forwarded to the next-hop(s) used by that route entry according to the switching path in use on the router.  A benefit of this is that load balancing can be used.  Since we are using process switching, load balancing will take place per packet.  We will create a loopback on R3 and multiple static routes to it on R2.  We will then set the recursive next-hop address to R3’s loopback.  The static route left over from before will also be removed:

R3:
interface Loopback3
 ip address 3.3.3.3 255.255.255.0

R2:
no ip route 10.0.34.0 255.255.255.0 FastEthernet0/0
ip route 3.3.3.0 255.255.255.0 10.1.23.3
ip route 3.3.3.0 255.255.255.0 10.2.23.3
ip route 3.3.3.0 255.255.255.0 10.3.23.3
!
route-map pbr permit 10
 no set interface FastEthernet0/0
 set ip next-hop recursive 3.3.3.3

1-R2-Route-Table

Note: Using a recursive lookup with PBR to send traffic to a route entry does not count as a “turn” for that path in the load-balancing equation if the destination address is outside the range of the route.  If no traffic is ever sent to the actual destination of the route, all PBR traffic will be sent to the path that is next in line, and no load-balancing will occur.  Therefore to demonstrate load-balancing, a packet is sent to 3.3.3.3 in between each packet from R1 loopback #1.

R1#ping 10.0.34.3 repeat 1 source loopback1
R1#ping 3.3.3.3 repeat 1 source s0/0
R1#ping 10.0.34.3 repeat 1 source loopback1
R1#ping 3.3.3.3 repeat 1 source s0/0
R1#ping 10.0.34.3 repeat 1 source loopback1
R1#ping 3.3.3.3 repeat 1 source s0/0
R1#ping 10.0.34.3 repeat 1 source loopback1

R2#debug ip policy
R2#debug ip packet
Mar 1 01:46:47.595: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 01:46:47.595: IP: route map pbr, item 10, permit
Mar 1 01:46:47.599: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.3), len 100, policy routed
Mar 1 01:46:47.599: IP: Serial0/1 to Serial0/0.3 10.3.23.3
Mar 1 01:46:47.599: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.3), g=10.3.23.3, len 100, forward

 

Mar 1 01:47:06.951: IP: tableid=0, s=10.0.12.1 (Serial0/1), d=3.3.3.3 (Serial0/0.3), routed via RIB
Mar 1 01:47:06.955: IP: s=10.0.12.1 (Serial0/1), d=3.3.3.3 (Serial0/0.3), len 100, policy rejected -- normal forwarding
Mar 1 01:47:06.955: IP: s=10.0.12.1 (Serial0/1), d=3.3.3.3 (Serial0/0.3), g=10.3.23.3, len 100, forward

 

Mar 1 01:47:11.951: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 01:47:11.951: IP: route map pbr, item 10, permit
Mar 1 01:47:11.955: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.2), len 100, policy routed
Mar 1 01:47:11.955: IP: Serial0/1 to Serial0/0.2 10.2.23.3
Mar 1 01:47:11.955: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.2), g=10.2.23.3, len 100, forward

 

Mar 1 01:47:15.055: IP: tableid=0, s=10.0.12.1 (Serial0/1), d=3.3.3.3 (Serial0/0.2), routed via RIB
Mar 1 01:47:15.055: IP: s=10.0.12.1 (Serial0/1), d=3.3.3.3 (Serial0/0.2), len 100, policy rejected -- normal forwarding
Mar 1 01:47:15.055: IP: s=10.0.12.1 (Serial0/1), d=3.3.3.3 (Serial0/0.2), g=10.2.23.3, len 100, forward

 

Mar 1 01:47:17.439: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 01:47:17.439: IP: route map pbr, item 10, permit
Mar 1 01:47:17.439: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.1), len 100, policy routed
Mar 1 01:47:17.439: IP: Serial0/1 to Serial0/0.1 10.1.23.3
Mar 1 01:47:17.443: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.1), g=10.1.23.3, len 100, forward

 

Mar 1 01:47:22.635: IP: tableid=0, s=10.0.12.1 (Serial0/1), d=3.3.3.3 (Serial0/0.1), routed via RIB
Mar 1 01:47:22.635: IP: s=10.0.12.1 (Serial0/1), d=3.3.3.3 (Serial0/0.1), len 100, policy rejected -- normal forwarding
Mar 1 01:47:22.639: IP: s=10.0.12.1 (Serial0/1), d=3.3.3.3 (Serial0/0.1), g=10.1.23.3, len 100, forward

 

Mar 1 01:47:24.615: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 01:47:24.615: IP: route map pbr, item 10, permit
Mar 1 01:47:24.615: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.3), len 100, policy routed
Mar 1 01:47:24.619: IP: Serial0/1 to Serial0/0.3 10.3.23.3
Mar 1 01:47:24.619: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.3), g=10.3.23.3, len 100, forward

 

 

Each of the set commands that we have looked at so far with the exception of set ip next-hop recursive allow multiple interfaces or next-hop addresses to be listed in a single set statement.  If the 1st interface (or interface associated with the next-hop address, if a next-hop is used) is down or L2 address is unknown, the rest of the interfaces/addresses in the list are tried in order.  Let’s look at an example using set interface.  We will have R2 attempt to route traffic from R1 loopback #1 out S0/0.1, then F0/0, then S0/0.2.  All previous static routes and set commands have also been removed:

R2:
route-map pbr permit 10
 set interface Serial0/0.1 FastEthernet0/0 Serial0/0.2

R1#ping 10.0.34.3 repeat 1 source loopback1

R2#debug ip policy
R2#debug ip packet

Mar 1 02:28:19.723: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 02:28:19.723: IP: route map pbr, item 10, permit
Mar 1 02:28:19.727: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.1), len 100, policy routed
Mar 1 02:28:19.727: IP: Serial0/1 to Serial0/0.1 10.0.34.3
Mar 1 02:28:19.727: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.1), g=10.0.34.3, len 100, forward

R2:
interface Serial0/0.1 point-to-point
 shutdown

R1#ping 10.0.34.3 repeat 1 source loopback1

R2#debug ip policy
R2#debug ip packet

Mar 1 02:29:07.211: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 02:29:07.211: IP: route map pbr, item 10, permit
Mar 1 02:29:07.211: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.2), len 100, policy routed
Mar 1 02:29:07.215: IP: Serial0/1 to Serial0/0.2 10.0.34.3
Mar 1 02:29:07.215: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.2), g=10.0.34.3, len 100, forward

R2:
interface Serial0/0.2 point-to-point
 shutdown

R1#ping 10.0.34.3 repeat 1 source loopback1

R2#debug ip policy
R2#debug ip packet
Mar 1 02:29:54.835: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 02:29:54.835: IP: route map pbr, item 10, permit
Mar 1 02:29:54.835: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy rejected -- normal forwarding
Mar 1 02:29:54.839: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, unroutable

The first packet is sent out S0/0.1 since it is the first interface in the list and is up.  When the 2nd packet arrives S0/0.1 is shutdown.  F0/0 is next in the list, but R2 does not know what next-hop address to use so it is skipped also and the 2nd packet is sent out S0/0.2.  When the 3rd packet arrives S0/0.2 is shut down also.  PBR has no interfaces left to try, so the RIB is tried next.  R2 has no route to the destination so it is dropped and an unreachable sent to R1.

 

 

Yet another available command with PBR is set ip next-hop verify-availability.  This can either be used with CDP or object tracking.  We will take a look at it’s use with CDP first.  When used with CDP, reachability to the next-hop is verified by confirming that the device is a CDP neighbor.  To use it in this way, set ip next-hop verify-availability is entered without any arguments.  The set ip next-hop command is also required, which contains the list of next-hop addresses to be verified against the CDP neighbor table.  If only a subset of next-hops should use CDP verification a separate route-map entry must be used for the ones that should not be checked because all set ip next-hop commands are combined into a single line and set ip next-hop verify-availability will enable the CDP verification check for all of them.  Let’s try an example:

R2:
no route-map pbr
route-map pbr permit 10
 set ip next-hop 10.0.234.3
 set ip next-hop verify-availability
!
ip route 10.0.34.0 255.255.255.0 Serial0/0.1

2-R2-CDP

R1#ping 10.0.34.3 repeat 1 source loopback1

R2#debug ip policy
R2#debug ip packet
Mar 1 00:22:48.959: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 00:22:48.959: IP: route map pbr, item 10, permit
Mar 1 00:22:48.959: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), len 100, policy routed
Mar 1 00:22:48.959: IP: Serial0/1 to FastEthernet0/0 10.0.234.3
Mar 1 00:22:48.959: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), g=10.0.234.3, len 100, forward

R2 has an entry for R3 in it’s CDP table and the packet is successfully policy routed to the configured next-hop on F0/0.  Next we will shutdown R3 F0/0:

R3:
interface FastEthernet0/0
 shutdown

After the CDP holdtime for R3 expires it is removed from R2’s CDP neighbors table. When another packet arrives from R1 loopback #1, the CDP verification fails and R2 attempts to use normal routing which results in the packet being sent on S0/0.1:

3-R2-CDP

R1#ping 10.0.34.3 repeat 1 source loopback1

R2#debug ip policy
R2#debug ip packet

Mar 1 00:33:43.483: IP: tableid=0, s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.1), routed via RIB
Mar 1 00:33:43.483: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 00:33:43.483: IP: route map pbr, item 10, permit
Mar 1 00:33:43.483: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.1), len 100, policy rejected -- normal forwarding
Mar 1 00:33:43.483: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.1), g=10.0.34.3, len 100, forward

Without using next-hop verification in this example, R2 would not have fallen back to normal routing when R3 F0/0 went down.  Instead R2 would either keep sending traffic to the configured next-hop address unsuccessfully (if the L2 address is known) or keep trying to ARP for 10.0.234.3 (if the L2 address is unknown):

R2:
route-map pbr permit 10
 no set ip next-hop verify-availability
 

4-R2-ARP

R1#ping 10.0.34.3 repeat 2 source loopback1

R2#debug ip policy
R2#debug ip packet
Mar 1 00:53:57.391: IP: tableid=0, s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.1), routed via RIB
Mar 1 00:53:57.391: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 00:53:57.391: IP: route map pbr, item 10, permit
Mar 1 00:53:57.391: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), len 100, policy routed
Mar 1 00:53:57.395: IP: Serial0/1 to FastEthernet0/0 10.0.234.3
Mar 1 00:53:57.395: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), g=10.0.234.3, len 100, forward
Mar 1 00:53:57.399: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), len 100, encapsulation failed
Mar 1 00:53:59.423: IP: tableid=0, s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.1), routed via RIB
Mar 1 00:53:59.423: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 00:53:59.423: IP: route map pbr, item 10, permit
Mar 1 00:53:59.423: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), len 100, policy routed
Mar 1 00:53:59.427: IP: Serial0/1 to FastEthernet0/0 10.0.234.3
Mar 1 00:53:59.427: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), g=10.0.234.3, len 100, forward
Mar 1 00:53:59.431: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), len 100, encapsulation failed
 

Using CDP verification can also be used with the default keyword by using a combination of set ip default next-hop and set ip default next-hop verify-availability.  Like the other examples, this reverses the order of operation so that normal routing is attempted first.  The second way to use set ip next-hop verify-availability is with object tracking.  This configuration does not use the addresses listed in set ip next-hop.  Instead a next-hop address, sequence number, and object number are listed as arguments to the set ip next-hop verify-availability command.  Only 1 next-hop can be listed per command but the use of sequence numbers allows multiple next-hops to be used and attempted in order.  The next-hop address is considered reachable if the tracked object is up.  We will create a loopback on R4 and track reachability to it on R2 using an IP SLA:

R4:
interface Loopback4
 ip address 4.4.4.4 255.255.255.0

R2:
ip route 4.4.4.0 255.255.255.0 10.0.234.4
!
ip sla 1
 icmp-echo 4.4.4.4
ip sla schedule 1 life forever start-time now
!
track 1 rtr 1 reachability

Next we will create the route-map on R2.  R2 should send traffic from R1 to R4 whenever R4’s loopback is confirmed as reachable by the SLA:

R2:
no route-map pbr
route-map pbr permit 10
 set ip next-hop verify-availability 10.0.234.4 10 track 1

Let’s test it out:

R1#ping 10.0.34.3 repeat 1 source loopback1

R2#debug ip policy
R2#debug ip packet

Mar 1 01:36:25.071: IP: tableid=0, s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.1), routed via RIB
Mar 1 01:36:25.071: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 01:36:25.075: IP: route map pbr, item 10, permit
Mar 1 01:36:25.075: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), len 100, policy routed
Mar 1 01:36:25.075: IP: Serial0/1 to FastEthernet0/0 10.0.234.4
Mar 1 01:36:25.079: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), g=10.0.234.4, len 100, forward

R2 policy routes the packet using the configured next-hop of 10.0.234.4.  Next we will shutdown R4’s loopback:

R4:
interface Loopback4
 shutdown

When the next echo is sent by R2’s SLA, no response is received and the tracked object changes state to down:

Mar 1 01:37:13.487: %TRACKING-5-STATE: 1 rtr 1 reachability Up->Down

If another ping is sent by R1, it does not pass the reachability check by PBR on R2. R2 resorts to normal forwarding and uses the static route out S0/0.1:

R1#ping 10.0.34.3 repeat 1 source loopback1

R2#debug ip policy
R2#debug ip packet

Mar 1 01:37:49.415: IP: tableid=0, s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.1), routed via RIB
Mar 1 01:37:49.415: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 01:37:49.419: IP: route map pbr, item 10, permit
Mar 1 01:37:49.419: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.1), len 100, policy rejected -- normal forwarding
Mar 1 01:37:49.419: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.1), g=10.0.34.3, len 100, forward

Using object tracking to verify reachability of a next-hop is not supported with the default keyword in front of it.

 

 

 

PBR can contain multiple set commands within a single route-map sequence number.  If multiple set commands are configured, the order that PBR will attempt them in is:

1. set ip next-hop verify-availability  – with object tracking only

2. set ip next-hop – either alone or with CDP verification

3. set ip next-hop recursive

4. set interface

5. Route table

6. set ip default next-hop – either alone or with CDP verification

7. set default interface

The set commands will automatically be arranged into this order in the configuration regardless of what order they are entered in.  To demonstrate this, we will configure several set commands using each method, with each one using a different exit interface or next-hop.  We will then cause them to fail in order of preference to verify that the next preferred method is used.  Assume that all loopback interfaces on R3 and R4, and all static routes, route-maps, IP SLAs, and tracked objects on R2 have been deleted prior to the configuration below in order to make the configuration easier to follow.  An explanation of why each packet is routed the way that it is is shown at the end:

R3:
interface Loopback3
 ip address 3.3.3.3 255.255.255.0

R4:
interface Loopback4
 ip address 4.4.4.4 255.255.255.255

!
interface Loopback5
 ip address 4.4.4.5 255.255.255.255

R2:
ip route 1.1.1.0 255.255.255.0 10.0.12.1
ip route 3.3.3.0 255.255.255.0 Serial0/0.6
ip route 4.4.4.0 255.255.255.0 10.0.234.4
ip route 10.0.34.0 255.255.255.0 Serial0/0.4
!
ip sla 1
 icmp-echo 4.4.4.4
ip sla 2
 icmp-echo 4.4.4.5
ip sla schedule 1 life forever start-time now
ip sla schedule 2 life forever start-time now
!
track 1 rtr 1 reachability
track 2 rtr 2 reachability
!
route-map pbr permit 10
 set ip next-hop verify-availability 10.0.234.4 100 track 1
 set ip next-hop verify-availability 10.0.234.3 200 track 2
 set ip next-hop 10.7.23.3
 set ip next-hop verify-availability
 set ip next-hop recursive 3.3.3.3
 set interface Serial0/0.5
 set ip default next-hop 10.3.23.3 10.2.23.3
 set ip default next-hop verify-availability
 set default interface Serial0/0.1

 

R1#ping 10.0.34.3 repeat 1 source loopback1

#1
Mar 1 02:08:50.927: IP: tableid=0, s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.4), routed via RIB
Mar 1 02:08:50.927: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 02:08:50.931: IP: route map pbr, item 10, permit
Mar 1 02:08:50.931: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), len 100, policy routed
Mar 1 02:08:50.931: IP: Serial0/1 to FastEthernet0/0 10.0.234.4
Mar 1 02:08:50.935: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), g=10.0.234.4, len 100, forward

R4:
interface Loopback4
 shutdown

#2
Mar 1 02:10:43.883: IP: tableid=0, s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.4), routed via RIB
Mar 1 02:10:43.883: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 02:10:43.887: IP: route map pbr, item 10, permit
Mar 1 02:10:43.887: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), len 100, policy routed
Mar 1 02:10:43.887: IP: Serial0/1 to FastEthernet0/0 10.0.234.3
Mar 1 02:10:43.891: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (FastEthernet0/0), g=10.0.234.3, len 100, forward

R4:
interface Loopback5
 shutdown

#3
Mar 1 02:12:30.903: IP: tableid=0, s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.4), routed via RIB
Mar 1 02:12:30.903: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 02:12:30.903: IP: route map pbr, item 10, permit
Mar 1 02:12:30.907: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.7), len 100, policy routed
Mar 1 02:12:30.907: IP: Serial0/1 to Serial0/0.7 10.7.23.3
Mar 1 02:12:30.907: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.7), g=10.7.23.3, len 100, forward

R2:
interface Serial0/0.7 point-to-point
 shutdown

#4
Mar 1 02:13:05.751: IP: tableid=0, s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.4), routed via RIB
Mar 1 02:13:05.751: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 02:13:05.755: IP: route map pbr, item 10, permit
Mar 1 02:13:05.755: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.6), len 100, policy routed
Mar 1 02:13:05.755: IP: Serial0/1 to Serial0/0.6 3.3.3.3
Mar 1 02:13:05.755: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.6), g=3.3.3.3, len 100, forward

R2:
no ip route 3.3.3.0 255.255.255.0 Serial0/0.6

#5
Mar 1 02:13:35.219: IP: tableid=0, s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.4), routed via RIB
Mar 1 02:13:35.219: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 02:13:35.223: IP: route map pbr, item 10, permit
Mar 1 02:13:35.223: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.5), len 100, policy routed
Mar 1 02:13:35.223: IP: Serial0/1 to Serial0/0.5 10.0.34.3
Mar 1 02:13:35.227: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.5), g=10.0.34.3, len 100, forward

R2:
interface Serial0/0.5 point-to-point
 shutdown

#6
Mar 1 02:13:47.243: IP: tableid=0, s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.4), routed via RIB
Mar 1 02:13:47.243: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 02:13:47.247: IP: route map pbr, item 10, permit
Mar 1 02:13:47.247: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.4), len 100, policy rejected -- normal forwarding
Mar 1 02:13:47.247: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.4), g=10.0.34.3, len 100, forward

R2:
no ip route 10.0.34.0 255.255.255.0 Serial0/0.4

#7
Mar 1 02:15:58.471: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 02:15:58.471: IP: route map pbr, item 10, permit
Mar 1 02:15:58.475: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.3), len 100, policy routed
Mar 1 02:15:58.475: IP: Serial0/1 to Serial0/0.3 10.3.23.3
Mar 1 02:15:58.475: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.3), g=10.3.23.3, len 100, forward

R2:
interface Serial0/0.3 point-to-point
 no cdp enable

#8
Mar 1 02:19:05.127: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 02:19:05.127: IP: route map pbr, item 10, permit
Mar 1 02:19:05.127: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.2), len 100, policy routed
Mar 1 02:19:05.127: IP: Serial0/1 to Serial0/0.2 10.2.23.3
Mar 1 02:19:05.127: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.2), g=10.2.23.3, len 100, forward

R2:
interface Serial0/0.2 point-to-point
 shutdown

#9
Mar 1 02:19:15.831: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 02:19:15.831: IP: route map pbr, item 10, permit
Mar 1 02:19:15.831: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.1), len 100, policy routed
Mar 1 02:19:15.835: IP: Serial0/1 to Serial0/0.1 10.0.34.3
Mar 1 02:19:15.835: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3 (Serial0/0.1), g=10.0.34.3, len 100, forward

R2:
interface Serial0/0.1 point-to-point
 shutdown

#10
Mar 1 02:19:26.767: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy match
Mar 1 02:19:26.767: IP: route map pbr, item 10, permit
Mar 1 02:19:26.767: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, policy rejected -- normal forwarding
Mar 1 02:19:26.771: IP: s=1.1.1.1 (Serial0/1), d=10.0.34.3, len 100, unroutable

 

#1. Of all the set commands configured, set ip next-hop verify-availability with object tracking takes precedence.  The lowest sequence number is 100 and the tracked object is up so it is policy routed to 10.0.234.4.

#2. The tracked object for sequence number 100 is down.  Sequence number 200 is up so the packet is policy routed to 10.0.234.3.

#3. Both next-hops that use object tracking are down.  The next method to use is set ip next-hop.  The next-hop is in the CDP neighbor table so the packet is policy routed to 10.7.23.3.

#4. The interface where 10.7.23.3 was found is now down.  The next attempted method is set ip next-hop recursive.  R2 has a static route to the configured recursive next-hop address out of S0/0.6, so the packet is policy routed out S0/0.6.

#5. R2 no longer has a route to reach the recursive next-hop so the next method, set interface, is used and the packet is policy routed out S0/0.5.

#6. All remaining set commands use the default keyword, so R2 uses normal routing to send the packet out S0/0.4.

#7. There is no matching route in the route table anymore.  The next method to use is set ip default next-hop.  10.3.23.3 is the first listed next-hop and is in the CDP neighbor table so the packet is policy routed to 10.3.23.3.

#8. CDP has been disabled on S0/0.3 so availability can no longer be verified.  The next address listed in set ip default next-hop is still in the CDP neighbor table so the packet is policy routed to 10.2.23.3.

#9. The only remaining set command that is still valid, set default interface, is used and the packet is policy routed out S0/0.1.

#10. PBR fails and normal routing fails so the packet is dropped.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: