Cisconinja’s Blog

GLBP Weights, Load Balancing, and Redirection

Posted by Andy on February 11, 2009

This post will take a look at how weighting, load balancing, the redirect timer, and the timeout timer work in GLBP.  The topology for these tests is shown below:

glbp-topology2

All routers will be configured to preempt and each router will be configured with GLBP priority 100 + X where X is the router number so that the highest numbered router that is online will become the AVG for the group.  Each router’s interface MAC address will be changed to 0000.0000.000X where X is the router number for the sake of clarity.  We will configure each of the routers in order (the order is important for how forwarders are assigned) and will wait to configure GLBP on R5 for now.  The configuration is:

R1:
interface FastEthernet0/0
 ip address 10.1.1.1 255.255.255.0
 mac-address 0000.0000.0001
 glbp 1 preempt
 glbp 1 priority 101
 glbp 1 ip 10.1.1.100

R2:
interface FastEthernet0/0
 ip address 10.1.1.2 255.255.255.0
 mac-address 0000.0000.0002
 glbp 1 preempt
 glbp 1 priority 102
 glbp 1 ip 10.1.1.100

R3:
interface FastEthernet0/0
 ip address 10.1.1.3 255.255.255.0
 mac-address 0000.0000.0003
 glbp 1 preempt
 glbp 1 priority 103
 glbp 1 ip 10.1.1.100

R4:
interface FastEthernet0/0
 ip address 10.1.1.4 255.255.255.0
 mac-address 0000.0000.0004
 glbp 1 preempt
 glbp 1 priority 104
 glbp 1 ip 10.1.1.100

R5:
interface FastEthernet0/0
 ip address 10.1.1.5 255.255.255.0
 mac-address 0000.0000.0005

R4 has become the AVG due to it’s higher priority and preemption enabled.  We can also see that R1 is active for AVF #1, R2 for AVF #2, R3 for AVF #3, and R4 for AVF #4 since they were configured in that order:

glbp-1

Now we will bring R5 online:

R5:
interface FastEthernet0/0
 glbp 1 preempt
 glbp 1 priority 105
 glbp 1 ip 10.1.1.100

R5 becomes the AVG but remains in a Listen state for all 4 AVFs:

glbp-2

Since GLBP has a limit of 4 AVFs per group, no new ones were created for R5.  Let’s try increasing R5′s weighting value from the default of 100:

R5:
interface FastEthernet0/0
 glbp 1 weighting 200

We can see that the weighting on R5 is now 200 and that R5 knows each of the other AVFs are using the default value of 100.  We can also see that R5 is (by default) configured to preempt forwarders after a 30 second delay.  Despite these facts, R5 still remains in the Listen state for all 4 AVFs:

glbp-3

Let’s create a loopback interface on R1 and configure GLBP to track it and decrement the weight value when it goes down:

R1:
interface Loopback0
 ip address 1.1.1.1 255.255.255.0
!
track 1 interface Loopback0 line-protocol
!
interface FastEthernet0/0
 glbp 1 weighting 100 lower 90 upper 95
 glbp 1 weighting track 1 decrement 20

This configuration on R1 essentially says “Start with a weight value of 100.  If Loopback0 goes down, decrement the weight by 20.  If the the weight falls below 90, this router is no longer allowed to be an AVF.  Once the weight has fallen below 90, do not allow the router to become the AVF again until the weight is at least 95.”  Now we will shutdown Loopback 0 on R1:

R1:
interface Loopback0
 shutdown

After R5′s 30 second AVF preemption timer expires, R5 takes over the role of active for AVF #1 and R1 transitions to listening:

glbp-5

glbp-4

Now let’s bring R1′s loopback interface back up:

R1:
interface Loopback0
 no shutdown

After 30 seconds, R1 preempts R5 and reclaims AVF #1 even though it’s weight (100) is lower than R5′s (200):

glbp-6

glbp-7

As we can see, the owner ID for AVF #1 is set to R1′s MAC address since it was the first GLBP router to come online.  As soon as R1′s weighting value is greater than or equal to the upper threshold it is allowed to reclaim it’s AVF role as long as forwarder preemption is enabled on it.

Let’s look at another scenario where the AVF that we are trying to preempt is not the owner of that AVF.  We will shutdown R5′s F0/0 interface and then shut down R1′s loopback interface.  This causes R1 to fall below the lower weighting threshold again, and since R5 is not available to take over AVF #1, one of the other routers must take over AVF #1 in addition to it’s own AVF:

R5:
interface FastEthernet0/0
shutdown

R1:
interface Loopback0
shutdown

After the 30 second preemption delay expires on R2, R3, and R4, each of them becomes active for AVF #1 for a few milliseconds before hearing each other’s hellos and deciding on one of them to remain active.  (I’m not sure how this is determined exactly; after shutting down and re-enabling R1′s loopback interface 5 different times, R3 took over AVF #1 two times and R4 took over AVF #1 three times.  Cisco does not seem to have much information available on GLBP so it is hard to know for sure, but it is unimportant for this test as long as 1 of the 3 remaining routers takes over AVF #1.  If we needed it to be deterministic, we could simply configure a lower forwarder preemption delay on 1 of the 3 routers).  We can now see that R4 has taken over AVF #1 in addition to it’s own:

glbp-8

Now we will bring R5 back online:

R5:
interface FastEthernet0/0
 no shutdown

Even though R5 has a higher weight than R4 and R4 is not the owner of AVF #1, R5 still does not preempt R4 and take over AVF #1:

glbp-9

glbp-10

For the last test related to weights, we will look at what happens when the weighting value of a router falls below the lower threshold and no other routers in the group are configured for forwarder preemption.  We will re-enable R1′s loopback and verify that it becomes active for AVF #1 again:

R1:
interface Loopback0
 no shutdown

glbp-111

Next we will disable forwarder preemption on all other routers and then shutdown R1′s loopback, causing the weight on R1 to fall below the lower threshold:

R2:
interface FastEthernet0/0
 no glbp 1 forwarder preempt

R3:
interface FastEthernet0/0
 no glbp 1 forwarder preempt

R4:
interface FastEthernet0/0
 no glbp 1 forwarder preempt

R5:
interface FastEthernet0/0
 no glbp 1 forwarder preempt

R1:
interface Loopback0
 shutdown

R1 indicates that it has crossed the lower threshold, but it still remains active for AVF #1:

glbp-12

Without forwarder preemption configured on any other routers in the group, crossing the lower threshold does not cause that router to lose it’s active state.

 

Next we will look at the different types of load balancing that can be used for ARP replies to clients.  First we will re-enable R1′s loopback and re-enable forwarder preemption on the other routers:

R1:
interface Loopback0
 no shutdown

R2:
interface FastEthernet0/0
 glbp 1 forwarder preempt

R3:
interface FastEthernet0/0
 glbp 1 forwarder preempt

R4:
interface FastEthernet0/0
 glbp 1 forwarder preempt

R5:
interface FastEthernet0/0
 glbp 1 forwarder preempt

At this point R5 is the AVG for the group and R1-4 are each AVFs for the forwarder that they own:

glbp-13 

There are three ways GLBP can be used to load balance ARP replies to clients: round-robin, weighted, and host dependent.  We will look at round-robin (the default) first.  We will also enable the GLBP client-cache, which keeps track of the IP and MAC received from a client in an ARP request and the forwarder to which the client was assigned based on the virtual MAC sent in the ARP reply:

R5:
interface FastEthernet0/0
 glbp 1 client-cache maximum 30

Now we will generate ARP requests from various MAC and IP addresses on a PC attached to the network using ColaSoft Packet Builder.  The ARP requests will use MAC address 0000.0000.00XX and IP address 10.1.1.XX, where XX is a number written in decimal starting at 10 and incrementing to 29.  This will give us a total of 20 unique ARP requests being sent to the GLBP virtual IP address at a rate of 1 per second.  After sending the 20 ARP requests, we can see the results on R5:

glbp-14

show glbp detail gives a lot of useful information (show glbp client-cache can also be used to view just the client cache).  We can see that round-robin is the load balancing method and that 20 of 30 cache entries are being used.  We can also see exactly how the load balancing of ARP replies has taken place: AVF #1 received the first client (10.1.1.10) and every fourth client after that.  Age shows the amount of time since an ARP request was last received and updates shows the number of ARP requests received from the client.  Next let’s send 20 duplicate ARP requests of the first entry only (MAC 0000.0000.0010, IP 10.1.1.10).  The results on R5 look like this:

glbp-15

The ‘Client selection count’ keeps track of the number of replies sent with that vMAC in response to ARP requests, whether those ARP requests are unique or not.  We can see that 10 replies with each vMAC have been sent, which equals the 20 unique ARP requests initially sent plus the 20 duplicates.  The client cache only contains unique client entries; if repeat ARP requests are received from a client the age timer is reset and update counter is incremented.  Since AVF#4 received the last client in the previous test, AVF#1 is next in line to have it’s vMAC used in the ARP reply.  After 20 requests, the round-robin process ends on AVF#4 again and 10.1.1.10 is now a client of AVF#4, resulting in an unequal number of clients per AVF.  Next we will change the load balancing method on R5 to use weighted load balancing.  First the GLBP configuration on R5 is removed to clear the counters of ARP replies issued, then the load balancing method is changed to weighted.  We will also change the weights on some of the other routers since they are all currently using the default of 100:
 
R5:
interface FastEthernet0/0
 no glbp 1 ip
 no glbp 1 priority
 no glbp 1 preempt
 no glbp 1 weighting
 no glbp 1 client-cache

A few seconds later…

R5:
interface FastEthernet0/0
 glbp 1 ip 10.1.1.100
 glbp 1 priority 105
 glbp 1 preempt
 glbp 1 client-cache maximum 30
 glbp 1 load-balancing weighted

R1:
interface FastEthernet0/0
 glbp 1 weighting 200
 

R2:
interface FastEthernet0/0
 glbp 1 weighting 100

R3:
interface FastEthernet0/0
 glbp 1 weighting 80

R4:
interface FastEthernet0/0
 glbp 1 weighting 20

 After sending the same 20 unique ARP requests again, the results on R5 are:

glbp-16

Each AVF has it’s vMAC used in a percentage of the ARP replies equal to it’s weight divided by the sum of all AVF weights.  In this case, that means each AVF receives:

R1:    200 / (200 + 100 + 80 + 20)    = 50%, or 10 clients

R2:    100 / (200 + 100 + 80 + 20)    = 25%, or 5 clients

R3:    80 / (200 + 100 + 80 + 20)    = 20%, or 4 clients

R4:    20 / (200 + 100 + 80 + 20)    = 5%, or 1 client

For the last GLBP load balancing method, we will change it to host dependent after clearing the GLBP counters on R5:

R5:
interface FastEthernet0/0
 no glbp 1 ip
 no glbp 1 priority
 no glbp 1 preempt
 no glbp 1 weighting
 no glbp 1 client-cache
 default glbp 1 load-balancing


 
A few seconds later…

R5:
interface FastEthernet0/0
 glbp 1 ip 10.1.1.100
 glbp 1 priority 105
 glbp 1 preempt
 glbp 1 client-cache maximum 30
 glbp 1 load-balancing host-dependent

After sending the same 20 unique ARP requests again, the results on R5 are:

glbp-171

AVF #1 and #2 have each received 6 clients, while #3 and #4 have each received 4 clients.  Now let’s send 20 duplicate ARP requests like we did earlier from MAC: 0000.0000.0010, IP: 10.1.1.10.  This client is currently assigned to AVF #1.  After sending the ARP requests, R5 looks like this:

glbp-18

The ‘Client selection count’ has increased from 6 to 26 for AVF#1, while the other remained the same.  We can also see in the client cache that this client still uses AVF#1.  Now let’s look at another interesting aspect of host-dependent load balancing.  Notice that the clients were not completely evenly distributed to begin with; instead AVF#1 and #2 received 6 each while #3 and #4 received 4 each.  Also, there appears to be a pattern in the client addresses assigned to each AVF: 

AVF#1 has all clients whose MAC/IP ends in 0, 4, or 8

AVF#2 has all clients whose MAC/IP ends in 1, 5, or 9

AVF#3 has all clients whose MAC/IP ends in 2 or 6

AVF#4 has all clients whose MAC/IP ends in 3 or 7

Let’s test this theory by resetting R5′s GLBP configuration to clear the counters and clients, and then sending ARP requests from only clients whose MAC/IP ends in a 0, 4, or 8.  Just to be certain that there is no way R5 is somehow remembering which AVF the clients were assigned to before, we will use 10 MAC/IP addresses that the router has never seen before starting at MAC: 0000.0000.0030, IP: 10.1.1.30

R5:
interface FastEthernet0/0
 no glbp 1 ip
 no glbp 1 priority
 no glbp 1 preempt
 no glbp 1 client-cache
 default glbp 1 load-balancing

A few seconds later…

R5:
interface FastEthernet0/0
 glbp 1 ip 10.1.1.100
 glbp 1 priority 105
 glbp 1 preempt
 glbp 1 client-cache maximum 30
 glbp 1 load-balancing host-dependent

Here are the results on R5:

glbp-19

All 10 new clients received the vMAC of AVF#1 in their ARP reply.  Therefore, GLBP does not need to have already received an ARP request from a client in order to know which AVF to assign it to – the results are deterministic even if it is a brand new client.  In fact, even if the GLBP client cache is disabled (tested but not shown), the results still come out the same every time when a new client generates an ARP request.  The assignment of clients to AVFs with host-dependent load balancing appears to be based on some function of the client’s MAC and/or IP address rather than a database of which AVF the client was previously assigned to.

 

Next we will look at how the redirect and timeout timers function in GLBP.  The redirect timer in GLBP controls when the AVG will stop replying to client ARP requests with the virtual MAC address of an AVF that has been taken over by a router other than the original owner.  The timeout timer controls when the AVG removes the AVF entirely if the owner has not reclaimed it, and clients that had obtained that virtual MAC address through ARP (either before the AVF was taken over by a router other than the owner, or after it was taken over by a router other than the owner but before the redirection timer expired) must obtain a different virtual MAC address to use for the gateway.  First we will look at what happens when there are 4 or less routers in the GLBP group.  We will take R5 offline so that R4 becomes the new AVG:

 R5:
interface FastEthernet0/0
 shutdown

glbp-20

We will decrease the redirect and timeout timers to 60 and 660 seconds (the timeout must be at least 600 seconds more than the redirect) so that we can see the results more quickly:

R4:
interface FastEthernet0/0
 glbp 1 timers redirect 60 660

Now we will pretend R1 fails by shutting down it’s interface:

R1:
interface FastEthernet0/0
 shutdown

After the 10 second holddown timer expires, R4 becomes active for AVF#1 in addition to it’s own.  The redirection timer for AVF#1 begins counting down from 60 seconds from the time R4 last heard a hello from R1:

glbp-21

If we generate 20 ARP requests from the PC before the redirection timer expires, we find that R4 includes AVF#1 in it’s round robin cycle and each AVF handles 5 client requests:

glbp-22

If we generate another 20 ARP requests after the redirection timer has expired for AVF#1, AVF#1 is no longer included in the round-robin cycle of ARP replies and the new clients are split up between the 3 remaining AVFs:

glbp-23

After 660 total seconds, the timeout timer expires and AVF#1 is removed:

glbp-24

If instead we had 5 routers, when R1 failed R5 would take over AVF#1.  When the timeout timer for AVF#1 expires the AVF is removed, but instead of the group continuing to function with 3 AVFs, a new AVF is created after a few seconds with R5′s MAC as the owner ID:

R5:
02:35:17.098: %GLBP-6-FWDSTATECHANGE: FastEthernet0/0 Grp 1 Fwd 1 state Active -> Disabled
02:35:28.898: %GLBP-6-FWDSTATECHANGE: FastEthernet0/0 Grp 1 Fwd 1 state Listen -> Active

glbp-25

About these ads

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
Follow

Get every new post delivered to your Inbox.

%d bloggers like this: