Posted by: Richard @ Configureterminal.com | September 17, 2008

“Variance” – Understanding The Balancing Act

This is an extract from a comment left by a visitor to a previous post on this blog:

“i was going through the bsci section and found about load balancing …

i set up a network in gns3 and tested loadbalancing with bandwidth and variance command

what exactly is the difference between those two ? both are used for load balancing right..

when we execute bandwidth command it changes bandwidth to our specified value…

when we execute a command say variance 2what happens inside the router? what factors does variance depends upon … does it depend upon k values or any other more factors”


I’ve found in the past that an answer to a question like this immediately pops into my head (it’s etched ‘in-there’ from CCNA/CCNP), but then, after answering it, I would start to think about it in a little bit more detail and before I know it I had a few niggly questions in my head – you know, the kind that stick with you for days… That’s where reading and labbing to find ALL of the answers can be a really useful thing to do. So, here’s a bit of a breakdown…


Actually… before I start, I’ll try to halt any confusion around the terms I will be using now:
– Where “Route” or “Prefix” could be used, I will use the word “Route”
– Where “Distance” or “Metric”/”Metric Value” or “Cost” could be used, I will use the word “Distance” (to keep with the EIGRP theme – I would usually use Cost)

By default, EIGRP will load-balance across all equal distance paths.
Up-to 4 paths will be installed in the routing table by default, but this can be increased up-to a maximum of 6 using the maximum-paths command under the EIGRP process. So far, this is typical of all of the routing protocols except BGP (and static) on the CCIE blueprint. (Just in case you are interested BGP = 1 and static = 6).

The network below is a starting point for an explanation of the specifics of EIGRP load-balancing – specifically support for unequal distance load-balancing (inc. what the variance, bandwidth, delay commands trigger):

In the network above, there is not one but two, equal, best paths between R5 and 10.1.0.0/24:
R5 – R2 – R1
R5 – R3 – R1


Just in case you’re thinking “it all looks the same to me” –> R4 to R5 = a delay of 30000 vs 20000 (that’s R5’s S1/2)

Both routes have been installed into the routing table, the third route hasn’t been installed because the default variance of 1 is being used by R5 –>
A multiplier of “1” = Use routes that are less than the route(s) with the smallest distance multiplied by “1”

The third route hasn’t been discarded by R5, it remains in the EIGRP topology database as a “Feasible Successor” or “FS” (a backup that can be installed in the event of a failure of the primary route(s)) but not in the routing table, here is the output of show ip eigrp topology and show ip route on R5:

R5#show ip eigrp topology
IP-EIGRP Topology Table for AS(100)/ID(192.168.45.5)

Codes: P – Passive, A – Active, U – Update, Q – Query, R – Reply,
r – reply Status, s – sia Status

P 10.1.0.0/24, 2 successors, FD is 27136000
via 192.168.25.2 (27136000/26624000), Serial1/0
via 192.168.35.3 (27136000/26624000), Serial1/1
via 192.168.45.4 (27392000/26624000), Serial1/2
P 192.168.45.0/24, 1 successors, FD is 26368000
via Connected, Serial1/2
P 192.168.35.0/24, 1 successors, FD is 26112000
via Connected, Serial1/1
P 192.168.12.0/24, 1 successors, FD is 26624000
via 192.168.25.2 (26624000/26112000), Serial1/0
P 192.168.13.0/24, 1 successors, FD is 26624000
via 192.168.35.3 (26624000/26112000), Serial1/1
P 192.168.14.0/24, 1 successors, FD is 26880000
via 192.168.45.4 (26880000/26112000), Serial1/2
via 192.168.25.2 (27136000/26624000), Serial1/0
via 192.168.35.3 (27136000/26624000), Serial1/1
P 192.168.25.0/24, 1 successors, FD is 26112000
via Connected, Serial1/0

R5#show ip route
<lines omitted>
D 192.168.12.0/24 [90/26624000] via 192.168.25.2, 00:00:01, Serial1/0
D 192.168.13.0/24 [90/26624000] via 192.168.35.3, 00:00:01, Serial1/1
D 192.168.14.0/24 [90/26880000] via 192.168.45.4, 00:00:01, Serial1/2
C 192.168.45.0/24 is directly connected, Serial1/2
C 192.168.25.0/24 is directly connected, Serial1/0
10.0.0.0/24 is subnetted, 1 subnets
D 10.1.0.0 [90/27136000] via 192.168.35.3, 00:00:01, Serial1/1
[90/27136000] via 192.168.25.2, 00:00:01, Serial1/0
C 192.168.35.0/24 is directly connected, Serial1/1


The Advertised Distance (AD) received from R2 and R3 for 10.1.0.0/24 = 26624000
The Feasible Distance (FD) to 10.1.0.0/24 via R2 and R3 from R5’s perspective = 27136000


The difference between the AD and the FD of a route is the distance to reach the neighbour advertising the route. Therefore, you could say that the FD is the “real distance” and this is reflected in the routing table.

Let’s work out the math for 10.1.0.0/24…

The default (k’s 10100) EIGRP formula for calculating distance =
256(10000000/’SmallestBW’ + ‘SumOfDelays’/10)

R1-R2-R5 Advertised Distance:
10000000/100 = 100000 <– “10000000/’SmallestBW’”
20000 + 20000 = 40000
<– “SumOfDelays”
40000/10 = 4000
<– “/10”
100000 + 4000 = 104000
<– “+”

104000 x 256 = 26624000 <– “256(”

R1-R2-R5 Feasible Distance:
10000000/100 = 100000
20000 + 20000 + 20000 = 60000
60000/10 = 6000
100000 + 6000 = 106000

106000 x 256 = 27136000

Just to confirm the AD vs FD statement above –> 27136000 – 26624000 = 512000 –> 512000/256 = 2000

Have you noticed anything in the calculations above?
The bandwidth portion of the formula hasn’t changed at-all between the two calculations, this is because the lowest bandwidth along the path is used – and once only … this allows me to address one of the sub-questions from the comment; if you are forcing the lowest bandwidth along the entire path it is possible to affect whether a path will or will not be included in load balancing, if it’s not the lowest nothing will change!

Let’s look into that a bit more –> The interface that connects R5 to R3 will now be increased to a bandwidth of 200Kbps (R5 S1/1), the lowest bandwidth along the path remains as 100 (R1 to R3):

As you can now see below, both of the previously installed routes are still the same (and equal), and remain in the routing table:

R5#show ip eigrp topology
IP-EIGRP Topology Table for AS(100)/ID(192.168.45.5)

Codes: P – Passive, A – Active, U – Update, Q – Query, R – Reply,
r – reply Status, s – sia Status

P 10.1.0.0/24, 2 successors, FD is 27136000
via 192.168.25.2 (27136000/26624000), Serial1/0
via 192.168.35.3 (27136000/26624000), Serial1/1
via 192.168.45.4 (27392000/26624000), Serial1/2

P 192.168.45.0/24, 1 successors, FD is 26368000
via Connected, Serial1/2
P 192.168.35.0/24, 1 successors, FD is 13312000
via Connected, Serial1/1
P 192.168.12.0/24, 1 successors, FD is 26624000
via 192.168.25.2 (26624000/26112000), Serial1/0
P 192.168.13.0/24, 1 successors, FD is 26624000
via 192.168.35.3 (26624000/26112000), Serial1/1
P 192.168.14.0/24, 1 successors, FD is 26880000
via 192.168.45.4 (26880000/26112000), Serial1/2
via 192.168.25.2 (27136000/26624000), Serial1/0
via 192.168.35.3 (27136000/26624000), Serial1/1
P 192.168.25.0/24, 1 successors, FD is 26112000
via Connected, Serial1/0

R5#show ip route
<lines omitted>
D 192.168.12.0/24 [90/26624000] via 192.168.25.2, 00:00:01, Serial1/0
D 192.168.13.0/24 [90/26624000] via 192.168.35.3, 00:00:01, Serial1/1
D 192.168.14.0/24 [90/26880000] via 192.168.45.4, 00:00:01, Serial1/2
C 192.168.45.0/24 is directly connected, Serial1/2
C 192.168.25.0/24 is directly connected, Serial1/0
10.0.0.0/24 is subnetted, 1 subnets
D 10.1.0.0 [90/27136000] via 192.168.35.3, 00:00:01, Serial1/1
[90/27136000] via 192.168.25.2, 00:00:01, Serial1/0

C 192.168.35.0/24 is directly connected, Serial1/1


Now, let’s change the bandwidth back to 100Kbps but increase the delay to 3000:

R5#show ip eigrp topology
IP-EIGRP Topology Table for AS(100)/ID(192.168.45.5)

Codes: P – Passive, A – Active, U – Update, Q – Query, R – Reply,
r – reply Status, s – sia Status

P 10.1.0.0/24, 1 successors, FD is 27136000
via 192.168.25.2 (27136000/26624000), Serial1/0
via 192.168.35.3 (27392000/26624000), Serial1/1
via 192.168.45.4 (27392000/26624000), Serial1/2

P 192.168.45.0/24, 1 successors, FD is 26368000
via Connected, Serial1/2
P 192.168.35.0/24, 1 successors, FD is 26368000
via Connected, Serial1/1
P 192.168.12.0/24, 1 successors, FD is 26624000
via 192.168.25.2 (26624000/26112000), Serial1/0
P 192.168.13.0/24, 1 successors, FD is 26624000
via 192.168.35.3 (26880000/26112000), Serial1/1
P 192.168.14.0/24, 1 successors, FD is 26880000
via 192.168.45.4 (26880000/26112000), Serial1/2
via 192.168.25.2 (27136000/26624000), Serial1/0
via 192.168.35.3 (27392000/26624000), Serial1/1
P 192.168.25.0/24, 1 successors, FD is 26112000
via Connected, Serial1/0

Because delay always has an effect on a route’s distance, a clear ‘winner’ now exists; just one route is now in the routing table:

R5#show ip route
<lines omitted>
D 192.168.12.0/24 [90/26624000] via 192.168.25.2, 00:00:01, Serial1/0
D 192.168.13.0/24 [90/26880000] via 192.168.35.3, 00:00:01, Serial1/1
D 192.168.14.0/24 [90/26880000] via 192.168.45.4, 00:00:01, Serial1/2
C 192.168.45.0/24 is directly connected, Serial1/2
C 192.168.25.0/24 is directly connected, Serial1/0
10.0.0.0/24 is subnetted, 1 subnets

D 10.1.0.0 [90/27136000] via 192.168.25.2, 00:00:01, Serial1/0
C 192.168.35.0/24 is directly connected, Serial1/1


The path of R5 – R2 – R1 will be used for traffic flowing from R5 to 10.1.0.0/24, the route via R3 has been relegated to a FS; this means no load-balancing:

When we have this kind of situation it allows us to have a play with the variance command. To use all three paths in a load-balanced setup we need to compare the ‘worst’ distance with the ‘best’ (aka the Successor):

Worst FD = 27392000
Best FD = 27136000

27136000 x 1 = 27136000
Is 27392000<27136000?
NO

27136000 x 2 = 54272000
Is 27392000<54272000?
YES

To include all three paths in a load-balanced setup we need R5 to have a variance value of 2 set, here is the outcome of configuring this:

R5#conf t
Enter configuration commands, one per line. End with CNTL/Z.
R5(config)#router eigrp 100
R5(config-router)#variance 2
R5(config-router)#do show ip eigrp topology
IP-EIGRP Topology Table for AS(100)/ID(192.168.45.5)

Codes: P – Passive, A – Active, U – Update, Q – Query, R – Reply,
r – reply Status, s – sia Status

P 10.1.0.0/24, 1 successors, FD is 27136000
via 192.168.25.2 (27136000/26624000), Serial1/0
via 192.168.35.3 (27392000/26624000), Serial1/1
via 192.168.45.4 (27392000/26624000), Serial1/2
P 192.168.45.0/24, 1 successors, FD is 26368000
via Connected, Serial1/2
P 192.168.35.0/24, 1 successors, FD is 26368000
via Connected, Serial1/1
P 192.168.12.0/24, 1 successors, FD is 26624000
via 192.168.25.2 (26624000/26112000), Serial1/0
P 192.168.13.0/24, 1 successors, FD is 26624000
via 192.168.35.3 (26880000/26112000), Serial1/1
P 192.168.14.0/24, 1 successors, FD is 26880000
via 192.168.45.4 (26880000/26112000), Serial1/2
via 192.168.25.2 (27136000/26624000), Serial1/0
via 192.168.35.3 (27392000/26624000), Serial1/1
P 192.168.25.0/24, 1 successors, FD is 26112000
via Connected, Serial1/0
R5(config-router)#do show ip route
<lines omitted>
D 192.168.12.0/24 [90/26624000] via 192.168.25.2, 00:00:01, Serial1/0
D 192.168.13.0/24 [90/27392000] via 192.168.45.4, 00:00:01, Serial1/2
[90/26880000] via 192.168.35.3, 00:00:01, Serial1/1
[90/27136000] via 192.168.25.2, 00:00:01, Serial1/0
D 192.168.14.0/24 [90/26880000] via 192.168.45.4, 00:00:01, Serial1/2
[90/27392000] via 192.168.35.3, 00:00:01, Serial1/1
[90/27136000] via 192.168.25.2, 00:00:01, Serial1/0
C 192.168.45.0/24 is directly connected, Serial1/2
C 192.168.25.0/24 is directly connected, Serial1/0
10.0.0.0/24 is subnetted, 1 subnets
D 10.1.0.0 [90/27392000] via 192.168.45.4, 00:00:01, Serial1/2
[90/27392000] via 192.168.35.3, 00:00:01, Serial1/1
[90/27136000] via 192.168.25.2, 00:00:01, Serial1/0

C 192.168.35.0/24 is directly connected, Serial1/1


And there you go, all three paths will be used:

Notes. –>
– Variance = Whole numbers only (aka Integers)
– When I say “Worst” I mean the least-preferred FS that you wish to use in load-balancing
– Traffic will by-default be distributed proportionately to the ratios of the distances that are associated with the different routes
— On a per-destination basis

Other stuff

From my understanding of things, delay has is a direct relationship with the default bandwidth/clock rate of an interface – the higher the default bandwidth, the lower the delay. I assume the original intentions were for the delay value to be dynamic in nature – based on Round Trip Time? but, that doesn’t seem to have happened. If we want to affect EIGRPs calculations in its default state we have the option of adjusting bandwidth or delay – delay is usually chosen because it doesn’t affect other functions running on the router – e.g. QoS calculations. Delay is also usually the weapon of choice because it guarantees a change to a route’s distance (see above). If an additional ‘metric contributor’ (k value) is added to the EIGRP distance calculations it can affect whether a path is or is not included in load balancing by effecting a route’s distance.
(Mini-disclaimer –> An offset-list can also be used to affect a route’s distance…)


The Feasibility Condition applies at all times – a route is only eligible for load balancing if it has an AD that is less than the FD of the Successor route(s).

EIGRP also supports load sharing (intelligent load-balancing), multi-interface load splitting allows you to efficiently control traffic that travels across multiple interfaces to the same destination:

traffic-share min
If multiple paths are available to the same destination, only paths with the minimum distance will be used, FS routes will remain in the routing table for quick convergence in the event of a failure. (never more than six and can be controlled by dynamic routing protocol max paths). Extra paths are discarded. If paths are removed from the routing table, pending routes are added automatically.

traffic-share min across-interfaces
Multi-interface load sharing on different interfaces with equal distance paths.
When the maximum path limit has been reached and a new path is installed, the router compares the installed paths.

traffic-share balanced [across-interfaces]
The router distributes traffic proportionately to the ratios of the distances that are associated with different routes (rounds down).
– balanced is the default

To summarise/answer the questions in the comment:

“i set up a network in gns3 and tested loadbalancing with bandwidth and variance command what exactly is the difference between those two ? both are used for load balancing right”
The only link between the bandwidth of an interface (inc. the command itself) and EIGRP is the fact that by default bandwidth is taken into consideration when EIGRP calculates a given route’s distance – hence load balancing can be affected by bandwidth

“when we execute bandwidth command it changes bandwidth to our specified value”
Yes – we enter it in Kbps form, and it is used as a ‘logical’ reference by the router (i.e. you could set the bandwidth of an interface higher than its clock rate if you really wanted to)


“when we execute a command say variance 2 what happens inside the router? what factors does variance depends upon … does it depend upon k values or any other more factors”
Configuring a variance of 2 would tell the router to load balance across any paths that have a distance less than 2x the distance of the best route available (i.e. the successor. Note. the feasible condition still applies). K values affect distance calculation and therefore could affect whether a route’s distance is within 2 x ‘best path’s distance’ or not.

Thank you to Rakesh for the comment/question – it gave me the opportunity to make sure I know the theory. And like always, if anybody notices an error – please let me know J

Advertisements

Responses

  1. now i got it …

    if you are forcing the lowest bandwidth along the entire path it is possible to affect whether a path will or will not be included in load balancing, if it’s not the lowest nothing will change!

    this is the point of the day …

    thank you for your time and explanation … its simply great …

  2. one final doubt …. i was messing up with ip cef command and its usgae and got this … is my understanding of the ip cef correct following this paragraph?

    Recent Cisco IOS releases have Cisco Express Forwarding (CEF) enabled by
    default. CEF allows fast switching of packets based on a per-destination
    switching architecture. The first packet in a flow is routed, and the rest are
    switched. This is the preferred behavior in most circumstances, because it
    allows load balancing in fast-switching architectures. However, if we were to
    ping the destination network, we would not see load balancing occurring on a
    packet level because CEF treats the entire series of pings as one flow.
    CEF on R3 overrides the per-packet balancing behavior of process switching
    with per-destination load balancing…

    my understanding is as follows :

    enabling ip cef which is by default means that .. let us say we have two best routes , the router primarily routes the packet seeing its routing table in two ways and from there on it sends the packets in that two ways without any comparision ?

    or … will it route let us say 10 packets to line 1 and 10 packets to line 2 .. is this what is meant by
    per-destination load balancing…

    disabling ipcef makes the router / eigrp to route each and every packet …. to the destination…

    though it is not advised to turn off ip-cef i see one advantage as we can see perfect load balancing which helps in determinig underused links while period of congestion as protocol routes every packet ….?

    thank you richard and i give you my word this is the last doubt (of this month lol)….

    take care and good luck

    regards
    rakesh

  3. FYI – I believe the maximum-paths is now as high as 16 depending on the IOS in use. Nice article though, excellent information in there!

  4. Hi rakesh,

    you can take a look at this blog about CEF load-sharing:

    http://blog.ioshints.info/2007/02/unequal-cost-load-sharing.html

    http://blog.ioshints.info/2006/12/fine-tuning-cef-load-sharing.html

  5. Derek, thanks, but don’t confuse me! I have to lock-away all of those “I’m actually answering this incorrectly” thoughts for the exam – in my mind the maximum number of paths is *8* for the next few months 🙂

  6. Thanks so much Carlos, I had allocated 1/2 hr tonight to make sure I was going to post the correct answer for Rakesh, and someone else had already done it – I should have checked.

    1/2 hr more OSPF tonight 🙂 – oh, wait….


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: