CEF allows for load balancing or load sharing of traffic among multiple outgoing links. CEF needs multiple outgoing links as next hops in the routing table to perform load balancing. The command maximum-paths specifies how many paths or next hops are allowed per prefix in the routing table for the specific routing protocol. For instance, if you configure maximum-path 2 under the routing protocol Open Shortest Path First (OSPF), only two OSPF paths per prefix are allowed in the routing table. Those two paths are then shown in the CEF table as outgoing paths. In CEF, the two main load balancing schemes are per-packet or per-destination. If you configure the per-packet load balancing scheme, the load balancing of all packets is round-robin packet per packet on the outgoing links. The per-packet load balancing is configured with the interface command ip load-sharing per-packet.

You need to configure this command on all the outbound interfaces if you want to configure per-packet CEF load balancing. The default CEF load balancing scheme is per-destination. This terminology is a bit misleading, though, because the CEF per-destination load balancing is done by hashing the destination and source IP address. In contrast, the per-destination load balancing that fast switching does is strictly by looking at the destination IP address. Per-destination load sharing is the default load sharing scheme for CEF. It is the default load sharing method for CEF because the per-packet load sharing scheme can send consecutive packets of the same flow (this means the same source/destination IP address pair) across different paths and hence might lead to a reordering problem of the IP packets at the destination. This can lead to problems for traffic such as VoIP because a performance hit or quality degradation can occur if the packets arrive out of sequence, as packets might be considered lost. In addition, it adds jitter. Per-packet load sharing, however, gives a perfect load sharing distribution on the outgoing paths, whereas the per-destination load sharing is only a statistical method of distributing flows per pairs of (source IP address, destination IP address). Therefore, the load sharing of traffic with the perdestination method can only give a good result (a good distribution among all possible outgoing links) if enough different pairs of source and destination addresses make up the traffic toward the different destinations that are outbound on the outgoing links. Even then, if some flows are present, with considerably more traffic on than some others, which are on one path, the distribution might still be uneven.

Unequal Cost Load Balancing

It is possible to have unequal cost load balancing in CEF. In that case, the 16 hash buckets are not evenly distributed among all possible paths. Example 6-9 shows a router running Enhanced Interior Gateway Routing Protocol (EIGRP) as the routing protocol and variance being configured for EIGRP. Variance allows EIGRP to perform unequal cost load balancing because it enables routes to be installed in the routing table that are not the best. (These routes do not have the lowest metric.) In short, the variance number allows all routes that have a metric that is smaller than that of the best route multiplied by the variance to be installed in the routing table. One additional check is needed: The reported distance (metric of the route as reported by the EIGRP neighbor) of a route has to be smaller than the feasible distance (FD) in EIGRP for it to be eligible for installation in the routing table. In Example 6-9, the best route for prefix 10.200.254.4/32 has a metric of 2323456, and the second best route has a metric of 8697856. This latter metric is 3.74 times bigger than the metric of the best route, and this is reflected in the number of hash buckets assigned to each of the two paths. This ratio is also seen in the traffic share count in the routing table for the prefix (15/4). The best path pointing to Ethernet 1/2 has 13 hash buckets, whereas the path pointing to Ethernet 1/3 has 3 hash buckets. Obviously, 16 is a small number; as a result, a perfect distribution of the hash buckets according to the metric is not always possible. The distribution will always be approximate.

Load Balancing Labeled Packets

If the Multiprotocol Label Switching payload is an IPv4 or IPv6 packet, Cisco IOS uses the CEF hashing algorithm to determine the outgoing interface, in the case of per-destination load balancing. The load balancing is done only between labeled paths. This means that if an IP (unlabeled) and a labeled path have the same cost, only the labeled path is used to forward the packets. The command to verify which path a labeled IPv4 packet will take in the case of per-destination load balancing is show Multiprotocol Label Switching forwarding-table labels label exact-path ipv4 source-address destination-address. Following are the general rules for load balancing labeled packets on a non-IPv6-capable Cisco IOS router:

■ If the Multiprotocol Label Switching payload is an IPv4 packet, the load balancing is done by hashing the source and destination IP address of the IPv4 header.

■ If the Multiprotocol Label Switching payload is not an IPv4 packet, the load balancing is done by looking at the value of the bottom label.

How does an Multiprotocol Label Switching-enabled router know what the Multiprotocol Label Switching payload is? The router that assigned the label can figure this out by looking at the label, because this router assigned a label to the particular Forwarding Equivalence Class that the packet belongs to. However, if the stack holds more than one label, the P router in the Multiprotocol Label Switching network did not assign the bottom label. Because Multiprotocol Label Switching does not yet have a protocol identifier field in the label stack, the P router cannot easily identify what the Multiprotocol Label Switching payload is. In Cisco IOS, the router can look at the first nibble following the Multiprotocol Label Switching label stack. If the first nibble has the value 4, Cisco IOS considers this an IPv4 packet and performs IPv4 CEF hashing. Newer Cisco IOS software that is also capable of running IPv6 can check whether the first nibble is 6. If so, the Multiprotocol Label Switching payload is considered IPv6, and the router performs IPv6 CEF hashing. The load balancing is then based on the source and destination address in the IPv6 header. The algorithm for load balancing the Multiprotocol Label Switching packets becomes as follows:

■ If the Multiprotocol Label Switching payload is an IPv4 packet, the load balancing is done by hashing the source and destination IP address of the IPv4 header.

■ If the Multiprotocol Label Switching payload is not an IPv4 or IPv6 packet, the load balancing is done by looking at the value of the bottom label.

Troubleshooting CEF

When packets do not get to their destination in an Multiprotocol Label Switching network, it might be because CEF failed to label the packets correctly on the ingress PE router or forwarded them to the wrong adjacent router. You should have some technical troubleshooting skills to debug such CEF problems. You can disable and enable CEF on the interface by using the command ip route-cache cef. Toggling CEF on the interface can often indicate whether the problem is with CEF. If the problem lies with the Layer 2 rewrite of the packets, you can check the adjacency information with the show adjacency command or clear the adjacency with the clear adjacency command. The debug command debug ip cef drops [access-list] tells you if IP packets are dropped on the ingress PE router. You can specify an access list 1 to 99 to help narrow the debug output to one or more specific prefixes.