CEF and the FIB from Zero to CCIE: How a Cisco Router Really Forwards Packets
From Routing Table to Forwarding Table
In the previous module we treated the routing table as the router's final answer. That is true for the control plane, but it is not where packets are actually switched. A busy edge router can receive tens of millions of packets per second, and on a 100 Gbps interface full of minimum-size frames that number reaches 148.8 million packets per second. No router can afford to walk the routing table, resolve recursive next hops and look up a MAC address for each of them. Cisco solves this with Cisco Express Forwarding, CEF, and the two tables it builds: the Forwarding Information Base, the FIB, and the adjacency table.
CEF did not arrive first. Early Cisco routers used process switching, where the main CPU handled every packet with a full routing table lookup. Then came fast switching, which processed the first packet of a flow and stored the result in a route cache, so later packets to the same destination could reuse it. The cache helped, but it had a serious weakness. Routing changes invalidated cache entries, and in a network with many destinations and frequent changes the router spent much of its time rebuilding the cache. Traffic patterns that touched huge numbers of destinations, which is exactly what an Internet-facing BGP router sees, made it worse.
CEF removes the cache idea entirely. Instead of learning forwarding decisions from traffic, it builds them in advance from the routing table. Every entry in the RIB has a matching entry in the FIB, so there is nothing to learn and nothing to age out. When the RIB changes, the FIB changes with it. CEF has been enabled by default on most platforms since IOS 12.0, and on modern IOS XE and IOS XR it is simply how the router forwards.
The FIB is the first half of the design. It is a copy of the routing table optimised for lookup, with every recursive next hop already resolved. A BGP route that says "via 3.3.3.3" in the RIB appears in the FIB already pointing at the real next hop and interface that reach 3.3.3.3. The packet path never has to recurse.
The adjacency table is the second half. It holds the Layer 2 rewrite for each directly connected next hop: on Ethernet, the destination MAC, the source MAC and the EtherType that must be written in front of the packet. Because the adjacency table is built separately from the FIB, a change of MAC address on a neighbour does not invalidate any FIB entry. Only the one adjacency is updated, and every prefix that points to it uses the new rewrite at once.
R1#show ip cef 192.0.2.0/24 detail
192.0.2.0/24, epoch 2
recursive via 3.3.3.3
nexthop 10.1.12.2 GigabitEthernet0/0
R1#show adjacency
Protocol Interface Address
IP GigabitEthernet0/0 10.1.12.2(7)
IP GigabitEthernet0/1 10.1.13.3(5)
The first output is the FIB's view of an iBGP prefix. The recursion through 3.3.3.3 has already been resolved down to 10.1.12.2 on GigabitEthernet0/0. The second output lists the adjacencies the FIB can point to. Together they are everything the data plane needs: one lookup in the FIB, one pointer to an adjacency, one rewrite, and the packet is gone.
Reading the FIB and the Adjacency Table
With the architecture in place, the next step is to read what CEF actually holds, because the FIB contains entries that never appear in "#show ip route".
R1#show ip cef
Prefix Next Hop Interface
0.0.0.0/0 no route
0.0.0.0/8 drop
0.0.0.0/32 receive
1.1.1.1/32 receive Loopback0
2.2.2.2/32 10.1.12.2 GigabitEthernet0/0
3.3.3.3/32 10.1.12.2 GigabitEthernet0/0
10.1.12.0/24 attached GigabitEthernet0/0
10.1.12.0/32 receive GigabitEthernet0/0
10.1.12.1/32 receive GigabitEthernet0/0
10.1.12.2/32 attached GigabitEthernet0/0
10.1.12.255/32 receive GigabitEthernet0/0
127.0.0.0/8 drop
192.0.2.0/24 10.1.12.2 GigabitEthernet0/0
224.0.0.0/4 drop
224.0.0.0/24 receive
240.0.0.0/4 drop
255.255.255.255/32 receive
Each "Next Hop" word tells you how the data plane treats a match. An address in that column means a normal forwarding entry pointing at an adjacency. "attached" means the prefix is a directly connected subnet, and packets are delivered straight to the destination host on that link. "receive" means the packet is for the router itself, for example its own interface address, the subnet and broadcast addresses of a connected network, or 255.255.255.255. Those packets are punted to the CPU instead of being forwarded. "drop" marks ranges that are never forwarded, such as 127.0.0.0/8 and the multicast and reserved blocks in the unicast FIB. "no route" on 0.0.0.0/0 means there is no default route, so a packet with no other match is dropped.
Notice also the /32 entry for 10.1.12.2 marked "attached". When CEF learns the MAC address of a neighbour on a connected subnet, it creates a host entry for that neighbour with a complete adjacency. Before that happens, a packet to a host on a connected subnet hits a glean adjacency. A glean adjacency means "this subnet is connected, but there is no MAC rewrite yet", so the router triggers ARP and builds the host adjacency when the reply arrives.
CEF has a few more special adjacencies worth knowing by name. A null adjacency is created for routes to Null0, and packets that use it are dropped, which makes Null0 an efficient filter. A punt adjacency sends packets to the next higher switching level, because CEF cannot handle them itself. Discard and drop adjacencies drop packets for other reasons. When a route seems correct in the RIB but traffic fails, the adjacency type in the FIB is often the explanation.
The adjacency table can also be read per interface, which shows the actual rewrite string.
R1#show adjacency GigabitEthernet0/0 detail
Protocol Interface Address
IP GigabitEthernet0/0 10.1.12.2(7)
0 packets, 0 bytes
epoch 0
sourced in sev-epoch 1
Encap length 14
5254000A1B2C5254000C3D4E0800
ARP
The long hexadecimal string is the 14-byte Ethernet header CEF will write: the neighbour's MAC address, the local interface MAC address and EtherType 0800 for IPv4. The last line, ARP, says where the rewrite came from.
Why CEF Matters for BGP at CCIE Level
Once the FIB and the adjacency table are clear, it becomes obvious why CEF is part of a BGP course and not only a switching topic. BGP lives entirely in the control plane. It decides which paths are best and offers them to the RIB, and from there CEF decides how fast and how reliably traffic moves. Three areas connect the two directly.
The first is scale. A full Internet table holds roughly a million IPv4 prefixes, and every one of them that wins in the RIB becomes a FIB entry. On a software platform that costs memory. On a hardware platform such as a Catalyst 9000 switch it consumes forwarding hardware resources, and those resources are finite. When they are exhausted, existing routes keep forwarding, but new or changed prefixes cannot be programmed and become unreachable, and the switch logs a %FED_L3_ERRMSG-3-RSRC_ERR message. The control plane looks perfectly healthy while part of the data plane is silently broken.
The second is recursion. Almost every iBGP route points at a remote next hop, often a PE loopback. If the FIB stored every BGP prefix with its final interface copied in, a single IGP change towards that PE would force the router to rewrite hundreds of thousands of entries. Cisco's hierarchical FIB avoids this. Each BGP prefix points to a shared next hop object, and only that object points to the adjacency. When the path to the PE changes, one object is updated and every prefix behind it follows instantly. This is also what makes BGP Prefix Independent Convergence possible, because a backup path can be stored in the same shared structure. On IOS XE, PIC Core behaviour is enabled by default, and the command "#cef table output-chain build favor memory-utilization" disables it to save memory.
The third is load sharing. When BGP multipath installs several equal paths, CEF decides which packets use which path. By default it shares traffic per destination with a hash, not per packet, so the result depends on the traffic mix rather than on simple round robin. That behaviour has its own post in this series, because it causes some of the most confusing "why is one link full and the other empty" tickets.
The practical habit to build now is simple. When a BGP route looks right in "#show ip bgp" and right in "#show ip route" but traffic still goes the wrong way, check "#show ip cef" for the destination. The FIB is what forwards packets, and it is the last place where control plane intent and data plane reality can disagree.