Kernel-bypass networking

Deep dive

How it works

In: packets from NIC receive rings · Out: packets dropped, rewritten, forwarded or handed to the program · State: per-core queues, packet buffer pools (mbuf/UMEM), flow tables · Cost: O(1) per packet; budget 67 ns at 10 GbE, 6.7 ns at 100 GbE (64-byte frames)

DPDK (user-space poll-mode drivers):

1. Bind the NIC to vfio-pci; reserve hugepages (2 MB or 1 GB) so packet buffers have stable physical addresses and few TLB misses.
2. Create an mbuf mempool (fixed-size buffers, per-core caches) on the NIC's NUMA node.
3. Each worker core owns an RX/TX queue pair; RSS in the NIC hashes flows to queues, so cores share nothing.
4. Loop: rte_eth_rx_burst (up to 32 descriptors), process, rte_eth_tx_burst; refill descriptors in bulk.

XDP / AF_XDP (in-kernel hook + optional user space):

1. Attach an eBPF program to the driver RX path (native mode). It sees raw frame bytes and returns XDP_DROP, XDP_PASS (normal stack), XDP_TX (bounce out the same NIC) or XDP_REDIRECT (another NIC, CPU or AF_XDP socket).
2. An AF_XDP socket registers a UMEM (user memory split into frames) and four rings: fill and completion (frame ownership), RX and TX (descriptors). In zero-copy mode the NIC DMAs straight into UMEM.

NIC RX ring --DMA--> [XDP prog] --DROP/TX--> done
                         |--REDIRECT--> AF_XDP RX ring --> user space (UMEM frames)
                         +--PASS-------> skb --> kernel stack --> sockets
DPDK: NIC RX ring --DMA--> hugepage mbufs --rx_burst--> app core (no kernel at all)

Code

XDP program (C, built with clang -O2 -g -target bpf -c xdp_udp.c): drop memcached reflection, steer one UDP port to AF_XDP, pass everything else.

#include <linux/bpf.h>
#include <linux/if_ether.h>
#include <linux/in.h>
#include <linux/ip.h>
#include <linux/udp.h>
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_endian.h>

struct { __uint(type, BPF_MAP_TYPE_XSKMAP); __uint(max_entries, 64);
         __type(key, __u32); __type(value, __u32); } xsks SEC(".maps");
struct { __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY); __uint(max_entries, 1);
         __type(key, __u32); __type(value, __u64); } drops SEC(".maps");

SEC("xdp")
int xdp_udp(struct xdp_md *ctx) {
    void *data = (void *)(long)ctx->data, *end = (void *)(long)ctx->data_end;
    struct ethhdr *eth = data;
    if ((void *)(eth + 1) > end || eth->h_proto != bpf_htons(ETH_P_IP)) return XDP_PASS;
    struct iphdr *ip = (void *)(eth + 1);
    if ((void *)(ip + 1) > end || ip->protocol != IPPROTO_UDP) return XDP_PASS;
    if (ip->ihl < 5) return XDP_DROP;
    struct udphdr *udp = (void *)ip + ip->ihl * 4;   // the verifier demands every bounds check
    if ((void *)(udp + 1) > end) return XDP_PASS;
    if (udp->source == bpf_htons(11211)) {           // reflection traffic comes FROM port 11211
        __u32 k = 0;
        __u64 *c = bpf_map_lookup_elem(&drops, &k);  // per-CPU counter: no atomics needed
        if (c) (*c)++;
        return XDP_DROP;
    }
    if (udp->dest == bpf_htons(9000))                // AF_XDP socket bound to this RX queue
        return bpf_redirect_map(&xsks, ctx->rx_queue_index, XDP_PASS); // fallback if none
    return XDP_PASS;
}
char LICENSE[] SEC("license") = "GPL";

DPDK core loop (C, DPDK ≥ 21.11): MAC-swap reflector.

#include <rte_eal.h>
#include <rte_ethdev.h>
#include <rte_mbuf.h>
#define BURST 32

int main(int argc, char **argv) {
    if (rte_eal_init(argc, argv) < 0) rte_exit(EXIT_FAILURE, "EAL init failed\n");
    uint16_t port = 0;
    struct rte_mempool *pool = rte_pktmbuf_pool_create("mbufs", 8191, 256, 0,
        RTE_MBUF_DEFAULT_BUF_SIZE, rte_eth_dev_socket_id(port));   // NUMA-local buffers
    struct rte_eth_conf conf = {0};
    if (rte_eth_dev_configure(port, 1, 1, &conf) < 0 ||
        rte_eth_rx_queue_setup(port, 0, 1024, rte_eth_dev_socket_id(port), NULL, pool) < 0 ||
        rte_eth_tx_queue_setup(port, 0, 1024, rte_eth_dev_socket_id(port), NULL) < 0 ||
        rte_eth_dev_start(port) < 0)
        rte_exit(EXIT_FAILURE, "port setup failed\n");
    struct rte_mbuf *bufs[BURST];
    for (;;) {                                       // busy-poll: 100 % of this core, always
        uint16_t n = rte_eth_rx_burst(port, 0, bufs, BURST);
        for (uint16_t i = 0; i < n; i++) {
            struct rte_ether_hdr *eth = rte_pktmbuf_mtod(bufs[i], struct rte_ether_hdr *);
            struct rte_ether_addr t = eth->src_addr;
            eth->src_addr = eth->dst_addr;
            eth->dst_addr = t;
        }
        uint16_t sent = rte_eth_tx_burst(port, 0, bufs, n);
        for (uint16_t i = sent; i < n; i++) rte_pktmbuf_free(bufs[i]); // TX ring full: drop
    }
}

Complexity and constants

• Line rate, 64-byte frames (84 bytes on the wire with preamble and gap): 10 GbE 14.88 Mpps = 67 ns; 25 GbE 37.2 Mpps; 100 GbE 148.8 Mpps = 6.7 ns. At 3 GHz, 6.7 ns is 20 cycles: one LLC miss (order of 30-40 ns) blows it; 100 GbE small packets need several cores.
• A DRAM cache miss costs order of 80-100 ns; DDIO places DMA'd packets in LLC, so process them before they are evicted (small RX rings, 512-2048 descriptors).
• Batching 32 packets amortizes descriptor ring reads, doorbell writes (MMIO, order of 100s of ns) and function calls.
• Latency: order of a few µs app-to-app with bypass vs order of 10-20 µs through kernel sockets, mostly from interrupts, wakeups and syscalls.

Efficiency

• Cost: the budget is the packet interval: 67 ns at 10 GbE, 6.7 ns (20 cycles at 3 GHz) at 100 GbE with 64-byte frames. One DRAM miss is order of 80-100 ns, so work per packet must stay in cache. Order of a few µs app-to-app with bypass versus order of 10-20 µs through kernel sockets. The bound is cache and memory latency, then PCIe.
• Levers:
  • Burst of 32 packets — amortizes descriptor reads, doorbell MMIO writes and calls (Complexity) — all poll-mode drivers.
  • Pin one queue per isolated core on the NIC's NUMA node — avoids order of 30-50 % cross-NUMA loss (Pitfalls) — multi-core data planes.
  • Small RX rings (512-2048) with DDIO — packets stay in LLC until processed — receive-heavy loops.
  • XDP native mode / AF_XDP zero-copy — stays in kernel or skips copies, keeps the kernel stack for the rest (Høiland-Jørgensen et al., CoNEXT 2018) — filtering, load balancing, drop.
• Naive → standard → state of the art:
  • Kernel sockets · interrupts, skb allocation and copies cost order of 10s of µs and cap one core at order of 1-2 Mpps.
  • Tuned kernel (busy poll, GRO, SO_REUSEPORT) · often enough for 1-5 Mpps.
  • DPDK/AF_XDP poll mode, or RDMA/SmartNIC offload · tens of Mpps per core, line rate at 100 GbE.

Implementation notes

• Pin each worker to an isolated core (isolcpus, nohz_full), on the NIC's NUMA node; never let two cores share a queue.
• Use RSS with a symmetric Toeplitz key if both directions of a flow must land on the same core.
• Prefetch the next packet's header while processing the current one (rte_prefetch0).
• Keep per-core state (counters, flow tables) in per-core arrays padded to cache lines to avoid false sharing; hand work between cores with SPSC rings (rte_ring).
• XDP: per-CPU maps for counters, BPF_MAP_TYPE_LRU_HASH for flow tables, bpf_xdp_adjust_head to encapsulate (GUE/IPIP for L4 load balancers).
• AF_XDP: keep the fill ring stocked; an empty fill ring makes the NIC drop. Use XDP_ZEROCOPY and need_wakeup to avoid syscalls when busy.

Pitfalls

• Losing the kernel: no ARP, ICMP, routing, tcpdump or firewall on a DPDK port. Plan an exception path (TAP/virtio-user to the kernel) and your own monitoring.
• Mempool too small or mbufs leaked on error paths: RX silently drops (rx_nombuf counter).
• Cross-NUMA buffers or cores: order of 30-50 % throughput loss.
• XDP generic (skb) mode works on any driver but is slower than the kernel fast path; verify native mode (ip link shows xdp, not xdpgeneric).
• Benchmarking with 64-byte packets at one flow hides RSS imbalance; real traffic has elephant flows.

Variants and alternatives

In production

Meta Katran (XDP L4 load balancer), Cloudflare (XDP DDoS drop, Unimog load balancer), Cilium (XDP service load balancing), Google Snap (user-space host networking), fd.io VPP and OVS-DPDK (telco, NFV), Suricata (AF_XDP capture), Seastar/ScyllaDB (optional DPDK).

Implementations

Sources

• Høiland-Jørgensen et al., The eXpress Data Path: Fast Programmable Packet Processing in the Operating System Kernel, ACM CoNEXT, 2018. paper · code
• Rizzo, netmap: A Novel Framework for Fast Packet I/O, USENIX ATC, 2012. paper · code
• Belay et al., IX: A Protected Dataplane Operating System for High Throughput and Low Latency, OSDI, 2014. paper
• Marty et al., Snap: a Microkernel Approach to Host Networking, SOSP, 2019. paper
• DPDK Project, Programmer's Guide, doc.dpdk.org. docs · code