epoll event loop

Deep dive

How it works

In: a set of non-blocking fds with interest masks · Out: the ready fds, dispatched to handlers · State: the kernel epoll instance (tree of watched fds plus ready list) · Cost: epoll_wait O(ready), epoll_ctl O(log n)

1. epoll_create1(EPOLL_CLOEXEC) makes an epoll instance (a file).
2. epoll_ctl(ADD, fd, {events, data}) inserts an epitem into a red-black tree and hooks a callback on the socket's wait queue.
3. When a packet arrives, the socket wakes its wait queue; the callback appends the epitem to the ready list and wakes a waiter.
4. epoll_wait copies up to maxevents ready items to user space. Level-triggered items are re-added to the ready list if still ready; edge-triggered ones are not.
5. The loop dispatches by data.u64 (a token: fd, slab index or pointer), performs non-blocking reads/writes, and runs expired timers before waiting again.

 sockets --wakeup cb--> ready list --epoll_wait--> [ev0 ev1 ...] --> handler(token)
    ^                                                                 |
    +---------------- read()/write() until EAGAIN <-------------------+

Code

// Edge-triggered echo server. Cargo: libc = "0.2".
use std::collections::HashMap;
use std::io::{self, ErrorKind, Read, Write};
use std::net::{TcpListener, TcpStream};
use std::os::fd::{AsRawFd, RawFd};

fn ctl(ep: RawFd, op: i32, fd: RawFd, events: u32) -> io::Result<()> {
    let mut ev = libc::epoll_event { events, u64: fd as u64 };
    if unsafe { libc::epoll_ctl(ep, op, fd, &mut ev) } < 0 { return Err(io::Error::last_os_error()); }
    Ok(())
}

struct Conn { s: TcpStream, out: Vec<u8> }

/// Write as much of `out` as the socket takes; Ok(false) means the peer is gone.
fn flush(c: &mut Conn) -> io::Result<bool> {
    while !c.out.is_empty() {
        match c.s.write(&c.out) {
            Ok(n) => { c.out.drain(..n); }
            Err(e) if e.kind() == ErrorKind::WouldBlock => break, // wait for EPOLLOUT
            Err(e) if e.kind() == ErrorKind::Interrupted => continue,
            Err(_) => return Ok(false),
        }
    }
    Ok(true)
}

fn main() -> io::Result<()> {
    let lst = TcpListener::bind("0.0.0.0:7000")?;
    lst.set_nonblocking(true)?;
    let ep = unsafe { libc::epoll_create1(libc::EPOLL_CLOEXEC) };
    ctl(ep, libc::EPOLL_CTL_ADD, lst.as_raw_fd(), libc::EPOLLIN as u32)?; // level-triggered
    let mut conns: HashMap<RawFd, Conn> = HashMap::new();
    let mut evs = vec![libc::epoll_event { events: 0, u64: 0 }; 1024];
    let mut buf = vec![0u8; 64 * 1024];
    loop {
        let n = unsafe { libc::epoll_wait(ep, evs.as_mut_ptr(), evs.len() as i32, -1) };
        if n < 0 {
            let e = io::Error::last_os_error();
            if e.kind() == ErrorKind::Interrupted { continue; }
            return Err(e);
        }
        for i in 0..n as usize {
            let (fd, flags) = (evs[i].u64 as RawFd, evs[i].events); // by-value reads (struct is packed)
            if fd == lst.as_raw_fd() {
                loop {
                    match lst.accept() {
                        Ok((s, _)) => {
                            s.set_nonblocking(true)?;
                            s.set_nodelay(true)?;
                            let cfd = s.as_raw_fd();
                            // Register read+write once; ET means no epoll_ctl per state change.
                            let ev = (libc::EPOLLIN | libc::EPOLLOUT | libc::EPOLLRDHUP | libc::EPOLLET) as u32;
                            ctl(ep, libc::EPOLL_CTL_ADD, cfd, ev)?;
                            conns.insert(cfd, Conn { s, out: Vec::new() });
                        }
                        Err(e) if e.kind() == ErrorKind::WouldBlock => break,
                        Err(e) => { eprintln!("accept: {e}"); break; } // e.g. EMFILE
                    }
                }
                continue;
            }
            let Some(c) = conns.get_mut(&fd) else { continue };
            let mut alive = flags & (libc::EPOLLERR | libc::EPOLLHUP) as u32 == 0;
            if alive && flags & libc::EPOLLIN as u32 != 0 {
                loop { // ET: drain until EAGAIN or we never hear about this data again
                    match c.s.read(&mut buf) {
                        Ok(0) => { alive = false; break; }
                        Ok(k) => c.out.extend_from_slice(&buf[..k]),
                        Err(e) if e.kind() == ErrorKind::WouldBlock => break,
                        Err(e) if e.kind() == ErrorKind::Interrupted => continue,
                        Err(_) => { alive = false; break; }
                    }
                }
            }
            if alive { alive = flush(c)?; }
            if !alive { conns.remove(&fd); } // drop closes fd, which removes it from epoll
        }
        // Timers: compute the next deadline here and pass it as the epoll_wait timeout.
    }
}

Complexity and constants

• epoll_wait: order of 0.1-1 µs syscall base cost plus per-event copy; with Spectre/Meltdown mitigations syscalls cost more than on older kernels.
• epoll_ctl: one syscall, O(log n). A loop doing EPOLL_CTL_MOD per event (common with EPOLLONESHOT or toggling EPOLLOUT) doubles its syscall count.
• Kernel memory: order of 100-200 bytes per registered fd; fs.epoll.max_user_watches caps the total.
• Typical single-core echo/HTTP loop: order of 100k-1M small requests/s; beyond that the read/write syscalls, not epoll, dominate.
• select is limited to FD_SETSIZE = 1024 fds and scans bitmaps O(maxfd) each call.

Efficiency

• Cost: one epoll_wait is a syscall of order of 0.1-1 µs plus a copy per ready event; epoll_ctl is a syscall of O(log n). Kernel memory is order of 100-200 bytes per registered fd. A single core serves order of 100k-1M small requests/s; above that the read/write syscalls dominate, not the polling. The bound is syscall count (CPU), then memory bandwidth.
• Levers:
  • Drain until EAGAIN with edge-triggered mode — one epoll_ctl at registration instead of re-arming per event — busy connections.
  • Write immediately, register EPOLLOUT only on a full buffer — removes half the epoll_ctl calls (Implementation notes) — most responses.
  • maxevents 256-1024 — fewer epoll_wait calls per event — high connection counts.
  • One epoll plus SO_REUSEPORT listener per core — no shared lock or cache-line traffic — multi-core servers.
  • writev for header + body — one syscall instead of two — small responses.
• Naive → standard → state of the art:
  • Thread per connection · stack memory and context switches cap it at thousands of connections (Kegel, C10K).
  • select/poll · O(maxfd) scan per call, select capped at 1024 fds.
  • epoll, shared-nothing per core · O(ready) per wait; io_uring with multishot ops and provided buffers goes further by batching the I/O syscalls too.

Implementation notes

• Threading: shared-nothing, one epoll and one SO_REUSEPORT listener per core (nginx reuseport, Seastar style). Shared epoll across threads needs EPOLLEXCLUSIVE (Linux 4.5) on the listener to avoid thundering herd, or EPOLLONESHOT + re-arm for connections.
• Tokens: put a slab index plus a generation counter in data.u64, not a raw pointer, so stale events for a recycled fd are detected.
• Fairness: with ET, cap reads per connection per wakeup (e.g. 16 x 64 KB) and keep your own "still readable" list; otherwise one fast sender starves the rest.
• Timers: epoll timeout is in ms; use epoll_pwait2 (Linux 5.11, ns) or a timerfd, and a timing wheel for many connection timeouts.
• Batching: process all returned events before waiting; size maxevents to 256-1024.
• Writes: try write immediately; only rely on EPOLLOUT when the kernel buffer is full. Use writev to send header + body in one call.

Pitfalls

• ET without draining: the socket still has data but no new edge arrives; the connection hangs forever.
• epoll tracks the open file description, not the fd number. After dup/fork, closing your fd does not deregister it and events keep arriving for a dead number. Call EPOLL_CTL_DEL before close when fds can be shared.
• EMFILE on accept with a level-triggered listener spins at 100 % CPU; keep a spare fd to close-accept-close, or back off.
• Blocking work in a handler (DNS lookup, disk read, logging to a full pipe) stalls every connection on that thread.
• Level-triggered EPOLLOUT registered permanently fires on every wait: busy loop.

Variants and alternatives

In production

nginx, HAProxy, Redis (ae), memcached and Envoy (libevent), Node.js (libuv), Rust tokio (mio), Netty's native epoll transport, Java NIO Selector on Linux, Go's netpoller.

Implementations

Sources

• Linux man-pages, epoll(7), epoll_ctl(2), epoll_wait(2). man
• Banga, Mogul, Druschel, A Scalable and Explicit Event Delivery Mechanism for UNIX, USENIX ATC, 1999. paper
• Lemon, Kqueue: A Generic and Scalable Event Notification Facility, USENIX ATC (FREENIX), 2001. paper
• Kegel, The C10K problem, web article, 1999-2011. article