io_uring

Deep dive

How it works

In: submission entries (op, fd, buffer, user_data) · Out: completion entries (res, flags, user_data) · State: the shared rings plus registered files and buffers · Cost: O(1) per op, one io_uring_enter per batch (none with SQPOLL)

1. io_uring_setup(entries, params) returns an fd; mmap it three times: SQ ring (indices), SQE array, CQ ring (CQ is 2x SQ by default).
2. Submit: fill an SQE (opcode, fd, buffer, length, offset, flags, user_data), write its index into the SQ ring, then store-release the SQ tail.
3. io_uring_enter(fd, to_submit, min_complete, IORING_ENTER_GETEVENTS) makes the kernel consume SQEs. Ops that can finish inline (data already in the socket buffer) complete immediately; others are armed on internal poll or punted to io-wq worker threads.
4. Reap: load-acquire the CQ tail, read CQEs (user_data, res = bytes or -errno, flags), store-release the CQ head.
5. Buffers passed to an in-flight op must stay valid and unmoved until its CQE arrives.

user space                         kernel
 SQE array [op,fd,buf,len,ud] --> consumes at SQ head
 SQ ring  tail++ ----------------->
 CQ ring  <------------------------ tail++ [ud,res,flags]
          head++ (after reading)

Key features: SQPOLL (kernel thread polls the SQ; no enter call while busy), registered files/buffers (skip per-op fd lookup and page pinning), provided buffer rings (5.19: kernel picks a buffer at receive time, so idle connections hold no buffers), multishot accept/recv (5.19/6.0: one SQE, many CQEs), linked SQEs and link timeouts, SEND_ZC (6.0 zero-copy send), SINGLE_ISSUER + DEFER_TASKRUN (6.1) to cut cross-CPU work.

Code

// TCP echo server with io_uring. Cargo: io-uring = "0.7", slab = "0.4", libc = "0.2".
use io_uring::{opcode, squeue, types, IoUring};
use slab::Slab;
use std::{net::TcpListener, os::fd::AsRawFd};

enum Op { Accept, Recv(usize), Send(usize) }
struct Conn { fd: i32, buf: Box<[u8]>, len: usize, sent: usize }

fn ud(op: &Op) -> u64 {
    match *op { Op::Accept => 0, Op::Recv(i) => (i as u64) << 2 | 1, Op::Send(i) => (i as u64) << 2 | 2 }
}

fn push(ring: &mut IoUring, e: squeue::Entry) {
    // SAFETY: every buffer referenced by `e` lives in the slab until its CQE is handled.
    while unsafe { ring.submission().push(&e) }.is_err() { ring.submit().unwrap(); }
}

fn recv(ring: &mut IoUring, conns: &mut Slab<Conn>, i: usize) {
    let c = &mut conns[i];
    let e = opcode::Recv::new(types::Fd(c.fd), c.buf.as_mut_ptr(), c.buf.len() as u32).build();
    push(ring, e.user_data(ud(&Op::Recv(i))));
}

fn send(ring: &mut IoUring, conns: &mut Slab<Conn>, i: usize) {
    let c = &conns[i];
    let p = unsafe { c.buf.as_ptr().add(c.sent) };
    let e = opcode::Send::new(types::Fd(c.fd), p, (c.len - c.sent) as u32).build();
    push(ring, e.user_data(ud(&Op::Send(i))));
}

fn main() -> std::io::Result<()> {
    let lst = TcpListener::bind("0.0.0.0:7000")?;
    let lfd = types::Fd(lst.as_raw_fd());
    let mut ring = IoUring::new(1024)?;
    let mut conns: Slab<Conn> = Slab::new();
    let accept = opcode::Accept::new(lfd, std::ptr::null_mut(), std::ptr::null_mut()).build();
    push(&mut ring, accept.clone().user_data(0));
    loop {
        ring.submit_and_wait(1)?; // submits everything queued and sleeps until >= 1 completion
        let done: Vec<(u64, i32)> = ring.completion().map(|c| (c.user_data(), c.result())).collect();
        for (u, res) in done {
            let i = (u >> 2) as usize;
            match u & 3 {
                0 => {
                    if res >= 0 {
                        let i = conns.insert(Conn { fd: res, buf: vec![0; 16 * 1024].into(), len: 0, sent: 0 });
                        recv(&mut ring, &mut conns, i);
                    }
                    push(&mut ring, accept.clone().user_data(0)); // re-arm (or use AcceptMulti)
                }
                1 if res > 0 => {
                    let c = &mut conns[i];
                    c.len = res as usize; c.sent = 0;
                    send(&mut ring, &mut conns, i);
                }
                2 if res >= 0 => {
                    let c = &mut conns[i];
                    c.sent += res as usize;
                    if c.sent < c.len { send(&mut ring, &mut conns, i) } // short send: resubmit tail
                    else { recv(&mut ring, &mut conns, i) }
                }
                _ => { // EOF (res == 0) or -errno on recv/send: close and free the slot
                    let c = conns.remove(i);
                    unsafe { libc::close(c.fd) };
                }
            }
        }
    }
}

Complexity and constants

• Per op: one SQE write and one CQE read (a few ns each) plus the kernel work; syscalls amortized over the batch.
• Syscall base cost is order of 0.1-1 µs; a loop handling 64 completions per io_uring_enter pays that once instead of 64-128 times.
• Storage: order of 1M+ 4 KB random-read IOPS per core with registered buffers and IORING_SETUP_IOPOLL; Axboe's tuned benchmarks reach order of 10M per core.
• Networking: over a well-written batched epoll loop, gains are typically order of 10s of %, mostly from fewer syscalls, multishot ops and provided buffers.
• SQPOLL burns a core while busy (sq_thread_idle ms before it sleeps).

Efficiency

• Cost: per operation one SQE write and one CQE read (a few ns each) plus the kernel work; the io_uring_enter syscall (order of 0.1-1 µs) is shared by the whole batch. Order of 1M+ 4 KB random-read IOPS per core with registered buffers and IORING_SETUP_IOPOLL; Axboe's tuned runs reach order of 10M per core. On networking, gains over a good epoll loop are order of 10s of %. The bound is syscall count for small ops, device or memory bandwidth for large ones.
• Levers:
  • Batch submissions — one syscall per 64+ ops instead of 64-128 (Complexity) — many concurrent ops.
  • Registered files and buffers — skips per-op fd lookup and page pinning (man io_uring_register(2)) — hot sockets and fixed I/O buffers.
  • Multishot accept/recv with a provided buffer ring — one SQE yields many CQEs, memory scales with in-flight data — many connections.
  • SQPOLL — no submit syscall, costs a polling core — latency-critical, saturated rings.
• Naive → standard → state of the art:
  • Blocking read/write thread per request · one syscall and one context switch per op.
  • epoll plus non-blocking syscalls · readiness batched, but every data transfer is still a syscall.
  • io_uring with registered resources and multishot · syscalls amortized over the batch; adds kernel attack surface, so check kernel.io_uring_disabled.

Implementation notes

• Ring size: 256-4096 entries; set IORING_SETUP_CQSIZE larger than SQ when using multishot ops, and handle CQ overflow (IORING_SQ_CQ_OVERFLOW).
• One ring per thread (IORING_SETUP_SINGLE_ISSUER), like one epoll per thread.
• Provided buffer ring: register N buffers of 4-16 KB per ring; recv CQEs carry the buffer id in flags >> 16; recycle it after processing. Memory scales with in-flight data, not connections.
• Use registered (fixed) files for hot sockets: types::Fixed(idx) instead of types::Fd.
• Cancellation: AsyncCancel by user_data; you still must wait for the original op's CQE (-ECANCELED) before freeing its buffer.
• Check feature availability with io_uring_register(IORING_REGISTER_PROBE); don't assume from the version number.

Pitfalls

• Freeing or moving a buffer while its op is in flight: silent memory corruption. Rust futures that are dropped mid-flight are the classic case; completion-based runtimes (tokio-uring, glommio, monoio) own buffers for this reason.
• Ops that cannot complete inline on files without O_DIRECT (or on old kernels) go to io-wq threads; thousands of them can spawn hundreds of kernel workers. Bound with IORING_REGISTER_IOWQ_MAX_WORKERS.
• submit_and_wait while the CQ is full and not reaped can deadlock your loop; always reap before waiting.
• Container default seccomp profiles and kernel.io_uring_disabled (6.6+) often disable it; keep an epoll fallback.
• Benchmarks with one connection lie: batching only pays off with many concurrent ops.

Variants and alternatives

In production

liburing (reference library), tokio-uring, glommio (Datadog), monoio (ByteDance), TigerBeetle, PostgreSQL 18 (io_method = io_uring), RocksDB MultiGet, QEMU block layer, libuv (file ops), Netty io_uring transport, Seastar.

Implementations

Sources

• Axboe, Efficient IO with io_uring, kernel.dk, 2019.
• Linux man-pages / liburing, io_uring(7), io_uring_setup(2), io_uring_enter(2). man · code
• Koczka, Learnings from kCTF VRP's 42 Linux kernel exploits submissions, Google Security Blog, 2023. post