Context switching

Deep dive

How it works

In: a running task A, a reason to leave the CPU (blocking call, timer tick, preemption) and a runnable task B · Out: B running with its own registers, stack and address space; A resumable later · State: per task: saved registers, kernel stack pointer, FPU/SIMD save area; per process: page-table root · Cost: O(1) work; measured on this Mac (Apple M4 Pro) about 1.8-2.0 µs per hop of a thread ping-pong, wakeup included (see Code).

1. Enter the kernel by syscall, interrupt or timer tick (syscalls, interrupts). With page-table isolation on x86 the entry also switches CR3 to the kernel tables, "on the order of a hundred cycles" per move (PTI).
2. Save the old task: callee-saved registers and stack pointer; the calling convention already put the rest on the stack. Linux x86-64 pushes r15-r12, rbx, rbp and the return address (inactive_task_frame); arm64 stores x19-x28, fp, sp, lr (cpu_switch_to).
3. Pick the next task (scheduler).
4. Switch address space if the next task has a different mm. Linux context_switch() calls switch_mm_irqs_off() only when going to a user task; kernel threads borrow the previous mm (core.c). x86 writes CR3 with a PCID so entries survive; arm64 writes TTBR0_EL1 with an ASID and flushes only when the ASID generation rolls over (arm64 context.c).
5. Switch kernel stack and registers: load the new stack pointer and callee-saved registers, then ret into the new task.
6. FPU/SIMD state: x86 Linux saves the old task's state at switch-out and defers the restore to the return to user mode, skipping it if the same CPU still holds that task's registers (sched.h).
7. Return to user; with PTI, CR3 switches back to the user tables.

User-space switching drops steps 1, 3 (the runtime schedules), 4 and most of 6: Go's gobuf holds just sp, pc, g, ctxt, lr, bp (runtime2.go). Boost.Context states a thread switch "can cost more than thousand CPU cycles" on x86 against "few CPU cycles" for a user-space transfer (docs).

Code

Part 1 models the bookkeeping (registers survive, the root loads only on a process change). Part 2 measures the real OS: two threads hand a message back and forth over channels (each hop is a wakeup plus a switch), against the same channel traffic on one thread.

use std::sync::mpsc::channel;
use std::{thread, time::Instant};

#[derive(Clone, Copy, Default, PartialEq, Debug)]
struct Regs { callee_saved: [u64; 4], sp: u64, pc: u64 }
struct Task { saved: Regs, mm: u32 } // mm = page-table root id
struct Cpu { regs: Regs, mm: u32, cur: usize, root_loads: u32 }

impl Cpu {
    fn switch(&mut self, tasks: &mut [Task]) {
        tasks[self.cur].saved = self.regs;        // 1. save old registers and sp
        self.cur = (self.cur + 1) % tasks.len();  // 2. scheduler: round robin
        let next = &tasks[self.cur];
        if next.mm != self.mm {                   // 3. only a different process
            self.mm = next.mm;                    //    loads a new page-table root
            self.root_loads += 1;
        }
        self.regs = next.saved;                   // 4. restore, resume at pc
    }
}

fn model_test() {
    let mut tasks: Vec<Task> = (0..3u64).map(|i| Task {
        saved: Regs { callee_saved: [i; 4], sp: 0x1000 * (i + 1), pc: i },
        mm: if i < 2 { 7 } else { 8 } }).collect(); // tasks 0,1 share a process
    let mut cpu = Cpu { regs: tasks[0].saved, mm: 7, cur: 0, root_loads: 0 };
    let mut work = [0u64; 3];
    for _ in 0..300 {
        let w = cpu.cur;
        assert_eq!(cpu.regs.callee_saved, [work[w] + w as u64; 4]); // survived the others
        work[w] += 1;
        cpu.regs.callee_saved = [work[w] + w as u64; 4];
        cpu.switch(&mut tasks);
    }
    assert_eq!(cpu.root_loads, 200); // 2 of every 3 switches cross processes
}

fn pingpong(n: u32) -> f64 { // seconds per round trip, two threads
    let (to_b, from_a) = channel::<u32>();
    let (to_a, from_b) = channel::<u32>();
    let b = thread::spawn(move || {
        while let Ok(v) = from_a.recv() { if to_a.send(v + 1).is_err() { break; } }
    });
    let t = Instant::now();
    let mut v = 0;
    for _ in 0..n { to_b.send(v).unwrap(); v = from_b.recv().unwrap(); }
    let dt = t.elapsed().as_secs_f64();
    assert_eq!(v, n);
    drop(to_b); b.join().unwrap();
    dt / n as f64
}

fn same_thread(n: u32) -> f64 { // same channel traffic, no second thread
    let (tx, rx) = channel();
    let t = Instant::now();
    let mut v = 0;
    for _ in 0..n { tx.send(v).unwrap(); v = rx.recv().unwrap() + 1; }
    assert_eq!(v, n);
    t.elapsed().as_secs_f64() / n as f64
}

fn summary(mut xs: Vec<f64>) -> (f64, f64) { // (min, median) in ns
    xs.sort_by(|a, b| a.partial_cmp(b).unwrap());
    (xs[0] * 1e9, xs[xs.len() / 2] * 1e9)
}

fn main() {
    model_test();
    let n = 200_000;
    let (pmin, pmed) = summary((0..9).map(|_| pingpong(n)).collect());
    let (smin, smed) = summary((0..9).map(|_| same_thread(n)).collect());
    println!("two threads: min {pmin:.0} ns, median {pmed:.0} ns per round trip");
    println!("one thread : min {smin:.0} ns, median {smed:.0} ns per round trip");
}

Complexity and constants

Constant work per switch: a handful of registers, one root load. Fixed sizes: the AVX-512 components alone are 64 B (opmask) + 512 B (ZMM_Hi256) + 1024 B (Hi16_ZMM) = 1600 B per task, from the struct definitions in fpu/types.h. Linux disables global pages for kernel structures under PTI; the doc says the loss never exceeds 1 %.

Measured on this Mac (Apple M4 Pro), macOS, release build, repeated runs of 9 x 200,000 round trips each: two threads min 3.5-3.7 µs and median 3.7-4.0 µs per round trip (two hops); one thread 7 ns. A hop therefore costs about 1.8-2.0 µs. This includes the channel's wake-up call, the scheduler and possibly a cross-core wake (not controlled; macOS has no thread pinning). It is an upper bound on the switch itself, and macOS is not Linux: do not compare against Linux numbers.

Efficiency

• Cost: direct cost is a fixed store/load sequence (not measured separately here); the observable cost is the wake-up path, 1.8-2.0 µs per hop here, plus cache and TLB refill afterwards (studied in Li, Ding, Shen, ExpCS 2007; findings not verified here). Bound: latency and memory, not bandwidth.
• Levers:
  • Do not switch: one thread per core with non-blocking tasks — Seastar states one thread per core matters because context switches are slow — work that can be written non-blocking.
  • Switch in user space — "few cycles" against "more than a thousand" (Boost.Context) — many cooperative tasks.
  • Tagged TLB (PCID/ASID) — no full flush on a root change (PTI) — default on supporting hardware.
  • Pin threads — fewer migrations, warmer caches (no figure here; pinning) — latency-critical or HPC loops.
  • Batch handoffs — one wake-up per batch, not per item (not measured here) — queue consumers.
• Naive → standard → state of the art: thread or process per request with blocking I/O · two kernel switches per request, each ~2 µs of wake-up path here; thread pool plus event loop · switches only when idle; shared-nothing thread per core with cooperative tasks · no kernel switch on the fast path.

Implementation notes

• Count switches with getrusage(2): ru_nvcsw = voluntary (task gave up the CPU, usually to wait for a resource), ru_nivcsw = involuntary (higher-priority task became runnable, or time slice used up) (man page). Linux also exposes voluntary_ctxt_switches and nonvoluntary_ctxt_switches in /proc/<pid>/status (proc_pid_status(5)). Many voluntary switches point to blocking I/O or locks; many involuntary ones to CPU oversubscription.
• Keep a task on one core where possible so it finds warm lines when it returns.
• Do not block the audio callback or a game's render thread: a blocking call is a voluntary switch whose return time you do not control.

Pitfalls

• Reading the ping-pong number as "the switch cost": it contains wake-up and scheduling latency.
• Lazy FPU restore: Linux made eager the default in 4.6 for speed (clearing CR0.TS was slow) and removed lazy mode later (commit). CVE-2018-3665 describes lazy FP state restore on some Intel cores as a way for a local process to infer another process's data through a speculative side channel (NVD); see side channels.
• Large vector state: AMX tile data is not allocated per task until its first use traps in the kernel (xstate doc), so the first use is slow.
• Oversubscription (more runnable threads than cores) turns every wake-up into a preemption.

Variants and alternatives

In production

• OS schedulers (Linux, macOS, Windows), Go's runtime, Seastar and the databases built on it, HPC jobs that pin ranks to cores, DAWs and game engines with dedicated audio threads.

Implementations

Sources

• Li, Ding, Shen, Quantifying the cost of context switch, ExpCS 2007. paper
• Linux docs and source cited inline: PTI, XSTATE, kernel/sched/core.c, arch/*/ switch code; man pages getrusage(2), proc_pid_status(5); CVE-2018-3665; Seastar tutorial.