Process sandboxing

Deep dive

How it works

Parts:

• broker — trusted, long-lived; owns credentials, files, sockets; decides policy
• worker — untrusted, short-lived or pooled; minimal code; only reads requests, writes replies
• channel — pipe or socketpair with length-prefixed frames; the only door
• confinement set — rlimits, no_new_privs, namespaces, file rules, syscall filter, dropped identity

Steps:

1. Broker spawns the worker with a cleared environment, no inherited descriptors except the channel, and a fixed working directory.
2. Between fork and exec (or in the worker's first lines) apply limits: setrlimit for CPU, file size, open files, core dumps.
3. Drop identity: switch to an unprivileged uid (setuid) or enter a fresh user namespace; set PR_SET_NO_NEW_PRIVS so execve can never regain privilege.
4. Cut ambient reach: new mount, pid and net namespaces (empty network), a Landlock ruleset or chroot, and a syscall filter.
5. Do privileged startup first, then lock down; a handle leaked past lock-down can be abused (Chromium doc).
6. Broker sends one frame; worker replies with one frame; broker checks the length before allocating and enforces a deadline and an output cap.
7. Broker kills and reaps the worker, validates the reply as untrusted data.

 untrusted input
      |
 +----v-----+  frames over pipe   +--------------------------+
 |  broker  |<------------------->|  worker                  |
 | creds,   |  length <= cap      |  no env, no net, no write|
 | files,   |  deadline           |  rlimits + no_new_privs  |
 | policy   |                     |  ns / landlock / seccomp |
 +----------+  kill on violation  +--------------------------+

OS mechanisms (what each restricts)

• Linux user, mount, pid, net namespaces: separate view of ids, mounts, processes, interfaces; user namespaces give capabilities only inside the namespace (man7). See cgroups and namespaces.
• Landlock: an unprivileged process restricts itself with a ruleset of filesystem rules (and TCP, from ABI v4, and UDP rules in newer ABIs); enforcement needs no_new_privs or CAP_SYS_ADMIN; probe the ABI version and degrade gracefully (kernel docs).
• no_new_privs: cannot be unset, inherited by children, kept across execve (man7).
• rlimits: per-process caps; RLIMIT_NPROC counts all processes of the real user, not just the worker (getrlimit(2)); cgroups add memory, pids and CPU caps for a whole group.
• OpenBSD pledge: declare needed subsystems (about three dozen promises); anything else kills the process with an uncatchable SIGABRT; promises only shrink. unveil limits visible paths and should be locked after setup (pledge(2), unveil(2)).
• macOS App Sandbox: entitlement-based access limits per app, mandatory for Mac App Store apps (Apple); the command-line sandbox-exec is marked deprecated in its own man page.
• Windows: restricted token, job object, alternate desktop, integrity level, and AppContainer, which denies network unless a capability is granted (Chromium design doc, Microsoft).

Code

Portable broker/worker (macOS and Linux): cleared env, closed stdin after the request, rlimits, framed protocol, deadline, size cap. Namespaces, Landlock and seccomp are Linux-only (prose); add them in the same hook.

use std::io::{self, Read, Write};
use std::os::unix::process::CommandExt;
use std::process::{Command, Stdio};
use std::sync::mpsc;
use std::time::{Duration, Instant};

const MAX_FRAME: usize = 1 << 20;

fn write_frame(w: &mut impl Write, body: &[u8]) -> io::Result<()> {
    w.write_all(&(body.len() as u32).to_be_bytes())?;
    w.write_all(body)?;
    w.flush()
}

// check the length BEFORE allocating
fn read_frame(r: &mut impl Read) -> io::Result<Vec<u8>> {
    let mut h = [0u8; 4];
    r.read_exact(&mut h)?;
    let n = u32::from_be_bytes(h) as usize;
    if n > MAX_FRAME {
        return Err(io::Error::new(io::ErrorKind::InvalidData, "frame too large"));
    }
    let mut b = vec![0u8; n];
    r.read_exact(&mut b)?;
    Ok(b)
}

fn setlim(res: libc::c_int, v: u64) -> io::Result<()> {
    let l = libc::rlimit { rlim_cur: v as libc::rlim_t, rlim_max: v as libc::rlim_t };
    if unsafe { libc::setrlimit(res, &l) } != 0 { return Err(io::Error::last_os_error()); }
    Ok(())
}

/// Broker side: run one request in a fresh, confined worker process.
fn call(request: &[u8], timeout: Duration) -> io::Result<Vec<u8>> {
    let mut cmd = Command::new(std::env::current_exe()?);
    cmd.arg("worker")
        .env_clear()
        .stdin(Stdio::piped()).stdout(Stdio::piped()).stderr(Stdio::null());
    unsafe {
        cmd.pre_exec(|| {
            setlim(libc::RLIMIT_CPU, 2)?;  // CPU seconds
            setlim(libc::RLIMIT_CORE, 0)?; // no core dumps
            setlim(libc::RLIMIT_FSIZE, 0)?;// cannot create files with content
            setlim(libc::RLIMIT_NOFILE, 16)
        });
    }
    let mut child = cmd.spawn()?;
    let mut stdin = child.stdin.take().unwrap();
    let mut stdout = child.stdout.take().unwrap();
    write_frame(&mut stdin, request)?;
    drop(stdin);                           // EOF: the worker's stdin is closed after the request
    let (tx, rx) = mpsc::channel();
    std::thread::spawn(move || { let _ = tx.send(read_frame(&mut stdout)); });
    let out = match rx.recv_timeout(timeout) {
        Ok(Ok(b)) => Ok(b),
        Ok(Err(e)) => Err(e),
        Err(_) => Err(io::Error::new(io::ErrorKind::TimedOut, "worker timed out")),
    };
    let _ = child.kill();
    let _ = child.wait();
    out
}

//// Worker side: the untrusted parser.
fn worker() -> io::Result<()> {
    let req = read_frame(&mut io::stdin().lock())?;
    let reply: Vec<u8> = match req.first() {
        Some(b'u') => req[1..].to_ascii_uppercase(),
        Some(b'e') => std::env::vars().count().to_string().into_bytes(),
        Some(b's') => loop { std::hint::spin_loop() },
        Some(b'f') => {
            io::stdout().write_all(&u32::MAX.to_be_bytes())?; return Ok(());
        }
        _ => b"?".to_vec(),
    };
    write_frame(&mut io::stdout().lock(), &reply)
}

fn main() {
    if std::env::args().nth(1).as_deref() == Some("worker") {
        std::process::exit(worker().is_err() as i32);
    }
    unsafe { std::env::set_var("SECRET_TOKEN", "do-not-leak") }; // broker secret
    let t = Duration::from_millis(1500);
    assert_eq!(call(b"uhello", t).unwrap(), b"HELLO");
    assert_eq!(call(b"e", t).unwrap(), b"0");                    // env cleared
    let e = call(b"s", Duration::from_millis(300)).unwrap_err();
    assert_eq!(e.kind(), io::ErrorKind::TimedOut);               // timeout kills it
    let e = call(b"f", t).unwrap_err();
    assert_eq!(e.kind(), io::ErrorKind::InvalidData);            // size cap
    let n = 200;
    let t0 = Instant::now();
    for _ in 0..n { assert_eq!(call(b"uabc", t).unwrap(), b"ABC"); }
    println!("spawn+roundtrip: {:?} per call (n={n})", t0.elapsed() / n);
    println!("ok");
}

Complexity and constants

• Memory: one process image per live worker plus a broker frame buffer capped at MAX_FRAME.
• Syscalls per request: fork/exec, a handful of setrlimit, 2 pipe writes and reads, kill, wait; no per-byte syscall beyond the frame.
• Measured on this Mac: 3.6-4.2 ms per spawn plus one tiny round trip (200 calls, release build, three runs). Linux numbers differ; measure your own.
• Namespace and mount setup add work per spawn.

Efficiency

Cost: spawn-bound, 3.6-4.2 ms per fresh worker here; once a worker is warm, one message is two pipe copies and two context switches, with CPU idle in between.

Levers:

• Reuse the worker for many requests — removes the per-call spawn — (measured above, spawn dominates) — when requests are from one trust domain; recycle after N requests or on any anomaly.
• Pre-start a pool of already-confined workers — moves spawn off the request path — when latency matters.
• Lock down once, early — the filter then costs nothing at setup per request — (Chromium: restrictions applied after startup, doc) — long-lived workers.
• Pass big inputs as a file descriptor or shared memory, not through the pipe — avoids copying megabytes — when payloads are large.

Naive → standard → state of the art:

• Parse in the main process · zero cost, full blast radius
• Fresh sandboxed process per request · about 4 ms here, strongest isolation per input
• Pooled, pre-confined workers with fd passing · about the cost of a pipe round trip, recycled on schedule

Implementation notes

• Close every inherited descriptor except the channel; set close-on-exec by default.
• In a multi-threaded broker only async-signal-safe calls belong in pre_exec; otherwise use a small launcher.
• Kill by process group; on Linux a pid namespace makes the whole tree disappear with the worker.
• Treat replies as untrusted: bound sizes, validate structure.

Pitfalls

• Sandbox applied too late: secrets, descriptors or directory handles opened earlier stay usable.
• Broker that proxies too much (generic "open any path for me") recreates the privilege it removed.
• Check-then-open races in the broker.s path checks.
• Environment variables, argv, shared /tmp or shared memory bridging the boundary.
• Limits that do not mean what you think: RLIMIT_NPROC is per user; use cgroups or job objects for groups.
• Unbounded length prefixes: allocation chosen by the attacker.
• Silent downgrade on older kernels (missing Landlock ABI, user namespaces disabled): fail closed.

Variants and alternatives

In production

• Browsers: Chromium separates a privileged broker (the browser process) from sandboxed renderers, on Windows with token, job and desktop restrictions; Linux and macOS variants are documented separately.
• Remote access: OpenSSH privilege separation runs the pre-authentication phase in an unprivileged child chrooted to an empty directory (sshd(8), Provos et al.).
• Operating systems and services: systemd units add NoNewPrivileges, ProtectSystem, PrivateTmp and DynamicUser (hardening list); OpenBSD programs pledge themselves.
• Desktop apps and CI: bubblewrap builds unprivileged sandboxes for Flatpak-style apps and build jobs; code-execution and agent tools wrap interpreters in the same mechanisms (see agent sandboxing).

Implementations

Sources

• Chromium project, Sandbox design document. doc
• Kernel documentation, Landlock: unprivileged access control. doc
• Linux man-pages: namespaces(7), user_namespaces(7), getrlimit(2), PR_SET_NO_NEW_PRIVS(2const). man7
• OpenBSD manual: pledge(2), unveil(2), sshd(8). pledge
• Provos, Friedl, Honeyman, Preventing Privilege Escalation, USENIX Security 2003. paper
• Apple, App Sandbox. doc · Microsoft, AppContainer isolation. doc