Hot reloading

Deep dive

How it works

Parts:

• Host — owns all state and the single indirection point (table pointer); never reloaded.
• Module — versioned bundle of entry points (function table) plus the state layout version it expects; replaceable.
• Loader — builds or maps the new module (compile, dlopen of a uniquely named file, class bytes, patch object).
• Migrator — pure function from state layout N to N+1, per module.
• Safe point — a moment with no activation of old code: between frames, requests, messages or syscalls.

1. Edit; the build produces a new module version. Rebuild only the changed unit (linking and relocation still has to resolve its imports).
2. Loader checks compatibility: ABI/table shape, declared state version. Reject early, keep running the old code.
3. Wait for quiescence: finish the in-flight request or frame (lock, frame boundary, per-thread epoch, kernel stack check).
4. Migrate the state on a copy (or with undo). If it fails, abort; nothing changed.
5. Store the new table pointer with release ordering. Callers that load it afterwards run new code; callers already inside old code finish there.
6. Retire the old module once nobody runs it (no threads, no return addresses, no callbacks registered). Unload last; leaking old code is the safe default.

 callers --load--> [ table ptr ] --> Module v1 { handle, migrate }   (in flight: finishes in v1)
                        |  swap at safe point                
                        +---------> Module v2 { handle', migrate' }  new calls
 state (host heap): [count] --migrate 1->2--> [count, sum=0]

The indirection varies: PLT/GOT-style trampolines, vtables, a patched function prologue (ftrace slot in Linux), JavaScript module re-evaluation, or the VM's own dispatch (Erlang external calls, JVM method entries).

Code

Host with a swappable function table, migration v1 to v2, an ABI check, and a concurrent reload. Modules are 'static structs of fn pointers standing in for a loaded library; the mutex is the safe point.

use std::sync::atomic::{AtomicPtr, Ordering};
use std::sync::Mutex;

const ABI: u32 = 1; // layout of Module itself; change = not reloadable

struct Module {
    abi: u32,
    code_version: u32,
    state_version: u32, // layout of the state words this code expects
    handle: fn(&mut Vec<u64>, u64) -> u64,
    /// Convert state from layout `from` to ours in place; false = cannot.
    migrate: fn(&mut Vec<u64>, u32) -> bool,
}

// v1 state layout: [count]
fn h1(s: &mut Vec<u64>, req: u64) -> u64 { s[0] += 1; req + 1 }
fn m1(_: &mut Vec<u64>, from: u32) -> bool { from == 1 }
static V1: Module = Module { abi: ABI, code_version: 1, state_version: 1, handle: h1, migrate: m1 };

// v2 state layout: [count, sum]
fn h2(s: &mut Vec<u64>, req: u64) -> u64 { s[0] += 1; s[1] += req; req * 2 }
fn m2(s: &mut Vec<u64>, from: u32) -> bool {
    match from {
        1 => { s.push(0); true } // new field gets a default
        2 => true,
        _ => false,
    }
}
static V2: Module = Module { abi: ABI, code_version: 2, state_version: 2, handle: h2, migrate: m2 };
static BAD: Module = Module { abi: 99, code_version: 3, state_version: 2, handle: h2, migrate: m2 };

struct Host {
    table: AtomicPtr<Module>,      // the one indirection point
    state: Mutex<(u32, Vec<u64>)>, // (state_version, words): owned by the host, not the module
}

impl Host {
    fn new(m: &'static Module) -> Host {
        let s = vec![0; m.state_version as usize]; // toy init: one word per version
        Host { table: AtomicPtr::new(m as *const _ as *mut _), state: Mutex::new((m.state_version, s)) }
    }
    /// One request. The lock is the safe point: reload cannot run mid-request.
    fn serve(&self, req: u64) -> u64 {
        let mut st = self.state.lock().unwrap();
        // SAFETY: only ever points at 'static Modules (old ones are never freed here).
        let m = unsafe { &*self.table.load(Ordering::Acquire) };
        (m.handle)(&mut st.1, req)
    }
    fn reload(&self, new: &'static Module) -> Result<(), String> {
        if new.abi != ABI { return Err(format!("abi {} != {}", new.abi, ABI)); }
        let mut st = self.state.lock().unwrap(); // wait for quiescence
        let mut next = st.1.clone();             // migrate a copy: failure leaves old state intact
        if !(new.migrate)(&mut next, st.0) { return Err("cannot migrate state".into()); }
        *st = (new.state_version, next);
        self.table.store(new as *const _ as *mut _, Ordering::Release); // atomic switch
        Ok(())
    }
    fn code_version(&self) -> u32 { unsafe { (*self.table.load(Ordering::Acquire)).code_version } }
}

fn main() {
    let h = Host::new(&V1);
    assert_eq!(h.serve(10), 11);
    assert_eq!(h.serve(20), 21);
    assert!(h.reload(&BAD).is_err());                   // incompatible module rejected
    assert_eq!(h.code_version(), 1);                    // still on v1
    assert_eq!(h.serve(5), 6);
    h.reload(&V2).unwrap();
    assert_eq!(h.code_version(), 2);
    assert_eq!(h.state.lock().unwrap().1, vec![3, 0]);  // count survived, sum defaulted
    assert_eq!(h.serve(7), 14);                         // new call uses new code
    assert_eq!(h.state.lock().unwrap().1, vec![4, 7]);
}

The lock makes quiescence trivial but serializes requests; a real host would use per-thread safe points or RCU-style grace periods.

Complexity and constants

• Call: one extra load of the table pointer and an indirect jump; an indirect call cannot be inlined across the boundary, which usually costs more than the load itself.
• Swap: O(1). Migration: O(state size) and runs at the safe point, so it sets the pause. Prefer lazy, per-object migration (version tag on each record) over a stop-the-world rewrite.
• Reload latency is dominated by the build, not the swap.

Efficiency

• Cost: measured by me on an Apple M4 Pro, release build, 200M calls of a tiny dependent function: about 1.6 ns direct, about 2.5-2.9 ns through an AtomicPtr load plus indirect call, so roughly +1 ns per call (order of; in a tight dependent chain, not a mix). The swap is one store. Bound: build time and migration size, not dispatch.
• Levers:
  • Reload at coarse boundaries (per frame or request, not per call) — dispatch cost stays off the inner loop (measured above) — hot loops live inside one module.
  • Rebuild only the changed module (separate dynamic library or one compilation unit) — avoids a full relink and restart (design rationale, no number) — large codebases.
  • Keep state outside the module in plain-old-data layouts — migration is a copy or a field default, no pointer fix-up — game state, request contexts.
  • Lazy migration with a version tag per record — no pause proportional to state size — big in-memory stores.
• Naive → standard → state of the art:
  • Kill and restart, rebuild everything · seconds to minutes plus lost state.
  • Reload a dynamic library at a frame or request boundary, state in the host · one incremental link plus O(state) pause.
  • Function-level patching of the changed code with consistent per-task switching (kernel livepatch) or VM-level redefinition with state-preserving UI frameworks · no pause visible to users, restricted to compatible changes.

Implementation notes

• Dynamic library reload: copy the library to a fresh versioned name before dlopen (the OS caches by path and the build tool may hold the file open on Windows), dlclose only when no thread is inside. Statics and thread-locals inside the library are reinitialized: do not keep state there.
• Cross the boundary with repr(C) structs and fixed-width integers; never share Rust types or allocators between separately built modules.
• Memory pages of freshly mapped code need the usual write/execute handling, see executable memory.
• Function pointers saved elsewhere (callbacks, vtables, threads' return addresses) keep pointing at old code: register callbacks through the host table, or keep old code mapped.

Pitfalls

• State holding pointers into the old module (fn pointers, vtables, static data, trait objects) becomes dangling after unload.
• Silent layout mismatch: new code reads old bytes. Always carry a layout version and reject unknown ones.
• Reloading mid-call: a thread blocked in old code resumes in unmapped memory after dlclose.
• Migration is irreversible: migrate a copy, or keep an undo, so a failed reload can fall back.
• Erlang: a third load purges the old code and kills processes still running it.

Variants and alternatives

In production

• Telecom and backends: Erlang/OTP upgrades running switches and message brokers.
• Web frontends: webpack and Vite dev servers push module updates to the browser.
• Mobile apps: Flutter stateful hot reload on devices and emulators.
• Games: engine editors and C++ live-patching tools iterate on gameplay code in a running level.
• OS kernels: Linux livepatch applies security fixes without reboot.

Implementations

Sources

• Erlang/OTP docs, Code Loading (two versions, purge). docs
• Erlang/OTP docs, gen_server (code_change/3). docs
• Oracle, JVM Tool Interface, RedefineClasses restrictions. spec
• Linux kernel docs, Livepatch (ftrace, consistency model). docs
• Vite, HMR API. docs
• Flutter, Hot reload. docs
• Microsoft, Hot Reload (Visual Studio, .NET). docs
• Live++ product site. site