Deep dive
How it works
Parts:
- Client — samples input, sends
(seq, inputs), renders snapshots. - Input queue — per client, ordered by sequence, jitter buffer of 1-2 inputs.
- Simulation — the single owner of state, advanced by a fixed dt.
- Snapshot builder — per-client delta of relevant entities plus
last_processed_input_seq.
- Fixed timestep: the server advances in ticks of dt = 1/tick_rate, using an accumulator so wall-clock jitter never changes the simulation step.
- Inputs: each client sends
(seq, tick_hint, buttons, aim)every client frame or tick. The server queues them per client in sequence order and drops duplicates. - Per tick: for each client, pop the next input (jitter buffer of 1-2 inputs). If none arrived, reuse the last one (the client will predict the same). Clamp and validate: speed, cooldowns, ammo, line of sight.
- Simulate the world one dt: physics, AI, projectiles, hits (with lag compensation).
- Snapshot: at the send rate (every tick or every N ticks), build a per-client snapshot of relevant entities plus
last_processed_input_seq, delta-encoded against that client's last acked snapshot. - Clients reconcile their predicted player with the snapshot and interpolate everyone else.
client --input seq 41,42,43--> [jitter buf] -> tick 1000: apply 42 -> simulate -> snapshot{tick 1000, ack 42}
client <-------------------------------------------------------- snapshot (delta vs acked baseline)
Code
// Minimal authoritative tick loop over UDP (std only). 60 Hz simulation, snapshots every tick.
use std::collections::{HashMap, VecDeque};
use std::net::{SocketAddr, UdpSocket};
use std::time::{Duration, Instant};
const DT: f32 = 1.0 / 60.0;
const MAX_SPEED: f32 = 6.0; // m/s, enforced on the server, never trusted from the client
#[derive(Clone, Copy, Default)]
struct Input { seq: u32, mx: i8, my: i8 } // movement axes in [-127, 127]
struct Player { id: u16, x: f32, y: f32, inputs: VecDeque<Input>, last: Input, last_seen: Instant }
fn parse_input(b: &[u8]) -> Option<Input> {
if b.len() != 7 || b[0] != 1 { return None; }
Some(Input { seq: u32::from_le_bytes(b[1..5].try_into().ok()?), mx: b[5] as i8, my: b[6] as i8 })
}
fn main() -> std::io::Result<()> {
let sock = UdpSocket::bind("0.0.0.0:27015")?;
sock.set_nonblocking(true)?;
let mut players: HashMap<SocketAddr, Player> = HashMap::new();
let (mut tick, mut next_id) = (0u32, 0u16);
let start = Instant::now();
let mut buf = [0u8; 1500];
loop {
// 1. Drain the socket: queue inputs in order, ignore stale or duplicate ones.
while let Ok((n, from)) = sock.recv_from(&mut buf) {
let Some(inp) = parse_input(&buf[..n]) else { continue };
let p = players.entry(from).or_insert_with(|| {
next_id += 1;
Player { id: next_id, x: 0.0, y: 0.0, inputs: VecDeque::new(), last: Input::default(), last_seen: Instant::now() }
});
p.last_seen = Instant::now();
let newest = p.inputs.back().map_or(p.last.seq, |i| i.seq);
if inp.seq > newest && p.inputs.len() < 8 { p.inputs.push_back(inp); } // cap = anti speed-hack
}
players.retain(|_, p| p.last_seen.elapsed() < Duration::from_secs(10));
// 2. Simulate one tick: at most one input per player per tick.
for p in players.values_mut() {
// Late client: if the buffer grew, consume 2 this tick to catch up, never more.
let take = if p.inputs.len() > 2 { 2 } else { 1 };
for _ in 0..take {
if let Some(i) = p.inputs.pop_front() { p.last = i; }
let (dx, dy) = (p.last.mx as f32 / 127.0, p.last.my as f32 / 127.0);
let len = (dx * dx + dy * dy).sqrt().max(1.0); // clamp diagonal speed
p.x += dx / len * MAX_SPEED * DT;
p.y += dy / len * MAX_SPEED * DT;
}
}
tick += 1;
// 3. Snapshot per client: tick, its last processed input, all player positions.
for (addr, me) in &players {
let mut s = vec![2u8];
s.extend_from_slice(&tick.to_le_bytes());
s.extend_from_slice(&me.last.seq.to_le_bytes()); // client replays inputs after this
for p in players.values() {
s.extend_from_slice(&p.id.to_le_bytes());
s.extend_from_slice(&p.x.to_le_bytes());
s.extend_from_slice(&p.y.to_le_bytes());
}
let _ = sock.send_to(&s, addr);
}
// 4. Sleep until the next tick boundary (absolute schedule: no drift).
let next = start + Duration::from_secs_f64(tick as f64 / 60.0);
if let Some(d) = next.checked_duration_since(Instant::now()) { std::thread::sleep(d); }
}
}
Complexity and constants
- Tick budget: 15.6 ms at 64 Hz, 7.8 ms at 128 Hz for receive + simulate + build all snapshots. A 100-player match must fit physics and per-client relevancy into that budget on 1-2 cores.
- Server CPU scales roughly linearly with tick rate; 128 Hz costs about 2x the hardware of 64 Hz.
- Upstream per client: 60 inputs/s × (28 B UDP/IP + order of 10-40 B payload) ≈ 20-30 kbit/s.
- Downstream: N entities × bytes per entity × send rate; e.g. 50 visible entities × 6 B (delta, quantized) × 30 Hz ≈ 72 kbit/s.
- Added latency seen by the player for authoritative actions: RTT + input buffer (1-2 ticks) + interpolation delay.
Efficiency
Cost: one tick must fit receive + simulate + snapshot build: 15.6 ms at 64 Hz, 7.8 ms at 128 Hz (1/rate); CPU-bound on one core per match. A single-threaded in-memory state machine reached 6 million orders/s on a 3 GHz Nehalem server (Fowler, LMAX). Upstream is order of 20-30 kbit/s per client; downstream is the dominant cost.
Levers:
- Delta-encode against the client's acked snapshot — order of 10s-100s of kbit/s per client with the next lever (see snapshot-delta-compression) — whenever state is large and mostly static.
- Send only what each client can see — cuts snapshot size and hidden-information leaks (interest-management) — large worlds.
- Repeat the last 2-3 inputs in every packet — a single loss costs nothing (order of) — lossy links.
- Time dilation: tell the client to run a few % faster or slower so inputs arrive just in time (Ford, GDC 2017) — to keep the input buffer at 1-2 ticks.
- Keep each simulation single-threaded and scale across matches/shards — no locks in the tick (Fowler) — many independent rooms or instruments.
Naive → standard → state of the art:
- Trust client-reported state, server relays · O(1) per message, no cheat protection
- Fixed-tick authoritative server with input queues and delta snapshots · 7.8-15.6 ms tick budget, 10s-100s kbit/s per client
- Single-thread event-sourced processor with replicated journal and snapshots · 6 million events/s, restart under a minute (Fowler, LMAX)
Implementation notes
- One process per match, several per core-pair on a host; orchestrate with Agones (Kubernetes), GameLift, or your own allocator. Keep the simulation single-threaded and deterministic per tick; parallelize across matches.
- Use the reliable UDP layer for events (kills, chat) and unreliable snapshots that are superseded by the next tick.
- Send inputs redundantly: each packet carries the last 3 inputs, so a single loss costs nothing.
- Time dilation (Overwatch): if a client's input buffer runs dry or grows, tell it to simulate slightly faster or slower (a few %) so its inputs arrive just in time.
- Shared simulation code between client and server (same language, same fixed dt) is what makes prediction match.
- Validate everything with server-side state: positions (speed, collision), fire rate, inventory, and the lag-compensated view time the client claims.
Pitfalls
- Variable timestep on the server: physics depends on frame time, predictions diverge, and "fast" servers behave differently.
- Accepting unlimited queued inputs: a client sending 2x inputs moves 2x faster (speed hack). Cap inputs consumed per tick.
- Trusting client-sent positions or hit claims "for responsiveness": the root of most shooter cheats.
- Information leaks: sending positions of hidden enemies enables wallhacks; cull by visibility (interest management).
- Single-threaded loop blocked by I/O (database writes, logging): ticks slip and everyone rubber-bands. Push side effects to another thread.
Variants and alternatives
In production
- Games: Quake/QuakeWorld lineage, Valve Source and Source 2 (CS2, Dota 2), Unreal Engine dedicated servers (Fortnite), Overwatch, Valorant, Apex Legends, Rocket League; hosting via Agones (Kubernetes), Amazon GameLift, Unity Multiplay.
- Collaborative software: Figma's server is "the ultimate authority on what the document looks like"; clients apply their own edits at once and discard server values that conflict with unacknowledged ones.
- Offline-first web apps: Replicache clients run mutators locally; the server re-executes them against the authoritative datastore and clients rebase pending mutations on its patch.
- Finance: LMAX's retail trading platform runs all business logic in one in-memory thread fed by a journaled input stream (6 million orders/s in Fowler's benchmark).
Implementations
| name | lang | repo | install | verdict | licence | health | as_of | notes |
|---|---|---|---|---|---|---|---|---|
| lightyear | Rust | cBournhonesque/lightyear | cargo add lightyear | — | — | — | — | Server-authoritative multiplayer for Bevy: replication, prediction, interpolation, rooms |
| bevy_replicon | Rust | simgine/bevy_replicon | cargo add bevy_replicon | — | — | — | — | Server-authoritative ECS replication for Bevy |
| Colyseus | TypeScript | colyseus/colyseus | npm install colyseus | — | — | — | — | Authoritative multiplayer framework for Node.js; game logic runs on the server |
| Nakama | Go | heroiclabs/nakama | n/a (service; Docker image) | — | — | — | — | Game server with authoritative match handlers |
| Agones | Go | agones-dev/agones | helm repo add agones https://agones.dev/chart/stable | — | — | — | — | Hosts and scales dedicated game servers on Kubernetes |
Sources
- Valve Developer Community, Source Multiplayer Networking. page
- Bernier, Latency Compensating Methods in Client/Server In-game Protocol Design and Optimization, GDC, 2001. page
- Ford, Overwatch Gameplay Architecture and Netcode, GDC, 2017. talk
- Fiedler, What Every Programmer Needs To Know About Game Networking and Fix Your Timestep!, gafferongames.com, 2010 and 2004. post · post
- Gambetta, Fast-Paced Multiplayer (series), gabrielgambetta.com. series
- Fowler, The LMAX Architecture, martinfowler.com, 2011. article
- Figma, How Figma's multiplayer technology works, 2019. post
- Replicache, How Replicache Works. docs