HTTP range requests

Deep dive

How it works

In: Range header (plus optional If-Range) and the current representation length and validator · Out: 200 (whole), 206 with one range or multipart/byteranges, or 416 · State: none per request; the client keeps its partial bytes and the validator · Cost: parse O(header), body O(bytes served)

1. Client sends GET with Range: bytes=spec,spec,... (a spec is a-b, a- or -n), optionally If-Range: <strong ETag or Last-Modified>. Range applies to GET only.
2. Server sends a plain 200 if it ignores Range: unsupported, unit not bytes, invalid syntax (for example first > last), too many ranges, or an If-Range mismatch (evaluated first).
3. Per spec: a-b becomes a..min(b, len-1); a- becomes a..len-1; -n becomes max(len-n, 0)..len-1. Specs with a >= len or -0 are unsatisfiable and dropped.
4. If no spec is satisfiable: 416 with Content-Range: bytes */len.
5. The server MAY coalesce overlapping or nearby ranges (a multipart part costs about 80 bytes).
6. One range: 206, Content-Range: bytes first-last/len, Content-Length of the slice. Several: 206, Content-Type: multipart/byteranges; boundary=…, each part with its own Content-Type and Content-Range, and no top-level Content-Range.
7. The client MUST read every Content-Range: ranges may be coalesced or reordered. A 200 means Range was ignored and the body starts at byte 0: skip the bytes already held, or restart; never append.
8. Pieces may be combined only if they share the same strong validator.

Accept-Ranges: bytes advertises support (none says no); a client may try without it. An If-Range ETag must be strong (no W/); a Last-Modified date counts as strong only if the file cannot have changed twice in that second (RFC 9110 §8.8.2.2).

Code

/// What the server does with a `Range` header, given the representation length.
#[derive(Debug, PartialEq)]
pub enum Outcome {
    Full,                      // ignore Range: plain 200 with the whole body
    Partial(Vec<(u64, u64)>),  // 206: ascending, disjoint, inclusive (first, last)
    Unsatisfiable,             // 416 with `Content-Range: bytes */len`
}

/// Decimal digits only; saturates instead of overflowing (RFC 9110 §14.1.2).
fn num(s: &str) -> Option<u64> {
    if s.is_empty() || !s.bytes().all(|b| b.is_ascii_digit()) { return None; }
    Some(s.bytes().fold(0u64, |a, b| a.saturating_mul(10).saturating_add((b - b'0') as u64)))
}

/// Parse `bytes=a-b, c-, -n` and resolve against `len`. `max_specs` caps the list length.
pub fn resolve(header: &str, len: u64, max_specs: usize) -> Outcome {
    let Some((unit, set)) = header.split_once('=') else { return Outcome::Full };
    if !unit.trim().eq_ignore_ascii_case("bytes") { return Outcome::Full; } // unknown unit
    let mut out: Vec<(u64, u64)> = Vec::new();
    for (i, spec) in set.split(',').enumerate() {
        if i >= max_specs { return Outcome::Full; }            // too many ranges: ignore
        let Some((a, b)) = spec.trim().split_once('-') else { return Outcome::Full };
        let r = if a.is_empty() {                              // suffix-range: last n bytes
            match num(b) { None => return Outcome::Full, Some(0) => continue,
                           Some(n) => (len.saturating_sub(n), len.wrapping_sub(1)) }
        } else {
            let Some(first) = num(a) else { return Outcome::Full };
            let last = if b.is_empty() { u64::MAX } else {
                match num(b) { Some(l) if l >= first => l, _ => return Outcome::Full } // first > last: invalid
            };
            if first >= len { continue; }                      // unsatisfiable spec
            (first, last.min(len - 1))                         // clamp last to len-1
        };
        if len > 0 { out.push(r); }
    }
    if len == 0 { return Outcome::Full; }       // empty body: server MAY ignore Range
    if out.is_empty() { return Outcome::Unsatisfiable; }
    out.sort_unstable();                        // coalesce overlapping and adjacent ranges
    let mut m: Vec<(u64, u64)> = Vec::with_capacity(out.len());
    for (f, l) in out {
        match m.last_mut() {
            Some(p) if f <= p.1.saturating_add(1) => p.1 = p.1.max(l),
            _ => m.push((f, l)),
        }
    }
    Outcome::Partial(m)
}

pub fn content_range(first: u64, last: u64, len: u64) -> String { format!("bytes {first}-{last}/{len}") }
pub fn unsatisfied(len: u64) -> String { format!("bytes */{len}") }

fn main() {
    use Outcome::*;
    let p = |h, n| resolve(h, n, 16);
    assert_eq!(p("bytes=0-499", 10000), Partial(vec![(0, 499)]));
    assert_eq!(p("bytes=9500-", 10000), p("bytes=-500", 10000));     // open and suffix agree
    assert_eq!(p("bytes=-500", 100), Partial(vec![(0, 99)]));        // suffix longer than file
    assert_eq!(p("bytes=0-0,-1", 10000), Partial(vec![(0, 0), (9999, 9999)]));
    assert_eq!(p("bytes= 0-999, 4500-5499, -1000", 10000),
               Partial(vec![(0, 999), (4500, 5499), (9000, 9999)]));
    assert_eq!(p("bytes=500-600,601-999", 10000), Partial(vec![(500, 999)])); // adjacent merge
    assert_eq!(p("bytes=0-9,5-20,3-4,0-0", 100), Partial(vec![(0, 20)]));     // overlaps merge
    assert_eq!(p("bytes=900-5000", 1000), Partial(vec![(900, 999)]));        // last clamped
    assert_eq!(p("bytes=1000-", 1000), Unsatisfiable);                       // first >= len
    assert_eq!(p("bytes=-0", 1000), Unsatisfiable);                          // zero suffix
    assert_eq!(p("bytes=2000-3000,0-9", 1000), Partial(vec![(0, 9)]));       // one good spec suffices
    assert_eq!(p("bytes=5-4", 1000), Full);                                  // first > last: invalid
    assert_eq!(p("bytes=0-9", 0), Full);                                     // empty file
    assert_eq!(p("items=0-9", 1000), Full);                                  // unknown unit
    assert_eq!(p("bytes=a-b", 1000), Full);
    assert_eq!(p("bytes=0-1,2-3", 1000), Partial(vec![(0, 3)]));
    assert_eq!(resolve("bytes=0-1,5-6,9-9", 1000, 2), Full);                 // spec cap
    assert_eq!(p("bytes=99999999999999999999999-", 1000), Unsatisfiable);    // numeral overflow
    assert_eq!(content_range(42, 1233, 1234), "bytes 42-1233/1234");
    assert_eq!(unsatisfied(1234), "bytes */1234");
}

Complexity and constants

• Parsing is one pass over a header of tens of bytes; sorting and merging cost O(k log k) for k specs, so cap k (Apache MaxRanges default 200, Go ServeContent default 200, nginx max_ranges unlimited by default).
• Overhead: about 80 bytes per multipart part (RFC 9110 §15.3.7.2). The real cost of ranged reads is round trips: latency, not bandwidth, bounds many small reads.

Efficiency

• Cost: parsing and resolving a header with 1 range took 56-68 ns, 3 ranges 95-111 ns, 8 ranges 204-226 ns (the code above, rustc 1.99.1 -O, Apple M4 Pro, 5 million parses per header, two runs, measured by me; not profiled, likely dominated by the two small Vec allocations). Negligible next to serving: the bound is the I/O of the slice (one pread or sendfile per range) and, for clients, one round trip per request.
• Levers:
  • One covering range instead of many small ones — saves a round trip each and about 80 bytes per multipart part (RFC 9110 §15.3.7.2) — gaps small against round-trip time.
  • Speculative read of the file's tail (columnar footer, zip central directory) — one request instead of two — metadata at the end.
  • Parallel ranges on separate connections or HTTP/2 streams — throughput up to the link limit — large files, high latency.
  • sendfile for one range — no user-space copy (Zero-copy I/O) — static files.
  • If-Range — no second request when the file changed (RFC 9110 §13.1.5) — resumes.
• Naive → standard → state of the art:
  • Restart the whole download after a drop · O(file) bytes re-sent.
  • Resume with Range: bytes=N- and If-Range: "etag" · only the missing bytes plus one round trip.
  • Index-driven reads: read the footer or index first, merge nearby targets, fetch the merged ranges in parallel · bytes ≈ what is needed plus gaps, about 2 round trips deep.

Implementation notes

• Parse numerals with saturation: RFC 9110 §14.1.2 requires recipients to survive values beyond u64 without overflow.
• Resolve once, then stream: compute the final Content-Length (multipart: part headers, boundary lines and slices) before sending.
• Emit a strong ETag per file version so If-Range works.
• Range: bytes=0-0 is a cheap probe: 206 shows support, Content-Range the total length.
• Ask for Accept-Encoding: identity on ranged reads: a compressing server's encoded bytes may differ between requests.
• A resuming client checks that the 206's first byte equals the bytes it holds and that the validator is unchanged; a 416 with */len equal to the local size means the file is complete.

Pitfalls

• Appending a 200 body to a partial file because the server ignored Range: silent corruption.
• Resuming without If-Range or a validator check: the head from version 1 and the tail from version 2 form a file that never existed.
• Off by one: last is inclusive, so a slice has last - first + 1 bytes; Content-Length of a 206 is the slice, not the file.
• Weak ETag (W/"…") in If-Range: the strong comparison fails and you get the whole file.
• Unbounded range lists: thousands of overlapping 0- specs are a cheap request for gigabytes (RFC 9110 §17.15). Cap the count, coalesce, or ignore the header; Go also ignores a request whose ranges sum to more than the file size.
• Assuming a 206 holds the ranges you asked for in your order: it can be coalesced, reordered or partial.
• Caches: one that does not support partial content must not store a 206 (RFC 9111 §3.3); stored pieces combine only on the same strong validator.

Variants and alternatives

In production

nginx, Apache httpd and Go ServeContent serve ranges of static files; S3 GetObject takes one Range; browsers seek media with ranges; HLS EXT-X-BYTERANGE playlists (RFC 8216 §4.3.2.2) and DASH segment indexes address byte ranges (see Low-latency HLS and DASH); Parquet and zip readers fetch the tail first (Columnar file formats, Object storage); cloud-optimised GeoTIFF readers fetch only needed tiles; PDF viewers on linearised files; BAM/CRAM readers slice by index; pip's lazy wheel metadata fetch; curl -r.

Implementations

Sources

• Fielding, Nottingham, Reschke, RFC 9110: HTTP Semantics, 2022: §13.1.5 If-Range, §14 Range Requests, §15.3.7 206, §15.5.17 416, §17.15 DoS. rfc
• Fielding, Nottingham, Reschke, RFC 9111: HTTP Caching, 2022: §3.3 Storing Incomplete Responses, §3.4 Combining Partial Content. rfc
• Pantos, May, RFC 8216: HTTP Live Streaming, 2017: §4.3.2.2 EXT-X-BYTERANGE. rfc
• Mozilla, HTTP range requests, MDN Web Docs. docs
• Amazon Web Services, GetObject, Amazon S3 API Reference (single Range only). docs
• nginx, ngx_http_core_module: max_ranges. docs · Apache, core: MaxRanges. docs
• Go authors, net/http fs.go (ServeContent). code