Adaptive bitrate streaming

Deep dive

How it works

Parts:

• Manifest — variants (bitrate, resolution, codecs) and segment URLs: HLS playlists, DASH MPD (manifests).
• Ladder — renditions with aligned segments (ladder); BANDWIDTH must be honest.
• Segment server / CDN — static files, identical for all clients, so cacheable.
• Estimator — download rates into one number (harmonic mean, EWMA, sliding percentile).
• Buffer — seconds downloaded but not yet played.
• ABR rule — (estimate, buffer, last rung) to next rung, plus startup, switch limits, abandon check.

Operations

1. Fetch the manifest; start low or from a prior bandwidth guess.
2. Download segment k at rung i; record bytes and time. Playback starts once a small buffer exists.
3. Update the estimate; buffer += segment length − download time.
4. Run the rule: throughput (highest rung ≤ safety × estimate), buffer (BBA rate map; BOLA utility score) or hybrid/MPC (predict, then search the next few segments for best QoE).
5. Hysteresis: delay up-switches until the buffer is healthy; allow fast down-switches.
6. Buffer full: wait. Download far too slow: abandon and refetch lower.
7. Repeat; on seek or stall re-enter startup.

 manifest ──> ladder ─┐
 CDN cache ──segments─┼─> download ─> throughput samples ─> estimator ─┐
                      │       │                                        v
                      │       └─> buffer (s) ───────────────────> ABR rule ─> next rung
                      └────────────── request segment k+1 <──────────────┘

Rules in more detail.

• Estimate. A harmonic mean of recent samples is dominated by slow ones (FESTIVE; recalled from the paper, not re-read). dash.js defaults to EWMA with half-lives 3 s and 8 s; hls.js and Shaka take the minimum of a fast and a slow EWMA (smoothing): drop quickly, rise slowly. Media3 uses a weighted sliding percentile.
• BBA (Huang et al.): below a reservoir (90 s of a 240 s buffer there) pick the lowest rate; above it, rate rises with buffer across a cushion. Throughput only at startup, when the buffer is empty.
• BOLA (Spiteri et al.): with v_m = ln(S_m/S_1), pick the rung maximizing (V·(v_m + γp) − Q)/S_m, Q the buffer in segments; no throughput prediction. dash.js ships it with startup and steady states and can alternate it with the throughput rule.
• MPC (Yin et al.): combines throughput prediction and buffer state, choosing the bitrate sequence that maximizes QoE over a short horizon (MPC).
• Learned (Pensieve): a reinforcement-learned network replaces the rule; 12-25 % better average QoE than prior schemes in their tests.

QoE trade-off. Utilities reward quality (BOLA: log of segment size) and penalize rebuffering, quality changes and startup delay. Weights set behavior: heavy rebuffer penalty gives conservative rungs; heavy switch penalty gives sticky ones.

Low latency. LL-HLS (RFC 8216bis draft) adds partial segments, blocking playlist reload and preload hints; CMAF chunked transfer does it for DASH (details). Chunks arrive at the encoder's pace with idle gaps, so bytes/time measures that pace, not the link; Bentaleb et al. report "grossly inaccurate" measurements and measure over sliding windows of chunk bursts (ACTE).

Code

Rust, std only: a player over a synthetic link (6000, then 500, 3000, 8000 kbit/s phases of 120 s, ±20 % noise per second), 5-rung ladder, 4 s segments, 24 s buffer cap. Policies: harmonic mean of 5 samples × 0.8, and BOLA-BASIC.

const LADDER: [f64; 5] = [300., 750., 1200., 2400., 4800.]; // kbit/s
const SEG: f64 = 4.0; // segment duration, s
const MAX_BUF: f64 = 24.0; // player buffer cap, s

/// Synthetic link: kbit/s at time t. Step changes + deterministic +-20 % noise per second.
fn bandwidth(t: f64, seed: u64) -> f64 {
    let base = match t as u32 % 480 {
        0..=119 => 6000., 120..=239 => 500., 240..=359 => 3000., _ => 8000.,
    };
    let mut z = (t as u64).wrapping_add(seed).wrapping_mul(0x9E37_79B9_7F4A_7C15);
    z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9);
    base * (0.8 + 0.4 * ((z >> 40) as f64 / (1u64 << 24) as f64))
}

enum Policy { Throughput, Bola }

/// Pick a rung index from the player state. `hist` = recent segment throughputs (kbit/s).
fn choose(p: &Policy, hist: &[f64], buf: f64) -> usize {
    match p {
        Policy::Throughput => {
            if hist.is_empty() { return 0; }
            let w = &hist[hist.len().saturating_sub(5)..];
            let est = 0.8 * w.len() as f64 / w.iter().map(|x| 1.0 / x).sum::<f64>(); // harmonic mean x safety
            (0..LADDER.len()).rev().find(|&i| LADDER[i] <= est).unwrap_or(0)
        }
        Policy::Bola => {
            // BOLA-BASIC: maximize (V*(v_m + gp) - Q) / S_m, v_m = ln(S_m/S_1), Q in segments.
            let gp = 5.0;
            let v = (MAX_BUF / SEG - 1.0) / ((LADDER[4] / LADDER[0]).ln() + gp);
            let score = |i: usize| (v * ((LADDER[i] / LADDER[0]).ln() + gp) - buf / SEG) / (LADDER[i] * SEG);
            (0..LADDER.len()).max_by(|&a, &b| score(a).total_cmp(&score(b))).unwrap()
        }
    }
}

#[derive(Debug, Default)]
struct Stats { rebuf_s: f64, mean_kbps: f64, switches: u32, max_kbps: f64 }

fn play(p: &Policy, segments: usize, seed: u64) -> Stats {
    let (mut t, mut buf, mut playing) = (0.0, 0.0, false);
    let (mut hist, mut last, mut s) = (Vec::new(), usize::MAX, Stats::default());
    for _ in 0..segments {
        let i = choose(p, &hist, buf);
        let mut bits = LADDER[i] * SEG; // kbit still to download
        let t0 = t;
        while bits > 0.0 { bits -= bandwidth(t, seed) * 0.05; t += 0.05; } // integrate the link, 50 ms steps
        let dl = t - t0;
        hist.push(LADDER[i] * SEG / dl);
        if playing { if buf >= dl { buf -= dl } else { s.rebuf_s += dl - buf; buf = 0.0 } }
        buf += SEG;
        playing = true; // start after the first segment
        if buf > MAX_BUF { t += buf - MAX_BUF; buf = MAX_BUF; } // buffer full: idle, playback drains it
        if last != usize::MAX && last != i { s.switches += 1; }
        last = i;
        s.mean_kbps += LADDER[i] / segments as f64;
        s.max_kbps = s.max_kbps.max(LADDER[i]);
    }
    s
}

fn main() {
    let mut res = Vec::new();
    for (name, p) in [("throughput", Policy::Throughput), ("bola", Policy::Bola)] {
        let s = play(&p, 120, 1); // 120 segments = 480 s of video: covers every trace phase
        println!("{name:10} rebuffer {:5.1} s  mean {:6.0} kbit/s  switches {:3}", s.rebuf_s, s.mean_kbps, s.switches);
        assert!(s.max_kbps <= LADDER[4]); // never above the top rung
        res.push(s);
    }
    assert!(res[1].rebuf_s > res[0].rebuf_s); // measured here: after a 12x drop the buffer rule stalls MORE
    // mean over many noise seeds, and speed
    let n = 2000;
    for (name, p) in [("throughput", Policy::Throughput), ("bola", Policy::Bola)] {
        let t = std::time::Instant::now();
        let r: Vec<Stats> = (0..n).map(|k| play(&p, 120, k)).collect();
        let e = t.elapsed().as_secs_f64();
        let m = |f: fn(&Stats) -> f64| r.iter().map(f).sum::<f64>() / n as f64;
        println!("{name:10} {n} seeds: rebuffer {:.1} s, mean {:.0} kbit/s, switches {:.1}; {:.0} segments/s simulated",
            m(|s| s.rebuf_s), m(|s| s.mean_kbps), m(|s| s.switches as f64), (n * 120) as f64 / e);
    }
}

Run (cargo run --release), measured on this Mac (Apple M4 Pro), synthetic trace, 120 segments (480 s) per session, 2000 noise seeds:

The asserts pass. The buffer rule stalled more here: after the 12x drop it held a full buffer and had just requested a 4800 kbit/s segment (2.4 MB), about 38 s at 500 kbit/s, longer than the 24 s buffer. Abandoning such requests would help; this sketch lacks it. It also switches more. Speed: about 1.2×10⁷ (throughput) and 6×10⁶ (BOLA) simulated segments/s, dominated by the 50 ms link integration.

Complexity and constants

• Decision: O(rungs) for throughput and BOLA; MPC enumerates rungs^horizon plans (5 rungs and horizon 5 give 3,125 plans; arithmetic).
• A 4 s segment at 4800 kbit/s is 2.4 MB (arithmetic), one request; segment length sets reaction time.

Efficiency

• Cost: the rule costs nanoseconds to microseconds on a phone CPU; the real cost is network: the chosen rendition's bytes, one request per segment, and stall time when wrong. Bound: link bandwidth and latency. CDN cache hits serve all clients from one segment.
• Levers:
  • Buffer-based rule instead of throughput-only — 10-20 % lower rebuffer rate at similar average rate (Huang et al.) — steady state with a sizeable buffer.
  • Chunk-aware bandwidth estimation — 65 % fewer stalls and 49 % higher QoE vs other schemes (ACTE, as reported by the authors) — chunked low-latency delivery.
  • Learned policy — 12-25 % average QoE over best prior (Pensieve, trace-driven and real tests) — when training traces match the audience.
  • Abandon slow in-flight requests — avoids the stall shown above (dash.js AbandonRequestsRule; no figure) — large segments, sudden drops.
• Naive → standard → state of the art:
  • Last-sample throughput, no safety factor · oscillates (reasoning)
  • Smoothed estimate × safety factor + up-switch limits · what hls.js, Shaka, dash.js and Media3 configure
  • Buffer-based + throughput hybrid, MPC or learned, chunk-aware for low latency · best reported QoE, more tuning

Implementation notes

• Ignore tiny downloads: Shaka drops samples under 16 kB.
• Hysteresis in time: Media3 needs 10 s of buffer to switch up and switches down only below 25 s (defaults); Shaka allows one switch per 8 s.
• Start low or from a stored estimate (hls.js default: 500 kbit/s); count audio in the bandwidth.

Pitfalls

• Oscillation: neighbouring rungs flip each segment when the estimate sits between them; add hysteresis and minimum dwell.
• Idle gaps: samples after a full-buffer idle may not show capacity (reasoning, not sourced).
• Chunked transfer: bytes/time equals the live encoding rate, so the estimate never rises above the current bitrate.
• Shared links: competing players can split unfairly (FESTIVE's motivation, from memory).
• Different keys per rung: switching to a rung whose CENC key is not loaded adds a licence round trip.
• VBR: declared BANDWIDTH is a peak; choosing by average underestimates risk.

Variants and alternatives

In production

dash.js, hls.js, Shaka Player, Media3/ExoPlayer, GStreamer adaptivedemux2 (bandwidth-target-ratio, connection-speed, min-bitrate), Netflix's deployment in Huang et al.

Implementations

Sources

• R. Pantos, W. May, HTTP Live Streaming, RFC 8216, 2017. RFC
• R. Pantos (ed.), HTTP Live Streaming 2nd Edition, IETF draft rfc8216bis (LL-HLS). draft
• ISO/IEC 23009-1, Dynamic adaptive streaming over HTTP (DASH), Part 1 (cite as standard only).
• T.-Y. Huang, R. Johari, N. McKeown, M. Trunnell, M. Watson, A Buffer-Based Approach to Rate Adaptation: Evidence from a Large Video Streaming Service, SIGCOMM 2014. DOI
• K. Spiteri, R. Urgaonkar, R. K. Sitaraman, BOLA: Near-Optimal Bitrate Adaptation for Online Videos, IEEE/ACM ToN 2020. arXiv
• X. Yin, A. Jindal, V. Sekar, B. Sinopoli, A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTP, SIGCOMM 2015. DOI
• J. Jiang, V. Sekar, H. Zhang, Improving Fairness, Efficiency, and Stability in HTTP-based Adaptive Video Streaming with FESTIVE, CoNEXT 2012. DOI
• H. Mao, R. Netravali, M. Alizadeh, Neural Adaptive Video Streaming with Pensieve, SIGCOMM 2017. DOI
• A. Bentaleb, C. Timmerer, A. Begen, R. Zimmermann, Bandwidth Prediction in Low-Latency Chunked Streaming, NOSSDAV 2019 (ACTE). DOI