RTP and RTCP

Deep dive

How it works

In: a UDP datagram, RTP data or compound RTCP · Out: payload with order and sampling time; per-source loss, jitter, RTT; RTCP reports · State: per SSRC: base and highest sequence, cycles, received, jitter, last SR time; per session: members, senders, average RTCP size, next send time · Cost: O(1) per packet; reports O(sources)

1. Send. Sequence +1 per packet, random start. Timestamp = sampling instant in the payload clock: 8 000 Hz for PCMU (PT 0) and, by historic error, G.722 (RFC 3551); 90 000 Hz for video, where a frame's packets share it. One random SSRC per stream.
2. Validate (A.1). Version 2, known PT. A new SSRC counts after MIN_SEQUENTIAL = 2 consecutive packets; a jump over MAX_DROPOUT = 3000 waits for confirmation; up to MAX_MISORDER = 100 behind is reordering.
3. Count (A.3). A wrap adds 65536 to cycles. expected = extended max - base + 1; lost = expected - received (negative with duplicates, 24-bit clamp); fraction = 256 * lost / expected since the last report.
4. Jitter. D = (Rj - Ri) - (Sj - Si) in timestamp units; J += (|D| - J) / 16.
5. Report. SR (200): NTP time, RTP timestamp, packet and octet counts, report blocks; RR (201) without sender info; SDES (202) with CNAME; BYE (203); APP (204). Compound packets start with SR or RR.
6. RTT. The receiver echoes the middle 32 bits of the last SR's NTP time (LSR) and its holding time (DLSR, 1/65536 s). Sender at arrival A: rtt = A - LSR - DLSR.
7. Lip-sync. An SR pairs NTP with RTP time: wallclock = SR NTP + (ts - SR ts) / clock rate. Equal CNAME: sync these streams (A/V sync).
8. Schedule. Td = max(Tmin, n * avg_rtcp_size / share): share 5 % of session bandwidth (senders 25 % of it), Tmin 5 s (2.5 s first). T uniform in [0.5, 1.5] * Td, divided by e - 3/2 = 1.21828; at expiry send only if tp + T <= now, else reschedule (§6.3, A.7).

Header. V=2, P (last byte counts padding), X, CC (CSRC count), M, PT, then sequence, timestamp, SSRC, CSRCs. X=1 adds profile ID, length in words, data. RFC 8285 elements: profile 0xBEDE = one-byte form (ID 1-14, length field = bytes - 1, 15 stops, 0 pads); 0x100 = two-byte form (ID 1-255). IDs bind via a=extmap:<id> <URI> in SDP.

Payload types. 0-95 static (RFC 3551), 96-127 dynamic, bound in SDP (RFC 8866): a=rtpmap:111 opus/48000/2. Separate streams by SSRC, not PT.

Feedback (RFC 4585, a=rtcp-fb:96 nack pli). RTPFB 205 FMT 1 Generic NACK: PID plus a 16-bit mask of the next 16 sequence numbers; PSFB 206 FMT 1 PLI: send an intra picture. Early mode drops the 5 s minimum. Reduced-size RTCP (RFC 5506, a=rtcp-rsize) permits feedback without SR/RR/SDES. RTCP mux (RFC 5761, a=rtcp-mux): types 200-204 equal PT 72-76 with M=1, so PT 64-95 are barred; one port, one NAT binding.

Collisions. A source seeing its SSRC used from another address sends BYE and picks a new one (§8.2).

Code

Std only: header parse and write (CSRC, extension, padding), A.1/A.3 sequence tracking and loss report, LSR/DLSR round trip on the RFC 3550 Fig. 2 vector.

/// RFC 3550 s5.1 fixed header, CSRC list, s5.3.1 extension.
#[derive(Debug, PartialEq, Default, Clone)] pub struct Header { pub marker: bool, pub pt: u8, pub seq: u16, pub ts: u32, pub ssrc: u32,
                    pub csrc: Vec<u32>, pub ext: Option<(u16, Vec<u8>)>, pub pad: u8 }
impl Header {
    /// Header and payload range without padding; None if malformed.
    pub fn parse(p: &[u8]) -> Option<(Header, std::ops::Range<usize>)> {
        if p.len() < 12 || p[0] >> 6 != 2 { return None; }
        let be32 = |o: usize| u32::from_be_bytes(p[o..o + 4].try_into().unwrap());
        let (cc, mut off) = ((p[0] & 15) as usize, 12 + 4 * (p[0] & 15) as usize);
        if p.len() < off { return None; }
        let ext = if p[0] & 0x10 != 0 {
            let n = 4 * u16::from_be_bytes(p.get(off + 2..off + 4)?.try_into().ok()?) as usize;
            let e = (u16::from_be_bytes([p[off], p[off + 1]]), p.get(off + 4..off + 4 + n)?.to_vec());
            off += 4 + n;
            Some(e)
        } else { None };
        let pad = if p[0] & 0x20 != 0 { *p.last()? } else { 0 };
        if (p[0] & 0x20 != 0 && pad == 0) || off + pad as usize > p.len() { return None; }
        let h = Header { marker: p[1] & 0x80 != 0, pt: p[1] & 0x7f, seq: u16::from_be_bytes([p[2], p[3]]),
                         ts: be32(4), ssrc: be32(8), csrc: (0..cc).map(|i| be32(12 + 4 * i)).collect(), ext, pad };
        Some((h, off..p.len() - pad as usize))
    }
    pub fn write(&self, payload: &[u8]) -> Vec<u8> {
        let mut v = vec![0x80 | self.ext.is_some() as u8 * 0x10 | self.csrc.len() as u8, (self.marker as u8) << 7 | self.pt];
        for w in [&self.seq.to_be_bytes()[..], &self.ts.to_be_bytes(), &self.ssrc.to_be_bytes()] { v.extend(w); }
        self.csrc.iter().for_each(|c| v.extend(c.to_be_bytes()));
        if let Some((prof, d)) = &self.ext { // d.len() % 4 == 0
            v.extend(prof.to_be_bytes()); v.extend(((d.len() / 4) as u16).to_be_bytes()); v.extend(d);
        }
        v.extend(payload); v // padding not written here
    }
}
/// Per-source state, RFC 3550 App. A.1 and A.3.
#[derive(Default)] pub struct Source { max_seq: u16, cycles: u32, base_seq: u32, bad_seq: u32, probation: u32,
                    received: u32, received_prior: u32, expected_prior: u32 }
const MAX_DROPOUT: u16 = 3000; const MAX_MISORDER: u32 = 100; const MIN_SEQUENTIAL: u32 = 2;
impl Source {
    fn init_seq(&mut self, seq: u16) { *self = Source { max_seq: seq, base_seq: seq as u32, bad_seq: (1 << 16) + 1, ..Source::default() }; }
    pub fn new(first: u16) -> Source {
        let mut s = Source::default(); s.init_seq(first);
        (s.max_seq, s.probation) = (first.wrapping_sub(1), MIN_SEQUENTIAL); s
    }
    /// false = on probation, or a jump awaiting confirmation.
    pub fn update_seq(&mut self, seq: u16) -> bool {
        let udelta = seq.wrapping_sub(self.max_seq);
        if self.probation > 0 {
            if seq == self.max_seq.wrapping_add(1) {
                self.probation -= 1; self.max_seq = seq;
                if self.probation == 0 { self.init_seq(seq); self.received += 1; return true; }
            } else { self.probation = MIN_SEQUENTIAL - 1; self.max_seq = seq; }
            return false;
        } else if udelta < MAX_DROPOUT {
            if seq < self.max_seq { self.cycles += 1 << 16; } // wrapped
            self.max_seq = seq;
        } else if udelta as u32 <= (1 << 16) - MAX_MISORDER {
            if seq as u32 == self.bad_seq { self.init_seq(seq); } // restart
            else { self.bad_seq = (seq as u32 + 1) & 0xffff; return false; }
        } // else duplicate or reordered: counted
        self.received += 1; true
    }
    /// (fraction lost /256, cumulative lost, extended highest seq)
    pub fn report(&mut self) -> (u8, i32, u32) {
        let ext_max = self.cycles + self.max_seq as u32;
        let expected = ext_max - self.base_seq + 1;
        let lost = (expected as i64 - self.received as i64).clamp(-0x80_0000, 0x7f_ffff) as i32;
        let (exp_i, rcv_i) = (expected - self.expected_prior, self.received - self.received_prior);
        (self.expected_prior, self.received_prior) = (expected, self.received);
        let lost_i = exp_i as i64 - rcv_i as i64;
        (if exp_i == 0 || lost_i <= 0 { 0 } else { ((lost_i << 8) / exp_i as i64) as u8 }, lost, ext_max)
    }
}
/// Middle 32 bits of a 64-bit NTP timestamp (the LSR field).
pub fn ntp_mid(sec: u32, frac: u32) -> u32 { sec << 16 | frac >> 16 }
/// RTT in 1/65536 s from arrival time A, LSR, DLSR (16.16 fixed point).
pub fn rtt(a: u32, lsr: u32, dlsr: u32) -> u32 { a.wrapping_sub(lsr).wrapping_sub(dlsr) }
#[cfg(test)]
mod tests {
    use super::*;
    #[test]
    fn header() {
        let pkt = [0x92, 0xE0, 0x12, 0x34, 0, 1, 0x5F, 0x90, 0xDE, 0xAD, 0xBE, 0xEF, 0x11, 0x11, 0x11, 0x11,
                   0x22, 0x22, 0x22, 0x22, 0xBE, 0xDE, 0, 1, 0x11, 0xAB, 0xCD, 0, 1, 2, 3]; // CC=2 X=1 M=1 PT=96
        let (h, pl) = Header::parse(&pkt).unwrap();
        assert!(h.marker && h.pt == 96 && h.seq == 0x1234 && h.ts == 90_000 && h.ssrc == 0xDEAD_BEEF);
        assert_eq!(h.csrc, [0x1111_1111, 0x2222_2222]);
        let (prof, d) = h.ext.clone().unwrap();
        assert_eq!((prof, d), (0xBEDE, vec![0x11, 0xAB, 0xCD, 0])); // RFC 8285: ID 1, 2 bytes, 1 pad
        assert_eq!((&pkt[pl.clone()], h.write(&pkt[pl])), (&[1, 2, 3][..], pkt.to_vec()));
        let mut q = pkt.to_vec(); q[0] |= 0x20; q.extend([0, 0, 0, 4]); // P bit, 4 pad bytes
        assert_eq!(Header::parse(&q).map(|(h, r)| (h.pad, r)), Some((4, 28..31)));
        assert!(Header::parse(&pkt[..20]).is_none() && Header::parse(&[0x40; 12]).is_none());
    }
    #[test]
    fn loss_rtt_sync() {
        let mut s = Source::new(65531);
        assert!(!s.update_seq(65531) && s.update_seq(65532)); // probation
        assert!(s.update_seq(65533) && s.update_seq(65535)); // 65534 missing
        assert_eq!(s.report(), (64, 1, 65535)); // 256/4
        assert!(s.update_seq(1) && s.update_seq(4) && s.update_seq(65534)); // wrap; 0, 2, 3 lost; 65534 late
        assert_eq!(s.report(), (102, 3, 65540)); // 512/5
        assert!(!s.update_seq(30000) && s.update_seq(30001)); // jump, restart
        assert_eq!(s.report(), (0, 0, 30001));
        assert_eq!(ntp_mid(0xb44d_b705, 0x2000_0000), 0xb705_2000); // RFC 3550 s6.4.1, Fig. 2
        assert_eq!(rtt(0xb710_8000, 0xb705_2000, 0x0005_4000), 0x0006_2000); // 6.125 s
    }
}

Complexity and constants

• Measured on this Mac (Apple M4 Pro), release build, one thread, three runs: plain header parse 3.9 ns, no allocation; with 2 CSRC and a 4-byte extension 32-35 ns (two Vec copies); update_seq 0.87-0.88 ns, 1 packet in 8 reordered. State per source 32 bytes (size_of).
• SR = 28 + 24n bytes, RR = 8 + 24n (n blocks). Cost driver: one syscall per datagram.

Efficiency

• Cost: parse plus sequence tracking about 5 ns per packet (measured on this Mac, Apple M4 Pro, sum of the above); bound is the datagram path (syscall, copy, SRTP). Overhead 12 bytes; RTCP about 5 % of bandwidth.
• Levers:
  • Borrow CSRC and extension slices — avoids the 3.9 vs 32-35 ns gap (measured) — packets with extensions.
  • Generic NACK bitmask — 16 bytes name up to 17 lost packets (RFC 4585 §6.2.1) — bursty loss.
  • Reduced-size RTCP — no SR/RR/SDES per feedback (RFC 5506) — frequent NACK and PLI.
• Naive → standard → state of the art:
  • Two ports, RTCP every 5 s, no feedback · loss concealed only.
  • AVPF NACK and PLI, mux · repair costs one round trip.
  • Early reduced-size feedback, congestion-control reports, FEC · repair before playout.

Implementation notes

• Use wrapping arithmetic for sequence numbers and timestamps; key on the extended sequence number.
• Validate RTCP first (A.2): version, SR or RR first, lengths summing to the datagram.

Pitfalls

• DLSR is 1/65536 s, not ms; LSR is the middle 32 NTP bits; 0 means no SR yet.
• Duplicates raise received; cumulative lost can go negative.

Variants and alternatives

In production

• Browsers, SIP phones, RTSP cameras, conferencing servers.

Implementations

Sources

• Schulzrinne et al., RTP: A Transport Protocol for Real-Time Applications, RFC 3550, 2003. rfc
• Schulzrinne, Casner, RTP Profile for Audio and Video Conferences with Minimal Control, RFC 3551, 2003. rfc
• Singer et al., A General Mechanism for RTP Header Extensions, RFC 8285, 2017. rfc
• Ott et al., Extended RTP Profile for RTCP-Based Feedback, RFC 4585, 2006. rfc
• Perkins, Westerlund, Multiplexing RTP Data and Control Packets on a Single Port, RFC 5761, 2010. rfc
• Johansson, Westerlund, Support for Reduced-Size RTCP, RFC 5506, 2009. rfc
• Begen et al., SDP: Session Description Protocol, RFC 8866, 2021. rfc