Device fingerprinting

Deep dive

How it works

Parts:

• Collectors — server side (header order, TLS fingerprint, IP prefix) and client side (screen, GPU renderer, time zone, languages, cores, fonts).
• Canonicaliser — fixed key set, lowercase, collapsed whitespace, sorted sets, empties dropped.
• Stable hash — digest of the stable attributes; the exact-lookup key.
• Matcher — weighted per-attribute similarity (exact, Jaccard, version distance) with a coverage check.
• Policy — thresholds give same, similar, ambiguous or new; output feeds the risk score.

1. Collect and canonicalise; absent attributes are dropped, not counted as mismatches.
2. Hash the stable subset (length-prefixed fields); look up candidates by that hash.
3. No candidates: scan, or use a blocking index on rare attributes, since a stable trait may have changed.
4. Score each candidate: Σ wᵢ·simᵢ / Σ wᵢ over attributes present on both sides, wᵢ about the attribute's entropy in bits. Decide only if the shared weight is at least 50 %.
5. Volatile attributes earn partial credit: sets by Jaccard, browser versions by major-version distance, IP prefix low weight.
6. Classify: score ≥ 0.85 same, 0.6-0.85 similar, else new; two near-equal top candidates is ambiguous.
7. Emit features (matched device, score, accounts on it, device age) to the risk model; store or update the vector.

request ─► collect ─► canonicalise ─► stable hash ─► candidates ─► weighted match ─► verdict
                                       (exact key)   (or scan)     + coverage check   └─► risk score

Code

use std::collections::{BTreeMap, BTreeSet, HashMap};
#[derive(Clone, Copy, PartialEq)]
enum Kind { Exact, Version, Set }
/// (name, weight ~ bits of entropy, stable?, comparison). Stable ones form the exact-lookup hash.
const SPEC: &[(&str, f64, bool, Kind)] = &[
    ("gpu", 7.0, true, Kind::Exact),
    ("screen", 5.0, true, Kind::Exact),
    ("cores", 2.5, true, Kind::Exact),
    ("tz", 3.0, true, Kind::Exact),
    ("platform", 2.0, true, Kind::Exact),
    ("fonts", 8.0, false, Kind::Set),     // installs change it
    ("langs", 3.0, false, Kind::Set),
    ("ua", 4.0, false, Kind::Version),    // "chrome/126": updates every ~4 weeks
    ("ip24", 2.0, false, Kind::Exact),
];
pub type Attrs = BTreeMap<String, String>;
/// Canonicalise: known keys only, lowercase, collapsed whitespace, sets sorted + deduped, empties dropped.
pub fn canon(raw: &[(&str, &str)]) -> Attrs {
    let mut out = Attrs::new();
    for (k, v) in raw {
        let k = k.trim().to_ascii_lowercase();
        let Some(&(_, _, _, kind)) = SPEC.iter().find(|s| s.0 == k) else { continue };
        let v = v.split_whitespace().collect::<Vec<_>>().join(" ").to_ascii_lowercase();
        if v.is_empty() { continue; }
        let v = if kind == Kind::Set {
            let s: BTreeSet<&str> = v.split(',').map(str::trim).filter(|x| !x.is_empty()).collect();
            s.into_iter().collect::<Vec<_>>().join(",")
        } else { v };
        out.insert(k, v);
    }
    out
}
fn fnv1a(bytes: &[u8]) -> u64 { // non-cryptographic; use a keyed HMAC if the id leaves the service
    bytes.iter().fold(0xcbf29ce484222325, |h, b| (h ^ *b as u64).wrapping_mul(0x100000001b3))
}
/// Exact-lookup id over the stable attributes only, with length-prefixed fields (no ambiguity).
pub fn stable_hash(a: &Attrs) -> u64 {
    let mut buf = Vec::new();
    for (name, _, stable, _) in SPEC {
        if !*stable { continue; }
        let v = a.get(*name).map(String::as_str).unwrap_or("");
        buf.extend_from_slice(&(name.len() as u32).to_le_bytes());
        buf.extend_from_slice(name.as_bytes());
        buf.extend_from_slice(&(v.len() as u32).to_le_bytes());
        buf.extend_from_slice(v.as_bytes());
    }
    fnv1a(&buf)
}
fn sim(kind: Kind, a: &str, b: &str) -> f64 {
    match kind {
        Kind::Exact => (a == b) as u8 as f64,
        Kind::Set => {
            let (x, y): (BTreeSet<&str>, BTreeSet<&str>) = (a.split(',').collect(), b.split(',').collect());
            x.intersection(&y).count() as f64 / x.union(&y).count() as f64 // Jaccard
        }
        Kind::Version => { // family/major: same family, small major distance = browser update
            let p = |s: &str| { let (f, m) = s.split_once('/').unwrap_or((s, "0")); (f.to_string(), m.parse::<i32>().unwrap_or(0)) };
            let ((fa, ma), (fb, mb)) = (p(a), p(b));
            if fa != fb { 0.0 } else { (1.0 - 0.1 * (ma - mb).abs() as f64).max(0.0) }
        }
    }
}
/// Weighted similarity over attributes present on both sides; also returns the weight coverage.
pub fn similarity(a: &Attrs, b: &Attrs) -> (f64, f64) {
    let (mut num, mut den, mut total) = (0.0, 0.0, 0.0);
    for (name, w, _, kind) in SPEC {
        total += w;
        if let (Some(x), Some(y)) = (a.get(*name), b.get(*name)) { num += w * sim(*kind, x, y); den += w; }
    }
    (if den > 0.0 { num / den } else { 0.0 }, den / total)
}
#[derive(Debug, PartialEq)]
pub enum Verdict { Same(usize, f64), Similar(usize, f64), Ambiguous(f64), New }
#[derive(Default)]
pub struct Registry { devices: Vec<Attrs>, by_hash: HashMap<u64, Vec<usize>> }
impl Registry {
    pub fn add(&mut self, a: Attrs) -> usize {
        let id = self.devices.len();
        self.by_hash.entry(stable_hash(&a)).or_default().push(id);
        self.devices.push(a);
        id
    }
    pub fn lookup(&self, q: &Attrs) -> Verdict {
        let all: Vec<usize> = (0..self.devices.len()).collect();
        let cand = self.by_hash.get(&stable_hash(q)).unwrap_or(&all);
        let mut scored: Vec<(f64, usize)> = cand.iter()
            .map(|&i| (similarity(q, &self.devices[i]), i))
            .filter(|((_, cov), _)| *cov >= 0.5) // too few shared attributes: no decision
            .map(|((s, _), i)| (s, i)).collect();
        scored.sort_by(|a, b| b.0.partial_cmp(&a.0).unwrap());
        match scored.as_slice() {
            [] => Verdict::New,
            [(s, _), (s2, _), ..] if *s >= 0.6 && s - s2 < 0.05 => Verdict::Ambiguous(*s), // clone images
            [(s, i), ..] if *s >= 0.85 => Verdict::Same(*i, *s),
            [(s, i), ..] if *s >= 0.6 => Verdict::Similar(*i, *s),
            _ => Verdict::New,
        }
    }
}
fn main() {
    let base = [("gpu", "ANGLE (Apple M2)"), ("screen", "1512x982x2"), ("cores", "8"), ("tz", "Europe/Berlin"),
        ("platform", "MacIntel"), ("fonts", "Arial, Menlo, Helvetica"), ("langs", "en-US,de"), ("ua", "chrome/126"), ("ip24", "198.51.100")];
    let mut reg = Registry::default();
    let id = reg.add(canon(&base));
    // Case, spacing, key order and unknown keys do not change the hash.
    let messy = [("GPU", " angle (apple  m2) "), ("screen", "1512x982x2"), ("cores", "8"), ("tz", "europe/berlin"), ("platform", "macintel"), ("junk", "x")];
    assert_eq!(stable_hash(&canon(&messy)), stable_hash(&canon(&base)));
    // Drift (browser update, new font, new /24): still the same device.
    let mut d = base.to_vec();
    d[5].1 = "Arial, Menlo, Helvetica, Inter"; d[7].1 = "chrome/127"; d[8].1 = "203.0.113";
    assert!(matches!(reg.lookup(&canon(&d)), Verdict::Same(i, s) if i == id && s > 0.85));
    // Different hardware: new. A bare User-Agent: too little to decide.
    let mut o = base.to_vec();
    o[0].1 = "ANGLE (NVIDIA RTX 3060)"; o[1].1 = "1920x1080x1"; o[3].1 = "America/New_York";
    assert_eq!(reg.lookup(&canon(&o)), Verdict::New);
    assert_eq!(reg.lookup(&canon(&[("ua", "chrome/126")])), Verdict::New);
    // Two identical corporate images: ambiguous, never a confident link.
    reg.add(canon(&base));
    assert!(matches!(reg.lookup(&canon(&base)), Verdict::Ambiguous(_)));
}

Complexity and constants

• Canonicalise and hash: O(attributes), order of 100-300 bytes. Match: O(candidates × attributes); a changed stable attribute defeats the hash and forces a scan.
• Cost drivers: bytes per stored vector (a few hundred), per-pair allocations, cache misses on candidates.
• Measured (Apple Silicon Mac, release build, 100,000 devices, 9 attributes): hash about 0.28 µs, similarity about 0.8 µs per pair, full scan about 80 ms.

Efficiency

Cost: about 0.8 µs per candidate pair and 0.28 µs per hash (measured, code above); bound by allocation and memory, not arithmetic.

Levers:

• hash-first lookup — one bucket instead of an ≈ 80 ms scan of 100,000 (measured) — stable attributes unchanged.
• blocking index on rare attributes — candidates from N to dozens (order of) — when stable attributes change.
• precomputed sets — removes per-pair allocations — at thousands of lookups per second.
• match once per session and cache the verdict — no repeat work — until the vector changes.

Naive → standard → state of the art:

• scan all devices, compare strings · O(N × attributes) per request
• stable hash plus weighted fuzzy match on candidates · about 1 µs per candidate
• blocking index plus learned linking (FP-Stalker hybrid) · cheap rules first, ML only on ambiguous pairs

Implementation notes

• Stable for months: GPU renderer, screen, cores, platform, time zone. Volatile: User-Agent version (monthly releases), fonts, languages, IP prefix.
• Absence is not mismatch; require shared-weight coverage before deciding.
• Length-prefix fields; plain concatenation makes ab+c equal a+bc. FNV here is not cryptographic; use a keyed HMAC (SHA-2/BLAKE3) if ids leave the service.
• Derive weights from your own traffic; recompute when a browser change shrinks an attribute.
• Age out stale devices; decay old matches (time decay).

Pitfalls

• Shared and cloned devices: corporate images, school labs, kiosks, popular phone models collide. Count accounts per fingerprint, flag only above what the population explains, and return ambiguous rather than a link.
• Browser anti-fingerprinting (Firefox, Safari, Brave, Chrome's User-Agent reduction) coarsens or removes attributes, so entropy falls and false matches rise. Re-measure regularly.
• Published uniqueness figures describe 2010 and 2016 browsers; do not reuse them.
• Tune thresholds on labelled same-device and different-device pairs, including identical images.

Privacy and legal limits

Accessing information on a user's terminal falls under ePrivacy Art. 5(3): consent, unless "strictly necessary" for a service the user asked for; whether fraud defence qualifies is contested, so take legal advice per jurisdiction. Under GDPR an account-linked fingerprint is personal data; Recital 47 names strictly necessary fraud prevention as a legitimate interest, which still needs a balancing test, transparency, minimisation and a retention limit. Collect the fewest attributes, store hashed values, offer an appeal, and do not reuse the data for advertising.

Variants and alternatives

In production

Fraud teams at banks, marketplaces, game publishers and social networks link accounts and score logins by device; vendors sell it as a service.

Implementations

Sources

• Peter Eckersley, How Unique Is Your Web Browser?, PETS 2010. paper · publisher
• Pierre Laperdrix, Walter Rudametkin, Benoit Baudry, Beauty and the Beast: Diverting modern web browsers to build unique browser fingerprints, IEEE S&P 2016 (AmIUnique). paper · code
• Antoine Vastel, Pierre Laperdrix, Walter Rudametkin, Romain Rouvoy, FP-STALKER: Tracking Browser Fingerprint Evolutions, IEEE S&P 2018. paper
• Pierre Laperdrix, Nataliia Bielova, Benoit Baudry, Gildas Avoine, Browser Fingerprinting: A Survey, ACM TWEB 2020. paper
• EDPB, Guidelines 2/2023 on Technical Scope of Art. 5(3) of ePrivacy Directive. guidelines
• Directive 2002/58/EC Art. 5(3); Regulation (EU) 2016/679 Recital 47. ePrivacy · GDPR
• Chrome, What is User-Agent reduction? docs