DNS and load balancers

Deep dive

Layers

Decision table

Code

HAProxy front for HTTP/2 + TLS with health checks and least-connections:

global
    maxconn 200000
    nbthread 4
    ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_CHACHA20_POLY1305_SHA256
defaults
    mode http
    timeout connect 3s
    timeout client  30s
    timeout server  30s
    timeout http-keep-alive 60s
    http-reuse safe                 # keep-alive pool to backends
frontend https
    bind :443 ssl crt /etc/haproxy/certs/ alpn h2,http/1.1
    http-request set-header X-Forwarded-Proto https
    default_backend app
backend app
    balance leastconn
    option httpchk GET /ready
    http-check expect status 200
    default-server inter 2s fall 3 rise 2 maxconn 2000 slowstart 30s
    server a1 10.0.0.11:8080 check
    server a2 10.0.0.12:8080 check
    server a3 10.0.0.13:8080 check

Route 53 failover pair (Terraform):

resource "aws_route53_health_check" "eu" {
  fqdn = "eu.api.example.com"
  type = "HTTPS"
  resource_path = "/ready"
  failure_threshold = 3
  request_interval  = 10
}
resource "aws_route53_record" "api_primary" {
  zone_id = var.zone_id
  name = "api.example.com"
  type = "A"
  ttl  = 60
  records = [var.eu_ip]
  set_identifier  = "eu"
  health_check_id = aws_route53_health_check.eu.id
  failover_routing_policy { type = "PRIMARY" }
}
resource "aws_route53_record" "api_secondary" {
  zone_id = var.zone_id
  name = "api.example.com"
  type = "A"
  ttl  = 60
  records = [var.us_ip]
  set_identifier = "us"
  failover_routing_policy { type = "SECONDARY" }
}

Efficiency

• Cost: an L7 balancer pays a TLS handshake and a request parse per new connection or request; an L4 balancer only forwards packets. HAProxy reports over 2 million HTTP requests/s on one Arm-based AWS Graviton2 instance (HAProxy blog, vendor benchmark). Route 53 costs $0.50 per hosted zone-month and $0.40 per million standard queries up to 1 billion (Route 53 pricing, as of 2026-10). Bound: CPU for TLS and parsing; for DNS, query count.
• Levers:
  • keep-alive pool to backends (http-reuse) — no TCP and TLS setup per request (HAProxy Configuration Manual) — all L7 setups
  • TLS 1.3 — one round trip for a full handshake (RFC 8446) — new client connections
  • least-request or power of two choices — lower tail load imbalance (Mitzenmacher 2001) — uneven request costs
  • long TTL on records that never fail over — fewer billed queries (Route 53 pricing) — static records
  • L4 consistent-hash tier (Maglev) — per-connection routing without parsing requests (Eisenbud 2016) — raw TCP/UDP, very high rates
• Naive → standard → state of the art:
  • round-robin DNS, no health checks · failover waits for resolver caches
  • L7 balancer with health checks, pooled backends, least-request · one handshake per client
  • anycast or L4 hash tier in front of an L7 pool · connection routing without request parsing

Implementation notes

• Keep-alive from balancer to backends; reconnecting per request burns CPU on TLS and ports.
• Preserve the client IP: PROXY protocol for L4, X-Forwarded-For for L7, and trust it only from the balancer's addresses.
• Health checks: readiness endpoint that checks the process, not its dependencies, or one DB blip fails every backend at once.
• Connection draining (deregistration delay) of 10-30 s for deploys; long-lived WebSockets need explicit migration logic.
• Low TTLs (30-60 s) only on records that fail over; long TTLs elsewhere save queries and resolver dependency.

Pitfalls

• Ephemeral port exhaustion on the balancer to a few backends (64k ports per source/dest pair); add backend IPs or source IPs.
• AWS ALB scales gradually; a sudden 10x spike gets 503s unless pre-warmed or fronted by a CDN.
• Apex domains can't be CNAMEs; use ALIAS/ANAME/CNAME-flattening.
• Health checks from many balancer nodes add load: 30 nodes × 1 check/s is 30 req/s per backend.
• DNS-only failover leaves clients that cached the old IP failing for minutes.

Implementations

Sources

• Daniel E. Eisenbud et al., Maglev: A Fast and Reliable Software Network Load Balancer, NSDI 2016. paper
• Michael Mitzenmacher, The Power of Two Choices in Randomized Load Balancing, IEEE TPDS, 2001. paper
• HAProxy, Configuration Manual. docs · AWS, Elastic Load Balancing pricing. pricing · AWS, Route 53 pricing. pricing
• RFC 8446, The Transport Layer Security (TLS) Protocol Version 1.3, 2018. rfc