FIX protocol (session layer)

Deep dive

How it works

In: FIX messages on a TCP stream · Out: an ordered, gap-free message sequence per direction, resends on request · State: two sequence numbers + persisted outbound messages per session · Cost: O(message bytes) to frame and checksum; resend O(gap)

8=FIX.4.4|9=nnn|35=D|49=CLIENT|56=BROKER|34=215|52=20261020-14:30:00.123|11=ord-1|55=AAPL|54=1|38=100|40=2|44=191.25|60=...|10=ddd|
          └──── BodyLength counts these bytes: from after "9=nnn|" through the SOH before "10=" ────┘

1. Logon (A): initiator sends 98=0 (EncryptMethod none), 108=HeartBtInt seconds (e.g. 30), optionally 141=Y (ResetSeqNumFlag: both sides restart at 1). Acceptor validates CompIDs (49 SenderCompID / 56 TargetCompID) and answers Logon.
2. Sequence check on every inbound message:
   • 34 == expected → process, expected += 1.
   • 34 > expected → send ResendRequest (35=2, 7=BeginSeqNo=expected, 16=EndSeqNo=0 meaning "to infinity" in FIX 4.2+); queue or drop the early message until the gap is filled.
   • 34 < expected and 43=Y → duplicate from a resend: ignore (or use for state check). Without 43=Y → Logout with reason, disconnect, alert a human.
3. Answering a ResendRequest: replay stored application messages with 43=Y (PossDupFlag) and 122=OrigSendingTime; replace admin messages (heartbeats, test requests) and anything you choose not to resend with SequenceReset-GapFill (35=4, 123=Y, 36=NewSeqNo).
4. SequenceReset: GapFill mode obeys sequence rules and moves expected to 36; Reset mode (123=N) forces it, for disaster recovery only. NewSeqNo must not go backwards.
5. Liveness: send Heartbeat (35=0) when nothing was sent for HeartBtInt; if nothing is received for HeartBtInt plus a margin, send TestRequest (35=1, 112=TestReqID), which the peer must answer with a Heartbeat echoing 112; no answer → disconnect.
6. Logout (5): exchange Logouts, then close TCP. Sequence numbers persist for the next connection unless the counterparty agreement says to reset daily.

Common application tags: 11 ClOrdID, 37 OrderID, 17 ExecID, 55 Symbol, 54 Side (1=Buy, 2=Sell), 38 OrderQty, 40 OrdType (1=Market, 2=Limit), 44 Price, 59 TimeInForce, 39 OrdStatus, 150 ExecType, 151 LeavesQty, 14 CumQty. Messages: D NewOrderSingle, F OrderCancelRequest, G OrderCancelReplaceRequest, 8 ExecutionReport, 9 OrderCancelReject.

Code

pub const SOH: u8 = 0x01;

fn put_u(out: &mut Vec<u8>, mut v: u64) {
    let mut b = [0u8; 20];
    let mut i = b.len();
    loop { i -= 1; b[i] = b'0' + (v % 10) as u8; v /= 10; if v == 0 { break; } }
    out.extend_from_slice(&b[i..]);
}
pub fn field(out: &mut Vec<u8>, tag: u32, val: &[u8]) { put_u(out, tag as u64); out.push(b'='); out.extend_from_slice(val); out.push(SOH); }
pub fn field_u(out: &mut Vec<u8>, tag: u32, v: u64) { put_u(out, tag as u64); out.push(b'='); put_u(out, v); out.push(SOH); }

/// Frame a message: standard header + `body` (already-encoded fields) + checksum. `scratch` is reused.
pub fn encode(out: &mut Vec<u8>, scratch: &mut Vec<u8>, msg_type: &[u8], sender: &[u8], target: &[u8],
              seq: u64, sending_time: &[u8], body: &[u8]) {
    scratch.clear();
    field(scratch, 35, msg_type);
    field(scratch, 49, sender);
    field(scratch, 56, target);
    field_u(scratch, 34, seq);
    field(scratch, 52, sending_time);
    scratch.extend_from_slice(body);
    out.clear();
    field(out, 8, b"FIX.4.4");
    field_u(out, 9, scratch.len() as u64);
    out.extend_from_slice(scratch);
    let sum = out.iter().fold(0u32, |a, &b| a + b as u32) % 256;
    out.extend_from_slice(b"10=");
    out.extend_from_slice(&[b'0' + (sum / 100) as u8, b'0' + (sum / 10 % 10) as u8, b'0' + (sum % 10) as u8]);
    out.push(SOH);
}

fn parse_u(s: &[u8]) -> Option<u64> {
    if s.is_empty() || s.len() > 19 { return None; }
    s.iter().try_fold(0u64, |a, &c| c.is_ascii_digit().then(|| a * 10 + (c - b'0') as u64))
}

/// TCP stream framing: Ok(Some(len)) when a complete, checksum-valid message starts at buf[0].
pub fn frame(buf: &[u8]) -> Result<Option<usize>, &'static str> {
    if buf.len() < 2 { return Ok(None); }
    if !buf.starts_with(b"8=") { return Err("must start with 8="); }
    let Some(e8) = buf.iter().position(|&c| c == SOH) else { return Ok(None) };
    let rest = &buf[e8 + 1..];
    if rest.len() < 2 { return Ok(None); }
    if !rest.starts_with(b"9=") { return Err("second field must be 9="); }
    let Some(e9) = rest.iter().position(|&c| c == SOH) else { return Ok(None) };
    let body_len = parse_u(&rest[2..e9]).ok_or("bad BodyLength")? as usize;
    let body_start = e8 + 1 + e9 + 1;
    let total = body_start + body_len + 7;                     // "10=ddd" + SOH
    if buf.len() < total { return Ok(None); }
    let t = &buf[body_start + body_len..total];
    if !t.starts_with(b"10=") || t[6] != SOH { return Err("BodyLength mismatch"); }
    let want = parse_u(&t[3..6]).ok_or("bad CheckSum")?;
    let got = buf[..body_start + body_len].iter().fold(0u32, |a, &b| a + b as u32) % 256;
    if got as u64 != want { return Err("CheckSum mismatch"); }
    Ok(Some(total))
}

/// Iterate (tag, value) pairs of a framed message.
pub fn fields(msg: &[u8]) -> impl Iterator<Item = (u32, &[u8])> {
    msg.split(|&c| c == SOH).filter(|f| !f.is_empty()).filter_map(|f| {
        let eq = f.iter().position(|&c| c == b'=')?;
        Some((parse_u(&f[..eq])? as u32, &f[eq + 1..]))
    })
}

#[derive(Debug, PartialEq)]
pub enum Verdict { Process, Duplicate, Resend { from: u64 }, Fatal, Reject }

pub struct Session { pub next_in: u64, pub next_out: u64 }

impl Session {
    pub fn check_in(&mut self, seq: u64, poss_dup: bool) -> Verdict {
        if seq == self.next_in { self.next_in += 1; Verdict::Process }
        else if seq > self.next_in { Verdict::Resend { from: self.next_in } } // 35=2|7=from|16=0
        else if poss_dup { Verdict::Duplicate }
        else { Verdict::Fatal }                                                 // Logout + disconnect
    }
    pub fn on_sequence_reset(&mut self, seq: u64, poss_dup: bool, gap_fill: bool, new_seq: u64) -> Verdict {
        if gap_fill {
            let v = self.check_in(seq, poss_dup);
            if v != Verdict::Process { return v; }
        }
        if new_seq < self.next_in { return Verdict::Reject; }                  // never move backwards
        self.next_in = new_seq;
        Verdict::Process
    }
}

Complexity and constants

• Parsing is a linear byte scan plus integer conversions; a 200-byte message is order of 100 ns to 1 µs depending on how many fields are converted. Binary protocols do the same in order of 10 ns.
• Typical NewOrderSingle: 150-250 bytes.
• Persistence: every outbound message must be stored (by sequence number) before or as it is sent so it can be resent; sequence numbers must survive restarts.

Efficiency

Cost: the bound is the network round trip and the outbound store write, not parsing. A 200-byte message parses in order of 100 ns to 1 µs depending on converted fields; a typical NewOrderSingle is 150-250 bytes. Every outbound message needs a durable write by sequence number.

Levers

• Batch several messages per fsync — fewer durable writes — a lost tail can be GapFilled; keep the file append-only.
• Scan once, record tag to offset, convert lazily, reuse buffers — nearer the 100 ns end of the range — fields needed are few.
• TCP_NODELAY, no per-message allocation, pinned thread — lower jitter — latency-sensitive paths; kernel bypass only for the most critical.
• Binary protocols (SBE or a native venue protocol) — decode in order of 10 ns — where the venue offers one.

Naive → standard → state of the art

• Naive: parse every field to owned strings per message · allocation per tag
• Standard: QuickFIX-family engine · fine at modest order rates
• State of the art: hand-rolled session on the hot path or native binary venue protocol · order of 10 ns decode

Implementation notes

• Outbound store: an append-only file indexed by sequence number (write-ahead log); persist next_out and next_in too. A crash that loses next_out makes the counterparty see sequence numbers too low: fatal.
• Resend policy for orders: replaying a stale NewOrderSingle hours later can be dangerous. Many firms GapFill over old orders instead of resending them, or rely on the venue rejecting stale orders. Decide explicitly.
• ClOrdID is the idempotency key of order entry: unique per order (per session/day), so a resent duplicate (43=Y) can be recognised as already processed (idempotency keys).
• Repeating groups: a count tag (e.g. 268 NoMDEntries) followed by groups that each start with a fixed delimiter tag; field order inside groups matters, so the parser needs the data dictionary for groups.
• Timestamps: SendingTime (52) and TransactTime (60) are UTC, YYYYMMDD-HH:MM:SS.sss, with microsecond/nanosecond precision allowed in newer versions.
• Validation: check CompIDs on every message, not just Logon; reject unknown MsgTypes with a session-level Reject (35=3, 45=RefSeqNum, 58=Text).
• No allocation on the hot path: reuse buffers, parse values lazily from slices, keep a tag → offset table for the fields you need.
• TLS: FIX itself has no encryption; run it over TLS or a private line.

Pitfalls

• Computing BodyLength including the 8= and 9= fields, or excluding the final SOH before 10=.
• Checksum written with fewer than three digits (10=87 instead of 10=087).
• Processing an out-of-order message before the gap is filled: orders get applied in the wrong sequence.
• Answering a ResendRequest with fresh sequence numbers instead of the original ones with 43=Y.
• Both sides sending ResendRequest at once after reconnect and each also resending: handle resend while a resend is outstanding.
• Infinite resend loops when a stored message is missing: GapFill over it instead.

Variants and alternatives

In production

QuickFIX (C++), QuickFIX/J, QuickFIX/n, and commercial engines; FIX order entry and drop copy at essentially every broker and exchange.

Implementations

Sources

• FIX Trading Community, FIX Protocol Specification, Version 4.4, 2003. spec package
• FIX Trading Community, FIX Session Protocol (FIXT 1.1) specification. spec
• FIX Trading Community, Simple Binary Encoding (SBE) specification. page · code