Deep dive
How it works
Parts:
- State row —
id, version, state, amounts, one per payment - Transition function — legal (state, input) pairs; stale or illegal inputs ignored or rejected
- Outbox — effects (processor calls, events) written in the same transaction as the transition
- Processor client — calls with derived idempotency keys, status query for unknown outcomes
- Webhook intake — applies only forward transitions, idempotently
submit approved capture confirmed
Created ─────────► Authorizing ───────► Authorized ───────────► Capturing ─────────► Captured ──refund──► (refund records)
│ │ ▲ │ void (partial: captured ≤ authorized)
declined │ │ │ not found: ▼
Declined ◄────────┘ │ │ resend, same key Voided
timeout ▼ │
AuthUnknown ── still unknown after N queries ──► reverse (void), then Failed
transient error ×N on Authorizing ──► Failed
- Create with amount, currency and an idempotency root key (from the client's idempotency key). Every call to the processor uses a derived key:
root:auth:1,root:capture:1,root:refund:2. - Authorize. Outcomes:
- approved →
Authorized(record auth code, authorized amount; a partial approval may authorize less than requested). - declined →
Declined. Hard declines (stolen card, invalid account) are final; soft declines (insufficient funds, issuer unavailable) may be retried later only under card-network retry rules. - transient error before the request reached the processor (connect refused, 503 with no side effect) → retry with backoff, same key.
- timeout / connection reset after sending →
AuthUnknown.
- approved →
- Resolve unknown: query the processor by the idempotency key or merchant reference. Found approved/declined → apply it. Not found → resend with the same key. Still unknown after the deadline → send a reversal/void (card networks support timeout reversals, ISO 8583 0400) and mark
Failed. - Capture any amount ≤ authorized (one or more partial captures if supported);
Capturinguntil confirmed. Auth holds expire after a network-specific number of days; capture before that or re-authorize. - Void an uncaptured authorization to release the hold. Refund creates its own small state machine per refund (requested → succeeded/failed), with Σ refunds ≤ captured.
- Disputes (chargebacks) are separate inbound flows against a captured payment; they move money via the ledger, not by rewinding the payment state.
- Ledger: post at capture (merchant payable, processor receivable) and at settlement; reconcile against processor reports.
Code
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum State { Created, Authorizing, AuthUnknown, Authorized, Capturing, Captured, Declined, Voided, Failed }
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum Op { Auth, Capture, Void }
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub struct Key { pub root: u128, pub op: Op, pub n: u32 } // sent to the processor as "root:op:n"
#[derive(Debug)]
pub enum Input {
Submit,
Approved { amount: i64 }, Declined, Transient, Timeout, // processor answers to Auth
QueryFound { approved: bool, amount: i64 }, QueryNotFound, QueryUnknown,
Capture { amount: i64 }, CaptureConfirmed { amount: i64 }, Void, VoidConfirmed,
}
#[derive(Debug, PartialEq)]
pub enum Effect { Call(Op, Key, i64), Query { key: Key, delay_ms: u64 }, Retry { key: Key, attempt: u32 }, Emit(State) }
pub struct Payment {
pub state: State, pub amount: i64, pub authorized: i64, pub captured: i64,
pub root: u128, pub attempts: u32, pub queries: u32, pub captures: u32, pub version: u64,
}
const MAX_RETRIES: u32 = 3;
const MAX_QUERIES: u32 = 5;
#[derive(Debug)]
pub struct Illegal { pub state: State }
pub fn step(p: &mut Payment, input: Input) -> Result<Vec<Effect>, Illegal> {
use State::*;
let auth = Key { root: p.root, op: Op::Auth, n: 1 }; // one logical auth: every retry reuses it
let before = p.state;
let mut fx = Vec::new();
match (p.state, input) {
(Created, Input::Submit) => { p.state = Authorizing; fx.push(Effect::Call(Op::Auth, auth, p.amount)); }
(Authorizing | AuthUnknown, Input::Approved { amount })
| (AuthUnknown, Input::QueryFound { approved: true, amount }) => { p.state = Authorized; p.authorized = amount; }
(Authorizing | AuthUnknown, Input::Declined)
| (AuthUnknown, Input::QueryFound { approved: false, .. }) => p.state = Declined,
(Authorizing, Input::Transient) if p.attempts < MAX_RETRIES => {
p.attempts += 1;
fx.push(Effect::Retry { key: auth, attempt: p.attempts }); // scheduler adds backoff + jitter
}
(Authorizing, Input::Transient) => p.state = Failed,
(Authorizing, Input::Timeout) => { p.state = AuthUnknown; fx.push(Effect::Query { key: auth, delay_ms: 1_000 }); }
(AuthUnknown, Input::QueryNotFound) => { p.state = Authorizing; fx.push(Effect::Call(Op::Auth, auth, p.amount)); }
(AuthUnknown, Input::QueryUnknown) if p.queries < MAX_QUERIES => {
p.queries += 1;
fx.push(Effect::Query { key: auth, delay_ms: 1_000 << p.queries });
}
(AuthUnknown, Input::QueryUnknown) => { // give up safely: reverse
p.state = Failed;
fx.push(Effect::Call(Op::Void, Key { root: p.root, op: Op::Void, n: 1 }, p.amount));
}
(Authorized | Captured, Input::Capture { amount }) if amount > 0 && p.captured + amount <= p.authorized => {
p.state = Capturing;
p.captures += 1;
fx.push(Effect::Call(Op::Capture, Key { root: p.root, op: Op::Capture, n: p.captures }, amount));
}
(Capturing, Input::CaptureConfirmed { amount }) => { p.captured += amount; p.state = Captured; }
(Authorized, Input::Void) => fx.push(Effect::Call(Op::Void, Key { root: p.root, op: Op::Void, n: 1 }, p.authorized)),
(Authorized, Input::VoidConfirmed) => p.state = Voided,
// Late or duplicate webhooks for a step we've already passed: accept, do nothing.
(Authorized | Capturing | Captured | Voided, Input::Approved { .. }) | (Captured, Input::CaptureConfirmed { .. }) => {}
(s, _) => return Err(Illegal { state: s }),
}
if p.state != before { p.version += 1; fx.push(Effect::Emit(p.state)); }
Ok(fx)
}
Persisting a transition (the effects become outbox rows executed by a worker that retries them with their keys):
BEGIN;
UPDATE payments SET state = $new_state, authorized = $a, captured = $c, version = version + 1
WHERE id = $id AND version = $expected_version; -- 0 rows: concurrent update, reload and re-run step()
INSERT INTO outbox (payment_id, effect, idem_key, run_at) VALUES ($id, $effect, $key, now());
COMMIT;
Complexity and constants
- One row update and a few outbox inserts per transition; the cost is dominated by processor latency (order of 100 ms-2 s for card authorization).
- State row: order of 200 bytes; keep the full transition history as events (event sourcing) or an audit table.
Efficiency
Cost: the bound is external latency, not your database. A transition is one conditional row update plus a few outbox inserts (state row order of 200 bytes); card authorization at the processor takes order of 100 ms to 2 s.
Levers
- Serialize per payment id only (version check or one worker per id) — different payments run fully in parallel — any volume.
- Single-message sale (authorize and capture together) — drops the capture round trip — goods delivered at once.
- Outbox draining in batches with
FOR UPDATE SKIP LOCKEDand concurrent processor calls — more throughput than one row per poll — backlog. - Webhooks over polling; poll only stuck unknown states with growing delay — fewer calls — most flows.
- Circuit breaker per processor (circuit breaker) — no waiting out timeouts during an outage; never retry hard declines.
Naive → standard → state of the art
- Naive: call the processor inline, retry on any error · double charges after timeouts
- Standard: persisted state machine +
AuthUnknown+ idempotency keys + outbox · one row update per step - State of the art: multi-processor orchestration with routing and failover (Hyperswitch) or durable execution (Temporal) · same transition cost, retries and unknowns handled by the platform
Implementation notes
- Single writer per payment: optimistic
versioncheck, or route all inputs for a payment id to one worker. Never mutate state from two webhook handlers without it. - Webhooks as hints: dedupe by provider event id (unique constraint), then either apply the event through
step()or, more robustly, fetch the current object from the provider and apply that. - Classify errors precisely: "request never left" (DNS failure, connect refused, 4xx validation) is safe to retry immediately; "sent, no answer" is unknown; processor-specific decline codes map to hard/soft.
- Retries use exponential backoff with jitter and a deadline; a circuit breaker per processor stops hammering an outage and enables failover routing. Failover to a second processor is only safe once the first attempt's outcome is known to be "not charged".
- Timers: auth-expiry capture deadlines, unknown-state query schedule and webhook-silence alarms are scheduled jobs keyed by payment id.
- Amounts in integer minor units with currency (money representation); store requested, authorized and captured separately because partial approvals and partial captures are normal.
- Reference states: Stripe PaymentIntents use
requires_payment_method,requires_confirmation,requires_action(e.g. 3-D Secure),processing,requires_capture,canceled,succeeded; mirror your provider's model at the edge, keep your own internal states.
Pitfalls
- Treating a timeout as a decline and letting the customer retry: double charge. Treating it as success: goods shipped without payment.
- New idempotency key per retry, or per worker restart.
- Applying webhooks in arrival order without state checks: a late
authorizedevent moves acapturedpayment backwards. - Capturing more than authorized, or refunding more than captured, because concurrent requests each checked a stale row.
- Booleans (
is_paid,is_refunded) instead of one state field: impossible combinations appear. - Retrying hard declines, or retrying soft declines too often: networks charge fees for excessive reattempts.
Variants and alternatives
pending periods and returns days later, so succeeded can still be followed by returned.In production
Payments: Stripe PaymentIntents, Adyen's payment lifecycle (authorisation, capture, cancellation, refund notifications), every payment orchestrator and PSP integration layer; card networks carry these steps as ISO 8583 messages (0100/0110 authorization, 0400 reversal). Cloud and orchestration: the EC2 instance lifecycle and the Kubernetes Pod lifecycle are persisted state machines with defined phases.
Implementations
| name | lang | repo | install | verdict | licence | health | as_of | notes |
|---|---|---|---|---|---|---|---|---|
| Hyperswitch | Rust | juspay/hyperswitch | n/a (service); self-host via the repo's Docker setup | — | — | — | — | open-source payment orchestrator, multi-processor routing and retries |
| Kill Bill | Java | killbill/killbill | n/a (service); Docker or Maven artifacts per its docs | — | — | — | — | open-source billing and payments platform with payment plugins |
| Temporal | Go (SDKs for several languages) | temporalio/temporal | brew install temporal (dev server CLI) | — | — | — | — | durable execution for the retry and unknown-outcome logic |
| statig | Rust | mdeloof/statig | cargo add statig | — | — | — | — | hierarchical state machine library |
| rust-fsm | Rust | eugene-babichenko/rust-fsm | cargo add rust-fsm | — | — | — | — | small FSM macro; the plain match above needs neither |
Sources
- Stripe documentation, How PaymentIntents work and Place a hold on a payment method. PaymentIntents · hold
- ISO 8583, Financial transaction card originated messages — Interchange message specifications.
- Chris Richardson, Microservices Patterns, Manning, 2018 (transactional outbox, sagas). book · outbox pattern
- Stripe documentation, Idempotent requests, API reference. docs