← Back
Home › Systems › AI and agent security

ai-red-teaming

automated red teaming of models and agents

garak, PyRIT

SystemTechniques · 25Primitives · 87 ai-red-teaming Prompt injection defenseagentic-ai · medium: Treat every untrusted string as data, track where values came from, and make tool calls and rendered output depend on provenance, not on what the model says.›User simulationagentic-ai · medium: LLM-driven simulated users following scripted goals to test conversational agentsDeterministic simulation testingdistributed · medium: Run the whole system in one thread under a seeded scheduler with simulated network, disk and clock, so every failure replays from its seed.›Durable job queuedistributed · medium: Persisted tasks that workers lease for a visibility timeout, ack on success and retry with backoff; at-least-once execution that survives worker and queue crashes.›Primary-backup replicationdistributed · medium: One node orders all writes and ships them to standbys; the acknowledgement rule (none, one, all) sets what a crash loses, and an external arbiter decides who takes over.›LLM-as-judgemachine-learning · medium: Grade open-ended outputs with a language model under a rubric, then correct its known biases by order swapping, voting and calibration against human labels.›Multi-armed banditsmachine-learning · medium: Choose among K options repeatedly while learning their payoffs: epsilon-greedy, UCB1 and Thompson sampling keep total regret low where a fixed split or a fixed exploration rate pays forever.›Template-based text generationmachine-learning · low: Realise facts as sentences by filling slots in alternative templates, with agreement, referring expressions and seeded variation; every word traces to data or a template.›Token samplingml-training · low: Turn a vector of scores into one chosen item with temperature, top-k, top-p, min-p, repetition penalties and a seed.›Exponential backoff with jittersystems · low: Retry after delays that grow geometrically and are randomized, so failed clients spread out instead of hammering a recovering service in synchronized waves.›Text normalizationmachine-learning · low: Rewrite messy text into one canonical form for matchers and models, or into spoken words for speech output (and back), keeping a map to the original offsets.›Agent evaluationagentic-ai · unknown: Agent loopagentic-ai · unknown: Tool callingagentic-ai · low: The model emits a structured call (id, name, JSON arguments) instead of prose; your code validates, runs it and returns a result tagged with the same id.›Hdr histogramdata-structures · low: Log-linear bucketed histograms with bounded relative error for latency percentilesProcess supervisionhardware-os · low: A parent process starts services, reaps their exits and restarts them under a delay and burst-limit policy, then stops them with SIGTERM before SIGKILL.›Pass at kmachine-learning · low: Unbiased pass@k estimator from n samples with c correct, for code and agent benchmarksReverse-mode automatic differentiat…ml-training · low: Record the forward computation on a tape, then sweep it backwards once to get the gradient of a scalar loss for all inputs.›Mutual TLSsecurity · low: TLS where the client also proves a certificate key, so every connection carries a verified, rotatable machine identity.›Bootstrap confidence intervalsstatistics · low: Resampling-based confidence intervals for arbitrary statisticsGraceful drainingsystems · low: On a termination signal, stop taking new work, finish in-flight work until a deadline, tell clients to reconnect elsewhere, then exit; clients resume from a token.›Html sanitizationsecurity · low: Allowlist-based HTML sanitizing of untrusted markupLeast privilege and isolationsecurity · low: Assume every component will be compromised and limit what it can then reach: narrow identities, separate accounts, sandboxed processes.›+ 18 more techniques+ 71 more primitives

Similar systems

Share the most building blocks with this one.

conversational-agent-platform 35 %mcp-host 29 %text-to-sql-agent 29 %chat-assistant 27 %dependency-updates 27 %