PyRIT (Python Risk Identification Tool) is an open-source AI security red teaming framework built by Microsoft’s AI Red Team. It has 3.4k GitHub stars, 661 forks, and 117 contributors.
The framework is MIT licensed and requires Python 3.10-3.13. The latest release is v0.11.0 (February 2026). Microsoft built PyRIT based on their experience red teaming production AI systems including Bing Chat and Copilot. The project has a published paper on arXiv (2410.02828).
What is PyRIT?
PyRIT automates the repetitive parts of AI red teaming so security professionals can focus on creative attack strategies. It provides orchestrators that manage multi-turn conversations with AI targets, converters that transform prompts to bypass filters, scorers that evaluate whether attacks succeeded, and a memory system that tracks everything.
The framework supports text, image, audio, and video modalities. You can test any AI system accessible via API, from OpenAI and Azure OpenAI to custom HTTP endpoints and browser-based targets via Playwright.
PyRIT’s architecture is modular. Each component (orchestrators, targets, converters, scorers, memory) can be swapped or extended independently, letting you build custom red teaming workflows for your specific needs.
Key Features
| Feature | Details |
|---|---|
| Orchestrators | Prompt sending (single-turn), red teaming (multi-turn), crescendo, Tree of Attacks with Pruning (TAP), XPIA, benchmarking |
| Targets | OpenAI Chat/Responses/Image/Video/TTS, Azure ML, HuggingFace, custom HTTP/WebSocket, Playwright browser |
| Converters | Text-to-text (Base64, ROT13, leetspeak, homoglyph, translation), audio, image, video, file converters, human-in-the-loop |
| Scorers | Azure Content Safety API, true/false classification, Likert scale, refusal detection, human-in-the-loop, batch scoring |
| Memory | SQLite (default), Azure SQL Database, labeling, export, embeddings |
| Modalities | Text, image, audio, video |
| Attack Types | Prompt injection, XPIA, crescendo, skeleton-key, many-shot jailbreak, role-play, multi-turn manipulation |
| Python Support | 3.10-3.13 |
| License | MIT |
Orchestrators
Orchestrators manage the flow of red teaming sessions. Each type handles a different attack pattern:
- Prompt Sending — sends a batch of prompts in a single turn, useful for baseline testing
- Red Teaming — multi-turn conversations where an attacker LLM generates follow-up prompts based on target responses
- Crescendo — gradually escalates requests across turns, starting innocuous and building toward the objective
- Tree of Attacks with Pruning (TAP) — explores multiple attack paths simultaneously, pruning unsuccessful branches
- XPIA — cross-domain prompt injection attacks that embed malicious instructions in external data sources
Converters
Converters transform prompts to bypass safety filters. They chain together for layered obfuscation:
- Text transforms — Base64 encoding, ROT13, leetspeak, homoglyph substitution, Unicode tricks, translation
- Cross-modal — convert text to images, audio, or video for multi-modal attacks
- Selective conversion — apply transformations to specific parts of a prompt while leaving the rest intact
- Human-in-the-loop — manual prompt modification when automated approaches need human creativity
Scoring and Evaluation
Scorers evaluate whether AI responses indicate a successful attack:
- Azure Content Safety — uses Microsoft’s content safety API to classify harmful content
- True/false scoring — binary classification of whether the response meets the attack objective
- Likert scale — graduated scoring for nuanced evaluation
- Refusal detection — identifies when the target model refuses to respond
- LLM-as-judge — uses a separate LLM to evaluate response quality
Memory System
The memory system stores every prompt, response, and metadata from red teaming sessions. You can analyze patterns across runs and reproduce tests later. Teams using Azure SQL as the backend can share sessions. Memory labeling lets you tag and organize findings.
Getting Started
pip install pyrit for the stable release (requires Python 3.10-3.13). For the easiest setup, use Docker which includes JupyterLab and all dependencies.AZURE_OPENAI_API_KEY and AZURE_OPENAI_ENDPOINT for Azure OpenAI).PromptSendingOrchestrator for single-turn tests, RedTeamingOrchestrator for multi-turn, or CrescendoOrchestrator for escalation attacks.When to use PyRIT
PyRIT fits teams that need programmatic, repeatable red teaming of AI systems. The orchestrator/converter/scorer architecture lets you build custom attack workflows that run consistently across different targets and over time.
The multi-modal support matters if you’re testing vision models, audio transcription, or document understanding beyond text-only LLMs. The memory system is useful for teams that need to track findings over multiple sessions and share results.
The 117 contributors and regular releases (17 total, latest in February 2026) show active development grounded in real-world AI security work.
For a broader overview of AI security risks, see our AI security guide. For simpler LLM testing with a CLI-first approach, Promptfoo has a lower barrier to entry. Garak focuses on vulnerability scanning with built-in probe modules. For runtime protection rather than testing, look at LLM Guard or NeMo Guardrails. DeepTeam provides unit-testing-style LLM vulnerability checks.
