NVIDIA NeMo Guardrails is an open-source AI security toolkit for adding programmable guardrails to LLM-based conversational applications. It has 5.6k GitHub stars and 597 forks.
The project is Apache 2.0 licensed and maintained by NVIDIA. The latest release is v0.20.0 (January 2026). It requires Python 3.10-3.13. The framework was published as a research paper at EMNLP 2023 (arXiv:2310.10501).
What is NeMo Guardrails?
NeMo Guardrails intercepts the traffic between your application and its LLM, applying configurable safety checks that block or modify content based on defined policies. The main difference from other guardrail tools is dialog management. Most solutions only filter individual inputs and outputs. NeMo Guardrails models entire conversation flows.
The toolkit uses Colang, a domain-specific language purpose-built for defining conversational guardrails. Colang lets you declaratively specify what conversations should look like, which topics are allowed, and how the system should respond to specific user intents. Both Colang 1.0 and the newer Colang 2.0 are supported.
Five rail types cover different stages of the LLM interaction: input rails process user messages, dialog rails control conversation flow, retrieval rails filter knowledge base results, execution rails gate tool/action calls, and output rails validate responses.
Key Features
| Feature | Details |
|---|---|
| Rail Types | Input, dialog, retrieval, execution, output |
| Colang Versions | 1.0 (default) and 2.0 |
| Input Rails | Jailbreak detection, prompt injection, content moderation, intent classification |
| Output Rails | Fact-checking, hallucination detection, sensitive data blocking, response validation |
| Dialog Rails | Topic boundaries, conversation flows, canonical forms, branching logic |
| LLM Providers | OpenAI, Azure, Anthropic, HuggingFace, NVIDIA NIM, LLaMA, Falcon, Vicuna, Mosaic |
| Frameworks | LangChain, LangGraph, custom chains |
| Deployment | Python API, FastAPI server, Docker, NeMo Microservice |
| Observability | OpenTelemetry tracing, structured logging, performance evaluation |
| License | Apache 2.0 |
Input Rails
Input rails process user messages before they reach the LLM. They handle jailbreak detection, prompt injection filtering, content moderation, and intent classification. You can use NVIDIA’s built-in safety models or plug in third-party providers.
Dialog Rails
Dialog rails are what make NeMo Guardrails different from input/output-only tools. They model the intended flow of conversations using Colang, keeping the LLM on track across multiple turns. If a user tries to steer the conversation off-topic or into restricted areas, dialog rails redirect it back.
Retrieval and Execution Rails
Retrieval rails filter and validate knowledge base results before they reach the LLM, reducing hallucination risk. Execution rails gate tool calls and actions, preventing the LLM from triggering operations it shouldn’t.
Output Rails
Output rails validate LLM responses before they reach the user. Built-in checks include fact-checking against knowledge bases, hallucination detection, sensitive data blocking, and response quality validation.
Colang Language
Colang is purpose-built for defining conversational guardrails. It uses a Python-like syntax where you define user intents, bot responses, and flows that connect them. Colang 2.0 adds events, actions, and more direct LLM integration.
Example Colang 1.0 flow:
define user express greeting
"Hello!"
"Good afternoon!"
define flow
user express greeting
bot express greeting
bot offer to help
Getting Started
pip install nemoguardrails. Requires Python 3.10-3.13 and a C++ compiler (needed for the Annoy library dependency).config/ folder with a config.yml file specifying your LLM provider and which rails to enable (input, output, dialog)..co files in your config directory to define user intents, bot responses, and conversation flows that enforce your safety policies.LLMRails Python API for direct integration, or launch the built-in FastAPI server with nemoguardrails server for HTTP access.When to use NeMo Guardrails
NeMo Guardrails makes sense when you need control over conversational AI behavior beyond input/output filtering. The dialog rails let you model how conversations should flow, which matters for customer service bots, enterprise assistants, or any application where multi-turn conversation control is needed.
The Colang language gives you declarative control that would require complex state management code in other frameworks. If you need to keep conversations on topic, implement branching dialog policies, or integrate fact-checking against knowledge bases, NeMo Guardrails handles these natively.
Organizations using NVIDIA’s AI ecosystem benefit from native compatibility with NVIDIA NIM models and NVIDIA AI Enterprise.
For a broader overview of LLM security risks, see our AI security guide. For input/output-only scanning without dialog management, LLM Guard is lighter weight. For adversarial testing and red teaming, look at Garak, PyRIT, or Promptfoo. Lakera Guard provides guardrails as a managed cloud API.
