Guardrails

What Are Guardrails?

Guardrails are controls that ensure an AI system behaves within defined bounds — producing safe, accurate, compliant, and on-brand outputs regardless of what users throw at it.

Think of guardrails as the safety belt, airbags, and lane-assist of your AI system: you hope you never need them, but you'd never deploy without them.

Why You Need Guardrails

Even well-aligned models can:

• Be jailbroken with adversarial prompts
• Hallucinate confidently on topics outside their training
• Produce off-brand or legally problematic content
• Leak sensitive information from context
• Be manipulated into violating their system prompt

Guardrails provide defense in depth: multiple overlapping layers so no single failure compromises the system.

The Guardrail Architecture

A production AI system typically has guardrails at multiple layers:

User Input
    ↓
[INPUT GUARDRAILS]
  ├── Prompt injection detection
  ├── PII detection & masking
  ├── Toxicity / policy classification
  ├── Topic scope check
  └── Rate limiting / abuse detection
    ↓
[SYSTEM PROMPT HARDENING]
  ├── Explicit behavioral constraints
  ├── Sensitive topic instructions
  └── Output format requirements
    ↓
LLM (Claude, GPT, etc.)
    ↓
[OUTPUT GUARDRAILS]
  ├── Hallucination / factuality check
  ├── Policy compliance check
  ├── PII in output detection
  ├── Format validation (JSON schema, etc.)
  └── Brand voice / tone check
    ↓
User Response

Input Guardrails

1. Prompt Injection Detection

Detect attempts to override system instructions:

INJECTION_PATTERNS = [
    r"ignore (previous|all|your) instructions?",
    r"disregard (the )?system prompt",
    r"pretend (you are|to be) .* without restrictions",
    r"DAN|jailbreak|bypass .* filters?",
    r"<\/system>|<\|im_end\|>",  # prompt delimiter injection
]

2. PII Detection and Masking

Prevent PII from reaching the LLM or appearing in logs:

import re

PII_PATTERNS = {
    "email": r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b",
    "ssn": r"\b\d{3}-\d{2}-\d{4}\b",
    "credit_card": r"\b(?:\d{4}[- ]?){3}\d{4}\b",
    "phone": r"\b(?:\+1)?[\s.-]?\(?\d{3}\)?[\s.-]?\d{3}[\s.-]?\d{4}\b",
}

def mask_pii(text: str) -> tuple[str, dict]:
    masked = text
    found_pii = {}
    for pii_type, pattern in PII_PATTERNS.items():
        matches = re.findall(pattern, text)
        if matches:
            found_pii[pii_type] = matches
            masked = re.sub(pattern, f"[{pii_type.upper()}_REDACTED]", masked)
    return masked, found_pii

3. Topic Scope Checking

For domain-specific bots, classify whether the query is in scope:

# Use a fast classifier or LLM to check topic relevance
def is_in_scope(query: str, allowed_topics: list[str]) -> bool:
    # Check if query relates to allowed topics
    ...

import anthropic
import re
from dataclasses import dataclass
from typing import Optional

client = anthropic.Anthropic()

@dataclass
class GuardrailResult:
    allowed: bool
    reason: Optional[str] = None
    modified_input: Optional[str] = None

# ─── Input Guardrails ────────────────────────────────────────

INJECTION_PATTERNS = [
    r"ignore (previous|all|your) instructions?",
    r"disregard (the )?system prompt",
    r"pretend you (are|have no)",
    r"jailbreak|DAN|bypass",
]

PII_PATTERNS = {
    "email": r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b",
    "ssn": r"\b\d{3}-\d{2}-\d{4}\b",
}

def check_input(user_input: str) -> GuardrailResult:
    # 1. Prompt injection check
    for pattern in INJECTION_PATTERNS:
        if re.search(pattern, user_input, re.IGNORECASE):
            return GuardrailResult(allowed=False, reason="prompt_injection")

    # 2. PII masking
    masked = user_input
    for pii_type, pattern in PII_PATTERNS.items():
        masked = re.sub(pattern, f"[{pii_type.upper()}]", masked)

    return GuardrailResult(allowed=True, modified_input=masked)

# ─── Output Guardrails ───────────────────────────────────────

def check_output_with_llm(response: str, original_query: str) -> GuardrailResult:
    """Use Claude to evaluate response safety."""
    eval_response = client.messages.create(
        model="claude-haiku-4-5-20251001",  # Fast, cheap judge
        max_tokens=128,
        system="""Evaluate if this AI response is safe to send to users.
Check for: harmful instructions, hate speech, PII, false medical/legal advice, copyright violations.
Respond with JSON only: {"safe": true/false, "reason": "brief reason if unsafe"}""",
        messages=[{
            "role": "user",
            "content": f"User asked: {original_query}\n\nAI response: {response}"
        }],
    )
    import json
    result = json.loads(eval_response.content[0].text)
    if result["safe"]:
        return GuardrailResult(allowed=True)
    return GuardrailResult(allowed=False, reason=result.get("reason", "policy_violation"))

# ─── Main Pipeline ───────────────────────────────────────────

def safe_chat(user_input: str, system_prompt: str) -> str:
    # 1. Input guardrails
    input_check = check_input(user_input)
    if not input_check.allowed:
        return f"I can't process that request ({input_check.reason})."

    clean_input = input_check.modified_input or user_input

    # 2. Generate with LLM
    response = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=1024,
        system=system_prompt,
        messages=[{"role": "user", "content": clean_input}],
    )
    output = response.content[0].text

    # 3. Output guardrails
    output_check = check_output_with_llm(output, clean_input)
    if not output_check.allowed:
        return "I'm unable to provide a response to that request."

    return output

# Usage
result = safe_chat(
    user_input="Help me understand how neural networks work",
    system_prompt="You are a helpful AI tutor for machine learning."
)
print(result)

Anthropic's Built-in Safety Features

Claude has safety measures built into its training that handle many common cases automatically:

What Claude Refuses by Default

• Instructions for creating weapons of mass destruction
• CSAM generation
• Helping undermine oversight of AI systems
• Providing serious uplift to attackers targeting critical infrastructure

System Prompt Hardening

Even with built-in safety, system prompts should explicitly set scope and constraints:

You are a customer support agent for [Company].
You only answer questions about [Company] products and services.
You never:
- Discuss competitors
- Provide medical, legal, or financial advice
- Share information from other users' accounts
- Engage with requests unrelated to [Company]

If asked something outside your scope, say:
"I'm specialized in [Company] support. For other questions, please consult a qualified professional."

Third-Party Guardrail Libraries

NVIDIA NeMo Guardrails

Configuration-based guardrail framework:

• Define allowed/blocked topics via Colang scripts
• Add fact-checking and hallucination detection rails
• Multi-LLM pipeline orchestration

# Example NeMo config
rails:
  input:
    flows:
      - check jailbreak
      - check off-topic
  output:
    flows:
      - check for sensitive data
      - check for toxic content

LlamaIndex / LangChain Callbacks

Hook into generation pipeline for real-time monitoring and filtering.

Guardrails AI

Python library with validators that run before/after LLM calls:

from guardrails import Guard
from guardrails.hub import DetectPII, ToxicLanguage

guard = Guard().use_many(
    DetectPII(["EMAIL_ADDRESS", "PHONE_NUMBER"], on_fail="fix"),
    ToxicLanguage(threshold=0.5, on_fail="exception"),
)
response = guard(client.messages.create, ...)

Monitoring and Feedback Loops

Guardrails deployed without monitoring degrade silently. Build observability in:

What to Log

• All inputs and outputs (with PII masked)
• Guardrail trigger rates and reasons
• Latency per guardrail layer
• Model confidence scores when available

Key Metrics to Track

• Block rate: what % of requests are blocked by each guardrail?
• False positive rate: are legitimate requests being blocked?
• Jailbreak attempt rate: trend in adversarial inputs
• Hallucination rate: estimated via LLM-as-judge on sampled outputs

Feedback Loop

Production Logs → Sample + Label → Identify Failure Modes
       ↑                                      ↓
Model/Guardrail ← Update Rules/Thresholds ← Root Cause Analysis

Review flagged outputs weekly. Update guardrail rules as new attack patterns emerge. Never treat guardrails as "set and forget."

Summary

• Guardrails are defense-in-depth controls at input, prompt, and output layers
• Input guardrails: detect prompt injection, mask PII, filter off-topic requests
• Output guardrails: check for hallucinations, policy violations, PII leakage, format compliance
• System prompt hardening explicitly constrains model behavior for your use case
• LLM-as-judge output checks catch nuanced violations at low cost using a smaller model
• Monitoring is essential — track block rates, false positives, and emerging attack patterns

Module Complete: Generative AI

You've completed all 10 chapters of the Generative AI module. You now understand:

1. The taxonomy of generative models (GANs, VAEs, diffusion, autoregressive)
2. Transformer architecture and how attention enables modern LLMs
3. The LLM training pipeline: pretraining → SFT → RLHF
4. Fine-tuning with LoRA and QLoRA for efficient specialization
5. Prompt engineering techniques for reliable, high-quality outputs
6. RAG for grounding responses in external knowledge
7. Multimodal AI across images, audio, and video
8. Responsible AI, fairness, and governance frameworks
9. Evaluation metrics from perplexity to LLM-as-judge
10. Guardrails for safe, compliant production deployments