​
Guardrails System

Guardrails provide output validation and quality assurance for agent tasks, ensuring results meet specified criteria before being accepted.

​
Overview

Guardrails ensure task outputs meet quality and safety criteria through:

  • Function-based validation for structured checks
  • LLM-based validation for natural language criteria
  • Automatic retry mechanisms for failed validations
  • Custom validation logic for specific requirements

​
Quick Start

from praisonaiagents import Agent, Task

# Define validation function
def validate_length(task_output):
    """Ensure output is at least 500 words"""
    word_count = len(task_output.raw.split())
    
    if word_count < 500:
        return False, f"Output too short: {word_count} words"
    
    return True, task_output

# Create task with guardrail
task = Task(
    description="Write a detailed article about AI",
    agent=writer_agent,
    guardrail=validate_length,
    expected_output="Article of at least 500 words"
)

​
Guardrail Types

​
Function-Based Guardrails

Function guardrails provide programmatic validation:

def validate_json(task_output):
    """Ensure output is valid JSON"""
    import json
    
    try:
        data = json.loads(task_output.raw)
        return True, task_output
    except json.JSONDecodeError as e:
        return False, f"Invalid JSON: {str(e)}"

​
LLM-Based Guardrails

LLM guardrails use natural language for validation:

task = Task(
    description="Write a children's story",
    agent=writer_agent,
    guardrail="The story must be appropriate for ages 5-8, use simple language, and have a positive message"
)

​
Advanced Features

​
Retry Configuration

Configure retry behaviour for failed validations:

task = Task(
    description="Generate validated content",
    agent=agent,
    guardrail=validation_function,
    max_retries=3,  # Maximum retry attempts
    retry_delay=2,  # Delay between retries (seconds)
    retry_with_feedback=True  # Pass failure reason to agent
)

​
Composite Guardrails

Combine multiple validation criteria:

def composite_guardrail(task_output):
    """Multiple validation checks"""
    # Check 1: Length
    if len(task_output.raw) < 100:
        return False, "Output too short"
    
    # Check 2: Format
    if not task_output.raw.strip().endswith('.'):
        return False, "Output must end with period"
    
    # Check 3: Content quality (using LLM)
    quality_check = LLMGuardrail(
        "Is this content professional and error-free?"
    )
    
    result = quality_check(task_output)
    if not result.success:
        return False, result.error
    
    return True, task_output

​
Guardrail Results

Access detailed validation results:

from praisonaiagents.guardrails import GuardrailResult

class CustomGuardrail:
    def __call__(self, task_output):
        # Perform validation
        if self.validate(task_output):
            return GuardrailResult(
                success=True,
                result=task_output
            )
        else:
            return GuardrailResult(
                success=False,
                error="Validation failed: specific reason",
                details={
                    "score": 0.3,
                    "issues": ["issue1", "issue2"]
                }
            )

​
Use Cases

Content Safety

Ensure generated content is safe and appropriate

guardrail="No offensive, harmful, or inappropriate content"

Data Validation

Validate analysis results and reports

def validate_analysis(output):
    # Check data accuracy
    # Verify calculations
    # Ensure completeness

Code Quality

Ensure generated code is safe and functional

def validate_code(output):
    # Syntax checking
    # Security scanning
    # Best practices

Compliance

Meet regulatory and policy requirements

guardrail="Must comply with GDPR, include privacy notice"

​
Integration Patterns

​
With Agents

from praisonaiagents import Agent, Task, PraisonAIAgents

# Define guardrails
def technical_accuracy(output):
    """Validate technical content"""
    # Implementation
    pass

# Create specialized agents
writer = Agent(
    name="Technical Writer",
    instructions="Write accurate technical content"
)

reviewer = Agent(
    name="Technical Reviewer",
    instructions="Review and improve technical content"
)

# Create tasks with guardrails
tasks = [
    Task(
        description="Write Python tutorial",
        agent=writer,
        guardrail=technical_accuracy,
        expected_output="Accurate Python tutorial"
    ),
    Task(
        description="Review and enhance the tutorial",
        agent=reviewer,
        guardrail="Ensure tutorial is beginner-friendly and error-free",
        expected_output="Polished tutorial"
    )
]

# Run with validation
agents = PraisonAIAgents(agents=[writer, reviewer], tasks=tasks)
result = agents.start()

​
With Dynamic Guardrails

class DynamicGuardrail:
    def __init__(self, config):
        self.min_length = config.get('min_length', 100)
        self.required_sections = config.get('sections', [])
        self.quality_threshold = config.get('quality', 0.8)
    
    def __call__(self, task_output):
        content = task_output.raw
        
        # Length check
        if len(content.split()) < self.min_length:
            return False, f"Minimum {self.min_length} words required"
        
        # Section check
        for section in self.required_sections:
            if section not in content:
                return False, f"Missing section: {section}"
        
        # Quality check (using LLM)
        quality_prompt = f"""
        Rate this content quality from 0-1:
        {content[:500]}...
        
        Consider: clarity, accuracy, completeness
        """
        
        # Implement quality scoring logic
        
        return True, task_output

# Use dynamic guardrail
guardrail = DynamicGuardrail({
    'min_length': 500,
    'sections': ['Introduction', 'Main Content', 'Conclusion'],
    'quality': 0.85
})

task = Task(
    description="Write comprehensive guide",
    agent=agent,
    guardrail=guardrail
)

​
Best Practices

Clear Criteria

  • Define specific, measurable criteria
  • Document validation requirements
  • Provide helpful error messages
  • Include examples of valid output

Balanced Approach

  • Use function guardrails for simple checks
  • Reserve LLM guardrails for complex validation
  • Implement caching for repeated validations

Retry Strategy

  • Set appropriate retry limits
  • Provide detailed failure reasons
  • Suggest corrections when possible

Testing

  • Log validation failures
  • Handle edge cases gracefully
  • Test guardrails independently
  • Verify both pass and fail cases
  • Check retry behaviour
  • Monitor validation performance

​
Complete Example

from praisonaiagents import Agent, Task, PraisonAIAgents
from praisonaiagents.guardrails import LLMGuardrail

# Define agents
researcher = Agent(
    name="Researcher",
    instructions="Research topics thoroughly and provide accurate information"
)

writer = Agent(
    name="Blog Writer",
    instructions="Write engaging, SEO-friendly blog posts"
)

editor = Agent(
    name="Editor",
    instructions="Polish and perfect blog posts"
)

# Define guardrails
def seo_validation(task_output):
    """Validate SEO requirements"""
    content = task_output.raw
    
    # Check title
    lines = content.split('\n')
    if not lines[0].startswith('#'):
        return False, "Missing H1 title"
    
    # Check meta description
    if 'Meta description:' not in content:
        return False, "Missing meta description"
    
    # Check keyword density (example: "AI" keyword)
    keyword_count = content.lower().count('ai')
    word_count = len(content.split())
    density = (keyword_count / word_count) * 100
    
    if density < 1 or density > 3:
        return False, f"Keyword density {density:.1f}% (target: 1-3%)"
    
    return True, task_output

# LLM-based quality guardrail
quality_guardrail = LLMGuardrail(
    description="""
    Evaluate if the blog post:
    1. Has a compelling introduction that hooks readers
    2. Provides valuable, actionable information
    3. Uses clear, concise language
    4. Includes relevant examples or case studies
    5. Has a strong conclusion with call-to-action
    6. Maintains consistent tone throughout
    7. Is free of factual errors
    8. Flows logically from point to point
    """,
    llm="gpt-4"
)

# Create tasks with guardrails
tasks = [
    Task(
        description="Research the topic: 'Future of AI in Healthcare'",
        agent=researcher,
        expected_output="Comprehensive research notes"
    ),
    Task(
        description="Write a 1000-word blog post based on the research",
        agent=writer,
        guardrail=seo_validation,
        expected_output="SEO-optimized blog post"
    ),
    Task(
        description="Edit and polish the blog post",
        agent=editor,
        guardrail=quality_guardrail,
        expected_output="Publication-ready blog post",
        context=[tasks[1]]  # Use previous task output
    )
]

# Run the blog generation pipeline
agents = PraisonAIAgents(
    agents=[researcher, writer, editor],
    tasks=tasks,
    verbose=True
)

# Execute with guardrails
result = agents.start()

# The system will:
# 1. Research the topic
# 2. Write the blog post (retry if SEO validation fails)
# 3. Edit the post (retry if quality standards not met)
# 4. Return the final, validated blog post

​
Next Steps

Learn about human-in-the-loop approvals

Explore advanced task configurations

​
======= description: “Implement validation and quality assurance for agent outputs” icon: “shield”

​
Guardrails

Guardrails provide validation and quality assurance mechanisms for agent outputs, supporting both function-based and LLM-based validation to ensure outputs meet specific criteria.

​
Overview

Guardrails allow you to:

  • Validate agent outputs before they’re returned
  • Implement custom validation logic
  • Use LLM-based validation with natural language criteria
  • Retry operations when validation fails
  • Transform outputs to meet requirements

​
Quick Start

from praisonaiagents import Agent, Task
from praisonaiagents.agents.guardrails import LLMGuardrail

# Create LLM-based guardrail
guardrail = LLMGuardrail(
    description="Ensure the response is professional and under 100 words",
    llm=None  # Uses agent's LLM
)

# Create agent with guardrailed task
agent = Agent(
    name="Customer Service",
    role="Support representative"
)

task = Task(
    description="Respond to angry customer email",
    agent=agent,
    guardrail=guardrail,
    max_retries=3  # Retry up to 3 times if validation fails
)

​
Guardrail Types

​
Function-Based Guardrails

Function guardrails provide programmatic validation:

from praisonaiagents import TaskOutput

def length_guardrail(output: TaskOutput) -> tuple[bool, any]:
    """Ensure output is within length limits"""
    if len(output.raw) > 500:
        return False, "Output too long, please summarize"
    if len(output.raw) < 50:
        return False, "Output too short, please elaborate"
    return True, output

task = Task(
    description="Write a product description",
    guardrail=length_guardrail
)

​
LLM-Based Guardrails

LLM guardrails use natural language validation:

from praisonaiagents.agents.guardrails import LLMGuardrail

# Simple string description
task = Task(
    description="Generate SQL query",
    guardrail="Ensure the SQL query only reads data and contains no DELETE, DROP, or UPDATE statements"
)

# Or using LLMGuardrail class
guardrail = LLMGuardrail(
    description="Validate that the response contains no personally identifiable information (PII) such as names, addresses, phone numbers, or email addresses",
    llm=agent.llm
)

​
Validation Process

​
How It Works

  1. Agent executes the task and produces output

  2. Guardrail validates the output

  3. If validation fails:

    • Task retries (up to max_retries times)
    • Agent receives feedback about what to fix

  4. If validation passes:

    • Output is returned as final result

​
Validation Response Format

Function guardrails return a tuple:

(success: bool, result: any)

Where:

  • success: Whether validation passed
  • result: Modified output or error message

​
Advanced Examples

​
Content Moderation

def content_guardrail(output: TaskOutput) -> tuple[bool, any]:
    """Ensure content is appropriate"""
    prohibited_words = ["spam", "scam", "illegal"]
    content_lower = output.raw.lower()
    
    for word in prohibited_words:
        if word in content_lower:
            return False, f"Content contains prohibited word: {word}"
    
    return True, output

agent = Agent(
    name="Content Creator",
    role="Blog writer"
)

task = Task(
    description="Write article about online safety",
    agent=agent,
    guardrail=content_guardrail
)

​
Data Format Validation

import json

def json_guardrail(output: TaskOutput) -> tuple[bool, any]:
    """Ensure output is valid JSON with required fields"""
    try:
        data = json.loads(output.raw)
        required_fields = ["id", "name", "price"]
        
        for field in required_fields:
            if field not in data:
                return False, f"Missing required field: {field}"
        
        if not isinstance(data["price"], (int, float)):
            return False, "Price must be a number"
            
        return True, output
    except json.JSONDecodeError:
        return False, "Output must be valid JSON"

task = Task(
    description="Generate product data in JSON format",
    guardrail=json_guardrail,
    max_retries=5
)

​
Combining Multiple Validations

def combined_guardrail(output: TaskOutput) -> tuple[bool, any]:
    """Apply multiple validation checks"""
    checks = [
        (lambda o: len(o.raw) < 1000, "Output too long"),
        (lambda o: "copyright" not in o.raw.lower(), "Contains copyright material"),
        (lambda o: o.raw.count("\n") < 10, "Too many line breaks")
    ]
    
    for check_func, error_msg in checks:
        if not check_func(output):
            return False, error_msg
    
    return True, output

​
Output Transformation

def transform_guardrail(output: TaskOutput) -> tuple[bool, any]:
    """Transform output to meet requirements"""
    # Clean up the output
    cleaned = output.raw.strip()
    cleaned = " ".join(cleaned.split())  # Normalize whitespace
    
    # Add required prefix/suffix
    if not cleaned.startswith("Summary:"):
        cleaned = f"Summary: {cleaned}"
    
    # Create new output with transformation
    new_output = TaskOutput(
        description=output.description,
        agent=output.agent,
        raw=cleaned,
        json_output=output.json_output,
        output_format=output.output_format
    )
    
    return True, new_output

​
Complex LLM Validation

​
Multi-Criteria Validation

guardrail = LLMGuardrail(
    description="""
    Validate the customer service response meets these criteria:
    1. Professional and empathetic tone
    2. Addresses all customer concerns mentioned
    3. Provides clear next steps or resolution
    4. Includes appropriate greeting and closing
    5. No grammar or spelling errors
    6. Between 50-150 words
    """,
    llm=agent.llm
)

​
Domain-Specific Validation

medical_guardrail = LLMGuardrail(
    description="""
    Ensure the medical information:
    - Contains disclaimer about consulting healthcare professionals
    - Avoids definitive diagnoses
    - References only peer-reviewed sources
    - Uses appropriate medical terminology
    - Includes no treatment recommendations without professional consultation
    """,
    llm=agent.llm
)

​
Integration Patterns

​
With PraisonAIAgents

from praisonaiagents import PraisonAIAgents

agents = PraisonAIAgents(
    agents=[agent1, agent2],
    tasks=[
        Task(
            description="Generate report",
            agent=agent1,
            guardrail="Ensure report includes executive summary, findings, and recommendations"
        ),
        Task(
            description="Review report",
            agent=agent2,
            guardrail=quality_guardrail
        )
    ]
)

​
Conditional Guardrails

def get_guardrail(task_type):
    """Return appropriate guardrail based on task type"""
    if task_type == "code":
        return "Ensure code is syntactically correct and includes comments"
    elif task_type == "email":
        return "Ensure email is professional and under 200 words"
    else:
        return None

task = Task(
    description="Write Python function",
    guardrail=get_guardrail("code")
)

​
Async Guardrails

async def async_guardrail(output: TaskOutput) -> tuple[bool, any]:
    """Async validation with external API"""
    # Check with external service
    is_valid = await check_content_api(output.raw)
    
    if not is_valid:
        return False, "Content failed external validation"
    
    return True, output

​
Error Handling

from praisonaiagents.agents.guardrails import GuardrailResult

try:
    # Task with guardrail
    result = task.execute()
except Exception as e:
    # Guardrail validation failed after all retries
    print(f"Task failed validation: {e}")

​
Best Practices

  1. Clear Criteria - Make validation criteria specific and measurable
  2. Helpful Feedback - Provide clear error messages for failed validations
  3. Appropriate Retries - Set max_retries based on task complexity
  4. Performance - Consider validation overhead, especially for LLM-based guardrails
  5. Combine Approaches - Use function guardrails for simple checks, LLM for complex validation
  6. Test Thoroughly - Test guardrails with various outputs including edge cases
  7. Fail Gracefully - Have fallback behavior when validation consistently fails