Skip to main content

​
eval

AI Agent PraisonAI Agents Evaluation Framework. Provides comprehensive evaluation capabilities for AI agents with zero performance impact when not in use through lazy loading. Evaluator Types:
  • AccuracyEvaluator: Compare output against expected output using LLM-as-judge
  • PerformanceEvaluator: Measure runtime and memory usage
  • ReliabilityEvaluator: Verify expected tool calls are made
  • CriteriaEvaluator: Evaluate against custom criteria
Example: >>> from praisonaiagents.eval import AccuracyEvaluator >>> evaluator = AccuracyEvaluator( … agent=my_agent, … input_text=“What is 2+2?”, … expected_output=“4” … ) >>> result = evaluator.run(print_summary=True)

​
Import

from praisonaiagents import eval

​
Constants

NameValue
_LAZY_IMPORTS{'BaseEvaluator': ('base', 'BaseEvaluator'), 'AccuracyEvaluator': ('accuracy', 'AccuracyEvaluator'), 'PerformanceEvaluator': ('performance', 'PerformanceEvaluator'), 'ReliabilityEvaluator': ('reliabil...

Evaluation Concept

LLM as Judge

Evaluation Loop