Skip to main content

Agent Observability

Monitor your Agents in production. Track every LLM call, tool execution, and decision your Agents make. Essential for debugging, optimization, and compliance.

Agent with Tracing

import { Agent, MemoryObservabilityAdapter, setObservabilityAdapter } from 'praisonai';

// Enable observability globally
const obs = new MemoryObservabilityAdapter();
setObservabilityAdapter(obs);

const agent = new Agent({
  name: 'Support Agent',
  instructions: 'Help users with their questions.',
  observability: true  // Enable tracing for this Agent
});

// All Agent actions are now traced
const response = await agent.chat('How do I reset my password?');

// View what happened
const traces = obs.getAllTraces();
for (const trace of traces) {
  console.log(`Trace: ${trace.name}`);
  for (const span of trace.spans) {
    console.log(`  ${span.type}: ${span.name} (${span.duration}ms)`);
  }
}

Multi-Agent Tracing

Track interactions between multiple Agents:
import { Agent, PraisonAIAgents, MemoryObservabilityAdapter, setObservabilityAdapter } from 'praisonai';

const obs = new MemoryObservabilityAdapter();
setObservabilityAdapter(obs);

const researcher = new Agent({
  name: 'Researcher',
  instructions: 'Research topics thoroughly.',
  observability: true
});

const writer = new Agent({
  name: 'Writer',
  instructions: 'Write clear summaries.',
  observability: true
});

const agents = new PraisonAIAgents({
  agents: [researcher, writer],
  tasks: [
    { agent: researcher, description: 'Research: {topic}' },
    { agent: writer, description: 'Summarize the research' }
  ],
  observability: true  // Trace the entire workflow
});

await agents.start({ topic: 'AI trends 2024' });

// Analyze the workflow
const traces = obs.getAllTraces();
console.log(`Total traces: ${traces.length}`);
console.log(`Total LLM calls: ${traces.flatMap(t => t.spans).filter(s => s.type === 'llm').length}`);

Agent Performance Monitoring

Track latency, tokens, and costs:
import { Agent, createTool, MemoryObservabilityAdapter, setObservabilityAdapter } from 'praisonai';

const obs = new MemoryObservabilityAdapter();
setObservabilityAdapter(obs);

const agent = new Agent({
  name: 'Analytics Agent',
  instructions: 'Analyze data and provide insights.',
  observability: true
});

// Run multiple queries
await agent.chat('Analyze sales trends');
await agent.chat('Compare Q1 vs Q2');
await agent.chat('Predict Q3 performance');

// Analyze Agent performance
const traces = obs.getAllTraces();
const llmSpans = traces.flatMap(t => t.spans).filter(s => s.type === 'llm');

const stats = {
  totalCalls: llmSpans.length,
  avgLatency: llmSpans.reduce((sum, s) => sum + s.duration, 0) / llmSpans.length,
  totalTokens: llmSpans.reduce((sum, s) => sum + (s.attributes?.tokens || 0), 0)
};

console.log('Agent Performance:');
console.log(`  LLM Calls: ${stats.totalCalls}`);
console.log(`  Avg Latency: ${stats.avgLatency.toFixed(0)}ms`);
console.log(`  Total Tokens: ${stats.totalTokens}`);

Agent Debugging

Debug Agent decisions and tool calls:
import { Agent, createTool, ConsoleObservabilityAdapter, setObservabilityAdapter } from 'praisonai';

// Console adapter logs everything in real-time
const obs = new ConsoleObservabilityAdapter();
setObservabilityAdapter(obs);

const searchTool = createTool({
  name: 'search',
  description: 'Search for information',
  parameters: {
    type: 'object',
    properties: { query: { type: 'string' } },
    required: ['query']
  },
  execute: async ({ query }) => `Results for: ${query}`
});

const agent = new Agent({
  name: 'Debug Agent',
  instructions: 'Search for information to answer questions.',
  tools: [searchTool],
  observability: true
});

await agent.chat('Find information about TypeScript');

// Console output shows:
// [TRACE START] agent-execution
//   [SPAN START] llm-call (llm)
//   [SPAN END] llm-call - completed (1234ms)
//   [SPAN START] tool-call (tool) - search
//   [SPAN END] tool-call - completed (56ms)
//   [SPAN START] llm-call (llm)
//   [SPAN END] llm-call - completed (987ms)
// [TRACE END] agent-execution - completed

Langfuse Integration

Send Agent traces to Langfuse for production monitoring:
import { Agent, LangfuseObservabilityProvider, setObservabilityAdapter } from 'praisonai';

const langfuse = new LangfuseObservabilityProvider({
  publicKey: process.env.LANGFUSE_PUBLIC_KEY!,
  secretKey: process.env.LANGFUSE_SECRET_KEY!,
  baseUrl: 'https://cloud.langfuse.com'
});

setObservabilityAdapter(langfuse);

const agent = new Agent({
  name: 'Production Agent',
  instructions: 'Handle customer inquiries.',
  observability: true
});

// All traces automatically sent to Langfuse
await agent.chat('I need help with my order');

// View in Langfuse dashboard:
// - Trace timeline
// - Token usage
// - Latency breakdown
// - Cost tracking

Custom Observability for Agents

Build custom monitoring:
import { Agent, ObservabilityAdapter, TraceContext, SpanContext } from 'praisonai';

class DatadogAgentAdapter implements ObservabilityAdapter {
  startTrace(name: string, metadata?: Record<string, any>): TraceContext {
    // Send to Datadog APM
    const traceId = datadogTracer.startSpan(name, { tags: metadata });
    return {
      traceId,
      name,
      startSpan: (spanName, type) => this.createSpan(traceId, spanName, type),
      end: (status) => datadogTracer.finishSpan(traceId, { status })
    };
  }
  
  private createSpan(traceId: string, name: string, type: string): SpanContext {
    const spanId = datadogTracer.startChildSpan(traceId, name, { type });
    return {
      spanId,
      name,
      type,
      setAttributes: (attrs) => datadogTracer.setTags(spanId, attrs),
      addEvent: (event, data) => datadogTracer.addEvent(spanId, event, data),
      end: (status) => datadogTracer.finishSpan(spanId, { status })
    };
  }
}

// Use with Agents
setObservabilityAdapter(new DatadogAgentAdapter());

Agent Audit Logging

Track Agent actions for compliance:
import { Agent, createTool, MemoryObservabilityAdapter, setObservabilityAdapter } from 'praisonai';

const obs = new MemoryObservabilityAdapter();
setObservabilityAdapter(obs);

const sensitiveDataTool = createTool({
  name: 'access_customer_data',
  description: 'Access customer information',
  parameters: {
    type: 'object',
    properties: { customerId: { type: 'string' } },
    required: ['customerId']
  },
  execute: async ({ customerId }) => {
    // Tool execution is automatically traced
    return `Customer data for ${customerId}`;
  }
});

const agent = new Agent({
  name: 'Support Agent',
  instructions: 'Help customers with their accounts.',
  tools: [sensitiveDataTool],
  observability: true
});

await agent.chat('Look up customer C-12345');

// Generate audit log
const auditLog = obs.getAllTraces().flatMap(t => 
  t.spans.filter(s => s.type === 'tool').map(s => ({
    timestamp: s.startTime,
    action: s.name,
    agent: t.metadata?.agentName,
    details: s.attributes
  }))
);

console.log('Audit Log:', JSON.stringify(auditLog, null, 2));

Environment Variables

# Langfuse
LANGFUSE_PUBLIC_KEY=pk-...
LANGFUSE_SECRET_KEY=sk-...