Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.praison.ai/llms.txt

Use this file to discover all available pages before exploring further.

Real API Testing CLI

Commands for running integration tests with real API keys.

Commands

Run All Real API Tests

export OPENAI_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py -v
Output:
tests/integration/test_real_api.py::TestAgentRealAPI::test_agent_simple_chat PASSED
tests/integration/test_real_api.py::TestAgentRealAPI::test_agent_with_tool PASSED
tests/integration/test_real_api.py::TestAgentRealAPI::test_agent_chat_history PASSED
tests/integration/test_real_api.py::TestLiteAgentRealAPI::test_lite_agent_with_openai PASSED
tests/integration/test_real_api.py::TestLiteAgentRealAPI::test_lite_agent_no_litellm_loaded PASSED

Run Specific Provider Tests

OpenAI Tests

export OPENAI_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py::TestAgentRealAPI -v

Anthropic Tests

export ANTHROPIC_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py::TestAnthropicAPI -v

Google Gemini Tests

export GOOGLE_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py::TestGoogleAPI -v

Run LiteAgent Tests

export OPENAI_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py::TestLiteAgentRealAPI -v

Quick Verification

Verify Agent Works

export OPENAI_API_KEY="your-key"
python -c "
from praisonaiagents import Agent

agent = Agent(
    name='Test',
    instructions='Reply with one word.',
    llm='gpt-4o-mini',
    output="silent"
)

response = agent.chat('Say hello')
print(f'Response: {response}')
print('Agent test: OK')
"

Verify LiteAgent Works

export OPENAI_API_KEY="your-key"
python -c "
from praisonaiagents.lite import LiteAgent, create_openai_llm_fn

llm_fn = create_openai_llm_fn(model='gpt-4o-mini')
agent = LiteAgent(name='Test', llm_fn=llm_fn)

response = agent.chat('Say hi')
print(f'Response: {response}')
print('LiteAgent test: OK')
"

Verify Tool Execution

export OPENAI_API_KEY="your-key"
python -c "
from praisonaiagents import Agent

def add(a: int, b: int) -> int:
    '''Add two numbers.'''
    return a + b

agent = Agent(
    name='Math',
    instructions='Use tools.',
    llm='gpt-4o-mini',
    tools=[add],
    output="silent"
)

response = agent.chat('What is 5+3?')
assert '8' in response, f'Expected 8 in response: {response}'
print('Tool test: OK')
"

Environment Variables

VariableDescriptionRequired
RUN_REAL_KEY_TESTSEnable real API testsYes
OPENAI_API_KEYOpenAI API keyFor OpenAI tests
ANTHROPIC_API_KEYAnthropic API keyFor Anthropic tests
GOOGLE_API_KEYGoogle API keyFor Gemini tests

CI/CD Integration

GitHub Actions

name: Real API Tests

on:
  workflow_dispatch:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - run: pip install -e ".[test]"
      - name: Run Tests
        env:
          RUN_REAL_KEY_TESTS: "1"
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: python -m pytest tests/integration/test_real_api.py -v

Shell Script

#!/bin/bash
set -e

export RUN_REAL_KEY_TESTS=1

if [ -z "$OPENAI_API_KEY" ]; then
    echo "Error: OPENAI_API_KEY not set"
    exit 1
fi

python -m pytest tests/integration/test_real_api.py -v
echo "All real API tests passed!"

Troubleshooting

Tests Skipped

If tests are skipped, ensure environment variable is set:
# Check if variable is set
echo $RUN_REAL_KEY_TESTS

# Set it
export RUN_REAL_KEY_TESTS=1

API Key Errors

# Verify key is set
python -c "import os; print('Key set:', bool(os.environ.get('OPENAI_API_KEY')))"

Rate Limits

If hitting rate limits, add delays between tests:
python -m pytest tests/integration/test_real_api.py -v --tb=short -x