Skip to main content

Real API Testing CLI

Commands for running integration tests with real API keys.

Commands

Run All Real API Tests

export OPENAI_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py -v
Output:
tests/integration/test_real_api.py::TestAgentRealAPI::test_agent_simple_chat PASSED
tests/integration/test_real_api.py::TestAgentRealAPI::test_agent_with_tool PASSED
tests/integration/test_real_api.py::TestAgentRealAPI::test_agent_chat_history PASSED
tests/integration/test_real_api.py::TestLiteAgentRealAPI::test_lite_agent_with_openai PASSED
tests/integration/test_real_api.py::TestLiteAgentRealAPI::test_lite_agent_no_litellm_loaded PASSED

Run Specific Provider Tests

OpenAI Tests

export OPENAI_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py::TestAgentRealAPI -v

Anthropic Tests

export ANTHROPIC_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py::TestAnthropicAPI -v

Google Gemini Tests

export GOOGLE_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py::TestGoogleAPI -v

Run LiteAgent Tests

export OPENAI_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py::TestLiteAgentRealAPI -v

Quick Verification

Verify Agent Works

export OPENAI_API_KEY="your-key"
python -c "
from praisonaiagents import Agent

agent = Agent(
    name='Test',
    instructions='Reply with one word.',
    llm='gpt-4o-mini',
    verbose=False
)

response = agent.chat('Say hello')
print(f'Response: {response}')
print('Agent test: OK')
"

Verify LiteAgent Works

export OPENAI_API_KEY="your-key"
python -c "
from praisonaiagents.lite import LiteAgent, create_openai_llm_fn

llm_fn = create_openai_llm_fn(model='gpt-4o-mini')
agent = LiteAgent(name='Test', llm_fn=llm_fn)

response = agent.chat('Say hi')
print(f'Response: {response}')
print('LiteAgent test: OK')
"

Verify Tool Execution

export OPENAI_API_KEY="your-key"
python -c "
from praisonaiagents import Agent

def add(a: int, b: int) -> int:
    '''Add two numbers.'''
    return a + b

agent = Agent(
    name='Math',
    instructions='Use tools.',
    llm='gpt-4o-mini',
    tools=[add],
    verbose=False
)

response = agent.chat('What is 5+3?')
assert '8' in response, f'Expected 8 in response: {response}'
print('Tool test: OK')
"

Environment Variables

VariableDescriptionRequired
RUN_REAL_KEY_TESTSEnable real API testsYes
OPENAI_API_KEYOpenAI API keyFor OpenAI tests
ANTHROPIC_API_KEYAnthropic API keyFor Anthropic tests
GOOGLE_API_KEYGoogle API keyFor Gemini tests

CI/CD Integration

GitHub Actions

name: Real API Tests

on:
  workflow_dispatch:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - run: pip install -e ".[test]"
      - name: Run Tests
        env:
          RUN_REAL_KEY_TESTS: "1"
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: python -m pytest tests/integration/test_real_api.py -v

Shell Script

#!/bin/bash
set -e

export RUN_REAL_KEY_TESTS=1

if [ -z "$OPENAI_API_KEY" ]; then
    echo "Error: OPENAI_API_KEY not set"
    exit 1
fi

python -m pytest tests/integration/test_real_api.py -v
echo "All real API tests passed!"

Troubleshooting

Tests Skipped

If tests are skipped, ensure environment variable is set:
# Check if variable is set
echo $RUN_REAL_KEY_TESTS

# Set it
export RUN_REAL_KEY_TESTS=1

API Key Errors

# Verify key is set
python -c "import os; print('Key set:', bool(os.environ.get('OPENAI_API_KEY')))"

Rate Limits

If hitting rate limits, add delays between tests:
python -m pytest tests/integration/test_real_api.py -v --tb=short -x