Documentation Index
Fetch the complete documentation index at: https://docs.praison.ai/llms.txt
Use this file to discover all available pages before exploring further.
Real API Testing CLI
Commands for running integration tests with real API keys.
Commands
Run All Real API Tests
export OPENAI_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py -v
Output:
tests/integration/test_real_api.py::TestAgentRealAPI::test_agent_simple_chat PASSED
tests/integration/test_real_api.py::TestAgentRealAPI::test_agent_with_tool PASSED
tests/integration/test_real_api.py::TestAgentRealAPI::test_agent_chat_history PASSED
tests/integration/test_real_api.py::TestLiteAgentRealAPI::test_lite_agent_with_openai PASSED
tests/integration/test_real_api.py::TestLiteAgentRealAPI::test_lite_agent_no_litellm_loaded PASSED
Run Specific Provider Tests
OpenAI Tests
export OPENAI_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py::TestAgentRealAPI -v
Anthropic Tests
export ANTHROPIC_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py::TestAnthropicAPI -v
Google Gemini Tests
export GOOGLE_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py::TestGoogleAPI -v
Run LiteAgent Tests
export OPENAI_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py::TestLiteAgentRealAPI -v
Quick Verification
Verify Agent Works
export OPENAI_API_KEY="your-key"
python -c "
from praisonaiagents import Agent
agent = Agent(
name='Test',
instructions='Reply with one word.',
llm='gpt-4o-mini',
output="silent"
)
response = agent.chat('Say hello')
print(f'Response: {response}')
print('Agent test: OK')
"
Verify LiteAgent Works
export OPENAI_API_KEY="your-key"
python -c "
from praisonaiagents.lite import LiteAgent, create_openai_llm_fn
llm_fn = create_openai_llm_fn(model='gpt-4o-mini')
agent = LiteAgent(name='Test', llm_fn=llm_fn)
response = agent.chat('Say hi')
print(f'Response: {response}')
print('LiteAgent test: OK')
"
export OPENAI_API_KEY="your-key"
python -c "
from praisonaiagents import Agent
def add(a: int, b: int) -> int:
'''Add two numbers.'''
return a + b
agent = Agent(
name='Math',
instructions='Use tools.',
llm='gpt-4o-mini',
tools=[add],
output="silent"
)
response = agent.chat('What is 5+3?')
assert '8' in response, f'Expected 8 in response: {response}'
print('Tool test: OK')
"
Environment Variables
| Variable | Description | Required |
|---|
RUN_REAL_KEY_TESTS | Enable real API tests | Yes |
OPENAI_API_KEY | OpenAI API key | For OpenAI tests |
ANTHROPIC_API_KEY | Anthropic API key | For Anthropic tests |
GOOGLE_API_KEY | Google API key | For Gemini tests |
CI/CD Integration
GitHub Actions
name: Real API Tests
on:
workflow_dispatch:
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- run: pip install -e ".[test]"
- name: Run Tests
env:
RUN_REAL_KEY_TESTS: "1"
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: python -m pytest tests/integration/test_real_api.py -v
Shell Script
#!/bin/bash
set -e
export RUN_REAL_KEY_TESTS=1
if [ -z "$OPENAI_API_KEY" ]; then
echo "Error: OPENAI_API_KEY not set"
exit 1
fi
python -m pytest tests/integration/test_real_api.py -v
echo "All real API tests passed!"
Troubleshooting
Tests Skipped
If tests are skipped, ensure environment variable is set:
# Check if variable is set
echo $RUN_REAL_KEY_TESTS
# Set it
export RUN_REAL_KEY_TESTS=1
API Key Errors
# Verify key is set
python -c "import os; print('Key set:', bool(os.environ.get('OPENAI_API_KEY')))"
Rate Limits
If hitting rate limits, add delays between tests:
python -m pytest tests/integration/test_real_api.py -v --tb=short -x