> ## Documentation Index
> Fetch the complete documentation index at: https://docs.praison.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Real API Testing CLI

> CLI commands for running integration tests with real API keys

# Real API Testing CLI

Commands for running integration tests with real API keys.

## Commands

### Run All Real API Tests

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
export OPENAI_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py -v
```

**Output:**

```
tests/integration/test_real_api.py::TestAgentRealAPI::test_agent_simple_chat PASSED
tests/integration/test_real_api.py::TestAgentRealAPI::test_agent_with_tool PASSED
tests/integration/test_real_api.py::TestAgentRealAPI::test_agent_chat_history PASSED
tests/integration/test_real_api.py::TestLiteAgentRealAPI::test_lite_agent_with_openai PASSED
tests/integration/test_real_api.py::TestLiteAgentRealAPI::test_lite_agent_no_litellm_loaded PASSED
```

### Run Specific Provider Tests

#### OpenAI Tests

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
export OPENAI_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py::TestAgentRealAPI -v
```

#### Anthropic Tests

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
export ANTHROPIC_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py::TestAnthropicAPI -v
```

#### Google Gemini Tests

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
export GOOGLE_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py::TestGoogleAPI -v
```

### Run LiteAgent Tests

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
export OPENAI_API_KEY="your-key"
export RUN_REAL_KEY_TESTS=1
python -m pytest tests/integration/test_real_api.py::TestLiteAgentRealAPI -v
```

## Quick Verification

### Verify Agent Works

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
export OPENAI_API_KEY="your-key"
python -c "
from praisonaiagents import Agent

agent = Agent(
    name='Test',
    instructions='Reply with one word.',
    llm='gpt-4o-mini',
    output="silent"
)

response = agent.chat('Say hello')
print(f'Response: {response}')
print('Agent test: OK')
"
```

### Verify LiteAgent Works

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
export OPENAI_API_KEY="your-key"
python -c "
from praisonaiagents.lite import LiteAgent, create_openai_llm_fn

llm_fn = create_openai_llm_fn(model='gpt-4o-mini')
agent = LiteAgent(name='Test', llm_fn=llm_fn)

response = agent.chat('Say hi')
print(f'Response: {response}')
print('LiteAgent test: OK')
"
```

### Verify Tool Execution

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
export OPENAI_API_KEY="your-key"
python -c "
from praisonaiagents import Agent

def add(a: int, b: int) -> int:
    '''Add two numbers.'''
    return a + b

agent = Agent(
    name='Math',
    instructions='Use tools.',
    llm='gpt-4o-mini',
    tools=[add],
    output="silent"
)

response = agent.chat('What is 5+3?')
assert '8' in response, f'Expected 8 in response: {response}'
print('Tool test: OK')
"
```

## Environment Variables

| Variable             | Description           | Required            |
| -------------------- | --------------------- | ------------------- |
| `RUN_REAL_KEY_TESTS` | Enable real API tests | Yes                 |
| `OPENAI_API_KEY`     | OpenAI API key        | For OpenAI tests    |
| `ANTHROPIC_API_KEY`  | Anthropic API key     | For Anthropic tests |
| `GOOGLE_API_KEY`     | Google API key        | For Gemini tests    |

## CI/CD Integration

### GitHub Actions

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
name: Real API Tests

on:
  workflow_dispatch:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - run: pip install -e ".[test]"
      - name: Run Tests
        env:
          RUN_REAL_KEY_TESTS: "1"
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: python -m pytest tests/integration/test_real_api.py -v
```

### Shell Script

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
#!/bin/bash
set -e

export RUN_REAL_KEY_TESTS=1

if [ -z "$OPENAI_API_KEY" ]; then
    echo "Error: OPENAI_API_KEY not set"
    exit 1
fi

python -m pytest tests/integration/test_real_api.py -v
echo "All real API tests passed!"
```

## Troubleshooting

### Tests Skipped

If tests are skipped, ensure environment variable is set:

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Check if variable is set
echo $RUN_REAL_KEY_TESTS

# Set it
export RUN_REAL_KEY_TESTS=1
```

### API Key Errors

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Verify key is set
python -c "import os; print('Key set:', bool(os.environ.get('OPENAI_API_KEY')))"
```

### Rate Limits

If hitting rate limits, add delays between tests:

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
python -m pytest tests/integration/test_real_api.py -v --tb=short -x
```

## Related

* [Real API Testing (Code)](/docs/features/real-api-testing)
* [Performance CLI](/docs/cli/performance)
* [Evaluation CLI](/docs/cli/eval)