Skip to main content

​
FailoverManager

Defined in the failover module.
Manages failover between multiple LLM auth profiles. Provides automatic failover when rate limits or errors occur, with configurable retry behavior and cooldown periods.

​
Constructor

​
config
Optional
No description available.

​
Methods

add_profile()

Add an auth profile.

remove_profile()

Remove a profile by name.

get_profile()

Get a profile by name.

list_profiles()

List all profiles.

get_next_profile()

Get the next available profile.

mark_failure()

Mark a profile as failed.

mark_success()

Mark a profile as successful.

on_failover()

Register a callback for failover events.

get_retry_delay()

Calculate retry delay for an attempt.

status()

Get failover manager status.

reset_all()

Reset all profiles to available status.

​
Usage

manager = FailoverManager()
    manager.add_profile(AuthProfile(
        name="openai-primary",
        provider="openai",
        api_key="sk-...",
        priority=0,
    ))
    manager.add_profile(AuthProfile(
        name="openai-backup",
        provider="openai",
        api_key="sk-...",
        priority=1,
    ))
    
    # Get next available profile
    profile = manager.get_next_profile()
    
    # On failure
    manager.mark_failure(profile, "Rate limit exceeded")
    
    # Get next profile (will return backup)
    profile = manager.get_next_profile()