Skip to main content

​
run_until

Method
This is a method of the Agent class in the agent module.
Run agent iteratively until output meets quality criteria. This method implements the “Ralph Loop” pattern: run agent → judge output → improve based on feedback → repeat until threshold met.

​
Signature

def run_until(prompt: str, criteria: str, threshold: float, max_iterations: int, mode: str, on_iteration: Optional[Callable[[Any], None]], verbose: bool) -> 'EvaluationLoopResult'

​
Parameters

​
prompt
str
required
The prompt to send to the agent
​
criteria
str
required
Evaluation criteria for the Judge (e.g., “Response is thorough”)
​
threshold
float
default:"8.0"
Score threshold for success (default: 8.0, scale 1-10)
​
max_iterations
int
default:"5"
Maximum iterations before stopping (default: 5)
​
mode
str
default:"'optimize'"
“optimize” (stop on success) or “review” (run all iterations)
​
on_iteration
Optional
Optional callback called after each iteration
​
verbose
bool
default:"False"
Enable verbose logging

​
Returns

​
Returns
'EvaluationLoopResult'
EvaluationLoopResult with iteration history and final score

​
Usage

agent = Agent(name="analyzer", instructions="Analyze systems")
    result = agent.run_until(
        "Analyze the auth flow",
        criteria="Analysis is thorough and actionable",
        threshold=8.0,
    )
    print(result.final_score)  # 8.5
    print(result.success)      # True

​
Uses

  • EvaluationLoop
  • loop.run

​
Source

View on GitHub

praisonaiagents/agent/agent.py at line 2371