Skip to main content

​
StreamingTracker

Defined in the profiler module.
Track streaming operations (LLM responses).

​
Constructor

​
name
str
required
No description available.

​
Methods

start()

Start tracking.

first_token()

Mark time to first token.

chunk()

Record a chunk received.

end()

End tracking and record to Profiler.

ttft_ms()

Get time to first token in ms.

elapsed_ms()

Get elapsed time in ms.

​
Usage

tracker = StreamingTracker("chat")
    tracker.start()
    tracker.first_token()  # Mark TTFT
    for chunk in stream:
        tracker.chunk()
    tracker.end(total_tokens=100)