Comparison
Inter-token latencyvsTime to first token (TTFT)
Inter-token latency
the first token came back fast and then the words trickled out, which is a different number from the one you were tracking.
The gap between successive streamed tokens; the steady-state speed after TTFT. Together with TTFT it fully describes streaming performance.
Full entry →Time to first token (TTFT)
the long prompt made the wait before the first word noticeably worse, even though generation ran at the same speed after that.
How long before the first output token arrives. Dominated by input processing, so long prompts raise it. The metric that matters most for perceived responsiveness.
Full entry →