Time to first token
The delay between sending a request and receiving the first token of the response.
By Lattice editorial · Entry last reviewed · method
What it implies
It is the latency number users actually perceive, and it is dominated by queueing rather than by model speed. A user-facing chatbot is judged on TTFT; a batch job is judged on total throughput. Conflating them is how teams end up optimising the wrong metric for months.