Comparison
Response timevsThroughput
Response time
you measure what the user actually experiences: request sent to last byte received, queueing and all.
The total elapsed time a client observes, comprising network latency, queueing delay and service time. It is the number that matters to users and the one to put in an SLO. Under load it is dominated by queueing rather than processing, which is why optimising the handler does nothing once the system is saturated.
Full entry →Throughput
you measure how many requests the system finishes per second, which is a different question from how long any one of them takes.
Completed work per unit time. It is the capacity number, and it trades against latency: batching, queueing and parallelism raise throughput while making individual requests slower. A system can have excellent throughput and unusable latency, which is why quoting one without the other means nothing.
Full entry →