Comparison
Rate limitingvsToken bucket
Rate limiting
you cap each client at a hundred requests a minute and start returning 429 once they go over.
Bounding how much a given caller can consume in a time window. It protects capacity, contains abuse, and stops one tenant from degrading everyone else. The limit needs to be per-identity and communicated in headers, or well-behaved clients cannot back off correctly and will simply retry into the wall.
Full entry →Token bucket
you let clients save up allowance while idle and spend it in a burst, as long as the long-run average stays under the limit.
A rate-limiting algorithm where tokens accumulate at a fixed rate up to a cap and each request consumes one. It permits bursts up to the bucket size while enforcing an average rate, which matches real traffic better than a hard per-second cap. Bucket size is the knob for how much burstiness you will tolerate.
Full entry →