Comparison
Rate limitingvsRate limits (RPM, TPM)
Rate limiting
you cap each client at a hundred requests a minute and start returning 429 once they go over.
Bounding how much a given caller can consume in a time window. It protects capacity, contains abuse, and stops one tenant from degrading everyone else. The limit needs to be per-identity and communicated in headers, or well-behaved clients cannot back off correctly and will simply retry into the wall.
Full entry →Rate limits (RPM, TPM)
your calls start coming back 429 once traffic climbs, and what you are hitting is requests per minute or tokens per minute, not a bug in your code.
Provider caps on requests per minute and tokens per minute, usually tiered by spend. Production code treats 429 responses as normal weather, not as errors to page on.
Full entry →