jargon

Comparison

Exponential backoffvsRate limits (RPM, TPM)

Exponential backoff

you retry the failed call, wait twice as long, retry again, and add a little randomness so your whole fleet does not retry in lockstep.

The standard retry strategy: on failure, wait, then retry with doubling delays plus jitter. Combined with a retry budget it handles rate limits and transient errors. You have written this before; it applies unchanged.

Full entry →

Rate limits (RPM, TPM)

your calls start coming back 429 once traffic climbs, and what you are hitting is requests per minute or tokens per minute, not a bug in your code.

Provider caps on requests per minute and tokens per minute, usually tiered by spend. Production code treats 429 responses as normal weather, not as errors to page on.

Full entry →

Related comparisons