Applied AI·Production patterns and cost
nobody is waiting on this call, so it goes on a queue with bounded concurrency and the 429 storm stops.
Queueing and async processing
Draft summary, pending review
Decoupling LLM work from request threads for anything a user is not actively awaiting: enqueue, process with controlled concurrency under the rate limit, deliver results. Smooths bursts and stops 429 storms.