Comparison
Batch APIvsStreaming
Batch API
nobody is waiting for these ten thousand answers, so you submit them as one file overnight and pay half.
An asynchronous endpoint: submit a file of requests, collect results within a service window (commonly 24 hours) at a large discount, typically 50 percent. The default choice for evals, backfills and any workload without a user waiting.
Full entry →Streaming
the twenty-second wait becomes an answer that visibly starts in under a second, without the model getting any faster.
Receiving the response token by token as it is generated instead of waiting for the whole thing. Essential for chat UX: it converts a 20 second wait into a response that visibly starts in under a second.
Full entry →