Data engineering·Orchestration and running it in anger
one malformed row fails the batch, the batch retries, and the same row fails it again every five minutes for two days.
Poison record
Also calledbad record, poison row
A single input that cannot be processed and blocks everything behind it because the failure is retried indefinitely. Retry logic that cannot distinguish a transient failure from a permanent one turns a single bad record into a stopped pipeline. The standard treatment is a bounded retry count followed by diverting the record somewhere it can be inspected, so progress resumes and nothing is lost.