Comparison
Keyed statevsStateful stream processing
Keyed state
the running total is kept per customer id, so scaling out just moves whole customers between workers.
State scoped to a partition key, so each worker owns a disjoint slice of it. Making state keyed is what makes a streaming job parallelisable at all, because two workers never need to see the same counter. The consequence is that your key choice is now a partitioning decision, an ordering decision and a memory-hotspot decision at once — one enormous customer becomes one enormous worker.
Full entry →Stateful stream processing
the job is holding four hundred gigabytes of per-user counters, and 'just restart it' stopped being free some time ago.
Streaming work whose output depends on records the job has already seen — aggregations, joins, deduplication, sessionisation. Once the job holds state, it stops being a stateless transformer you can kill freely and becomes a database with a very unusual query interface. Everything difficult in streaming operations follows from this: restarts need checkpoints, scaling needs state redistribution, and code changes need the old state to still be readable.
Full entry →