Comparison
Schema driftvsSchema evolution
Schema drift
the vendor added three columns and renamed one over the weekend, and nobody told anybody.
The source's structure changing without coordination — added, removed, renamed or retyped fields. It is distinct from planned schema evolution in exactly the way that matters: nobody agreed to it, so no compatibility rule was applied. Handling it means deciding in advance whether unknown fields are absorbed, ignored or fatal, and a pipeline that has never made that decision has made it implicitly by whichever the library defaults to.
Full entry →Schema evolution
you change the message format and have to make sure the consumers still on the old schema can read the new messages.
Managing changes to a serialisation contract so producers and consumers can be deployed independently. Formats like Avro and Protobuf encode the rules — optional fields, reserved tags, never reuse a field number — and a schema registry enforces them. The rules exist because in any rolling deploy the producer and consumer are briefly on different versions.
Full entry →