The case for Kafka cold storage

It is entirely possible that what I am going to describe here is an edge case not many people hit with their Kafka deployments. However, in my experience, when Kafka is used to ingest large volumes of data, it makes perfect sense. Considering that every now and then people ask for a cold storage feature on the Kafka mailing list, I am not the only one who would find this useful.
Read more

Apache Zookeeper authentication

The problem One of my Zookeeper clusters has to be available to everyone. But I want to make sure only specific known hosts are allowed to connect. The problem was, those known hosts are dynamic and I didn’t want any configuration for this component. The servers are running in the cloud, they come and go. At Technicolor, we use Chef to manage all our boxes. Every box is registered on the Chef server, we know every private and public IP address of every server within our setup.
Read more