Explained: Chaos Engineering

It's 2 a.m. Your on-call phone goes off. A payment service is down, and nobody can explain why. It worked fine in staging. It passed every test. But somewhere in the tangle of microservices, a database connection pool quietly exhausted itself, and the fallback that was supposed to kick in never did. Nobody knew the fallback was broken. It had never been tested in production. Not once. This is the failure mode that chaos engineering was built … Read More

How to Write a Kubernetes Operator

The previous post explained what a Kubernetes Operator is: a controller that watches a custom resource and continuously reconciles the actual cluster state toward the desired state. That explanation is useful. This one is different. This one ends with a working Operator. We're going to build one end-to-end: define a custom resource, write the reconciliation logic in Go, see the equivalent in Java, wire up the RBAC, and deploy it to a cluster.… Read More

Explained: Kubernetes Operators

You're deploying Kafka to Kubernetes. You write a Deployment for the brokers, a StatefulSet for ZooKeeper, ConfigMap s for configuration, Services to expose everything, and a handful of PersistentVolumeClaim s for storage. The cluster comes up. Then a broker dies and Kubernetes restarts it, but it comes back with the wrong broker ID and can't rejoin the cluster. Kubernetes did what it was told: restart the container. But it had no idea wha… Read More

Explained: Cookie Consent

You land on a news site to read one article. Before you can see a single word of it, a banner fills half the screen. "We value your privacy." There are three buttons: Accept All, Manage Preferences, and something like "Continue without accepting" that's grey and tiny and hard to find. You click Accept All because you just want to read the article. This happens roughly a dozen times a day for most people using the web. Most… Read More