What we Learned from Reading 100+ Kubernetes Post-Mortems

What's the #1 cause of Kubernetes outages? After analyzing over 100 post-mortems, the answer is surprisingly simple—and completely preventable.

#1about 6 minutes

Understanding the developer versus DevOps cultural divide

A story from a DevOps meetup illustrates the different goals and perspectives that create friction between developers and operations teams.

#2about 2 minutes

Bridge the gap with champions and failure stories

Delegate knowledge to developer champions and learn best practices by studying the post-mortem stories of other companies.

#3about 5 minutes

Common Kubernetes misconfigurations from real outages

Examples from Target and Zalando show how simple errors like incorrect CronJob concurrency policies or missing memory limits can cause major production failures.

#4about 3 minutes

How to introduce policy enforcement gradually

Avoid organizational friction by implementing new policies slowly, starting with a single pilot team to gain agreement and understanding before a wider rollout.

#5about 3 minutes

Categorizing the three types of Kubernetes failures

Kubernetes failures typically fall into three categories: simple syntax errors, gaps in knowledge of best practices, and misalignment with internal company policies.

#6about 2 minutes

Validating Kubernetes YAML for syntax and schema errors

Use tools like yq for YAML format validation and kubeconform for schema validation without requiring direct cluster access for developers.

#7about 4 minutes

The challenges of managing policies as code in Git

Managing policies in Git creates versioning nightmares and lacks features for permissions, dynamic adjustments, and providing clear remediation guidelines.

#8about 4 minutes