Noaa Barki

What we Learned from Reading 100+ Kubernetes Post-Mortems

What's the #1 cause of Kubernetes outages? After analyzing over 100 post-mortems, the answer is surprisingly simple—and completely preventable.

What we Learned from Reading 100+ Kubernetes Post-Mortems
#1about 6 minutes

Understanding the developer versus DevOps cultural divide

A story from a DevOps meetup illustrates the different goals and perspectives that create friction between developers and operations teams.

#2about 2 minutes

Bridge the gap with champions and failure stories

Delegate knowledge to developer champions and learn best practices by studying the post-mortem stories of other companies.

#3about 5 minutes

Common Kubernetes misconfigurations from real outages

Examples from Target and Zalando show how simple errors like incorrect CronJob concurrency policies or missing memory limits can cause major production failures.

#4about 3 minutes

How to introduce policy enforcement gradually

Avoid organizational friction by implementing new policies slowly, starting with a single pilot team to gain agreement and understanding before a wider rollout.

#5about 3 minutes

Categorizing the three types of Kubernetes failures

Kubernetes failures typically fall into three categories: simple syntax errors, gaps in knowledge of best practices, and misalignment with internal company policies.

#6about 2 minutes

Validating Kubernetes YAML for syntax and schema errors

Use tools like yq for YAML format validation and kubeconform for schema validation without requiring direct cluster access for developers.

#7about 4 minutes

The challenges of managing policies as code in Git

Managing policies in Git creates versioning nightmares and lacks features for permissions, dynamic adjustments, and providing clear remediation guidelines.

#8about 4 minutes

Using Datree for centralized policy management

Datree is an open-source tool that provides a centralized location for managing policies, which are then enforced locally and in CI for developers.

#9about 1 minute

The real meaning of shifting responsibility left

True shift-left culture is not just about tools but about delegating responsibility and empowering developers to own their configurations.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

The most dangerous trap for HR during company growth
04:20 MIN

The most dangerous trap for HR during company growth

From Data Keeper to Culture Shaper: The Evolution of HR Across Growth Stages

How to build structure and culture without killing agility
05:12 MIN

How to build structure and culture without killing agility

From Data Keeper to Culture Shaper: The Evolution of HR Across Growth Stages

Malware campaigns, cloud latency, and government IT theft
01:06 MIN

Malware campaigns, cloud latency, and government IT theft

Fake or News: Self-Driving Cars on Subscription, Crypto Attacks Rising and Working While You Sleep - Théodore Lefèvre

Building a learning culture instead of a failure culture
02:12 MIN

Building a learning culture instead of a failure culture

Turning People Strategy into a Transformation Engine

How AI-generated content is overwhelming open source maintainers
06:46 MIN

How AI-generated content is overwhelming open source maintainers

WeAreDevelopers LIVE – You Don’t Need JavaScript, Modern CSS and More

Organizing a developer conference for 15,000 attendees
01:32 MIN

Organizing a developer conference for 15,000 attendees

Cat Herding with Lions and Tigers - Christian Heilmann

Developing resilience by expanding your capacity for failure
04:57 MIN

Developing resilience by expanding your capacity for failure

What 2025 Taught Us: A Year-End Special with Hung Lee

Making accessibility tooling actionable and encouraging
03:58 MIN

Making accessibility tooling actionable and encouraging

Developer Time Is Valuable - Use the Right Tools - Kilian Valkhof

Featured Partners

Related Videos

See all videos
Answering the Million Dollar Question: Why did I Break Production?29:38

Answering the Million Dollar Question: Why did I Break Production?

Luís Ventura

Chaos in Containers - Unleashing Resilience27:52

Chaos in Containers - Unleashing Resilience

Maish Saidel-Keesing

Kubernetes Security - Challenge and Opportunity42:45

Kubernetes Security - Challenge and Opportunity

Marc Nimmerrichter

Kubernetes Security Best Practices23:08

Kubernetes Security Best Practices

Rico Komenda

Kubernetes Maestro: Dive Deep into Custom Resources to Unleash Next-Level Orchestration Power!29:21

Kubernetes Maestro: Dive Deep into Custom Resources to Unleash Next-Level Orchestration Power!

Um e Habiba

Mastering Kubernetes – Beginner Edition57:24

Mastering Kubernetes – Beginner Edition

Hannes Norbert Göring

I broke the production26:18

I broke the production

Arto Liukkonen

Hacking Kubernetes: Live Demo Marathon46:36

Hacking Kubernetes: Live Demo Marathon

Andrew Martin

Related Articles

View all articles
DC
Daniel Cranney
Dev Digest 188: CfP time, the risks of NPM and IKEA algorithms
Inside last week’s Dev Digest 188 . 🤖 GitHub Copilot CLI is now in public review 💻 Microsoft is bringing ‘vibe working’ to office apps 🎣 Attackers abuse AI tools to generate captchas in fishing attacks ⚠️ When LLMs autonomously attack 🧠 Common cause...
Dev Digest 188: CfP time, the risks of NPM and IKEA algorithms
Learning Kubernetes made easy with KubeCampus
Learning to use Kubernetes? KubeCampus by Kasten offers free educational content for all skill levels to get you started!Kubernetes is an open-source system for deploying, scaling and managing containerized applications. It allows you to deploy your ...
Learning Kubernetes made easy with KubeCampus
DC
Daniel Cranney
Dev Digest 196: AI Killed DevOps, LLM Political Bias & AI Security
Inside last week’s Dev Digest 196 . ⚖️ Political bias in LLMs 🫣 AI written code causes 1 in 5 security breaches 🖼️ Is there a limit to alternative text on images? 📝 CodeWiki - understand code better 🟨 Long tasks in JavaScript 👻 Scare yourself into n...
Dev Digest 196: AI Killed DevOps, LLM Political Bias & AI Security

From learning to earning

Jobs that call for the skills explored in this talk.