David Mosen

Deployed ML models need your feedback too

Is your deployed ML model slowly failing in production? Learn why feedback loops are the missing piece for monitoring true performance and preventing model decay.

Deployed ML models need your feedback too
#1about 2 minutes

The role of feedback in the MLOps lifecycle

MLOps extends traditional software engineering by integrating processes from data science and business to ensure deployed models perform correctly.

#2about 2 minutes

Understanding the three pillars of MLOps

MLOps adapts DevOps principles by adding continuous training to continuous integration and delivery, addressing the unique needs of ML models.

#3about 4 minutes

Architecting a mature and automated MLOps pipeline

A mature MLOps architecture automates the entire lifecycle from feature store to prediction service, but requires monitoring to close the loop.

#4about 8 minutes

Exploring the different layers of model monitoring

Effective monitoring covers multiple layers, including infrastructure, data drift, concept drift, model performance, and business KPIs.

#5about 4 minutes

Key characteristics of an effective feedback system

Designing a feedback system requires considering the delay, collection method, and correlation of feedback to predictions.

#6about 4 minutes

Evaluating the state of current monitoring solutions

While many tools exist for monitoring training, live performance monitoring is less mature, with platforms like Google Vertex AI and Seldon Core having limitations.

#7about 8 minutes

Demo of a unified model and business monitoring dashboard

An internal tool demonstrates how to integrate with cloud AI services like AWS Personalize to provide a unified view of model and business metrics.

#8about 5 minutes

Q&A on multi-tenant models and edge deployment

The discussion covers best practices for deploying models to different customers, handling unstructured data, and adapting monitoring concepts for edge devices.

Related jobs
Jobs that call for the skills explored in this talk.

Machine Learning Engineer

Picnic Technologies B.V.

Picnic Technologies B.V.
Amsterdam, Netherlands

Intermediate
Senior
Python
Structured Query Language (SQL)
+1

Matching moments

The evolving role of the machine learning engineer
02:20 MIN

The evolving role of the machine learning engineer

AI in the Open and in Browsers - Tarek Ziadé

Building an open source community around AI models
04:28 MIN

Building an open source community around AI models

AI in the Open and in Browsers - Tarek Ziadé

How AI-generated content is overwhelming open source maintainers
06:46 MIN

How AI-generated content is overwhelming open source maintainers

WeAreDevelopers LIVE – You Don’t Need JavaScript, Modern CSS and More

Why corporate AI adoption lags behind the hype
03:28 MIN

Why corporate AI adoption lags behind the hype

What 2025 Taught Us: A Year-End Special with Hung Lee

AI lawsuits, code flagging, and self-driving subscriptions
01:02 MIN

AI lawsuits, code flagging, and self-driving subscriptions

Fake or News: Self-Driving Cars on Subscription, Crypto Attacks Rising and Working While You Sleep - Théodore Lefèvre

Building and iterating on an LLM-powered product
05:03 MIN

Building and iterating on an LLM-powered product

Slopquatting, API Keys, Fun with Fonts, Recruiters vs AI and more - The Best of LIVE 2025 - Part 2

Increasing the value of talk recordings post-event
04:57 MIN

Increasing the value of talk recordings post-event

Cat Herding with Lions and Tigers - Christian Heilmann

Final advice for developers adapting to AI
03:07 MIN

Final advice for developers adapting to AI

WeAreDevelopers LIVE – AI, Freelancing, Keeping Up with Tech and More

Featured Partners

Related Videos

See all videos
Effective Machine Learning - Managing Complexity with MLOps45:45

Effective Machine Learning - Managing Complexity with MLOps

Simon Stiebellehner

DevOps for Machine Learning44:40

DevOps for Machine Learning

Hauke Brammer

The state of MLOps - machine learning in production at enterprise scale47:19

The state of MLOps - machine learning in production at enterprise scale

Bas Geerdink 

MLOps - What’s the deal behind it?29:24

MLOps - What’s the deal behind it?

Nico Axtmann

Is my AI alive but brain-dead? How monitoring can tell you if your machine learning stack is still performing48:07

Is my AI alive but brain-dead? How monitoring can tell you if your machine learning stack is still performing

Lina Weichbrodt

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow34:21

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow

Aarno Aukia

From Traction to Production: Maturing your GenAIOps step by step25:27

From Traction to Production: Maturing your GenAIOps step by step

Maxim Salnikov

AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment25:41

AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment

Joy Chatterjee

Related Articles

View all articles
BB
Benedikt Bischof
MLOps – What’s the deal behind it?
Welcome to this issue of the WeAreDevelopers Live Talk series. This article recaps an interesting talk by Nico Axtmann who introduced us to MLOpsAbout the speaker:Nico Axtmann is a seasoned machine learning veteran. Starting back in 2014 he observed ...
MLOps – What’s the deal behind it?
BB
Benedikt Bischof
MLOps And AI Driven Development
Welcome to this issue of the WeAreDevelopers Dev Talk Recap series. This article recaps an interesting talk by Natalie Pistunovic who spoke about the development of AI and MLOps. What you will learn:How the concept of AI became an academic field and ...
MLOps And AI Driven Development

From learning to earning

Jobs that call for the skills explored in this talk.