David Mosen

Deployed ML models need your feedback too

Is your deployed ML model slowly failing in production? Learn why feedback loops are the missing piece for monitoring true performance and preventing model decay.

Deployed ML models need your feedback too
#1about 2 minutes

The role of feedback in the MLOps lifecycle

MLOps extends traditional software engineering by integrating processes from data science and business to ensure deployed models perform correctly.

#2about 2 minutes

Understanding the three pillars of MLOps

MLOps adapts DevOps principles by adding continuous training to continuous integration and delivery, addressing the unique needs of ML models.

#3about 4 minutes

Architecting a mature and automated MLOps pipeline

A mature MLOps architecture automates the entire lifecycle from feature store to prediction service, but requires monitoring to close the loop.

#4about 8 minutes

Exploring the different layers of model monitoring

Effective monitoring covers multiple layers, including infrastructure, data drift, concept drift, model performance, and business KPIs.

#5about 4 minutes

Key characteristics of an effective feedback system

Designing a feedback system requires considering the delay, collection method, and correlation of feedback to predictions.

#6about 4 minutes

Evaluating the state of current monitoring solutions

While many tools exist for monitoring training, live performance monitoring is less mature, with platforms like Google Vertex AI and Seldon Core having limitations.

#7about 8 minutes

Demo of a unified model and business monitoring dashboard

An internal tool demonstrates how to integrate with cloud AI services like AWS Personalize to provide a unified view of model and business metrics.

#8about 5 minutes

Q&A on multi-tenant models and edge deployment

The discussion covers best practices for deploying models to different customers, handling unstructured data, and adapting monitoring concepts for edge devices.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

Understanding the role and challenges of MLOps
01:01 MIN

Understanding the role and challenges of MLOps

The Road to MLOps: How Verivox Transitioned to AWS

Using MLOps infrastructure to implement model governance
09:27 MIN

Using MLOps infrastructure to implement model governance

Model Governance and Explainable AI as tools for legal compliance and risk management

Four pillars for deploying successful machine learning systems
02:58 MIN

Four pillars for deploying successful machine learning systems

Model Governance and Explainable AI as tools for legal compliance and risk management

Overcoming the challenges of productionizing AI models
24:42 MIN

Overcoming the challenges of productionizing AI models

Navigating the AI Revolution in Software Development

The lifecycle for operationalizing AI models in business
00:20 MIN

The lifecycle for operationalizing AI models in business

Detecting Money Laundering with AI

Understanding the machine learning workflow and MLOps
06:34 MIN

Understanding the machine learning workflow and MLOps

Machine Learning in ML.NET

The challenge of operationalizing production machine learning systems
00:11 MIN

The challenge of operationalizing production machine learning systems

Model Governance and Explainable AI as tools for legal compliance and risk management

The convergence of ML and DevOps in MLOps
01:58 MIN

The convergence of ML and DevOps in MLOps

AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment

Featured Partners

Related Videos

See all videos
Effective Machine Learning - Managing Complexity with MLOps45:45

Effective Machine Learning - Managing Complexity with MLOps

Simon Stiebellehner

DevOps for Machine Learning44:40

DevOps for Machine Learning

Hauke Brammer

The state of MLOps - machine learning in production at enterprise scale47:19

The state of MLOps - machine learning in production at enterprise scale

Bas Geerdink 

MLOps - What’s the deal behind it?29:24

MLOps - What’s the deal behind it?

Nico Axtmann

Is my AI alive but brain-dead? How monitoring can tell you if your machine learning stack is still performing48:07

Is my AI alive but brain-dead? How monitoring can tell you if your machine learning stack is still performing

Lina Weichbrodt

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow34:21

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow

Aarno Aukia

From Traction to Production: Maturing your GenAIOps step by step25:27

From Traction to Production: Maturing your GenAIOps step by step

Maxim Salnikov

AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment25:41

AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment

Joy Chatterjee

Related Articles

View all articles
BB
Benedikt Bischof
MLOps – What’s the deal behind it?
Welcome to this issue of the WeAreDevelopers Live Talk series. This article recaps an interesting talk by Nico Axtmann who introduced us to MLOpsAbout the speaker:Nico Axtmann is a seasoned machine learning veteran. Starting back in 2014 he observed ...
MLOps – What’s the deal behind it?
BB
Benedikt Bischof
MLOps And AI Driven Development
Welcome to this issue of the WeAreDevelopers Dev Talk Recap series. This article recaps an interesting talk by Natalie Pistunovic who spoke about the development of AI and MLOps. What you will learn:How the concept of AI became an academic field and ...
MLOps And AI Driven Development

From learning to earning

Jobs that call for the skills explored in this talk.