Max Tkacz
The AI Agent Path to Prod: Building for Reliability
#1about 4 minutes
Why AI agents fail in production environments
AI agents often fail in production because the probabilistic nature of LLMs conflicts with the need for reliability at scale.
#2about 5 minutes
Scoping an AI agent for a specific business problem
Start by identifying a low-risk, high-impact task, like automating free trial extensions, to establish a viable solution scope.
#3about 3 minutes
Walking through the naive V1 customer support agent
The initial agent uses an LLM with tools to fetch user data and extend trials, but its reliability is unknown without testing.
#4about 4 minutes
Using evaluations to test the happy path case
Evaluations are introduced as a testing framework to run the agent against specific test cases, revealing inconsistencies even in the happy path.
#5about 4 minutes
Improving agent consistency with prompt engineering
By adding explicit rules and few-shot examples to the system prompt, the agent's tool usage and response quality become more consistent.
#6about 5 minutes
Testing for prompt injection and other edge cases
A new evaluation case for prompt injection reveals a security flaw, which is fixed by adding specific security rules to the system prompt.
#7about 6 minutes
Applying production guardrails beyond evaluations
Beyond evals, production readiness requires adding human-in-the-loop processes, custom error handling, rate limiting, and model redundancy.
Related jobs
Jobs that call for the skills explored in this talk.
Wilken GmbH
Ulm, Germany
Senior
Kubernetes
AI Frameworks
+3
Eltemate
Amsterdam, Netherlands
Intermediate
Senior
TypeScript
Continuous Integration
+1
Picnic Technologies B.V.
Amsterdam, Netherlands
Intermediate
Senior
Python
Structured Query Language (SQL)
+1
Matching moments
07:39 MIN
Prompt injection as an unsolved AI security problem
AI in the Open and in Browsers - Tarek Ziadé
06:33 MIN
The security challenges of building AI browser agents
AI in the Open and in Browsers - Tarek Ziadé
03:28 MIN
Why corporate AI adoption lags behind the hype
What 2025 Taught Us: A Year-End Special with Hung Lee
04:05 MIN
How AI code generators have become more reliable
AI in the Open and in Browsers - Tarek Ziadé
06:28 MIN
Using AI agents to modernize legacy COBOL systems
Devs vs. Marketers, COBOL and Copilot, Make Live Coding Easy and more - The Best of LIVE 2025 - Part 3
04:04 MIN
Shifting HR from standard products to AI-powered platforms
Turning People Strategy into a Transformation Engine
03:07 MIN
Final advice for developers adapting to AI
WeAreDevelopers LIVE – AI, Freelancing, Keeping Up with Tech and More
04:57 MIN
Increasing the value of talk recordings post-event
Cat Herding with Lions and Tigers - Christian Heilmann
Featured Partners
Related Videos
39:11Agents for the Sake of Happiness
Thomas Dohmke
24:22Beyond Chatbots: How to build Agentic AI systems
Philipp Schmid
25:32Beyond Prompting: Building Scalable AI with Multi-Agent Systems and MCP
Viktoria Semaan
29:03On a Secret Mission: Developing AI Agents
Jörg Neumann
24:05The Limits of Prompting: ArchitectingTrustworthy Coding Agents
Nimrod Kor
26:30Three years of putting LLMs into Software - Lessons learned
Simon A.T. Jiménez
24:11You are not my model anymore - understanding LLM model behavior
Andreas Erben
26:47The State of GenAI & Machine Learning in 2025
Alejandro Saucedo
Related Articles
View all articles
From learning to earning
Jobs that call for the skills explored in this talk.
BAS Group
Winterswijk, Netherlands
€4-5K
GIT
Python
Terraform
Amazon Web Services (AWS)
autonomous-teaming
München, Germany
Remote
API
React
Python
TypeScript
Starion Group
Municipality of Madrid, Spain
API
CSS
Python
Docker
Machine Learning
+1
Nteractive Consulting & Events Ltd
Staines-upon-Thames, United Kingdom
low-code
Machine Learning
T-Maxx International