Nimrod Kor

The Limits of Prompting: ArchitectingTrustworthy Coding Agents

Prompt engineering has its limits. Learn how a multi-agent architecture, enriched with deep context, boosted our AI agent's suggestion acceptance rate from 12% to over 60%.

The Limits of Prompting: ArchitectingTrustworthy Coding Agents
#1about 2 minutes

Prototyping a basic AI code review agent

A simple prototype using a GitHub webhook and a single LLM call reveals the potential for understanding code semantics beyond static analysis.

#2about 2 minutes

Iteratively improving prompts to handle edge cases

Simple prompts fail to consider developer comments or model knowledge cutoffs, requiring more detailed instructions to improve accuracy.

#3about 5 minutes

Establishing a robust benchmarking process for agents

A reliable benchmarking pipeline uses a large dataset, concurrent execution, and an LLM-as-a-judge (LLJ) to measure and track performance improvements.

#4about 2 minutes

Decomposing large tasks into specialized agents

To combat inconsistency and hallucinations, a single large task like code review is broken down into multiple smaller, specialized agents.

#5about 6 minutes

Leveraging codebase context for deeper insights

Moving beyond prompts, providing codebase context via vector similarity (RAG) and module dependency graphs (AST) unlocks high-quality, human-like feedback.

#6about 3 minutes

Introducing Awesome Reviewers for community standards

Awesome Reviewers is a collection of prompts derived from open-source projects that can be used to enforce team-specific coding standards.

#7about 1 minute

Key takeaways for building reliable LLM agents

The path to a reliable agent involves starting with a proof-of-concept, benchmarking rigorously, using prompt engineering for quick fixes, and investing in deep context.

Related jobs
Jobs that call for the skills explored in this talk.

Machine Learning Engineer

Picnic Technologies B.V.

Picnic Technologies B.V.
Amsterdam, Netherlands

Intermediate
Senior
Python
Structured Query Language (SQL)
+1

Matching moments

Prompt injection as an unsolved AI security problem
07:39 MIN

Prompt injection as an unsolved AI security problem

AI in the Open and in Browsers - Tarek Ziadé

Using AI agents to modernize legacy COBOL systems
06:28 MIN

Using AI agents to modernize legacy COBOL systems

Devs vs. Marketers, COBOL and Copilot, Make Live Coding Easy and more - The Best of LIVE 2025 - Part 3

Unlocking LLM potential with creative prompting techniques
04:59 MIN

Unlocking LLM potential with creative prompting techniques

WeAreDevelopers LIVE – Frontend Inspirations, Web Standards and more

Building and iterating on an LLM-powered product
05:03 MIN

Building and iterating on an LLM-powered product

Slopquatting, API Keys, Fun with Fonts, Recruiters vs AI and more - The Best of LIVE 2025 - Part 2

Making accessibility tooling actionable and encouraging
03:58 MIN

Making accessibility tooling actionable and encouraging

Developer Time Is Valuable - Use the Right Tools - Kilian Valkhof

The security challenges of building AI browser agents
06:33 MIN

The security challenges of building AI browser agents

AI in the Open and in Browsers - Tarek Ziadé

Using AI to overcome challenges in systems programming
02:49 MIN

Using AI to overcome challenges in systems programming

AI in the Open and in Browsers - Tarek Ziadé

The security risks of AI-generated code and slopsquatting
05:55 MIN

The security risks of AI-generated code and slopsquatting

Slopquatting, API Keys, Fun with Fonts, Recruiters vs AI and more - The Best of LIVE 2025 - Part 2

Featured Partners

Related Videos

See all videos
How we built an AI-powered code reviewer in 80 hours28:07

How we built an AI-powered code reviewer in 80 hours

Yan Cui

Three years of putting LLMs into Software - Lessons learned26:30

Three years of putting LLMs into Software - Lessons learned

Simon A.T. Jiménez

The AI Agent Path to Prod: Building for Reliability30:48

The AI Agent Path to Prod: Building for Reliability

Max Tkacz

Prompt Engineering - an Art, a Science, or your next Job Title?33:28

Prompt Engineering - an Art, a Science, or your next Job Title?

Maxim Salnikov

Bringing the power of AI to your application.32:26

Bringing the power of AI to your application.

Krzysztof Cieślak

Beyond Prompting: Building Scalable AI with Multi-Agent Systems and MCP25:32

Beyond Prompting: Building Scalable AI with Multi-Agent Systems and MCP

Viktoria Semaan

AI: Superhero or Supervillain? How and Why with Scott Hanselman25:17

AI: Superhero or Supervillain? How and Why with Scott Hanselman

Scott Hanselman

Using LLMs in your Product31:12

Using LLMs in your Product

Daniel Töws

Related Articles

View all articles

From learning to earning

Jobs that call for the skills explored in this talk.