Yan Cui

How we built an AI-powered code reviewer in 80 hours

Our AI prototype was built in a weekend. But productionizing it meant fighting collapsing context windows, high latency, and massive costs.

How we built an AI-powered code reviewer in 80 hours
#1about 2 minutes

An overview of an AI-powered code reviewer

The tool reviews pull requests, leaves inline comments, and provides a dashboard, with the talk focusing on lessons learned from building it.

#2about 3 minutes

The high-level serverless architecture for the application

The system uses GitHub webhooks, EventBridge, Lambda, Amazon Bedrock, and DynamoDB, with Clerk for auth and Stripe for payments.

#3about 4 minutes

Choosing Amazon Bedrock for security and privacy

Amazon Bedrock was selected for its strong security guarantees, data privacy policies, and serverless, token-based pricing model suitable for sensitive customer code.

#4about 5 minutes

The truth about LLM context window size and reasoning

Large context window sizes are misleading because a model's ability to reason over content collapses long before the advertised limit, forcing a one-prompt-per-file strategy.

#5about 3 minutes

Managing API rate limits and model availability

To overcome low default API rate limits, strategies include requesting limit increases, using cross-region inference, and implementing fallbacks to other models for reliability.

#6about 3 minutes

Strategies for controlling high LLM costs

The most effective cost control measure is to analyze only the changed lines in a pull request rather than the entire file, which also improves user experience.

#7about 4 minutes

Handling timeouts with durable execution in Lambda

A lightweight durable execution mechanism using checkpoints in DynamoDB prevents reprocessing files on Lambda retries, which are common due to slow LLM response times.

#8about 4 minutes

Dealing with different types of LLM hallucinations

Hallucinations range from simple invalid JSON to complex errors like suggesting fixes for outdated libraries, which can be mitigated with RAG but at a significant cost.

Related jobs
Jobs that call for the skills explored in this talk.

Machine Learning Engineer

Picnic Technologies B.V.

Picnic Technologies B.V.
Amsterdam, Netherlands

Intermediate
Senior
Python
Structured Query Language (SQL)
+1

Matching moments

How AI is changing the freelance developer experience
09:10 MIN

How AI is changing the freelance developer experience

WeAreDevelopers LIVE – AI, Freelancing, Keeping Up with Tech and More

Building and iterating on an LLM-powered product
05:03 MIN

Building and iterating on an LLM-powered product

Slopquatting, API Keys, Fun with Fonts, Recruiters vs AI and more - The Best of LIVE 2025 - Part 2

Using AI agents to modernize legacy COBOL systems
06:28 MIN

Using AI agents to modernize legacy COBOL systems

Devs vs. Marketers, COBOL and Copilot, Make Live Coding Easy and more - The Best of LIVE 2025 - Part 3

AI lawsuits, code flagging, and self-driving subscriptions
01:02 MIN

AI lawsuits, code flagging, and self-driving subscriptions

Fake or News: Self-Driving Cars on Subscription, Crypto Attacks Rising and Working While You Sleep - Théodore Lefèvre

The security risks of AI-generated code and slopsquatting
05:55 MIN

The security risks of AI-generated code and slopsquatting

Slopquatting, API Keys, Fun with Fonts, Recruiters vs AI and more - The Best of LIVE 2025 - Part 2

Using AI to overcome challenges in systems programming
02:49 MIN

Using AI to overcome challenges in systems programming

AI in the Open and in Browsers - Tarek Ziadé

How AI code generators have become more reliable
04:05 MIN

How AI code generators have become more reliable

AI in the Open and in Browsers - Tarek Ziadé

The emerging market for fixing AI-generated code
04:09 MIN

The emerging market for fixing AI-generated code

Devs vs. Marketers, COBOL and Copilot, Make Live Coding Easy and more - The Best of LIVE 2025 - Part 3

Featured Partners

Related Videos

See all videos
The Limits of Prompting: ArchitectingTrustworthy Coding Agents24:05

The Limits of Prompting: ArchitectingTrustworthy Coding Agents

Nimrod Kor

Three years of putting LLMs into Software - Lessons learned26:30

Three years of putting LLMs into Software - Lessons learned

Simon A.T. Jiménez

AI: Superhero or Supervillain? How and Why with Scott Hanselman25:17

AI: Superhero or Supervillain? How and Why with Scott Hanselman

Scott Hanselman

Leapter: The Reinvention of Software Development? A Future Built On AI Generated Code.04:17

Leapter: The Reinvention of Software Development? A Future Built On AI Generated Code.

Robert Werner

Panel discussion: Developing in an AI world - are we all demoted to reviewers? WeAreDevelopers WebDev & AI Day March202547:35

Panel discussion: Developing in an AI world - are we all demoted to reviewers? WeAreDevelopers WebDev & AI Day March2025

Laurie Voss, Rey Bango, Hannah Foxwell, Rizel Scarlett & Thomas Steiner

Evaluating AI models for code comprehension22:03

Evaluating AI models for code comprehension

Merrill Lutsky

Bringing the power of AI to your application.32:26

Bringing the power of AI to your application.

Krzysztof Cieślak

AI-Powered Code Documentation: Simplify the Complex28:22

AI-Powered Code Documentation: Simplify the Complex

Patrick Schnell

Related Articles

View all articles
DC
Daniel Cranney
Dev Digest 196: AI Killed DevOps, LLM Political Bias & AI Security
Inside last week’s Dev Digest 196 . ⚖️ Political bias in LLMs 🫣 AI written code causes 1 in 5 security breaches 🖼️ Is there a limit to alternative text on images? 📝 CodeWiki - understand code better 🟨 Long tasks in JavaScript 👻 Scare yourself into n...
Dev Digest 196: AI Killed DevOps, LLM Political Bias & AI Security

From learning to earning

Jobs that call for the skills explored in this talk.