Nathaniel Okenwa

Hello JARVIS - Building Voice Interfaces for Your LLMS

Can we escape the uncanny valley of conversation? Learn to build voice AIs that handle interruptions and latency like a true conversational partner.

Hello JARVIS - Building Voice Interfaces for Your LLMS
#1about 2 minutes

Introduction to building JARVIS-like voice interfaces

The goal of building a sophisticated voice AI assistant like Iron Man's JARVIS is now more achievable thanks to modern technologies.

#2about 5 minutes

Why natural voice AI has been so difficult

Fictional AI assistants set a high bar for natural voice interaction that early real-world technologies like Siri failed to meet until the arrival of LLMs.

#3about 3 minutes

Navigating the uncanny valley of AI conversations

To avoid the unsettling 'uncanny valley' in voice AI, systems must handle non-linear conversations, interruptions, and the subtle timing of human speech.

#4about 3 minutes

Architecting a composable text-based voice AI stack

A modern voice AI stack combines speech-to-text, an LLM, and text-to-speech, offering more control and better performance than current speech-to-speech models.

#5about 8 minutes

Live demo of handling user interruptions

The demo shows how to implement interruption handling by stopping the AI's audio output and feeding the context of the interruption back into the LLM prompt.

#6about 3 minutes

Using voice interstitials to manage processing delays

Voice interstitials are pre-emptive audio messages that inform the user an action is in progress, preventing the perception of a system failure during long tasks.

#7about 1 minute

Designing AI agents as a constellation of models

An effective voice AI system is not a single monolithic agent but a constellation of smaller, faster models for specific tasks like checking for wake words or playing interstitials.

#8about 2 minutes

Abstracting voice infrastructure with Twilio Conversation Relay

Twilio's Conversation Relay simplifies development by managing the complex audio pipeline, including speech-to-text, text-to-speech, and interruption handling, via a WebSocket API.

Related jobs
Jobs that call for the skills explored in this talk.

Machine Learning Engineer

Picnic Technologies B.V.

Picnic Technologies B.V.
Amsterdam, Netherlands

Intermediate
Senior
Python
Structured Query Language (SQL)
+1

Matching moments

Building and iterating on an LLM-powered product
05:03 MIN

Building and iterating on an LLM-powered product

Slopquatting, API Keys, Fun with Fonts, Recruiters vs AI and more - The Best of LIVE 2025 - Part 2

Unlocking LLM potential with creative prompting techniques
04:59 MIN

Unlocking LLM potential with creative prompting techniques

WeAreDevelopers LIVE – Frontend Inspirations, Web Standards and more

Writing authentic content in the age of LLMs
07:43 MIN

Writing authentic content in the age of LLMs

Slopquatting, API Keys, Fun with Fonts, Recruiters vs AI and more - The Best of LIVE 2025 - Part 2

AI lawsuits, code flagging, and self-driving subscriptions
01:02 MIN

AI lawsuits, code flagging, and self-driving subscriptions

Fake or News: Self-Driving Cars on Subscription, Crypto Attacks Rising and Working While You Sleep - Théodore Lefèvre

Using AI agents to modernize legacy COBOL systems
06:28 MIN

Using AI agents to modernize legacy COBOL systems

Devs vs. Marketers, COBOL and Copilot, Make Live Coding Easy and more - The Best of LIVE 2025 - Part 3

Exploring bizarre headlines about IoT and robotics
00:38 MIN

Exploring bizarre headlines about IoT and robotics

Fake or News: Coding on a Phone, Emotional Support Toasters, ChatGPT Weddings and more - Anselm Hannemann

The security challenges of building AI browser agents
06:33 MIN

The security challenges of building AI browser agents

AI in the Open and in Browsers - Tarek Ziadé

Increasing the value of talk recordings post-event
04:57 MIN

Increasing the value of talk recordings post-event

Cat Herding with Lions and Tigers - Christian Heilmann

Featured Partners

Related Videos

See all videos
Three years of putting LLMs into Software - Lessons learned26:30

Three years of putting LLMs into Software - Lessons learned

Simon A.T. Jiménez

Using LLMs in your Product31:12

Using LLMs in your Product

Daniel Töws

AI: Superhero or Supervillain? How and Why with Scott Hanselman25:17

AI: Superhero or Supervillain? How and Why with Scott Hanselman

Scott Hanselman

Bringing the power of AI to your application.32:26

Bringing the power of AI to your application.

Krzysztof Cieślak

Beyond Chatbots: How to build Agentic AI systems24:22

Beyond Chatbots: How to build Agentic AI systems

Philipp Schmid

AI & Ethics28:27

AI & Ethics

PJ Hagerty

On a Secret Mission: Developing AI Agents29:03

On a Secret Mission: Developing AI Agents

Jörg Neumann

Speak, Code, Deploy: Transforming Developer Experience with Voice Commands28:37

Speak, Code, Deploy: Transforming Developer Experience with Voice Commands

Sami Ekblad

Related Articles

View all articles
CH
Chris Heilmann
WWC24 Talk - Scott Hanselman - AI: Superhero or Supervillain?
Join Scott Hanselman at WWC24 to explore AI's role as a superhero or supervillain. Scott shares his 32 years of experience in software engineering, discusses AI myths, ethical dilemmas, and tech advancements. Engage with his live demos and insights o...
WWC24 Talk - Scott Hanselman - AI: Superhero or Supervillain?

From learning to earning

Jobs that call for the skills explored in this talk.