Zaid Zaim
Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!
#1about 3 minutes
Introducing the concept of an immersive AI copilot
The talk introduces building an interactive AI character inspired by Disney's WALL-E using immersive technologies and generative AI.
#2about 3 minutes
Using Microsoft Mesh for collaborative XR experiences
Microsoft Mesh integrates into Microsoft Teams to provide a platform for immersive meetings, trainings, and social events.
#3about 1 minute
The core technologies for building an immersive copilot
The project is built using three main components: the Unity game engine, the Mesh Toolkit for deployment, and Azure OpenAI for intelligence.
#4about 5 minutes
Integrating Azure OpenAI service with Unity via REST API
A C# script in Unity calls the Azure OpenAI REST API to handle communication, parse the JSON response, and manage text-to-speech output.
#5about 2 minutes
Using the Mesh Toolkit for cross-platform deployment
The Mesh Toolkit provides shaders, visual scripting, and an optimizer to prepare and deploy the Unity environment to the Mesh platform.
#6about 6 minutes
Live demo of the AI copilot on a HoloLens device
A live demonstration shows interaction with the AI character "Anna" using voice commands on a Microsoft HoloLens.
#7about 6 minutes
Live demo in a shared Microsoft Mesh environment
The AI copilot is demonstrated within a shared, multi-user Microsoft Mesh space, showing collaborative interaction with the character.
#8about 6 minutes
Integrating immersive spaces directly into Microsoft Teams
The final demo showcases how users can join a Microsoft Mesh immersive space directly from a standard Microsoft Teams meeting.
Related jobs
Jobs that call for the skills explored in this talk.
Matching moments
26:20 MIN
Live demo of the immersive AI assistant Anna
How Mixed Reality, Azure AI and Drones turned me into a Magician?
00:26 MIN
Fusing mixed reality, AI, and drones for magical experiences
How Mixed Reality, Azure AI and Drones turned me into a Magician?
03:58 MIN
Exploring the core technology stack for the project
How Mixed Reality, Azure AI and Drones turned me into a Magician?
07:16 MIN
A practical walkthrough of the Azure AI Foundry playground
How Mixed Reality, Azure AI and Drones turned me into a Magician?
16:05 MIN
Creating an intelligent cockpit experience with AI and AR
Software defines the vehicle: Why customers and developers will love cars even more
05:49 MIN
Getting started with the Azure AI Foundry platform
How Mixed Reality, Azure AI and Drones turned me into a Magician?
04:40 MIN
Exploring the tools for drone and MR development
Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!
14:41 MIN
Using AI to create adaptive and inclusive VR experiences
The End of Unconscious Bias Training. A Scientific Approach to Creating an Inclusive Culture
Featured Partners
Related Videos
35:26How Mixed Reality, Azure AI and Drones turned me into a Magician?
Zaid Zaim
38:28Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!
Zaid Zaim
31:39Livecoding with AI
Rainer Stropek
23:33Azure AI Foundry for Developers: Open Tools, Scalable Agents, Real Impact
Oliver Will
1:12:04WeAreDevelopers LIVE - the weekly developer show with Chris Heilmann and Daniel Cranney
1:09:49Inside the AI Revolution: How Microsoft is Empowering the World to Achieve More
Simi Olabisi
24:24Best practices: Building Enterprise Applications that leverage GenAI
Damir
26:35Agentic AI - From Theory to Practice: Developing Multi-Agent AI Systems on Azure
Ricardo
Related Articles
View all articles
From learning to earning
Jobs that call for the skills explored in this talk.
Cloud Solution Architecture - AI Application Development
Microsoft
Charing Cross, United Kingdom
Azure
DevOps
PostgreSQL
Amazon Web Services (AWS)
Remote Head of AI Development - Python, Azure, AI Powered Platform
Neologik
Windsor, United Kingdom
Remote
Senior
API
Azure
DevOps
Python
+5
AI Agent Builder & Experimenter (Fullstack)
autonomous-teaming
München, Germany
Remote
API
React
Python
TypeScript
AI Engineer
Codurance
Leeds, United Kingdom
£53K
Azure
Python
Agile Methodologies
Amazon Web Services (AWS)