Jodie Burchell
Vectorize all the things! Using linear algebra and NumPy to make your Python code lightning fast.
#1about 3 minutes
Why Python loops become slow at scale
Traditional loops over lists become a major performance bottleneck when processing large amounts of data.
#2about 5 minutes
Representing data with vectors, matrices, and NumPy arrays
Learn the fundamentals of linear algebra, where data points are vectors, datasets are matrices, and NumPy arrays provide the data structure.
#3about 3 minutes
A high-level overview of the KNN algorithm
The k-nearest neighbors algorithm classifies data points by finding the most common label among their closest neighbors in a vector space.
#4about 4 minutes
Coding a slow baseline KNN with Python lists
A walkthrough of an unoptimized k-nearest neighbors implementation demonstrates the severe performance issues caused by nested loops.
#5about 3 minutes
Vectorizing the distance calculation to remove a loop
Replace the inner loop for calculating Manhattan distance with a single vectorized subtraction operation on NumPy arrays.
#6about 9 minutes
Eliminating nested loops with NumPy array broadcasting
Use array reshaping and broadcasting to perform all pairwise distance calculations simultaneously, avoiding explicit replication and nested loops.
#7about 3 minutes
Optimizing neighbor selection with NumPy sorting and slicing
Gain final performance improvements by replacing Python's list sorting with NumPy's faster sorting algorithms and efficient array slicing.
#8about 6 minutes
How memory layout makes NumPy arrays so fast
NumPy arrays are faster because they store data in a contiguous block of memory, which is more efficient for the CPU to process than the scattered memory of Python lists.
#9about 22 minutes
Audience Q&A on performance and data science
Answers common questions about NumPy's underlying C implementation, hyperparameter tuning, memory management, and career paths in data science.
Related jobs
Jobs that call for the skills explored in this talk.
Picnic Technologies B.V.
Amsterdam, Netherlands
Intermediate
Senior
Python
Structured Query Language (SQL)
+1
Matching moments
08:07 MIN
Exploring modern JavaScript performance and new CSS features
WeAreDevelopers LIVE – AI, Freelancing, Keeping Up with Tech and More
02:33 MIN
Why you might not need JavaScript for everything
WeAreDevelopers LIVE – You Don’t Need JavaScript, Modern CSS and More
04:09 MIN
How Python became the dominant language for AI
AI in the Open and in Browsers - Tarek Ziadé
03:16 MIN
Improving the developer feedback loop with specialized tools
Developer Time Is Valuable - Use the Right Tools - Kilian Valkhof
02:55 MIN
Why developers often undervalue their time and paid tools
Developer Time Is Valuable - Use the Right Tools - Kilian Valkhof
02:49 MIN
Using AI to overcome challenges in systems programming
AI in the Open and in Browsers - Tarek Ziadé
09:10 MIN
How AI is changing the freelance developer experience
WeAreDevelopers LIVE – AI, Freelancing, Keeping Up with Tech and More
03:07 MIN
Final advice for developers adapting to AI
WeAreDevelopers LIVE – AI, Freelancing, Keeping Up with Tech and More
Featured Partners
Related Videos
56:46A beginner’s guide to modern natural language processing
Jodie Burchell
59:43Accelerating Python on GPUs
Paul Graham
1:05:44WeAreDevelopers LIVE - Vector Similarity Search Patterns for Efficiency and more
Chris Heilmann, Daniel Cranney, Raphael De Lio & Developer Advocate at Redis
21:02CUDA in Python
Andy Terrel
46:3030 Golden Rules of Deep Learning Performance
Anirudh Koul
57:46Overview of Machine Learning in Python
Adrian Schmitt
14:44Catching up on the basics you don't really need that much code
Chris Heilmann
50:38Python Data Visualization @ Deepnote (w/ PyViz overview)
Radovan Kavický
Related Articles
View all articles
From learning to earning
Jobs that call for the skills explored in this talk.
VECTOR Informatik
Stuttgart, Germany
Senior
Java
IT Security
VECTOR Informatik
Stuttgart, Germany
Senior
DevOps
Terraform
Kubernetes
Visonum GmbH
Remote
Junior
Intermediate
React
Redux
TypeScript
Forschungszentrum Jülich GmbH
Jülich, Germany
Intermediate
Senior
Linux
Docker
AI Frameworks
Machine Learning
Corriculo Ltd
Reading, United Kingdom
Remote
£40-60K
GIT
Linux
NumPy
+8
Corriculo Ltd
Oxford, United Kingdom
Remote
£40-60K
GIT
Linux
NumPy
+8
Corriculo Ltd
Oxford, United Kingdom
Remote
£60K
GIT
Linux
NumPy
+8
Corriculo Ltd
Milton, United Kingdom
Remote
£40-60K
GIT
Linux
NumPy
+8
Corriculo Ltd
Abingdon, United Kingdom
Remote
£40-60K
GIT
Linux
NumPy
+8