George Asafev

Swapping Low Latency Data Storage Under High Load

Miro's Redis instance couldn't scale for 700 engineers. Learn how they moved 140 million accounts to DynamoDB while handling 100,000 requests per second.

Swapping Low Latency Data Storage Under High Load
#1about 4 minutes

Migrating 140M users from a Redis-based monolith

Miro's user registry team faced migrating 140 million users from Redis, which was unsuitable as a persistent store and was tied to a difficult-to-maintain monolith.

#2about 3 minutes

Evaluating PostgreSQL vs DynamoDB for the new database

The team compared PostgreSQL's flexibility with DynamoDB's managed scaling, ultimately choosing DynamoDB paired with DAX for its performance on read-heavy workloads.

#3about 2 minutes

Load testing the database solutions under production load

To validate their choice, they load-tested PostgreSQL using pgbench and DynamoDB with a k6 and EC2 setup to simulate production traffic.

#4about 2 minutes

Understanding the limitations of DynamoDB and DAX

Adopting DynamoDB with DAX required navigating limitations like primary key access patterns, separate item and query caches, and difficult vertical scaling.

#5about 4 minutes

The phased migration strategy from Redis to DynamoDB

The migration was executed in controlled phases using feature flags, including dual writes, a background data copy, shadow calls for validation, and a gradual rollout.

#6about 2 minutes

Unexpected DAX performance issues and learnings

After going live, the team discovered that rapid connection spikes caused DAX throttling and that adding a fourth node to their cluster unexpectedly increased replication lag.

#7about 4 minutes

Results of the migration and Q&A

The successful migration reduced endpoint latency up to four times and enabled the team to move forward with their microservice architecture, followed by a Q&A on alternative solutions.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

Organizing a developer conference for 15,000 attendees
01:32 MIN

Organizing a developer conference for 15,000 attendees

Cat Herding with Lions and Tigers - Christian Heilmann

How to build structure and culture without killing agility
05:12 MIN

How to build structure and culture without killing agility

From Data Keeper to Culture Shaper: The Evolution of HR Across Growth Stages

Navigating cultural shifts during rapid growth and investment
06:01 MIN

Navigating cultural shifts during rapid growth and investment

From Data Keeper to Culture Shaper: The Evolution of HR Across Growth Stages

The growing importance of data and technology in HR
01:54 MIN

The growing importance of data and technology in HR

From Data Keeper to Culture Shaper: The Evolution of HR Across Growth Stages

Moving beyond headcount to solve business problems
04:27 MIN

Moving beyond headcount to solve business problems

What 2025 Taught Us: A Year-End Special with Hung Lee

Increasing the value of talk recordings post-event
04:57 MIN

Increasing the value of talk recordings post-event

Cat Herding with Lions and Tigers - Christian Heilmann

The business case for sustainable high performance
03:34 MIN

The business case for sustainable high performance

Sustainable High Performance: Build It or Pay the Price

Europe's push for digital independence from US tech
05:17 MIN

Europe's push for digital independence from US tech

WeAreDevelopers LIVE – AI, Freelancing, Keeping Up with Tech and More

Featured Partners

Related Videos

See all videos
Scaling: from 0 to 20 million users28:12

Scaling: from 0 to 20 million users

Josip Stuhli

Database Magic behind 40 Million operations/s24:22

Database Magic behind 40 Million operations/s

Jürgen Pilz

Single Server, Global Reach: Running a Worldwide Marketplace on Bare Metal in a Cloud-Dominated World50:06

Single Server, Global Reach: Running a Worldwide Marketplace on Bare Metal in a Cloud-Dominated World

Jens Happe

Reliable scalability: How Amazon.com scales on AWS22:24

Reliable scalability: How Amazon.com scales on AWS

Florian Mair

In-Memory Computing - The Big Picture30:36

In-Memory Computing - The Big Picture

Markus Kett

AWS Migration within 3 months37:08

AWS Migration within 3 months

Steffen Heilmann

The Road to MLOps: How Verivox Transitioned to AWS25:48

The Road to MLOps: How Verivox Transitioned to AWS

Elisabeth Günther

Advanced Caching Patterns used by 2000 microservices44:37

Advanced Caching Patterns used by 2000 microservices

Natan Silnitsky

Related Articles

View all articles
BB
Benedikt Bischof
Making Data Warehouses Fast: A Developer’s Story
Welcome to this issue of the WeAreDevelopers Live Talk series. This article recaps an interesting talk by Adnan Rahic who teaches the audience how to make data warehouses.About the Speaker: Adnan is senior developers advocate at Cube. His passion lie...
Making Data Warehouses Fast: A Developer’s Story

From learning to earning

Jobs that call for the skills explored in this talk.