Miki Lombardi

Automate everything via NodeJS and Puppeteer

How do you automate tasks on sites without a public API? See how NodeJS and Puppeteer can scrape data, publish content, and run continuous performance tests.

Automate everything via NodeJS and Puppeteer
#1about 2 minutes

An introduction to the Puppeteer library for Node.js

Puppeteer provides a high-level API to control headless or full Chrome instances through the DevTools Protocol.

#2about 2 minutes

The history and evolution of web automation tools

Web automation has evolved from cross-language frameworks like Selenium to JavaScript-based tools like PhantomJS and now Puppeteer.

#3about 3 minutes

Understanding Puppeteer's API and common capabilities

Puppeteer's readable, action-based API can be used for scraping, testing, generating PDFs, and analyzing performance.

#4about 3 minutes

Comparing Puppeteer with alternatives like Playwright

Playwright is a strong alternative for cross-browser end-to-end testing, while Puppeteer is more of a general-purpose automation tool.

#5about 4 minutes

Exploring common Puppeteer use cases for web applications

Puppeteer is well-suited for scraping dynamic sites, automating UI and visual regression tests, and building custom development utilities.

#6about 7 minutes

Case study on scraping Skyscanner for cheap flights

A personal project demonstrates how to automate flight searches by splitting code into modular functions and intercepting API responses.

#7about 5 minutes

Automating podcast uploads from YouTube to Anchor.fm

A GitHub Action uses Puppeteer to automate converting YouTube videos to audio and publishing them on Anchor.fm, saving significant manual effort.

#8about 7 minutes

Automating Lighthouse performance tests with authentication

A professional setup uses Puppeteer to handle authentication and run Lighthouse tests on protected pages, sending metrics to Prometheus and Grafana.

#9about 9 minutes

Q&A on Puppeteer vs Cypress and cloud deployment

The discussion covers when to choose Puppeteer over Cypress, running it in the cloud using containers instead of serverless, and its use in visual testing.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

Why you might not need JavaScript for everything
02:33 MIN

Why you might not need JavaScript for everything

WeAreDevelopers LIVE – You Don’t Need JavaScript, Modern CSS and More

Automating video post-production with local scripts
02:54 MIN

Automating video post-production with local scripts

Cat Herding with Lions and Tigers - Christian Heilmann

Improving the developer feedback loop with specialized tools
03:16 MIN

Improving the developer feedback loop with specialized tools

Developer Time Is Valuable - Use the Right Tools - Kilian Valkhof

Using AI to overcome challenges in systems programming
02:49 MIN

Using AI to overcome challenges in systems programming

AI in the Open and in Browsers - Tarek Ziadé

The value of progressive enhancement and semantic HTML
03:31 MIN

The value of progressive enhancement and semantic HTML

WeAreDevelopers LIVE – You Don’t Need JavaScript, Modern CSS and More

Ensuring accurate testing across different browsers and devices
07:12 MIN

Ensuring accurate testing across different browsers and devices

Developer Time Is Valuable - Use the Right Tools - Kilian Valkhof

Recreating React components using AI and dev tools
04:56 MIN

Recreating React components using AI and dev tools

WeAreDevelopers LIVE – AI, Freelancing, Keeping Up with Tech and More

Increasing the value of talk recordings post-event
04:57 MIN

Increasing the value of talk recordings post-event

Cat Herding with Lions and Tigers - Christian Heilmann

Featured Partners

Related Videos

See all videos
Boost Productivity with AI: Figma & Playwright MCP Workflows - Aris Markogiannakis45:48

Boost Productivity with AI: Figma & Playwright MCP Workflows - Aris Markogiannakis

From clicks to cribs - How to find your dream home with web scraping26:08

From clicks to cribs - How to find your dream home with web scraping

Alexander Lichter

One-click-to-production: Test and automate your application31:15

One-click-to-production: Test and automate your application

Bartosz Pietrucha

Frontend Performance Testing in practice36:50

Frontend Performance Testing in practice

Jonas Kröger

Measure and improve frontend performance by using test automation25:38

Measure and improve frontend performance by using test automation

Ramona Schwering

Catching up on the basics you don't really need that much code14:44

Catching up on the basics you don't really need that much code

Chris Heilmann

How to Automate your Murder Mystery22:00

How to Automate your Murder Mystery

Daphne Oakes and Tiger Oakes

Things I learned while writing high-performance JavaScript applications27:27

Things I learned while writing high-performance JavaScript applications

Michele Riva

Related Articles

View all articles
DC
Daniel Cranney
Dev Digest 195: End of Likes, JavaScript’s a Zoo, and Messing with Bots!
Inside last week’s Dev Digest 195 . 👎 No more external likes 🤗 Needy programs 📉 The worst selling Microsoft product 🟨 JavaScript engines zoo 🍞 No more toasts! 🤖 Messing with bots 👔 Beware of fake job interviews 🗞️ Join over 150,000 developers alread...
Dev Digest 195: End of Likes, JavaScript’s a Zoo, and Messing with Bots!

From learning to earning

Jobs that call for the skills explored in this talk.