Dr. Qiyang Duan

Alibaba Big Data and Machine Learning Technology

The world's largest shopping festival generates petabytes of data. Discover the integrated big data and AI platform Alibaba built to power its massive e-commerce empire.

Alibaba Big Data and Machine Learning Technology
#1about 4 minutes

How e-commerce scale drives Alibaba's big data technology

Alibaba's big data platform evolved to handle massive transaction volumes from events like the Double 11 festival, processing petabytes of data daily.

#2about 4 minutes

Exploring Alibaba's big data and AI product portfolio

The platform is structured around a typical data flow, including tools for data ingestion, storage, processing, visualization, and a dedicated AI platform called PAI.

#3about 4 minutes

Comparing core data processing engines on Alibaba Cloud

A look at key processing tools including MaxCompute for massive-scale SQL, E-MapReduce for Hadoop migrations, and Flink for real-time stream processing.

#4about 5 minutes

Using DataWorks as a unified IDE for big data

DataWorks provides a single environment for data development and operations, integrating various engines like MaxCompute and Hadoop and including data governance tools.

#5about 7 minutes

Developing models with the Platform for AI (PAI)

The PAI platform supports both visual drag-and-drop model building with PAI Studio and code-based development using the Data Science Workshop (DSW) with JupyterLab.

#6about 2 minutes

Announcing the Alibaba Cloud AI Champion 2020 competition

An overview of the AI Champion 2020 competition, which offers cash prizes and cloud credits for innovative projects built on the PAI platform.

#7about 5 minutes

Live demo of data analysis with DataWorks

A walkthrough of the DataWorks interface, demonstrating how to write and execute SQL queries, visualize results, and manage data integration jobs.

#8about 8 minutes

Live demo of building models on the PAI platform

A demonstration of building a recommendation model using the drag-and-drop PAI Studio and running a deep learning example in the PAI-DSW JupyterLab environment.

Related jobs
Jobs that call for the skills explored in this talk.

Featured Partners

Related Articles

View all articles
DC
Daniel Cranney
The State of WebDev AI 2025 Results: What Can We Learn?
Introduction The 2025 edition of The State of WebDev AI offers a detailed snapshot of how developers are using AI today, which tools have gained the most traction over the past year, and what these trends suggest about the future of the industry. In...
The State of WebDev AI 2025 Results: What Can We Learn?
CH
Chris Heilmann
With AIs wide open - WeAreDevelopers at All Things Open 2025
Last week our VP of Developer Relations, Chris Heilmann, flew to Raleigh, North Carolina to present at All Things Open . An excellent event he had spoken at a few times in the past and this being the “Lucky 13” edition, he didn’t hesitate to come and...
With AIs wide open - WeAreDevelopers at All Things Open 2025

From learning to earning

Jobs that call for the skills explored in this talk.

Data Scientist

Data Scientist

Ai-powered Stackadapt Marketing Platform

Remote
Machine Learning