Learn Big Data Analysis with PySpark

Course Curriculum

Chapter 1: Introduction

Lecture 1: Introduction to the Course

Lecture 2: What is PySpark

Lecture 3: Spark Features

Lecture 4: Resilient Distributed Dataset (RDD)

Chapter 2: Introduction to Python Development Environment

Lecture 1: Introduction to development platform (Colaboratory)

Lecture 2: Login to Colaboratory

Lecture 3: Create Lists in Python

Lecture 4: Create Tuples and Dictionaries in Python

Lecture 5: Create loops and functions in Python

Lecture 6: Introduction to Pandas Library in Python

Lecture 7: Import and use Pandas in Python

Lecture 8: Introduction to NumPy

Lecture 9: Import and use NumPy in Python

Lecture 10: Configure PySpark development environment to run Spark

Lecture 11: Course Rating

Chapter 3: Cleaning and Transforming Data in PySpark

Lecture 1: Create Resilient Distributed Dataset (RDD) in Spark

Lecture 2: Import data files in PySpark

Lecture 3: Show data in datasets

Lecture 4: Clean and transform data part_1 (replace nulls with values)

Lecture 5: Clean and transform data part_2 (remove null values)

Lecture 6: Clean and transform data part_3 (change data types)

Lecture 7: Clean and transform data part_4 (replace numeric nulls by the column mean)

Lecture 8: Clean and transform data part_5 (change columns names)

Lecture 9: Clean and transform data part_6 (change columns order in a dataset)

Lecture 10: Course Rating

Chapter 4: Performing Data Analysis in PySpark

Lecture 1: A reminder to upload data and run configuration code

Lecture 2: Data analysis part_1 (find sales volume per item using sum function)

Lecture 3: Data analysis part_2 (find number of orders per item using count function)

Lecture 4: Data analysis part_3 (find itemType data using aggregation functions)

Lecture 5: Data analysis part_4 (add a calculated column to datasets)

Lecture 6: Data analysis part_5 (find total profit per item_type and Sales_channel)

Lecture 7: Data analysis part_6 (find sales facts per country using conditions)

Lecture 8: Data analysis part_7 (find total profit per country)

Lecture 9: Data analysis part_8 (find countries generated total profit greater than 4 M)

Lecture 10: Run data analysis with SQL Part_1

Lecture 11: Run data analysis with SQL Part_2

Lecture 12: Course Rating