Problem Solving using PySpark – Regression & Classification

Course Curriculum

Chapter 1: Introduction

Lecture 1: Introduction

Lecture 2: Problem Solving with PySpark : Regression and Classification

Chapter 2: Data analysis and descriptive statistics with PySpark

Lecture 1: Setting up PySpark Environment in Google Colab

Lecture 2: Understanding Descriptive Statistics in PySpark

Lecture 3: Understanding Data Filtering and Slicing in PySpark

Lecture 4: Summary of Descriptive Statistics in PySpark and Quiz

Chapter 3: Data Cleaning with PySpark

Lecture 1: Introduction to Data Cleaning with PySpark

Lecture 2: Setting up PySpark Environment for Data Cleaning on Google Colab

Lecture 3: Understanding the Dataset : Explanatory Analysis and Data Cleaning with PySpark

Lecture 4: PySpark Data Cleaning : Assessment of Null Values and Outliers

Lecture 5: Data Cleaning with PySpark : Imputation Strategy Quiz

Lecture 6: Introduction to Pivot Tables in PySpark

Chapter 4: Predictive modeling with PySpark using Regression

Lecture 1: Introduction to Regression and Classification Problems in PySpark

Lecture 2: Understanding the Data Set through Explanatory Analysis

Lecture 3: Correlation Analysis and Data Preparation

Lecture 4: Modeling the data using Gradient Boosted Trees Regression

Lecture 5: Understanding Feature Importance

Lecture 6: Gradient Boosted Trees Regression – Quiz

Chapter 5: Predictive Modeling with PySpark using Classification

Lecture 1: Classification Problem Statement : Supervised Machine Learning

Lecture 2: Data Cleaning and Preparation for XGBoost Classification Model

Lecture 3: XGBoost Classification Model Pipeline using PySpark

Lecture 4: Summary of the segment on Spark XGBoost Classifier

Chapter 6: Text analytics using PySpark and Spark NLP

Lecture 1: Classification Model for Text Data

Lecture 2: Understanding the Data for Text Classification

Lecture 3: Word Cloud : Text Analytics Quiz

Lecture 4: Spark NLP Pipeline : Classification Model

Chapter 7: Time Series Analysis and Forecast with PySpark and Prophet

Lecture 1: Introduction to Time Series Analysis : Setting up the Google Colab Notebook

Lecture 2: Explanatory Analysis and Data Cleaning

Lecture 3: Analysis of time series components using advanced visualization techniques

Lecture 4: Use of Prophet Model for Time Series Forecasting

Lecture 5: Time Series Forecasting – Quiz

Chapter 8: Introduction to Spark SQL

Lecture 1: Introduction to Spark SQL Querying

Lecture 2: Comparison of PySpark statements and Spark SQL Query

Lecture 3: Join in Spark SQL

Lecture 4: Join in Spark SQL – Quiz