Advanced Apache Spark for Data Scientists and Developers

Course Curriculum

Chapter 1: Introduction to Advanced Apache Spark

Lecture 1: Introduction to Apache Spark

Lecture 2: Spark Installation

Lecture 3: IDE Installation

Chapter 2: Tuning and Debugging

Lecture 1: Introduction and Topics

Lecture 2: Spark Configuration with SparkConf

Lecture 3: Web User-Interface and Log Files

Lecture 4: Data Serialization

Lecture 5: Memory Tuning

Lecture 6: Level of Parallelism

Lecture 7: Section Topics

Chapter 3: Spark Streaming

Lecture 1: Introduction and Topics

Lecture 2: Overview of Spark Streaming

Lecture 3: Linking Input Sources

Lecture 4: Streaming Context

Lecture 5: Discretized Streams (DStreams)

Lecture 6: Input DStreams

Lecture 7: Hands-on Exercise 1: Spark Streaming

Lecture 8: Stateless Transformations on DStreams

Lecture 9: Stateful Transformations

Lecture 10: Hands-on Exercise 2: Spark Streaming

Lecture 11: Output Operations

Lecture 12: Hands-on Exercise 3: Spark Streaming

Lecture 13: Checkpointing

Lecture 14: Caching and Persisting

Lecture 15: Tuning and Debugging

Lecture 16: Section Topics

Chapter 4: Spark SQL

Lecture 1: Introduction to Spark SQL

Lecture 2: Spark SQL Overview

Lecture 3: The Spark Shell hands-on

Lecture 4: Hands-on Exercise 1: part a) Import CSV

Lecture 5: Schema Inference

Lecture 6: Data Query Select

Lecture 7: DataFrame.Reader DataFrame.Writer

Lecture 8: Hands-on Exercise 1: part b) Import JSON

Lecture 9: Data Query INNER JOINs

Lecture 10: Group By, Order By, Window Functions

Lecture 11: Data Query OUTER JOINs, SEMI JOIN

Lecture 12: Custom UDF (User Defined Function)

Lecture 13: API or SQL?

Lecture 14: Hands-on Exercise 2: Spark SQL

Chapter 5: Spark MLlib

Lecture 1: Introduction and Topics

Lecture 2: Machine Learning

Lecture 3: MLlib

Lecture 4: Basic Statistics

Lecture 5: Optimization

Lecture 6: Classification

Lecture 7: Hands-on Exercise 1: Spark MLlib: Classification

Lecture 8: Validation

Lecture 9: Regression

Lecture 10: Clustering

Lecture 11: Hands-on Exercise 2: Spark MLlib: Clustering

Lecture 12: Feature Extraction and Transformation

Lecture 13: Dimensionality Reduction

Lecture 14: Collaborative Filtering

Lecture 15: Evaluation Metrics

Chapter 6: Spark GraphX

Lecture 1: Introduction to Spark GraphX

Lecture 2: Graph creation examples

Lecture 3: Graph Operators Overview, Information about a Graph

Lecture 4: Information about a graph example

Lecture 5: Transform Graph Items

Lecture 6: Transform graph items examples

Lecture 7: Modify Graph Structure

Lecture 8: Modify graph structure example

Lecture 9: Graph Neighborhood Aggregations

Lecture 10: Neighborhood Aggregations Examples

Lecture 11: Graph Algorithms

Lecture 12: Triangle Count Example

Lecture 13: Pregel- Graph Parallel Computation

Lecture 14: Pregel Example

Lecture 15: Optimized Graph Representation

Lecture 16: Hands-on Exercise: Spark GraphX