Master Azure Databricks

Course Curriculum

Chapter 1: Introduction

Lecture 1: Course Structure

Chapter 2: Azure DataBricks

Lecture 1: What is Data Pipeline

Lecture 2: Azure databricks

Lecture 3: Azure Databricks Architecture

Lecture 4: Azure Account Setup

Lecture 5: WorkSpace Setup

Chapter 3: Workspace, Notebook, Job, DBFS

Lecture 1: Agenda

Lecture 2: Navigate the Workspace

Lecture 3: Databricks Runtimes

Lecture 4: Clusters Part 1

Lecture 5: Cluster Part 2

Lecture 6: Notebooks

Lecture 7: Libraries

Lecture 8: Repos for Git integration

Lecture 9: Databricks File System (DBFS)

Lecture 10: DBUTILS

Lecture 11: Widgets

Lecture 12: Workflows

Lecture 13: Metastore – Setup external Metastore

Lecture 14: Metastore – Setup external Metastore II

Chapter 4: RDD, Database, Tables

Lecture 1: Agenda

Lecture 2: RDD Introduction

Lecture 3: Lab : Creating RDD

Lecture 4: Lab : RDD transformations

Lecture 5: Lab :RDD Actions

Lecture 6: Lab : RDD Joins

Lecture 7: Lab : Pair RDD

Lecture 8: Lab : Broadcast Variables

Lecture 9: Lab : Accumulators

Lecture 10: Lab : Convert RDD to DataFrame

Lecture 11: Lab:Import & read data

Lecture 12: Lab: Create a table using the UI

Lecture 13: Lab : Create a table in a Notebook

Chapter 5: DataFrame

Lecture 1: Agenda

Lecture 2: What is DataFrame

Lecture 3: Lab : Create DataFrames

Lecture 4: Lab : Define Schema

Lecture 5: Lab : Functions

Lecture 6: Lab : Casting Operations

Lecture 7: Lab : Filter Transformation

Lecture 8: Lab: Update, Update ALL & UpdateByName

Lecture 9: Lab : OrderBy & SortBY

Lecture 10: Lab : GroupBy

Lecture 11: Lab : Remove Duplicates

Lecture 12: Lab : Window Functions

Lecture 13: Lab : Date and Timestamp Functions

Lecture 14: Lab : UDF (User Defined Function)

Lecture 15: Lab : JOIN

Lecture 16: Lab : Handle corrupt records using the badRecordsPath

Lecture 17: Lab:File metadata column

Chapter 6: Data Source

Lecture 1: Agenda

Lecture 2: Lab : Read and Write Parquet File

Lecture 3: Lab : Read and Write CSV Files

Lecture 4: Lab : Read and Write JSON Files

Lecture 5: Lab : Read and Write XML Files

Lecture 6: Lab : Read and Write Excel file

Lecture 7: Lab : Read and Write SQL databases using JDBC

Lecture 8: Lab : Read and Write Azure blob storage

Chapter 7: Spark Structure Streaming

Lecture 1: Agenda

Lecture 2: What is Spark Structure Streaming

Lecture 3: Data Source & Sink

Lecture 4: Lab : Rate & File Source

Lecture 5: Lab : Kafka Source

Lecture 6: Lab : Sink : Console, Memory, File & Custom

Lecture 7: Lab : Build Streaming ETL

Lecture 8: Lab : Setup Event Hub

Lecture 9: Lab : Event Hub Producer

Lecture 10: Lab : Integrate Event Hubs with Data Bricks

Lecture 11: Lab : Transformation

Lecture 12: Streaming ETL 5 : Ingest into Azure Data storage

Lecture 13: Twitter Sentiment Analysis – Introduction

Lecture 14: Setup Twitter Developer Account

Lecture 15: Twitter Sentiment Analysis – II

Lecture 16: Twitter Sentiment Analysis – III

Chapter 8: DataBricks SQL

Lecture 1: Agenda

Lecture 2: Components in Databricks SQL

Lecture 3: Lab : Configuring a SQL Endpoint

Lecture 4: Lab : Creating a Table from a CSV File

Lecture 5: Lab : Create Queries

Lecture 6: Lab : Parameterized Query

Lecture 7: Lab : Query Profile

Lecture 8: Lab : Building Visualization (Table, BAR & PIE )

Lecture 9: Lab : Building Line Chart & Counter Chart

Lecture 10: Lab : Adding Charts to Dashboards

Lecture 11: Lab : Defining a Query Alert

Lecture 12: Lab : Access Control on Databricks SQL Objects

Lecture 13: Lab: Data Object Access Control

Lecture 14: Lab : Transfer Ownership

Lecture 15: Lab : Access SQL Endpoint from Python