Apache Airflow: Complete Hands-On Beginner to Advanced Class

Apache Airflow: Complete Hands-On Beginner to Advanced Class, available at $74.99, has an average rating of 4.15, with 54 lectures, 8 quizzes, based on 425 reviews, and has 2442 subscribers.

You will learn about Core and Advanced Concepts in Airflow through Real-World Examples Architecture Components of Apache Airflow How to Set Up Connections to External Resources How to Load and Analyse Data in a Data Warehouse using Airflow How to Schedule PySpark jobs using Apache Airflow How to Extend Airflow with Custom Operators and Sensors How to Test Airflow DAGs and Operators How to Deploy Airflow Instances with Different Executors How to Set Up Error Tracking and Monitoring This course is ideal for individuals who are Data Engineers or Data Scientists or Python Developers Interested in Data Engineering or Data Analysts with Python Programming Knowledge It is particularly useful for Data Engineers or Data Scientists or Python Developers Interested in Data Engineering or Data Analysts with Python Programming Knowledge.

Enroll now: Apache Airflow: Complete Hands-On Beginner to Advanced Class

Summary

Title: Apache Airflow: Complete Hands-On Beginner to Advanced Class

Price: $74.99

Average Rating: 4.15

Number of Lectures: 54

Number of Quizzes: 8

Number of Published Lectures: 54

Number of Published Quizzes: 8

Number of Curriculum Items: 62

Number of Published Curriculum Objects: 62

Original Price: $124.99

Quality Status: approved

Status: Live

What You Will Learn

Core and Advanced Concepts in Airflow through Real-World Examples
Architecture Components of Apache Airflow
How to Set Up Connections to External Resources
How to Load and Analyse Data in a Data Warehouse using Airflow
How to Schedule PySpark jobs using Apache Airflow
How to Extend Airflow with Custom Operators and Sensors
How to Test Airflow DAGs and Operators
How to Deploy Airflow Instances with Different Executors
How to Set Up Error Tracking and Monitoring

Who Should Attend

Data Engineers
Data Scientists
Python Developers Interested in Data Engineering
Data Analysts with Python Programming Knowledge

Target Audiences

Data Engineers
Data Scientists
Python Developers Interested in Data Engineering
Data Analysts with Python Programming Knowledge

Hi there, my name is Alexandra Abbas. I’m an Apache Airflow Contributor and a Google Cloud Certified Data Engineer & Architect with over 3 years experience as a Data Engineer.

Are you struggling to learn Apache Airflow on your own? In this course I will teach you Airflow in a practical manner, with every lecture comes a full coding screencast. By the end of the course you will be able to use Airflow professionally and add Airflow to your CV.

This course includes 50 lectures and more than 4 hours of video, quizzes, coding exercises as well as 2 major real-life projects that you can add to your Github portfolio!

You will learn:

How to install and set up Airflow on your machine
Basic and advanced Airflow concepts
How to develop complex real-life data pipelines
How to interact with Google Cloud from your Airflow instance
How to extend Airflow with custom operators and sensors
How to test Airflow pipelines and operators
How to monitor your Airflow instance using Prometheus and Grafana
How to track errors with Sentry
How to set up and run Airflow in production

This course is for beginners. You do not need any previous knowledge of Apache Airflow, Data Engineering or Google Cloud. We will start right at the beginning and work our way through step by step.

You will get lifetime access to over 50 lectures plus corresponding cheat sheets, datasets and code base for the lectures!

Course Curriculum

Chapter 1: Introduction

Lecture 1: Your Airflow Journey

Lecture 2: What is Apache Airflow?

Lecture 3: Comparing Airflow to Other Tools

Lecture 4: Course Prerequisites

Lecture 5: Extra: Install Conda (Virtual Environment Manager)

Chapter 2: Getting Started with Apache Airflow

Lecture 1: Components of Airflow

Lecture 2: Install Airflow on MacOS (Video)

Lecture 3: Install Airflow on MacOS (Guide)

Lecture 4: Install Airflow on Linux

Lecture 5: Install Airflow on Windows (Video)

Lecture 6: Install Airflow on Windows (Guide)

Lecture 7: Install and Run Airflow with Docker

Lecture 8: Run Airflow Locally

Lecture 9: Introduction to the Airflow UI

Lecture 10: Introduction to the Airflow CLI

Chapter 3: Core Concepts in Apache Airflow

Lecture 1: What are DAGs?

Lecture 2: What are Default Arguments?

Lecture 3: What are Tasks and Operators?

Lecture 4: How to Define Dependencies?

Chapter 4: Loading Data to a Data Warehouse

Lecture 1: Use Case

Lecture 2: Set Up

Lecture 3: Connections

Lecture 4: Load Data from Storage to BigQuery

Lecture 5: Run SQL Query in BigQuery

Lecture 6: Use a Hook to List Storage Objects

Lecture 7: Cross-Task Communication (XComs)

Lecture 8: Jinja Templating and Macros

Lecture 9: Variables

Chapter 5: Analysing Data using PySpark

Lecture 1: Use Case

Lecture 2: Set Up

Lecture 3: Create Dataproc Hadoop Cluster

Lecture 4: Branching

Lecture 5: Submit a PySpark Job

Lecture 6: Subdags

Lecture 7: Trigger Rules

Lecture 8: DAG Documentation

Chapter 6: Extending Airflow with Custom Plugins

Lecture 1: Create a Custom Operator

Lecture 2: Create a Custom Sensor

Lecture 3: Run Custom Plugins

Chapter 7: Testing Airflow DAGs

Lecture 1: Load Test DAGs

Lecture 2: Unit Test DAGs and Operators

Lecture 3: Unit Test Custom Operators

Chapter 8: Airflow in Production

Lecture 1: Executors

Lecture 2: Configure Local Executor

Lecture 3: Configure Celery Executor

Lecture 4: Service Level Agreements (SLAs)

Lecture 5: Security: Authentication, Roles, Encryption

Lecture 6: Write Logs to a Remote Location

Lecture 7: Monitor Airflow with StatsD, Prometheus and Grafana

Lecture 8: Error Tracking with Sentry

Lecture 9: Managed Airflow Services

Chapter 9: Finale

Lecture 1: Clean Up in Google Cloud

Lecture 2: Additional Resources

Lecture 3: What's Next?

Instructors

Alexandra Abbas
Google Cloud Certified Data Engineer & Architect

Rating Distribution

1 stars: 12 votes
2 stars: 10 votes
3 stars: 52 votes
4 stars: 128 votes
5 stars: 223 votes

Frequently Asked Questions

How long do I have access to the course materials?

You can view and review the lecture materials indefinitely, like an on-demand channel.

Can I take my courses with me wherever I go?

Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don’t have an internet connection, some instructors also let their students download course lectures. That’s up to the instructor though, so make sure you get on their good side!

Menu