Practical Apache Airflow
Practical Apache Airflow, available at $39.99, has an average rating of 4.05, with 59 lectures, based on 22 reviews, and has 163 subscribers.
You will learn about Airflow, Developing Data Pipeline, Python, Pandas Hands on data pipeline development and comparison with other technology Installation, Configuration of Airflow DAG's, Creating workflow, Operators, Tasks, dependency management, Hooks, Connections Different Executors like Local, Celery and Sequential and differences Airflow Architecture in detail Advanced concepts like Xcoms, Branching, Variables and DAG Chaining Authentication and Log storage to S3 Airflow on Docker This course is ideal for individuals who are Students who want to learn developing data pipeline or Students and Professional wanting to learn Airflow or Professional looking to implement Airflow for their team and organization It is particularly useful for Students who want to learn developing data pipeline or Students and Professional wanting to learn Airflow or Professional looking to implement Airflow for their team and organization.
Enroll now: Practical Apache Airflow
Summary
Title: Practical Apache Airflow
Price: $39.99
Average Rating: 4.05
Number of Lectures: 59
Number of Published Lectures: 59
Number of Curriculum Items: 59
Number of Published Curriculum Objects: 59
Original Price: $19.99
Quality Status: approved
Status: Live
What You Will Learn
- Airflow, Developing Data Pipeline, Python, Pandas
- Hands on data pipeline development and comparison with other technology
- Installation, Configuration of Airflow
- DAG's, Creating workflow, Operators, Tasks, dependency management, Hooks, Connections
- Different Executors like Local, Celery and Sequential and differences
- Airflow Architecture in detail
- Advanced concepts like Xcoms, Branching, Variables and DAG Chaining
- Authentication and Log storage to S3
- Airflow on Docker
Who Should Attend
- Students who want to learn developing data pipeline
- Students and Professional wanting to learn Airflow
- Professional looking to implement Airflow for their team and organization
Target Audiences
- Students who want to learn developing data pipeline
- Students and Professional wanting to learn Airflow
- Professional looking to implement Airflow for their team and organization
Data engineering is a field that can be thought as a superset of business intelligence and data warehousing which brings more elements from software engineering. The reason data engineering exists today is because companies have massive treasure troves of data, but to provide value the data must be extracted. Data engineering provides the toolbox and is how we make sense of that data quickly and effectively.
When it comes to managing data collection, munging and consumption, data pipeline frameworks play a significant role and with the help of Apache Airflow, task of creating data pipeline is not only easy but its actually fun. Originated from AirBnb, Airflow soon became part of the very core of their tech stack.
The data infrastructure ecosystem has yet to show any sign of converging into something more manageable. It seems like we’re still in a huge phase of expansion where every new day bring new distributed database, new frameworks, new libraries and new teammates. As these systems get more complicated and evolve rapidly, it becomes even more important to have something like Apache Airflow that brings everything together in a sane place where every little piece of the puzzle can be orchestrated properly with sane APIs.
So in this course we will be learning as how to reach feature completeness with this amazing orchestration tool called Apache Airflow. You will not only learn to setup the environment but also learn how to create workflow pipeline with real world example so don’t wait and sign-up today and get started.
Looking forward to seeing you in this course!
Course Curriculum
Chapter 1: Introduction
Lecture 1: Introduction
Lecture 2: Who this course is for
Lecture 3: Dags and Pipeline
Lecture 4: What is Airflow?
Lecture 5: Why Use Airflow?
Lecture 6: Comparison between Luigi, Azkaban, Oozi
Lecture 7: Quiz Time
Chapter 2: Architecture
Lecture 1: Airflow Architecture
Lecture 2: Quiz Time
Chapter 3: Installation
Lecture 1: Installing Airflow
Lecture 2: Environment variable setup
Lecture 3: Hands on: setting environment variable and starting web server
Lecture 4: Hands on : setting encryption to secure connection secrets
Lecture 5: Quiz Time
Chapter 4: Airflow Configuration
Lecture 1: Airflow Configuration Overview
Lecture 2: Configuration options: ORM Configuration
Lecture 3: Configuration Option: Maximum Active Runs Explained
Lecture 4: Configuration Option: Maximum Active Runs Explained Continued
Lecture 5: Configuration Options: Additional Configuration Settings
Lecture 6: Quiz Time
Chapter 5: Developing Your First Data Pipeline
Lecture 1: Problem Statement
Lecture 2: Hands on: Project Setup
Lecture 3: Hands on: Data Retrieval from File System
Lecture 4: Hands on: Merging DataFrames
Lecture 5: Hands on: Aggregation Using Pandas
Lecture 6: Hands on: Database Connectivity
Lecture 7: Hands on: Creating Dags
Lecture 8: Summary
Chapter 6: DAG Chaining
Lecture 1: Dynamic Flow: Pattern -1 and Pattern – 2 Explained
Lecture 2: Dynamic Flow: Pattern – 3 Overview
Lecture 3: Dynamic Flow: Pattern – 4 Discussed
Lecture 4: Dynamic Flow: Exploring Pattern – 5
Lecture 5: Hands on: What are Airflow Hooks?
Lecture 6: Hands on: Exploring Task Branching
Lecture 7: Hands on: Passing and Modifying Variables
Lecture 8: Hands on: Xcom in Depth
Chapter 7: Authentication
Lecture 1: Airflow behavior with Python 2 vs Python 3
Lecture 2: Setting Authentication
Lecture 3: Setting Authentication Demo
Lecture 4: Hands On : Creating user with Python 3
Lecture 5: Creating user with Python2
Chapter 8: Airflow: Log storage to Cloud
Lecture 1: Writing Logs to S3
Lecture 2: Code: How to store Logs to S3
Lecture 3: Demo: Logs publishing to S3
Chapter 9: Airflow on Docker
Lecture 1: Intro to setting airflow in docker
Lecture 2: Installing Docker
Lecture 3: Building docker container for Airflow
Lecture 4: Building containers with Local and Celery Executors
Lecture 5: Dockerfile Explained
Lecture 6: Assignment
Lecture 7: Further on Airflow Containers
Chapter 10: REST API's
Lecture 1: Test REST API Server
Lecture 2: REST API Continued
Lecture 3: REST API
Chapter 11: SLA's
Lecture 1: What is an SLA?
Lecture 2: SLA Implementation
Lecture 3: SLA Demo
Lecture 4: (Optional) SLA – Implementation with Airflow explained
Chapter 12: Airflow Command Line
Lecture 1: Useful Command Lines for Airflow
Instructors
-
CGI GEN
Learn by Doing. Practical Approach to Solve Common Problems
Rating Distribution
- 1 stars: 2 votes
- 2 stars: 1 votes
- 3 stars: 7 votes
- 4 stars: 5 votes
- 5 stars: 7 votes
Frequently Asked Questions
How long do I have access to the course materials?
You can view and review the lecture materials indefinitely, like an on-demand channel.
Can I take my courses with me wherever I go?
Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don’t have an internet connection, some instructors also let their students download course lectures. That’s up to the instructor though, so make sure you get on their good side!
You may also like
- Top 10 Video Editing Courses to Learn in November 2024
- Top 10 Music Production Courses to Learn in November 2024
- Top 10 Animation Courses to Learn in November 2024
- Top 10 Digital Illustration Courses to Learn in November 2024
- Top 10 Renewable Energy Courses to Learn in November 2024
- Top 10 Sustainable Living Courses to Learn in November 2024
- Top 10 Ethical AI Courses to Learn in November 2024
- Top 10 Cybersecurity Fundamentals Courses to Learn in November 2024
- Top 10 Smart Home Technology Courses to Learn in November 2024
- Top 10 Holistic Health Courses to Learn in November 2024
- Top 10 Nutrition And Diet Planning Courses to Learn in November 2024
- Top 10 Yoga Instruction Courses to Learn in November 2024
- Top 10 Stress Management Courses to Learn in November 2024
- Top 10 Mindfulness Meditation Courses to Learn in November 2024
- Top 10 Life Coaching Courses to Learn in November 2024
- Top 10 Career Development Courses to Learn in November 2024
- Top 10 Relationship Building Courses to Learn in November 2024
- Top 10 Parenting Skills Courses to Learn in November 2024
- Top 10 Home Improvement Courses to Learn in November 2024
- Top 10 Gardening Courses to Learn in November 2024