Masterclass on Azure Data Factory – Data Engineer for 2024
Masterclass on Azure Data Factory – Data Engineer for 2024, available at $69.99, has an average rating of 4.54, with 71 lectures, based on 228 reviews, and has 5274 subscribers.
You will learn about You will learn how to build a real time data pipeline in Azure Data Factory (ADF). You will learn how to transform data using Data Flows in Azure Data Factory (ADF) and load into ADLS2, Blob storage. You will learn how to ingest JSON data from SQL Server to API end point. You will learn how to build production ready pipelines and good practices and naming standards. You will learn how to monitor pipelines using Azure Data Factory (ADF), Azure Monitor and Log Analytics with a real-world project. You will learn about manual/Triggers in Azure Data Factory (ADF) and how to use them to schedule the data pipelines. Create Data driven, fully integrated, dynamic and automated production grade pipeline creation and orchestration. Data Warehousing Concepts like Fact & dimension tables, SCD type1, type2, Incremental loading and implement using ADF. Connect & copy data from On Premise data stores, On premise SQL server via Self Hosted Integration Runtime to cloud. How to ingest data from sources such as REST API, Azure Blob Storage, SQL DB into Azure Data Lake Gen2 using Azure Data Factory (ADF) Learn Azure Data Factory(ADF) with real time projects – ADF, SQL Server, Blob Storage, Datalake G1,G2, REST API. This course is ideal for individuals who are For all levels. or Students, Working Professionals It is particularly useful for For all levels. or Students, Working Professionals.
Enroll now: Masterclass on Azure Data Factory – Data Engineer for 2024
Summary
Title: Masterclass on Azure Data Factory – Data Engineer for 2024
Price: $69.99
Average Rating: 4.54
Number of Lectures: 71
Number of Published Lectures: 69
Number of Curriculum Items: 71
Number of Published Curriculum Objects: 69
Original Price: ₹6,500
Quality Status: approved
Status: Live
What You Will Learn
- You will learn how to build a real time data pipeline in Azure Data Factory (ADF).
- You will learn how to transform data using Data Flows in Azure Data Factory (ADF) and load into ADLS2, Blob storage.
- You will learn how to ingest JSON data from SQL Server to API end point.
- You will learn how to build production ready pipelines and good practices and naming standards.
- You will learn how to monitor pipelines using Azure Data Factory (ADF), Azure Monitor and Log Analytics with a real-world project.
- You will learn about manual/Triggers in Azure Data Factory (ADF) and how to use them to schedule the data pipelines.
- Create Data driven, fully integrated, dynamic and automated production grade pipeline creation and orchestration.
- Data Warehousing Concepts like Fact & dimension tables, SCD type1, type2, Incremental loading and implement using ADF.
- Connect & copy data from On Premise data stores, On premise SQL server via Self Hosted Integration Runtime to cloud.
- How to ingest data from sources such as REST API, Azure Blob Storage, SQL DB into Azure Data Lake Gen2 using Azure Data Factory (ADF)
- Learn Azure Data Factory(ADF) with real time projects – ADF, SQL Server, Blob Storage, Datalake G1,G2, REST API.
Who Should Attend
- For all levels.
- Students, Working Professionals
Target Audiences
- For all levels.
- Students, Working Professionals
Welcome to the Course “Azure Data Factory – Data Engineer : Real Time Projects“.
Learn ETL and Big Data Processing on Azure with Azure Data Factory V2
Azure Data Factory (ADF) is one of the in-demand data engineering tool in the Cloud. This course has been taught with covering a data engineering solutions using ADF for real world scenarios.
Once you complete this course including assignments, quizzes, you will be in a position to take up or handle confidently any kind of projects one equal to 10+ years of experience.
Entire course built based on real time scenarios which are going to use very frequently and common to any type of projects on various domains.
This course will gives you necessary skills and help you greatly to pass for Certification exams like DP200 and DP203.
What this course will teach you?
-
You will learn how to build a real time data pipeline in Azure Data Factory (ADF).
-
You will learn how to transform data using Data Flows in Azure Data Factory (ADF) and load into ADLS2, Blob storage.
-
You will learn how to ingest JSON data from SQL Server to API end point.
-
You will learn how to build production ready pipelines and good practices and naming standards.
-
You will learn how to monitor pipelines using Azure Data Factory (ADF), Azure Monitor and Log Analytics with a real-world project.
-
You will learn about manual/Triggers in Azure Data Factory (ADF) and how to use them to schedule the data pipelines.
-
Create Data driven, fully integrated, dynamic and automated production grade pipeline creation and orchestration.
-
Data Warehousing Concepts like Fact & dimension tables, SCD type1, type2, Incremental loading and implement using ADF.
-
Connect & copy data from On Premise data stores, On premise SQL server via Self Hosted Integration Runtime to cloud.
-
How to ingest data from sources such as REST API, Azure Blob Storage, SQL DB into Azure Data Lake Gen2 using Azure Data Factory (ADF)
Course Curriculum
Chapter 1: Initial Concepts
Lecture 1: Data Factory creation
Lecture 2: Resource Group
Lecture 3: Top Level Concepts in ADF
Lecture 4: Linked Service and Datasets
Lecture 5: Blob Storage
Lecture 6: Variables in ADF
Lecture 7: Parameters
Lecture 8: JSON Structure
Lecture 9: Integration Runtime
Chapter 2: Pipeline Activities
Lecture 1: Copy Activity
Lecture 2: Append Variable Activity
Lecture 3: Delete Activity
Lecture 4: Execute Pipeline Activity
Lecture 5: Fail Activity
Lecture 6: Get Metadata Activity
Lecture 7: Lookup Activity
Lecture 8: Set Variable Activity
Lecture 9: Wait Activity
Chapter 3: Conditional and Loop Activities
Lecture 1: Switch Activity
Lecture 2: Filter Activity
Lecture 3: ForEach Activity
Lecture 4: Until Activity
Lecture 5: If Condition
Chapter 4: Data Flow Transformations
Lecture 1: Conditional Split Transformation
Lecture 2: Exists Transformation
Lecture 3: Union Transformation
Lecture 4: Join Transformation
Lecture 5: Aggregate Transformation
Lecture 6: Surrogate Key Transformation
Lecture 7: Select Transformation
Lecture 8: Lookup Transformation
Lecture 9: Derived Column Transformation
Lecture 10: Pivot Transformation
Lecture 11: Unpivot Transformation
Lecture 12: Rank Transformation
Lecture 13: Window Transformation
Lecture 14: Flatten Transformation
Lecture 15: Filter Transformation
Chapter 5: Real Time Scenarios
Lecture 1: Part 1 – Copy files dynamically from one storage location to other.
Lecture 2: Part 2 – Copy files dynamically from one storage location to other.
Lecture 3: Set Variable : self assignment handling.
Lecture 4: Fetch file names from folder and assign to variable
Lecture 5: Copying specific file pattern files from Source to Sink
Lecture 6: ADF pipeline logging
Lecture 7: Process CSV file and split single column into multiple columns
Lecture 8: Copy Missed files from Source to Destination
Lecture 9: Run ADF pipeline manaully-Call pipeline from other ADF pipeline (with REST API)
Chapter 6: Email Notification with ADF and Logic App
Lecture 1: Email notification theory
Lecture 2: Logic App Creation in Azure
Lecture 3: Logic App and ADF Configuration for Email notification
Lecture 4: Email notification configuration before and after pipeline with Stored procedure
Chapter 7: Data Load for Multiple Tables and multiple approaches
Lecture 1: Full Load and Incemental load Theory
Lecture 2: Copying multiple tables from database to database using ADF array.
Chapter 8: Slowly Changing Dimensions
Lecture 1: Fact and Dimension tables introduction
Lecture 2: Slowly Chaning Dimension – Type 1
Lecture 3: Slowly Changing Dimension – Type 2
Lecture 4: Slowly Changing Dimension – Type 6
Chapter 9: Logs Maintenece in ADF
Lecture 1: Logs Table Creation
Lecture 2: Starting Logs Stored Procedure Creation
Chapter 10: Azure Key Vault
Lecture 1: Azure Key Vault – Creation
Chapter 11: Azure Databricks – PySpark
Lecture 1: Azure Databricks Service -Creation
Lecture 2: JDBC Connection -How to connect to SQL server & read table from Azure Databricks
Lecture 3: Mount to ADLS or Blob from Azure Databricks using PySpark
Lecture 4: Read CSV file from DBFS with schema and header
Lecture 5: withColumn() Transformation Function
Chapter 12: Azure Databricks – Delta Lake & Delta Table
Lecture 1: Delta lake introduction
Lecture 2: Datawarehouse Vs Data Lake Vs Delta Lake
Lecture 3: Delta Table Architecture
Lecture 4: Delta Table creation and observing its structure
Instructors
-
Atchyut Kumar
Azure Data Engineer and Instructor
Rating Distribution
- 1 stars: 6 votes
- 2 stars: 7 votes
- 3 stars: 30 votes
- 4 stars: 84 votes
- 5 stars: 101 votes
Frequently Asked Questions
How long do I have access to the course materials?
You can view and review the lecture materials indefinitely, like an on-demand channel.
Can I take my courses with me wherever I go?
Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don’t have an internet connection, some instructors also let their students download course lectures. That’s up to the instructor though, so make sure you get on their good side!
You may also like
- Top 10 Language Learning Courses to Learn in November 2024
- Top 10 Video Editing Courses to Learn in November 2024
- Top 10 Music Production Courses to Learn in November 2024
- Top 10 Animation Courses to Learn in November 2024
- Top 10 Digital Illustration Courses to Learn in November 2024
- Top 10 Renewable Energy Courses to Learn in November 2024
- Top 10 Sustainable Living Courses to Learn in November 2024
- Top 10 Ethical AI Courses to Learn in November 2024
- Top 10 Cybersecurity Fundamentals Courses to Learn in November 2024
- Top 10 Smart Home Technology Courses to Learn in November 2024
- Top 10 Holistic Health Courses to Learn in November 2024
- Top 10 Nutrition And Diet Planning Courses to Learn in November 2024
- Top 10 Yoga Instruction Courses to Learn in November 2024
- Top 10 Stress Management Courses to Learn in November 2024
- Top 10 Mindfulness Meditation Courses to Learn in November 2024
- Top 10 Life Coaching Courses to Learn in November 2024
- Top 10 Career Development Courses to Learn in November 2024
- Top 10 Relationship Building Courses to Learn in November 2024
- Top 10 Parenting Skills Courses to Learn in November 2024
- Top 10 Home Improvement Courses to Learn in November 2024