Data Engineering with Google Dataflow and Apache Beam on GCP

Data Engineering with Google Dataflow and Apache Beam on GCP, available at $54.99, has an average rating of 4.33, with 21 lectures, based on 666 reviews, and has 3249 subscribers.

You will learn about Apache Beam ETL Python Google Cloud DataFlow Google Cloud Storage Big Query This course is ideal for individuals who are Data Engineers or Data Analysts or Business Intelligence Professionals or Open Source Fans or ETL Engineers It is particularly useful for Data Engineers or Data Analysts or Business Intelligence Professionals or Open Source Fans or ETL Engineers.

Enroll now: Data Engineering with Google Dataflow and Apache Beam on GCP

Summary

Title: Data Engineering with Google Dataflow and Apache Beam on GCP

Price: $54.99

Average Rating: 4.33

Number of Lectures: 21

Number of Published Lectures: 21

Number of Curriculum Items: 21

Number of Published Curriculum Objects: 21

Original Price: €34.99

Quality Status: approved

Status: Live

What You Will Learn

Apache Beam
ETL
Python
Google Cloud
DataFlow
Google Cloud Storage
Big Query

Who Should Attend

Data Engineers
Data Analysts
Business Intelligence Professionals
Open Source Fans
ETL Engineers

Target Audiences

Data Engineers
Data Analysts
Business Intelligence Professionals
Open Source Fans
ETL Engineers

This course wants to introduce you to the Apache Foundation’s newest data pipeline development framework: The Apache Beam, and how this feature is becoming popular in partnership with Google Dataflow. In a summary, we want to cover the following topics:

1. Understand your inner workings

2. What are your benefits

3. Explain how to use on your local machine without installation via Google Colab for development

4. Its main functions

5. Configure Apache Beam python SDK locallyvice

6. How to deploy this resource on Google Dataflow to a Batch pipeline

This course is dynamic, you will be receiving updates whenever possible.

It is important to remember that this course does not teach Python, but uses it. So, get comfortable with knowing Python basics, defining a function, creating objects and data types.

Also, if you are interested in learning section 4, which consists of deploying a pipeline on Google Dataflow, you will need to have a free counter in GCP. It’s a simple process, but it requires a credit card!

I kindly ask you you to consider all the efforts to put this course together and give a nice rate at the end of the course, even tough the course is simple, it was made with all good intent to share knowledge for cheap price. Thanks and hope you enjoy!

___________________________________________________________________________________________________________

Requirements:

· Basic knowledge of Python

· Have Python 3.7 or greater installed locally (from section 4)

· Free account at GCP (from section 4)

Schedule:

· Section 2 – Concepts

· Section 3 – Main Functions

· Section 4 – Apache Beam on Google Dataflow

Course Curriculum

Chapter 1: Apache Beam Concepts

Lecture 1: 2.1 What is Apache Beam ?

Lecture 2: 2.2 Apache Beam Architecture Overview

Lecture 3: 2.3 Apache Beam Pipeline Flow

Chapter 2: Apache Beam Main Functions

Lecture 1: 3.1 Using Apache Beam In Google Colab

Lecture 2: 3.2 Read Inputs

Lecture 3: 3.3 Write Outputs

Lecture 4: 3.4 beam.Map / beam.FlatMap

Lecture 5: 3.5 beam.Filter

Lecture 6: 3.6 beam.Flatten

Lecture 7: 3.7 beam.CombinePerKey

Lecture 8: 3.8 beam.combiners.Count.PerKey()

Lecture 9: 3.9 beam.CoGroupByKey

Lecture 10: 3.10 ParDo – Customizing Beam Functions

Chapter 3: Apache Beam + GCP = Dataflow

Lecture 1: 4.1 GCP Setup

Lecture 2: 4.2 Creating Service Account and a Bucket on GCP

Lecture 3: 4.3 Setup Apache Beam Local (SDK)

Lecture 4: 4.4 Direct Runner Execution + Saving to GCS

Lecture 5: 4.5 Creating a Dataflow Template

Lecture 6: 4.6 Executing Batch Job in Dataflow

Lecture 7: 4.7 Creating Dataflow Template to write in Big Query

Lecture 8: 4.8 Executing Batch Job to write in Big Query

Instructors

Cassio Alessandro de Bolba
Data Engineer

Rating Distribution

1 stars: 15 votes
2 stars: 22 votes
3 stars: 87 votes
4 stars: 218 votes
5 stars: 324 votes

Frequently Asked Questions

How long do I have access to the course materials?

You can view and review the lecture materials indefinitely, like an on-demand channel.

Can I take my courses with me wherever I go?

Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don’t have an internet connection, some instructors also let their students download course lectures. That’s up to the instructor though, so make sure you get on their good side!

Menu