Data Engineering for Beginner using Google Cloud & Python

Data Engineering for Beginner using Google Cloud & Python, available at $74.99, has an average rating of 4.27, with 77 lectures, based on 265 reviews, and has 2663 subscribers.

You will learn about Basic data engineering, what is data engineering, why needed, how to do it from zero Relational database model, database modelling for normalization design & hands-on using postgresql & python / pandas NoSQL database model, denormalization design & hands-on using elasticsearch & python / pandas Introduction to spark & spark cluster using google cloud platform This course is ideal for individuals who are Beginner python developer curious about data engineering or Software engineer who wants to take the path of becoming data engineer or Technical architect, engineering manager, who wants to know overview of data engineering It is particularly useful for Beginner python developer curious about data engineering or Software engineer who wants to take the path of becoming data engineer or Technical architect, engineering manager, who wants to know overview of data engineering.

Enroll now: Data Engineering for Beginner using Google Cloud & Python

Summary

Title: Data Engineering for Beginner using Google Cloud & Python

Price: $74.99

Average Rating: 4.27

Number of Lectures: 77

Number of Published Lectures: 77

Number of Curriculum Items: 77

Number of Published Curriculum Objects: 77

Original Price: $19.99

Quality Status: approved

Status: Live

What You Will Learn

Basic data engineering, what is data engineering, why needed, how to do it from zero
Relational database model, database modelling for normalization design & hands-on using postgresql & python / pandas
NoSQL database model, denormalization design & hands-on using elasticsearch & python / pandas
Introduction to spark & spark cluster using google cloud platform

Who Should Attend

Beginner python developer curious about data engineering
Software engineer who wants to take the path of becoming data engineer
Technical architect, engineering manager, who wants to know overview of data engineering

Target Audiences

Beginner python developer curious about data engineering
Software engineer who wants to take the path of becoming data engineer
Technical architect, engineering manager, who wants to know overview of data engineering

“Data is the new oil”.

You might have heard the quote before. Data in digital era is as valuable as oil in industrial era. However, just like oil, raw data itself is not usable. Rather, the value is created when it is gathered completely and accurately, connected to other relevant data, and done so in a timely manner.

Data engineers design and build pipelines that transform and transport data into a usable format. A different role, like data scientist or machine learning engineer then able to use the data into valuable business insight. Just like raw oil transformed into petrol to be used through complex process.

To be a data engineer requires a lot of data literacy and practice. This course is the first step for you who want to know about data engineering. In this course, we will see theories and hands-on to introduce you to data engineering. As data field is very wide, this course will show you the basic, entry level knowledge about data engineering process and tools.

This course is very suitable to build foundation for you to go to data field. In this course, we will learn about:

Introduction to data engineering
Relational & non relational database
Relational & non relational data model
Table normalization
Fact & dimension tables
Table denormalization for data warehouse
ETL (Extract Transform Load) & data staging using pyhton pandas
Elasticsearch basic
Data warehouse
Numbers every engineers should know & how it is related to big data
Hadoop
Spark cluster on google cloud dataproc
Data lake

Important Notes

Data field is HUGE! This course will be continuously updated, but for time being, this contains introduction to concept, and sample hands-on for data engineering.

For now, this course is intended for beginner on data engineering.

If you have some experience on programming and wonder about data engineering, this course is for you.

If you have experience in data engineering field, this course might be too basic for you (although I’m very happy if you still purchase the course)

If you never write python or SQL before, this course is not for you. To understand the course, you must have basic knowledge on SQL and pyhton.

Course Curriculum

Chapter 1: Introduction

Lecture 1: Welcome to This Course

Lecture 2: Course Structure & Coverage

Lecture 3: How To Get Maximum Value From This Course

Chapter 2: Introduction to Data Engineering

Lecture 1: What is Data Engineering?

Lecture 2: Data Engineering Example

Lecture 3: What is Data Modelling?

Chapter 3: Database

Lecture 1: What is Database

Lecture 2: Relational Database

Lecture 3: When Not To Use Relational Database?

Lecture 4: NoSQL Database

Lecture 5: Demo : Postgresql

Lecture 6: Demo : Python for Postgresql

Lecture 7: Demo : Elasticsearch

Lecture 8: Demo : Python for Elasticsearch

Chapter 4: Relational Database Model

Lecture 1: The Importance of Relational Data Model

Lecture 2: OLTP vs OLAP

Lecture 3: Database Normalization

Lecture 4: First Normal Form (1NF)

Lecture 5: Second Normal Form (2NF)

Lecture 6: Third Normal Form (3NF)

Lecture 7: Normalization Python Demo

Lecture 8: Normalization Tips

Lecture 9: Database Denormalization

Lecture 10: Denormalization Python Demo

Lecture 11: Fact & Dimension Tables

Lecture 12: Star Schema

Lecture 13: Star Schema Python Demo

Lecture 14: Snowflake Schema

Lecture 15: Galaxy Schema

Lecture 16: Extract Transform Load (ETL) & Staging Tables

Lecture 17: ETL & Staging Tables – Demo Overview

Lecture 18: ETL & Staging Tables – Python Demo 1

Lecture 19: ETL & Staging Tables – Python Demo 2

Lecture 20: To Insert or To Update?

Lecture 21: ETL & Staging Tables – Python Demo 3

Lecture 22: ETL & Staging Tables – Python Demo 4

Lecture 23: ETL & Staging Tables – Tips

Chapter 5: NoSQL Database Model

Lecture 1: Basic NoSQL Concept

Lecture 2: CAP Theorem

Lecture 3: Denormalization on Elasticsearch

Lecture 4: Elasticsearch Basic Usage

Lecture 5: Elasticsearch Index & Document

Lecture 6: Elasticsearch ETL – Overview

Lecture 7: Elasticsearch Query DSL

Lecture 8: Elasticsearch ETL – Python Demo

Chapter 6: Data Warehouse

Lecture 1: Business Perspective

Lecture 2: Technical Perspective

Lecture 3: More Fact & Dimension Table

Lecture 4: OLAP Cube

Lecture 5: On-Premise or Cloud?

Lecture 6: Various Techniques

Lecture 7: Demo Overview

Lecture 8: Demo 1 – PostgreSQL Data Warehouse

Lecture 9: Demo 2 – BigQuery Data Warehouse

Lecture 10: Demo 3 – Data Warehouse Operations

Chapter 7: Numbes Every Engineer Should Know

Lecture 1: Numbers Every Engineer Should Know

Lecture 2: Small Numbers

Lecture 3: Big Numbers

Chapter 8: Hadoop & Spark

Lecture 1: Hadoop Ecosystem

Lecture 2: Introducing Spark

Lecture 3: Spark Programming

Lecture 4: Data Formats

Lecture 5: Hello Spark

Lecture 6: Spark Demo – Dataframe

Lecture 7: Spark Demo – Spark SQL

Lecture 8: Spark & BigQuery – Setting Environment

Lecture 9: Spark & BigQuery – ETL Movies

Lecture 10: Spark & BigQuery – Lesson Learned

Chapter 9: Spark Cluster on Google Cloud (Dataproc)

Lecture 1: Spark Cluster – Overview

Lecture 2: Demo : Big Data

Lecture 3: Google Dataproc

Chapter 10: Data Lake

Lecture 1: Data Lake Overview

Lecture 2: Schema On Read

Lecture 3: Lake, not Swamp

Lecture 4: Google Data Catalog

Chapter 11: Resources & References

Lecture 1: Download Source Code & Datasets

Lecture 2: Bonus & Discount Codes

Instructors

Timotius Pamungkas
Java Software Engineer, Architect

Rating Distribution

1 stars: 4 votes
2 stars: 5 votes
3 stars: 29 votes
4 stars: 101 votes
5 stars: 126 votes

Frequently Asked Questions

How long do I have access to the course materials?

You can view and review the lecture materials indefinitely, like an on-demand channel.

Can I take my courses with me wherever I go?

Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don’t have an internet connection, some instructors also let their students download course lectures. That’s up to the instructor though, so make sure you get on their good side!

Menu