HOT
blank

Data Engineering – SSIS/ETL/Pipelines/Python/Web Scraping

Development

Data Engineering – SSIS/ETL/Pipelines/Python/Web Scraping, available at $59.99, has an average rating of 4.15, with 79 lectures, based on 29 reviews, and has 197 subscribers.

Rating4.2
Enroll Now

You will learn about Creates ETL pipelines Extract data from the web with Python Create SSIS Package Execute SSIS Package Build Web Scraping Script Prototype web scraping script Configure data source and data destination Clean and Transform Data Perform Data migration from SQL Server to Oracle This course is ideal for individuals who are Beginners to Data Engineering It is particularly useful for Beginners to Data Engineering.

Enroll now: Data Engineering – SSIS/ETL/Pipelines/Python/Web Scraping

Summary

Title: Data Engineering – SSIS/ETL/Pipelines/Python/Web Scraping

Price: $59.99

Average Rating: 4.15

Number of Lectures: 79

Number of Published Lectures: 79

Number of Curriculum Items: 79

Number of Published Curriculum Objects: 79

Original Price: $59.99

Quality Status: approved

Status: Live

What You Will Learn

  • Creates ETL pipelines
  • Extract data from the web with Python
  • Create SSIS Package
  • Execute SSIS Package
  • Build Web Scraping Script
  • Prototype web scraping script
  • Configure data source and data destination
  • Clean and Transform Data
  • Perform Data migration from SQL Server to Oracle

Who Should Attend

  • Beginners to Data Engineering

Target Audiences

  • Beginners to Data Engineering

A data engineer is someone who creates big data ETL pipelines, and makes it possible to take huge amounts of data and translate it into insights. They are focused on the production readiness of data and things like formats, resilience, scaling, and security.

SQL Server Integration Services is a component of the Microsoft SQL Server database software that can be used to perform a broad range of data migration tasks. SSIS is a platform for data integration and workflow applications. It features a data warehousing tool used for data extraction, transformation, and loading .

ETL, which stands for extract, transform and load, is a data integration process that combines data from multiple data sources into a single, consistent data store that is loaded into a data warehouse or other target system.

An ETL pipeline is the set of processes used to move data from a source or multiple sources into a database such as a data warehouse or  target databases.

SQL Server Integration Service (SSIS) provides an convenient and unified way to read data from different sources (extract), perform aggregations and transformation (transform), and then integrate data (load) for data warehousing and analytics purpose. When you need to process large amount of data (GBs or TBs), SSIS becomes the ideal approach for such workload.

Web scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites. The web scraping software may directly access the World Wide Web using the Hypertext Transfer Protocol or a web browser. While web scraping can be done manually by a software user, the term typically refers to automated processes implemented using a bot or web crawler. It is a form of copying in which specific data is gathered and copied from the web, typically into a central local database or spreadsheet, for later retrieval or analysis.

Course Curriculum

Chapter 1: SQL Server Setup

Lecture 1: Introduction

Lecture 2: What is SQL Server

Lecture 3: SQL Server Editions

Lecture 4: Download SQL Server

Lecture 5: Install SQL Server

Lecture 6: Install SQL Server Management Studio

Lecture 7: Connect SSMS to SQL Server

Lecture 8: Install Sample Database

Chapter 2: Visual Studio Setup

Lecture 1: What is Visual Studio

Lecture 2: Please Read

Lecture 3: Install Visual Studio

Lecture 4: Visual studio workloads

Lecture 5: Install Visual Studio Data Tools – SSDT

Lecture 6: Install SSIS Visual Studio Extension

Chapter 3: Building SSIS ETL Pipeline

Lecture 1: What is SSIS

Lecture 2: What is ETL

Lecture 3: What is an ETL Pipeline

Lecture 4: ETL Pipeline Requirements

Lecture 5: Install Source Database

Lecture 6: Create destination database

Lecture 7: Create a new SSIS Project

Lecture 8: Configure data source and data destination

Lecture 9: Data Transformation

Lecture 10: Running SSIS Package

Chapter 4: Python Setup

Lecture 1: What is Python

Lecture 2: Installing Python on Windows

Lecture 3: Installing Python on Macs

Lecture 4: Create a virtual environment on Windows

Lecture 5: Activate a virtual environment on Windows

Lecture 6: Create a virtual environment on Macs

Lecture 7: Activate a virtual environment on Macs

Lecture 8: Updating Pip

Lecture 9: Install Beautiful Soup

Lecture 10: Install Visual Studio Code

Chapter 5: What is Web Scraping

Lecture 1: What is web scraping

Lecture 2: Web Scraping and Web Crawling

Lecture 3: What is Robot.txt

Lecture 4: Legality of Web Scraping

Lecture 5: Checks before web scraping

Chapter 6: Scraping data from web with Python

Lecture 1: What we will scrape

Lecture 2: Building the web scraping script: part 1

Lecture 3: Building the web scraping script: part 2

Lecture 4: Prototyping the script: Part 1

Lecture 5: Prototyping the script: Part 2

Lecture 6: Prototyping the script: Part 3

Lecture 7: Prototyping the script: Part 4

Lecture 8: Prototyping the script: Part 5

Lecture 9: Testing the script

Chapter 7: Scraping data from Amazon Website

Lecture 1: Create and activate a virtual environment

Lecture 2: Install Python Packages

Lecture 3: Create a Python File

Lecture 4: Create Variables

Lecture 5: Send emails from Python

Lecture 6: Create functions: Part 1

Lecture 7: Create functions: Part 2

Lecture 8: Create functions: Part 3

Lecture 9: Testing the Script

Chapter 8: Scraping YouTube Data

Lecture 1: What is an API

Lecture 2: YouTube Data API

Lecture 3: Using Google Sheets to save scraped data

Lecture 4: Building the first Scraper: Part 1

Lecture 5: Building the first Scraper: Part 2

Lecture 6: Building the first Scraper: Part 3

Lecture 7: Testing the first scraper

Lecture 8: Building the second Scraper

Lecture 9: Testing the second scraper

Chapter 9: Oracle Database Setup

Lecture 1: Install Oracle

Lecture 2: Unlock Sample Schema

Lecture 3: Install Oracle SQL Developer

Lecture 4: Connect SQL Developer to Oracle

Chapter 10: Data Migration: Microsoft SQL Server to Oracle

Lecture 1: Creating database user

Lecture 2: Creating repository

Lecture 3: Configuring third party database connection: part 1

Lecture 4: Configuring third party database connection: part 2

Lecture 5: Capturing source data

Lecture 6: Converting captured model to Oracle

Lecture 7: Translating third party T-SQL Objects to Oracle

Lecture 8: Generating scripts to create target database

Lecture 9: Moving data from SQL Server into Oracle

Instructors

Rating Distribution

  • 1 stars: 0 votes
  • 2 stars: 1 votes
  • 3 stars: 7 votes
  • 4 stars: 10 votes
  • 5 stars: 11 votes

Frequently Asked Questions


How long do I have access to the course materials?


You can view and review the lecture materials indefinitely, like an on-demand channel.


Can I take my courses with me wherever I go?


Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don’t have an internet connection, some instructors also let their students download course lectures. That’s up to the instructor though, so make sure you get on their good side!

You may also like

Top 10 Ethical AI Courses to Learn in November 2024

Top 10 Game Development Courses to Learn in November 2024

Top 10 Software Testing Courses to Learn in November 2024