Pig For Wrangling Big Data

Pig For Wrangling Big Data, available at $19.99, has an average rating of 3.7, with 35 lectures, based on 102 reviews, and has 3367 subscribers.

You will learn about Work with unstructured data to extract information, transform it and store it in a usable form Write intermediate level Pig scripts to munge data Optimize Pig operations which work on large data sets This course is ideal for individuals who are Yep! Analysts who want to wrangle large, unstructured data into shape or Yep! Engineers who want to parse and extract useful information from large datasets It is particularly useful for Yep! Analysts who want to wrangle large, unstructured data into shape or Yep! Engineers who want to parse and extract useful information from large datasets.

Enroll now: Pig For Wrangling Big Data

Summary

Title: Pig For Wrangling Big Data

Price: $19.99

Average Rating: 3.7

Number of Lectures: 35

Number of Published Lectures: 35

Number of Curriculum Items: 35

Number of Published Curriculum Objects: 35

Original Price: $89.99

Quality Status: approved

Status: Live

What You Will Learn

Work with unstructured data to extract information, transform it and store it in a usable form
Write intermediate level Pig scripts to munge data
Optimize Pig operations which work on large data sets

Who Should Attend

Yep! Analysts who want to wrangle large, unstructured data into shape
Yep! Engineers who want to parse and extract useful information from large datasets

Target Audiences

Yep! Analysts who want to wrangle large, unstructured data into shape
Yep! Engineers who want to parse and extract useful information from large datasets

Prerequisites:�Working with Pig requires some basic knowledge of the SQL query language, a brief understanding of the Hadoop eco-system and MapReduce�

Taught by a team which includes 2 Stanford-educated, ex-Googlers and 2 ex-Flipkart Lead Analysts. This team has decades of practical experience in working with large-scale data processing jobs.�

Pig is aptly named, it is omnivorous, will consume any data that you throw at it and bring home the bacon!

Let’s parse that�

omnivorous:�Pig works with unstructured data. It has many operations which are very SQL-like but Pig can perform these operations on data sets which have no fixed schema. Pig is great at wrestling data into a form which is clean and can be stored in a data warehouse for reporting and analysis.

bring home the bacon:�Pig allows you to transform data in a way that makes is structured, predictable and useful, ready for consumption.

What’s Covered:�

Pig Basics: Scalar and Complex data types (Bags, Maps, Tuples), basic transformations such as Filter, Foreach, Load, Dump, Store, Distinct, Limit, Order by and other built-in functions.

Advanced Data Transformations and Optimizations:�The mind-bending Nested Foreach, Joins and their optimizations using “parallel”, “merge”, “replicated” and other keywords, Co-groups and Semi-joins, debugging using Explain and Illustrate commands

Real-world example:�Clean up server logs using Pig

Course Curriculum

Chapter 1: You, This Course and Us

Lecture 1: You, This Course and Us

Chapter 2: Where does Pig fit in?

Lecture 1: Pig and the Hadoop ecosystem

Lecture 2: Install and set up

Lecture 3: How does Pig compare with Hive?

Lecture 4: Pig Latin as a data flow language

Lecture 5: Pig with HBase

Chapter 3: Pig Basics

Lecture 1: Operating modes, running a Pig script, the Grunt shell

Lecture 2: Loading data and creating our first relation

Lecture 3: Scalar data types

Lecture 4: Complex data types – The Tuple, Bag and Map

Lecture 5: Partial schema specification for relations

Lecture 6: Displaying and storing relations – The dump and store commands

Chapter 4: Pig Operations And Data Transformations

Lecture 1: Selecting fields from a relation

Lecture 2: Built-in functions

Lecture 3: Evaluation functions

Lecture 4: Using the distinct, limit and order by keywords

Lecture 5: Filtering records based on a predicate

Chapter 5: Advanced Data Transformations

Lecture 1: Group by and aggregate transformations

Lecture 2: Combining datasets using Join

Lecture 3: Concatenating datasets using Union

Lecture 4: Generating multiple records by flattening complex fields

Lecture 5: Using Co-Group, Semi-Join and Sampling records

Lecture 6: The nested Foreach command

Lecture 7: Debug Pig scripts using Explain and Illustrate

Chapter 6: Optimizing Data Transformations

Lecture 1: Parallelize operations using the Parallel keyword

Lecture 2: Join Optimizations: Multiple relations join, large and small relation join

Lecture 3: Join Optimizations: Skew join and sort-merge join

Lecture 4: Common sense optimizations

Chapter 7: A real-world example

Lecture 1: Parsing server logs

Lecture 2: Summarizing error logs

Chapter 8: Installing Hadoop in a Local Environment

Lecture 1: Hadoop Install Modes

Lecture 2: Hadoop Standalone mode Install

Lecture 3: Hadoop Pseudo-Distributed mode Install

Chapter 9: Appendix

Lecture 1: [For Linux/Mac OS Shell Newbies] Path and other Environment Variables

Lecture 2: Setup a Virtual Linux Instance (For Windows users)

Instructors

Loony Corn
An ex-Google, Stanford and Flipkart team

Rating Distribution

1 stars: 5 votes
2 stars: 7 votes
3 stars: 19 votes
4 stars: 28 votes
5 stars: 43 votes

Frequently Asked Questions

How long do I have access to the course materials?

You can view and review the lecture materials indefinitely, like an on-demand channel.

Can I take my courses with me wherever I go?

Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don’t have an internet connection, some instructors also let their students download course lectures. That’s up to the instructor though, so make sure you get on their good side!

Menu