Real World Hadoop – Automating Hadoop install with Python!
Real World Hadoop – Automating Hadoop install with Python!, available at $19.99, has an average rating of 4.2, with 21 lectures, based on 26 reviews, and has 414 subscribers.
You will learn about Simply run a single command on your desktop, go for a coffee, and come back with a running distributed environment for cluster deployment Quickly build an environment where Cloudera and Hadoop software can be installed. Ability to automate the installation of software across multiple Virtual Machines This course is ideal for individuals who are Software engineers who want to expand their skills into the world of distributed computing or System Engineers that want to expand their skillsets beyond the single server or Developers who want to write/test their code against a valid distributed enviroment It is particularly useful for Software engineers who want to expand their skills into the world of distributed computing or System Engineers that want to expand their skillsets beyond the single server or Developers who want to write/test their code against a valid distributed enviroment.
Enroll now: Real World Hadoop – Automating Hadoop install with Python!
Summary
Title: Real World Hadoop – Automating Hadoop install with Python!
Price: $19.99
Average Rating: 4.2
Number of Lectures: 21
Number of Published Lectures: 21
Number of Curriculum Items: 21
Number of Published Curriculum Objects: 21
Original Price: £89.99
Quality Status: approved
Status: Live
What You Will Learn
- Simply run a single command on your desktop, go for a coffee, and come back with a running distributed environment for cluster deployment
- Quickly build an environment where Cloudera and Hadoop software can be installed.
- Ability to automate the installation of software across multiple Virtual Machines
Who Should Attend
- Software engineers who want to expand their skills into the world of distributed computing
- System Engineers that want to expand their skillsets beyond the single server
- Developers who want to write/test their code against a valid distributed enviroment
Target Audiences
- Software engineers who want to expand their skills into the world of distributed computing
- System Engineers that want to expand their skillsets beyond the single server
- Developers who want to write/test their code against a valid distributed enviroment
Note : This course is built on top of the “Real World Vagrant – Automate a Cloudera Manager Build – Toyin Akin” course
Deploy a Hadoop cluster (Zookeeper, HDFS, YARN, Spark) with Python! Instruct Cloudera Manager to do the work! Hands on. Here we use Python to instruct an already installed Cloudera Manager to deploy your Hadoop Services.
.The Cloudera Manager API provides configuration and service lifecycle management, service health information and metrics, and allows you to configure Cloudera Manager itself. The API is served on the same host and port as the Cloudera Manager Admin Console, and does not require an extra process or extra configuration. The API supports HTTP Basic Authentication, accepting the same users and credentials as the Cloudera Manager Admin Console.
.
Here are some of the cool things you can do with Cloudera Manager via the API:
Deploy an entire Hadoop cluster programmatically. Cloudera Manager supports HDFS, MapReduce, YARN, ZooKeeper, HBase, Hive, Oozie, Hue, Flume, Impala, Solr, Sqoop, Spark and Accumulo.
Configure various Hadoop services and get config validation.
Take admin actions on services and roles, such as start, stop, restart, failover, etc. Also available are the more advanced workflows, such as setting up high availability and decommissioning.
Monitor your services and hosts, with intelligent service health checks and metrics.
Monitor user jobs and other cluster activities.
Retrieve timeseries metric data.
Search for events in the Hadoop system.
Administer Cloudera Manager itself.
Download the entire deployment description of your Hadoop cluster in a json file.
Additionally, with the appropriate licenses, the API lets you:
Perform rolling restart and rolling upgrade.
Audit user activities and accesses in Hadoop.
Perform backup and cross data-center replication for HDFS and Hive.
Retrieve per-user HDFS usage report and per-user MapReduce resource usage report.
.
Here I present a curriculum as to the current state of my Cloudera courses.
My Hadoop courses are based on Vagrant so that you can practice and destroy your virtual environment before applying the installation onto real servers/VMs.
.
For those with little or no knowledge of the Hadoop eco system
Udemy course : Big Data Intro for IT Administrators, Devs and Consultants
.
I would first practice with Vagrant so that you can carve out a virtual environment on your local desktop. You don’t want to corrupt your physical servers if you do not understand the steps or make a mistake.
Udemy course : Real World Vagrant For Distributed Computing
.
I would then, on the virtual servers, deploy Cloudera Manager plus agents. Agents are the guys that will sit on all the slave nodes ready to deploy your Hadoop services
Udemy course : Real World Vagrant – Automate a Cloudera Manager Build
.
Then deploy the Hadoop services across your cluster (via the installed Cloudera Manager in the previous step). We look at the logic regarding the placement of master and slave services.
Udemy course : Real World Hadoop – Deploying Hadoop with Cloudera Manager
.
If you want to play around with HDFS commands (Hands on distributed file manipulation).
Udemy course : Real World Hadoop – Hands on Enterprise Distributed Storage.
.
You can also automate the deployment of the Hadoop services via Python (using the Cloudera Manager Python API). But this is an advanced step and thus I would make sure that you understand how to manually deploy the Hadoop services first.
Udemy course : Real World Hadoop – Automating Hadoop install with Python!
.
There is also the upgrade step. Once you have a running cluster, how do you upgrade to a newer hadoop cluster (Both for Cloudera Manager and the Hadoop Services).
Udemy course : Real World Hadoop – Upgrade Cloudera and Hadoop hands on
Course Curriculum
Chapter 1: Rational
Lecture 1: Rational
Lecture 2: Suggested course curriculum to follow …
Chapter 2: Anaconda (Python) Setup within Vagrant
Lecture 1: Course Notes and Resources
Lecture 2: Walking over the Cloudera / Hadoop Cluster Topology
Lecture 3: Part I – Installing Hadoop
Lecture 4: Part II – Installing Hadoop
Lecture 5: Automating the Installation of Anaconda
Lecture 6: Destroy our Hadoop Cluster. Python will now be taking over …
Chapter 3: Python – Automate Deployment of Cloudera Management Services
Lecture 1: Connect to Cloudera Manager via Python.
Lecture 2: Python – Obtain a handle to a new Clouderea Manager Services Container
Lecture 3: Python, Acquire database credentials in order to configure the CM Services
Lecture 4: Python – Deploy and start the Cloudera Manager Services
Lecture 5: Python – Verify the Cloudera Manager Services
Chapter 4: Python Automate Deployment of the Hadoop Cluster
Lecture 1: Python – Create Hadoop Cluster Container and deploy parcels
Lecture 2: Complete Cloudera Parcel setup – Deploying the Packaged Hadoop Binaries
Lecture 3: Creating directories in advance
Lecture 4: Python – Configure and Install a Multinode ZooKeeper Cluster.
Lecture 5: Python – Configure and Install a Multinode HDFS Cluster.
Lecture 6: Python – Configure and Install a Multinode YARN Cluster.
Lecture 7: Python – Configure and Install a Multinode SPARK Cluster.
Chapter 5: Conclusion
Lecture 1: Conclusion
Instructors
-
Toyin Akin
Big Data Engineer, Capital Markets FinTech Developer
Rating Distribution
- 1 stars: 3 votes
- 2 stars: 2 votes
- 3 stars: 1 votes
- 4 stars: 5 votes
- 5 stars: 15 votes
Frequently Asked Questions
How long do I have access to the course materials?
You can view and review the lecture materials indefinitely, like an on-demand channel.
Can I take my courses with me wherever I go?
Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don’t have an internet connection, some instructors also let their students download course lectures. That’s up to the instructor though, so make sure you get on their good side!
You may also like
- Top 10 Video Editing Courses to Learn in November 2024
- Top 10 Music Production Courses to Learn in November 2024
- Top 10 Animation Courses to Learn in November 2024
- Top 10 Digital Illustration Courses to Learn in November 2024
- Top 10 Renewable Energy Courses to Learn in November 2024
- Top 10 Sustainable Living Courses to Learn in November 2024
- Top 10 Ethical AI Courses to Learn in November 2024
- Top 10 Cybersecurity Fundamentals Courses to Learn in November 2024
- Top 10 Smart Home Technology Courses to Learn in November 2024
- Top 10 Holistic Health Courses to Learn in November 2024
- Top 10 Nutrition And Diet Planning Courses to Learn in November 2024
- Top 10 Yoga Instruction Courses to Learn in November 2024
- Top 10 Stress Management Courses to Learn in November 2024
- Top 10 Mindfulness Meditation Courses to Learn in November 2024
- Top 10 Life Coaching Courses to Learn in November 2024
- Top 10 Career Development Courses to Learn in November 2024
- Top 10 Relationship Building Courses to Learn in November 2024
- Top 10 Parenting Skills Courses to Learn in November 2024
- Top 10 Home Improvement Courses to Learn in November 2024
- Top 10 Gardening Courses to Learn in November 2024