Apache Spark 3 – Real-time Stream Processing using Scala

Apache Spark 3 – Real-time Stream Processing using Scala, available at $59.99, has an average rating of 4.7, with 33 lectures, based on 450 reviews, and has 7336 subscribers.

You will learn about Real-time Stream Processing Concepts Spark Structured Streaming APIs and Architecture Working with File Streams Working With Kafka Source and Integrating Spark with Kafka State-less and State-full Streaming Transformations Windowing Aggregates using Spark Stream Watermarking and State Cleanup Streaming Joins and Aggregation Handling Memory Problems with Streaming Joins Creating Arbitrary Streaming Sinks This course is ideal for individuals who are Software Engineers and Architects who are willing to design and develop a Bigdata Engineering Projects using Apache Spark or Programmers and developers who are aspiring to grow and learn Data Engineering using Apache Spark It is particularly useful for Software Engineers and Architects who are willing to design and develop a Bigdata Engineering Projects using Apache Spark or Programmers and developers who are aspiring to grow and learn Data Engineering using Apache Spark.

Enroll now: Apache Spark 3 – Real-time Stream Processing using Scala

Summary

Title: Apache Spark 3 – Real-time Stream Processing using Scala

Price: $59.99

Average Rating: 4.7

Number of Lectures: 33

Number of Published Lectures: 33

Number of Curriculum Items: 33

Number of Published Curriculum Objects: 33

Original Price: $19.99

Quality Status: approved

Status: Live

What You Will Learn

Real-time Stream Processing Concepts
Spark Structured Streaming APIs and Architecture
Working with File Streams
Working With Kafka Source and Integrating Spark with Kafka
State-less and State-full Streaming Transformations
Windowing Aggregates using Spark Stream
Watermarking and State Cleanup
Streaming Joins and Aggregation
Handling Memory Problems with Streaming Joins
Creating Arbitrary Streaming Sinks

Who Should Attend

Software Engineers and Architects who are willing to design and develop a Bigdata Engineering Projects using Apache Spark
Programmers and developers who are aspiring to grow and learn Data Engineering using Apache Spark

Target Audiences

Software Engineers and Architects who are willing to design and develop a Bigdata Engineering Projects using Apache Spark
Programmers and developers who are aspiring to grow and learn Data Engineering using Apache Spark

About the Course

I am creating Apache Spark 3 – Real-time Stream Processingusing the Scalacourse to help you understand the Real-time Stream processing using Apache Spark and apply that knowledge to build real-time stream processing solutions. This course is example-driven and follows a working session like approach. We will be taking a live coding approach and explain all the needed concepts along the way.

Who should take this Course?

I designed this course for software engineers willing to develop a Real-time Stream Processing Pipeline and application using the Apache Spark. I am also creating this course for data architects and data engineers who are responsible for designing and building the organization’s data-centric infrastructure. Another group of people is the managers and architects who do not directly work with Spark implementation. Still, they work with the people who implement Apache Spark at the ground level.

Spark Version used in the Course

This Course is using the Apache Spark 3.x. I have tested all the source code and examples used in this Course on Apache Spark 3.0.0 open-source distribution.

Course Curriculum

Chapter 1: Before you start

Lecture 1: About the Course

Lecture 2: Course Prerequisite

Lecture 3: Source Code and Other Resources

Chapter 2: Setup your Environment

Lecture 1: Spark Development Environment

Lecture 2: Spark Installation Prerequisites

Lecture 3: Installing Apache Spark

Lecture 4: Setup and test your IDE

Lecture 5: Install and run Apache Kafka

Chapter 3: Getting started with Spark Structured Streaming

Lecture 1: Introduction to Stream Processing

Lecture 2: Spark Streaming APIs – Dstream Vs Structured Streaming

Lecture 3: Creating your first stream processing application

Lecture 4: Stream processing model in Spark

Lecture 5: Working with Files and Directories

Lecture 6: Streaming Sources, Sinks and Output Mode

Lecture 7: Fault Tolerance and Restarts

Chapter 4: Spark Streaming with Kafka

Lecture 1: Streaming from Kafka Source

Lecture 2: Working with Kafka Sinks

Lecture 3: Multi-query Streams Application

Lecture 4: Kafka Serialization and Deserialization for Spark

Lecture 5: Creating Kafka AVRO Sinks

Lecture 6: Working with Kafka AVRO Source

Chapter 5: Windowing and Aggregates

Lecture 1: Stateless Vs Statefull transformations

Lecture 2: Event time and Windowing

Lecture 3: Tumbling Window aggregate

Lecture 4: Watermarking your windows

Lecture 5: Watermark and output modes

Lecture 6: Sliding Window

Chapter 6: Stream Processing and Joins

Lecture 1: Joining Stream to static source

Lecture 2: Joining Stream to another Stream

Lecture 3: Streaming Watermark

Lecture 4: Streaming Outer Joins

Chapter 7: Keep Learning

Lecture 1: Final Word

Lecture 2: Bonus Lecture : Get Extra

Instructors

Prashant Kumar Pandey
Architect, Author, Consultant, Trainer @ Learning Journal
Learning Journal
Online Training Company

Rating Distribution

1 stars: 2 votes
2 stars: 6 votes
3 stars: 26 votes
4 stars: 156 votes
5 stars: 260 votes

Frequently Asked Questions

How long do I have access to the course materials?

You can view and review the lecture materials indefinitely, like an on-demand channel.

Can I take my courses with me wherever I go?

Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don’t have an internet connection, some instructors also let their students download course lectures. That’s up to the instructor though, so make sure you get on their good side!

Menu