Learn to scrape any website with R

Course Curriculum

Chapter 1: Introduction

Lecture 1: Introduction

Lecture 2: Prerequisites

Lecture 3: Scrape a table from Wikipedia in 20 lines of code – part 1

Lecture 4: Scrape a table from Wikipedia in 20 lines of code – part 2

Chapter 2: Setup and resources

Lecture 1: Docker setup

Lecture 2: Resources

Chapter 3: Scrape your first (server-side rendered) website

Lecture 1: 3 HTML basics: tags, classes and id's

Lecture 2: Introduction to section: Scrape Trustpilot

Lecture 3: Read in HTML and create a list of reviews

Lecture 4: Extract the content of the reviews

Lecture 5: Extract the content about the reviewer

Lecture 6: Create a function and extract review data to a data frame

Lecture 7: Loop over all pages and collect all reviews

Lecture 8: Clean up the scraped data

Chapter 4: Scrabe a client-side rendered webpage

Lecture 1: First attempt – and fail

Lecture 2: Explained: Server-side vs Client-side rendering

Lecture 3: Scraping a client-side rendered webpage

Lecture 4: Extract a list of products

Lecture 5: Extract the details for each product

Lecture 6: Create a function and extract product data to a data frame

Lecture 7: Prepare to loop over all product pages

Lecture 8: Run loop and save HTML

Lecture 9: Parse the saved HTML to a data frame

Lecture 10: Clean the saved data – part 1

Lecture 11: Clean the saved data – part 2

Lecture 12: Tip: Save the page yourself

Chapter 5: Getting data from an API

Lecture 1: Example 1: ASDA API

Lecture 2: Example 2: Udemy API – a first look

Lecture 3: Example 2: Udemy API – get a thousand courses

Chapter 6: Wrap up

Lecture 1: Finishing notes