Modern Web Scraping with Python using Scrapy Splash Selenium

Web Scraping nowadays has become one of the hottest topics, there are plenty of paid tools out there in the market that don’t show you anything how things are done as you will be always limited to their functionalities as a consumer.

In this course you won’t be a consumer anymore, i’ll teach you how you can build your own scraping tool ( spider ) using Scrapy.

You will learn:

1. The fundamentals of Web Scraping

2. How to build a complete spider

3. The fundamentals of XPath & CSS Selectors

4. How to locate content/nodes from the DOM using XPath & CSS

5. How to store the data in JSON, CSV…and even to an external database(MongoDb & SQLite3)

6. How to write your own custom Pipeline

7. Fundamentals of Splash

8. How to scrape Javascriptwebsites using Scrapy Splash & Selenium

9. The Crawling behavior

10. How to build a CrawlSpider

11. How to avoid getting banned while scraping websites

12. How to build a custom Middleware

13. Web Scraping best practices

14. How to scrape APIs

15. How to use Request Cookies

16. How to scrape infinite scroll websites

17. Host spiders in Heroku for free

18. Run spiders periodically with a custom script

19. Prevent storing duplicated data

20. Deploy Splash to Heroku

21. Write data to Excel files

22. Login to websites using Scrapy

23. Download Files& Imagesusing Scrapy

24. Use Proxieswith Scrapy Spider

25. Use Crawlerawith Scrapy & Splash

26. Use Proxies with CrawlSpider

What makes this course different from the others, and why you should enroll ?

• First, this is the most updated course. You will be using Python 3.7, Scrapy 1.6 and Splash 3.0

• You will have an in-depth step by step guide on how to become a professional web scraper.

• You will learn how to use Splash & Selenium to scrape JavaScript websites and I can assure you, you won’t find any tutorials out there that teaches how to really use Splash like I’ll be doing in this course.

• You will learn how to host spiders in Heroku as well as Splash(Exclusive).

• You will learn how to create a custom script so spiders can run periodicallywithout any intervention from you.

• 30 days money back guarantee by Udemy

So whether you are a data analystwho wants to add web scraping to his tool set or someone else who wants to learn how to extract unstructured data from unstructured HTML web pages and then store back that data in a structured way to apply some data analysis on it then you are welcome to join this course.

**STUDENTS THOUGHTS ABOUT THIS COURSE **

“I was particularly looking for web scraping using XPATHs and this course is addressing that. It also covers dynamic paging. A proper mix of theory and practical. A must-have for those who wants to do web scraping . GREAT learning experience !!! “. By Hiran Kumar

“90% of what I was searching for!!! Great job!! Clear explanations and great communication with Ahmed”. By Raylyson Estanista 

“Admed’s Web scraping course is awesome . His approach using Python with scrapy and splash works well with all websites especially those that make heavy use of JavaScript. Ahmed is a gifted educator: expert communicator, passionate, conscientious and accessible to his students. I highly recommend this course and any of Ahmed Rafik’s Udemy courses. “. By Richard Blackmon

“Great course, and a nice introduction to Scrapy (I’m someone with no Python experience whatsoever).”. By I S

“Excellent course. Quick and thorough at the same time. Ahmed is incredibly responsive to the students and often replies to questions within minutes! Highest recommendation.” By Robert Nolte

“That course is very good and explanation is crystal clear! The instructor is very supportive in case of questions. Highly recommended.” By Shubina Ekaterina

“I like the course. Clear explanations and good comunication with Ahmed. All topics is interesting and full of information. I improved my skils in Scrapy. Author update course content by new videos. It’s a big bonus) Explained more advance topics I never see in other courses. Thank you, Ahmed. Waiting for new videos)”. By Ruslan Romanenko

Rating Distribution

• 1 stars: 43 votes
• 2 stars: 51 votes
• 3 stars: 285 votes
• 4 stars: 1025 votes
• 5 stars: 2360 votes

Frequently Asked Questions

How long do I have access to the course materials?

You can view and review the lecture materials indefinitely, like an on-demand channel.

Can I take my courses with me wherever I go?

Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don’t have an internet connection, some instructors also let their students download course lectures. That’s up to the instructor though, so make sure you get on their good side!