If you want to up your web scraping game, or have been learning python and web scraping for a while but haven't looked at Scrapy, I can help.


This video is a quick run down of the basic scrapy components, and contains a full HTML scraper including pagination. Using CSS selectors, tested with the Scrapy Shell, we are able to get this project underway quickly and easily under 50 lines of written code.


Although I show an equivalent scraper using BeautifulSoup its worth noting I am not comparing the two. Scrapy is a full web scraping framework, while BS4 is jsut an HTML parser, and a good one that at. It is still the best option in the right situations!



Support Me:
# Amazon US: https://amzn.to/2OzqL1M
# Amazon UK: https://amzn.to/2OYuMwo
# Hosting: Digital Ocean: https://m.do.co/c/c7c90f161ff6

# Gear Used: https://jhnwr.com/gear/ (NEW)
# Patreon: https://www.patreon.com/johnwatsonrooney (NEW)


-------------------------------------

Disclaimer: These are affiliate links and as an Amazon Associate I earn from qualifying purchases

-------------------------------------