Options and variety! If you are new to Scrapy and want to try it out but aren't confident with CSS selectors or XPATH, or perhaps you just want to move some scrapers over to Scrapy for the added benefits of the framework and don't want to rewrite the selectors, you can use BS4 very easily inside your Scrapy project. We can pass the response.text that comes back from the Scrapy request into BeautifulSoup and create a soup object that we can work with.

Keep it easy if you prefer the way BS4 parses HTML!

Support Me:

# Patreon: https://www.patreon.com/johnwatsonrooney (NEW)
# Amazon UK: https://amzn.to/2OYuMwo
# Hosting: Digital Ocean: https://m.do.co/c/c7c90f161ff6
# Gear Used: https://jhnwr.com/gear/ (NEW)


-------------------------------------
Disclaimer: These are affiliate links and as an Amazon Associate I earn from qualifying purchases
-------------------------------------