Python Code

Web Scraping Tools

webscraping tools

We’ve not included Javascript or Selenium and how others think about it may vary but time is finite, so concentrate on what gets the biggest benefit.

We could have put Python at the centre but we’ve assumed that’s a given.

How do you visualize your own process of web scraping?

Python Code

Fix : Module Not Found

If you use Scrapy, and CrawlerProcess then Python may not search the path that is required for Scrapy to find its “” file.

How do you fix this ‘ModuleNotFoundError‘ ?

Read on…

scrapy module not found lcoation

As we can see, Scrapy framework has the file in a directory above the spiders directoy

spider location

path append(Path = Project Name)
use path.append to add to the Scrapy search path