Categories
Python Code

Web Scraping Tools

webscraping tools

We’ve not included Javascript or Selenium and how others think about it may vary but time is finite, so concentrate on what gets the biggest benefit.

We could have put Python at the centre but we’ve assumed that’s a given.

How do you visualize your own process of web scraping?

Categories
Python Code

Fix : Module Not Found

If you use Scrapy, and CrawlerProcess then Python may not search the path that is required for Scrapy to find its “items.py” file.

How do you fix this ‘ModuleNotFoundError‘ ?

Read on…

scrapy items.py module not found
items.py lcoation

As we can see, Scrapy framework has the items.py file in a directory above the spiders directoy

spider location

path append(Path = Project Name)
use path.append to add items.py to the Scrapy search path