Let’s look at how to create complete documentation for your entire Python project using Sphinx Here is a guide to how to document a very small project, with 2 directories “src” and “docs” where all the source code is in “src” and the Sphinx specific files go inside “docs”. You’ll be able to view each […]
A Python generator is a “lazy Iterator” – a function that is a memory efficient way of providing an object to loop through We can define a Python generator function just like a normal function. This article will show some simple examples to illustrate how easy they are to use and how effective they can […]
If you want to learn about Data Analysis with Pandas and Python and you’re not familiar with Kaggle, check it out! Time to read article : 5 mins TLDR; We show how to use idxmax and apply with Pandas Introduction Here we will look at some functions in Pandas which will help with ‘EDA’ – […]
A common task is to track competitors prices and use that information as a guide to the prices you can charge, or if you are buying, you can spot when a product is at a new lowest price. The purpose of this article is to describe how to web scrape Amazon. Using Python, Scrapy, MySQL, […]
You may want to scrape Amazon for information about books about web scraping! We shorten what would have been a very very long selector, by using “contains” in our xpath : response.xpath(‘//*[contains(@class,”sg-col-20-of-24 s-result-item s-asin”)]’) The most important thing when starting to scrape is to establish what you want in your final output. Here are the […]
The challenge was to scrape a site where the class names for each element were out of order / randomised. So the only way to get the data in the correct sequence was to sort through the CSS styles by left, top, and match to the class names in the divs… The names were meaningless […]