Scrapy Form Login
The following is an article which will show you how to use Scrapy to log in to sites that have username and password authentication.
The important thing to remember is that there may be additional data that needs to be sent to the login page, data that is in addition to just username and password…
We are going to cover :
🔴 Identifying what data to POST
🔴 The Scrapy method to login
🔴 How to progress after logging in
Once you have seen that the Spider has logged in, you can proceed to scrape what you need. Remember, you are looking for “Logout” because that will mean you are logged in!
Conclusion : we have looked at how to use FormRequest to POST form data in a Scrapy spider, and we have extracted data from a site that is protected by a login form.
See the video on YouTube :
Scrapy Form Login | How to log in to sites using FormRequest | Web Scraping Tutorial
https://youtu.be/VvU9mR-WNSA
