Run Scrapy Spider
If you don’t already have a way to transfer the spider onto your Raspberry Pi you can simply use
git clone https://github.com/RGGH/scrapy6.git
Once all this is in place, you can run the spider with “nohup”
sudo nohup privatevpn
sudo python3 foodcom.py
Note: You also need to run the VPN with “nohup” as well, otherwise it will lose the connection when you exit from PuTTy / your Terminal session.
From your remote PC, eg a Windows PC, Mac, or Ubuntu, you can then copy the output file from the Pi, and open it, as you should have much more memory and of course a GUI!
To copy the results.csv from the Pi to my Ubuntu PC I used “rsync” but you could also use SCP.