Categories
Python Code Raspberry Pi Scrapy

Configure a Raspberry Pi for web scraping

Commands used to install the VPN:

Setting up the VPN (on the Pi Zero)

Important : Check disk free using “df -h

pi@pi_zero:~ $ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/root 15G 4.7G 9.3G 34% /
devtmpfs 213M 0 213M 0% /dev
tmpfs 217M 0 217M 0% /dev/shm
tmpfs 217M 6.2M 211M 3% /run
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 217M 0 217M 0% /sys/fs/cgroup
/dev/loop0 80M 80M 0 100% /snap/core/9069
/dev/mmcblk0p1 43M 23M 20M 53% /boot
tmpfs 44M 8.0K 44M 1% /run/user/1000

Next, get the VPN specific files (the ones in bold are for “Private VPN” yours may be different).

sudo apt-get update

sudo apt-get http://privatevpn.com/client/ca.crt

sudo wget https://privatevpn.com/client/install.sh

sudo ./install.sh

sudo chmod a+x install.sh

sudo apt-get install openvpn curl

pi@pi_zero:~ $ sudo ./install.sh
PrivateVPN Linux OpenVPN Installer v0.3

  • Checking for OpenVPN – OK
  • Enter login details for PrivateVPN
  • [username]: *****
  • [password]: ********
  • Installing conf to default location (/etc/openvpn), write c to edit installpath.
  • Continue [yes/no/c] yes
    Fetching the server list from https://privatevpn.com/serverlist/
  • Installation Complete, run “sudo privatvpn” to connect to PrivateVPN service.

sudo micro /etc/privatevpn.conf # add server details

sudo nohup privatvpn # nohup means it will keep running if you quit PuTTy or Terminal from where you launch it.

To check it’s connecting to the VPN server use :

curl ifconfig.me and check if the ip address is the same as the server you saw in the Private VPN published list.

vpn, webscraping, pi
Be sure to check your VPN ip is different to your router’s public IP

Use browser, whatsmyip.com to get your router ip address and curl ifconfig.me to find the ip address of your VPN connection.