Skip to the content Skip to the Navigation

Red And Green

  • Python Code
  • Rust Programming
  • Bitcoin Programming
  • Contact

July 2020

  1. HOME
  2. July 2020
July 27, 2020 / Last updated : February 15, 2023 admin Python Code

Scraping a page via CSS style data

The challenge was to scrape a site where the class names for each element were out of order / randomised. So the only way to get the data in the correct sequence was to sort through the CSS styles by left, top, and match to the class names in the divs… The names were meaningless […]

July 13, 2020 / Last updated : February 23, 2023 admin Python Code

Configure a Raspberry Pi for web scraping

Introduction The task was to scrape over 50,000 records from a website and be gentle on the site being scraped. A Raspberry Pi Zero was chosen to do this as speed was not a significant issue, and in fact, being slower makes it ideal for web scraping when you want to be kind to the […]

July 10, 2020 / Last updated : July 10, 2020 admin Python Code

Scraping “LOAD MORE”

Do you need to scrape a page that is dynamically loading content as “infinite scroll” ? Using self.nxp +=1 the value passed to “pn=” in the URL gets incremented “pn=” is the query – in your spider it may be different, you can always use urllib.parse to split up the URL into it’s parts. Test […]

July 2, 2020 / Last updated : July 2, 2020 admin Python Code

Extracting JSON from JavaScript in a web page

Why would you want to do that? Well, if you are web scraping using Python, and Scrapy for instance, you may need to extract reviews, or comments that are loaded from JavaScript. This would mean you could not use your css or xpath selectors like you can with regular html. Parse Instead, in your browser, […]

Recent Posts

lib.rs

Rust – use an upstream crate

December 1, 2023
studying Rust Code

Dissecting a Rust Program | Axum & Sled

November 30, 2023
Digital Signatures

Streamlit App – Digital Signature demo

November 28, 2023
bitcoin Script

Bitcoin Scripts and Script Language

November 28, 2023
from trait

Understanding Rust’s From Trait

November 27, 2023
bitcoin script

Learning Bitcoin Script/Miniscript and where to apply in a transaction

November 26, 2023
bitcoin Script

Explaining Bitcoin’s “Taproot”

November 25, 2023
axum-rust

Axum – testing API endpoints

November 22, 2023

BDK Wallet Descriptors

November 21, 2023
bitcoin script

Learn BDK – Bitcoin Dev Kit

November 21, 2023

Category

  • AI ML
  • automation
  • Bitcoin Programming
  • c
  • ebay api
  • email
  • JavaScript
  • LangChain
  • MySQL
  • Pandas
  • postgres
  • Python Code
  • Raspberry Pi
  • requests
  • Rust Programming
  • Scrapy
  • Selenium
  • Uncategorized
  • web scraping

Archive

  • December 2023
  • November 2023
  • October 2023
  • August 2023
  • July 2023
  • June 2023
  • May 2023
  • April 2023
  • March 2023
  • February 2023
  • January 2023
  • November 2022
  • September 2022
  • November 2021
  • September 2021
  • August 2021
  • May 2021
  • April 2021
  • March 2021
  • February 2021
  • January 2021
  • December 2020
  • November 2020
  • October 2020
  • September 2020
  • July 2020
  • June 2020
  • April 2020
  • March 2020
  • Rust – use an upstream crate
  • Dissecting a Rust Program | Axum & Sled
  • Streamlit App – Digital Signature demo
  • Bitcoin Scripts and Script Language
  • Understanding Rust’s From Trait

Archives

  • December 2023
  • November 2023
  • October 2023
  • August 2023
  • July 2023
  • June 2023
  • May 2023
  • April 2023
  • March 2023
  • February 2023
  • January 2023
  • November 2022
  • September 2022
  • November 2021
  • September 2021
  • August 2021
  • May 2021
  • April 2021
  • March 2021
  • February 2021
  • January 2021
  • December 2020
  • November 2020
  • October 2020
  • September 2020
  • July 2020
  • June 2020
  • April 2020
  • March 2020

Categories

  • AI ML
  • automation
  • Bitcoin Programming
  • c
  • ebay api
  • email
  • JavaScript
  • LangChain
  • MySQL
  • Pandas
  • postgres
  • Python Code
  • Raspberry Pi
  • requests
  • Rust Programming
  • Scrapy
  • Selenium
  • Uncategorized
  • web scraping

AI API Automation axum BDK bitcoin Bitcoin Core class Closure closures dictionaries dictionary from json Documentation enum form login generator Generators git clone GitHub GROUP BY huggingface impl javascript in scrapy json json from javascript langchain LLM load more mutable Pandas python Qdrant Rust Programming scrapy Sphinx SQL struct Upsert VectorDatabase Vectors webscraping web scraping xpath Yaml Yield

  • email
  • github
  • YouTube
This site is hosted with https://webdock.io/en Fast Cloud VPS Hosting Flat fee all-inclusive VPS with a Free Control Panel

Copyright © Red And Green All Rights Reserved.

Powered by WordPress & Lightning Theme by Vektor,Inc. technology.

MENU
  • Python Code
  • Rust Programming
  • Bitcoin Programming
  • Contact
Translate ยป