In this article I show some code to web scrape EBAY using Rust and a proxy provided by ScrapeOps The code below uses the ScrapeOps endpoint : “https://proxy.scrapeops.io/v1/” and the requests are forwarded on to your chosen url to scrape. How to get an API key? You can get a free one, with 1000 tokens […]
Web Scraping requires specific tools and techniques, let’s look at our favourite one, ScrapeOps : “ScrapeOps provides a suite of web scraping tools that makes it easier to deploy, scheduling and monitor your web scrapers and data extraction feeds in production.” The need for data is relentless – more and more AI projects rely on […]
We’ll convert png to jpg, but the same concept will apply to doing the reverse of this. You can grab png datasets from Kaggle If you just want to skip to the code, check out these links: What is a JPEG ? Let’s first understand the structure of a *.jpg file The structure of a […]
Introduction FastEmbed is a lightweight and fast Python library designed for generating high-quality text embeddings. FastEmbed supports a variety of popular text models and is continuously expanding to include more. At its core, FastEmbed utilizes the Flag Embedding model from the MTEB leaderboard, offering distinct “query” and “passage” prefixes for input text. This feature makes […]
Using serde_json to Handle Fake Data in Rust When working with JSON data in Rust, serde_json provides a convenient way to parse and manipulate JSON values. In this post, we’ll demonstrate how to use serde_json::Value to create some mock data, simulate a search result, and extract specific fields from it. Creating Fake Data and Extracting […]
This article delves into a Rust program that showcases custom error handling, parsing, and validation. By leveraging the thiserror crate, we create descriptive error types to manage various parsing scenarios robustly. We’ll explore how the program defines an Animal struct, parses and validates integer strings, and utilizes generics and traits for flexible error reporting. Whether […]
Introduction In this guide, we will compare Rust and Python for asynchronous and multithreaded operations, particularly focusing on uploading data to a Qdrant vector search engine. We will evaluate how using Arc in Rust and AsyncQdrantClient in Python impacts performance. Imagine you get approached by a company to speed up their code and you improve […]
Rust is an ideal choice for image preprocessing due to its combination of performance, safety, and concurrency. Its low-level control allows for highly efficient image manipulation, crucial for handling large datasets or performing complex transformations. Image preprocessing involves transforming raw image data into a format that is usable and meaningful. If you want a fast […]
To begin with, when learning about ML/AI you may just use the model specified in the tutorial you’re doing, but later on you’ll want to research, test, and evaluate different models based on size, performance and write your code to suit. Let’s compare output using 2 different models that I used in a recent project […]
