Qdrant & Rust – embeddings

While most examples for performing embeddings will be Python code, Rust is better when performance, safety, and scalability are critical. Most examples/tutorials stick to Python, maybe it’s easier to start with but let’s look beyond “getting started”.

Let’s explore Qdrant some more, this time we’ll delve into the Rust client plus fastembed.

(Note: If you prefer to see the code for embedding images, check out my other project : https://github.com/RGGH/qdrant_fastembed_rm)

Qdrant is: “The leading open source vector database and similarity search engine designed to handle high-dimensional vectors for performance and massive-scale AI applications.”

Concepts : https://qdrant.tech/documentation/concepts/

Qdrant has this terminolgy :

• Collections define a named set of points that you can use for your search.
  Payload
• A Payload describes information that you can store along with vectors.
  Points
• Points are a record which consists of a vector and an optional payload.

How to get Qdrant

Step 1 – Pull the Qdrant Docker image and start it

docker pull qdrant/qdrant

docker run -p 6333:6333 -p 6334:6334 \
    -v $(pwd)/qdrant_storage:/qdrant/storage:z \
    qdrant/qdrant

---

Now we have a running Qdrant vector database, let’s add some data. We’ll frequently refer to the Qdrant documentaton and then add some extra code nearer the end of this article to handle our own embeddings (data)

https://github.com/qdrant/rust-client?tab=readme-ov-file#qdrant-rust-client

Create a collection

use qdrant_client::qdrant::{CreateCollectionBuilder, Distance, VectorParamsBuilder};
use qdrant_client::{Qdrant, QdrantError};

#[tokio::main]
async fn main() -> Result<(), QdrantError> {
let client = Qdrant::from_url("http://localhost:6334").build()?;

client
    .create_collection(
        CreateCollectionBuilder::new("{collection_name}")
            .vectors_config(VectorParamsBuilder::new(100, Distance::Cosine)),
    )
    .await?;
    Ok(())
}

And if you run it a second time :

Error: ResponseError { status: Status { code: AlreadyExists, message: "Wrong input: Collection `{collection_name}` already exists!", metadata: MetadataMap { headers: {"content-type": "application/grpc", "date": "Thu, 28 Nov 2024 13:47:49 GMT", "content-length": "0"} }, source: None } }

We get an error !

Instead, let’s check for the existence of the collection name before we create it!

use qdrant_client::qdrant::{CreateCollectionBuilder, Distance, VectorParamsBuilder};
use qdrant_client::{Qdrant, QdrantError};

#[tokio::main]
async fn main() -> Result<(), QdrantError> {
    let client = Qdrant::from_url("http://localhost:6334").build()?;

    if !client.collection_exists("{collection_name}").await? {
        client
            .create_collection(
                CreateCollectionBuilder::new("{collection_name}")
                    .vectors_config(VectorParamsBuilder::new(100, Distance::Cosine)),
            )
            .await?;
    }
    Ok(())
}

Now the code will run without error

Upsert data (payload)

Performs the insert + update action on specified points. Any point with an existing {id} will be overwritten.

Rather than the upsert points example that Qdrant provide, let’s do a more interesting example:

let payload: Payload = serde_json::json!(
{
    "foo": "Bar",
    "bar": 12,
    "baz": {
        "qux": "quux"
    }
}
)
.try_into()
.unwrap();

You’ll need to do cargo add serde_json to add serde_json to Cargo.toml and also add Payload to the Qdrant client at the top of main.rs

use qdrant_client::{Qdrant, QdrantError,Payload};

[package]
name = "qd1"
version = "0.1.0"
edition = "2021"

[dependencies]
qdrant-client = "1.12.1"
serde_json = "1.0.133"
tokio = { version = "1.41.1", features = ["full"] }

Once we have a collection, we can insert (or rather upsert) points.

Points have an id, one or more vectors and a payload.

 let points = vec![PointStruct::new(0, vec![12.; 10], payload)];

The vectors, seen above as [12;10] would usually be created from the embedding model.

We’ll also need to add UpsertPointsBuilder and PointStruct to the qdrant client at the top of main.rs

Qdrant & Rust – the full code

Ok, this worked! We can check the “upsert” worked by following the url shown in the terminal

---

Part 2 – real embeddings

Now we know how to upsert, let’s use some actual embedded data, with the correct dimensions for the vector and pass the id, vector and payload into Qdrant

We’ll use fastembed to embed the text into numeric vector representation, based on what the LLM decided to do with each of the 4 lines of text, shown as strings inside a Rust vector.

Don’t confuse Rust’s vector type with the “Vector” in a vector database.

"passage: Hello, World!",
"query: Hello, World!",
"passage: This is an example passage.",
"passage: This is also an example passage.",

https://github.com/Anush008/fastembed-rs?tab=readme-ov-file#-installation

fastembed for Rust

Fastembed does dense AND sparse embeddings

FastEmbed performs inferencing. Specifically, it generates vector embeddings from textual data using machine learning models optimized for semantic understanding. These embeddings are numerical representations of text, enabling tasks like similarity search, clustering, and recommendation.

https://github.com/qdrant/fastembed

Ok, back to the code, we need to add fastembed and also add anyhow (to handle the “non qdrant” error).

cargo add fastembed anyhow

use fastembed::{EmbeddingModel, InitOptions, TextEmbedding};
use qdrant_client::qdrant::{
    CreateCollectionBuilder, Distance, PointStruct, UpsertPointsBuilder, VectorParamsBuilder,
};
use qdrant_client::{Payload, Qdrant};
use anyhow::Result;

#[tokio::main]
async fn main() -> Result<()> {
    // Initialize the embedding model with custom options
    let model = TextEmbedding::try_new(
        InitOptions::new(EmbeddingModel::AllMiniLML6V2).with_show_download_progress(true),
    )?;

    let documents = vec![
        "passage: Hello, World!",
        "query: Hello, World!",
        "passage: This is an example passage.",
        "passage: This is also an example passage.",
    ];

    // Generate embeddings
    let embeddings = model.embed(documents.clone(), None)?;

    // Connect to the Qdrant client
    let client = Qdrant::from_url("http://localhost:6334").build()?;

    let collection_name = "test";

    // Check if the collection exists, create it if it doesn't
    if !client.collection_exists(collection_name).await? {
        client
            .create_collection(
                CreateCollectionBuilder::new(collection_name)
                    .vectors_config(VectorParamsBuilder::new(embeddings[0].len() as u64, Distance::Cosine)),
            )
            .await?;
    } else {
        println!("Collection `{}` already exists!", collection_name);
    }

    // Prepare points with embeddings and corresponding documents as payload
    let points: Vec<PointStruct> = embeddings
        .into_iter()
        .enumerate()
        .map(|(id, vector)| {
            let payload: Payload = serde_json::json!({ "document": documents[id] })
                .try_into()
                .unwrap();
            PointStruct::new(id as u64, vector, payload)
        })
        .collect();

    // Upsert the points into the collection
    client
        .upsert_points(UpsertPointsBuilder::new(collection_name, points))
        .await?;

    Ok(())
}

If you’re used to Python, this might look a bit messy? – Rust may seem fussy, but it eliminates errors before the end user encounters them. You fix them as the developer, with the help of the Rust compiler and it’s very good hints.

Here’s the chatGPT explanation of what the map function is doing and why!

This code creates a vector of PointStruct instances from the embeddings and documents. Here’s a detailed breakdown:

---

Step-by-Step Explanation:

1. embeddings.into_iter()

• Converts the embeddings vector into an iterator that consumes the elements.
• Each element is an embedding vector (Vec<f32>), representing the numerical representation of a document.

2. .enumerate()

• Adds an index (id) to each element of the iterator.
• Produces tuples of (id, vector), where:
  • id is the index (0-based).
  • vector is the embedding vector for the corresponding document.

3. .map(|(id, vector)| { ... })

• Transforms each (id, vector) tuple into a PointStruct.
• Inside the map closure:
  1. Create a Payload:let payload: Payload = serde_json::json!({ "document": documents[id] }) .try_into() .unwrap();
     • Converts the corresponding document (documents[id]) into a JSON payload.
     • The payload is structured like: { "document": "passage: Hello, World!" }
     • try_into() converts the JSON to a Payload, which is the format expected by Qdrant.
     • unwrap() is used to handle conversion success; it panics if the conversion fails (for simplicity in this context).
  2. Create a PointStruct:PointStruct::new(id as u64, vector, payload)
     • A PointStruct is created using:
       • id as u64: A unique identifier for the point (cast to u64 as required by Qdrant).
       • vector: The embedding vector for the document.
       • payload: Metadata containing the document text.

4. .collect()

• Consumes the iterator and collects all PointStruct instances into a Vec<PointStruct>.

---

Result:

The points vector contains PointStruct instances, where each point combines:

1. A unique ID (starting from 0).
2. The embedding vector generated for the corresponding document.
3. A payload containing the original document.

---

Example:

Assume the following documents:

let documents = vec![
    "passage: Hello, World!",
    "query: Hello, World!",
    "passage: This is an example passage.",
];

Assume embeddings looks like:

vec![
    vec![0.1, 0.2, 0.3], // Embedding for "passage: Hello, World!"
    vec![0.4, 0.5, 0.6], // Embedding for "query: Hello, World!"
    vec![0.7, 0.8, 0.9], // Embedding for "passage: This is an example passage."
]

After this code, points will contain:

vec![
    PointStruct {
        id: 0,
        vector: vec![0.1, 0.2, 0.3],
        payload: { "document": "passage: Hello, World!" }
    },
    PointStruct {
        id: 1,
        vector: vec![0.4, 0.5, 0.6],
        payload: { "document": "query: Hello, World!" }
    },
    PointStruct {
        id: 2,
        vector: vec![0.7, 0.8, 0.9],
        payload: { "document": "passage: This is an example passage." }
    }
]

---

Why Use This Code?

• Organized Data: Each embedding is linked to its document through a payload.
• Unique IDs: id ensures every point is uniquely identifiable in Qdrant.
• Qdrant Readiness: Data is prepared in the format (PointStruct) that Qdrant requires for upsertion.

Final code

---

Python code comparison

Rust v Python

❯ cargo r

Model initialization took: 230.34ms
Embedding generation took: 8.71ms
Qdrant client connection took: 6.35µs
Collection creation took: 97.94ms
Upsert operation took: 706.09µs
done!

❯ python main.py
Model initialization took: 657.05ms
Embedding generation took: 71.84ms
Qdrant client connection took: 53003.45µs
Collection test created!
Collection creation took: 82.08ms
Upsert operation took: 5.52ms
done!

Part 3 – embedding data from a text file

And finally here is the code to import your text so you can embed it into Qdrant

And as a bonus, here is the code to do the search, in Rust :

https://github.com/RGGH/fe

    // The phrase or word you want to search for
    let text = "doggy"; // Replace with dynamic input if needed
    // Generate embeddings using FastEmbed
    let embeddings = model.embed(vec![text.to_string()], None)?;
    let vector = embeddings[0].clone(); // Get the first vector from the embeddings
    // Perform the search with the generated embedding
    let search_result = client
        .search_points(
            SearchPointsBuilder::new(collection_name, vector, 3) // Use the correct collection name
                //.filter(Filter::all([Condition::matches("document", "fox".to_string())])) // Optional filter
                .with_payload(true) // Include payload in the results
                .params(SearchParamsBuilder::default().exact(true)), // Optional search params
        )
        .await?;
    // Print the search result for debugging
    println!("{:#?}", search_result);