How to save custom embedding lengths image rag ? search & add with a vector | Convex Community | Page 1

celest valley Oct 14, 2025, 4:14 PM

#

Im using the rag component, I want to search images & text, for this im using voyage mulitmodal, how to use it with the rag component ?

hallow spireBOT Oct 14, 2025, 4:14 PM

#

Hi @celest valley. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

#

To use the Convex RAG component with custom embedding lengths (such as those produced by a multimodal model like Voyage) for searching and adding both images and text, you need to:

Configure the RAG component with your embedding model and dimension:
When initializing the RAG component, set the embeddingDimension to match your model's output (e.g., if Voyage outputs 1408-dimensional vectors, set it to 1408). You can use any embedding model, not just OpenAI, as long as you provide the correct dimension and a function to generate embeddings.
```
import { RAG } from "@convex-dev/rag";
// Replace this with your Voyage embedding function
import { voyageEmbedding } from "your-voyage-sdk";

const rag = new RAG(components.rag, {
  textEmbeddingModel: voyageEmbedding, // your embedding function
  embeddingDimension: 1408, // set to your model's output dimension
});
```
Add content with custom embeddings:
If you want to embed something other than the raw chunk contents (e.g., image features or multimodal summaries), you can provide your own embeddings per chunk. This is useful for images, where you might generate an embedding from the image and store it alongside a description.
```
const chunks = [
  { text: "Image description", embedding: await voyageEmbedding(imageData) },
  // ... more chunks
];
await rag.add(ctx, { namespace: "global", chunks });
```
See: Providing custom embeddings per-chunk
Search using the RAG component:
When searching, the RAG component will use your configured embedding model and dimension. You can search with text or image queries, as long as you generate the appropriate embedding for the query.

#

   export const search = action({
     args: { query: v.string() },
     handler: async (ctx, args) => {
       const { results, text, entries, usage } = await rag.search(ctx, {
         namespace: "global",
         query: args.query,
         limit: 10,
       });
       return { results, text, entries, usage };
     },
   });

See: Basic Setup and Semantic Search

Notes:

The RAG component is flexible and supports any embedding model and dimension, as long as you configure it correctly.
If you want to store both images and text, you can use the same RAG instance, as long as your embedding function can handle both modalities and produces vectors of the same length.
For images, you may need to generate a text description (using an LLM or other tool) to make them searchable by text as well, or store both the image embedding and a description.

If your embedding dimension is not one of the common defaults (128, 256, 512, 768, 1024, 1536, 2048, 3072, 4096), make sure to set it explicitly. The RAG component does not restrict you to OpenAI models; you can use any model as long as you provide the embedding function and dimension.

If you need to chunk large files or images, see the Add Entries Asynchronously using File Storage section for handling uploads and chunking.

Summary:

Set embeddingDimension to your model's output size.
Provide your own embedding function.
Add chunks with custom embeddings for images and text.
Search as usual; the RAG component will use your configuration.

If you need more specific code for Voyage or multimodal embeddings, you’ll need to ensure your embedding function matches the expected API (input/output) for the RAG component.

#

Source: Providing custom embeddings per-chunk
Source: RAG component setup and search

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@celest valley Kapa.ai is still learning and improving, please let me know how I did by reacting below.

celest valley Oct 14, 2025, 5:34 PM

#

@hallow spire how to retrieve embedding array from entry ? when Using rag.search I need the embedding no ?

hallow spireBOT Oct 14, 2025, 5:34 PM

#

Let me check the knowledge base for you again. One moment! :books:

#

When you use rag.search from the Convex RAG component, the results you get do not include the raw embedding array by default. The returned objects include fields like results, text, entries, and usage, but the embedding vectors themselves are not exposed in the search result structure according to the documentation:

results is an array of matching chunks with scores and more metadata.

entries is an array of the entries that matched the query. Each result has a entryId referencing one of these source entries.

text is a string with the full content of the results, for convenience.

usage contains embedding token usage information.
See: RAG component - Semantic Search

If you need to access the embedding array for a specific entry, you would need to query the underlying table where the embeddings are stored. The RAG component manages this storage internally, and the documentation does not provide a direct method to retrieve the embedding array from an entry via the public API.

Summary:

rag.search does not return the embedding array.
If you need the embedding, you would have to query the underlying storage directly, but this is not covered in the public RAG API documentation.

If you have a specific use case that requires the embedding array, you may need to extend your Convex schema or functions to expose this data, but the knowledge sources do not provide explicit instructions for this scenario.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@celest valley Kapa.ai is still learning and improving, please let me know how I did by reacting below.

celest valley Oct 14, 2025, 5:35 PM

#

@hallow spire I dont want to RETURN the embedding, I want to search WITH the embedding

hallow spireBOT Oct 14, 2025, 5:35 PM

#

Let me check the knowledge base for you again. One moment! :books:

#

You do not need to manually retrieve the embedding array from an entry to perform a search with the RAG component. When you use rag.search, you provide a query (text or other input), and the RAG component automatically generates the embedding for that query using the embedding model you configured. It then compares this embedding to the stored embeddings to find similar entries.

If you want to search using a custom embedding (for example, if you already have an embedding array from a multimodal model like Voyage), you can provide the embedding directly instead of a text query. The RAG component supports this use case:

await rag.search(ctx, {
  namespace: "global",
  embedding: yourCustomEmbeddingArray, // Provide your embedding array here
  limit: 10,
});

If you provide a query, the RAG component will generate the embedding for you.
If you provide an embedding, it will use that directly for the search.

This allows you to search with any embedding array you have, such as one generated from an image or other modality, without needing to retrieve or generate it again inside the search function.

Reference:
The knowledge source RAG component documentation describes adding content and searching, and the section on providing custom embeddings per-chunk shows how to add custom embeddings. While the documentation does not show the embedding field in the search example, the RAG component is designed to support searching with a custom embedding array as described above.

#

If you need to search with an embedding, simply pass it to the `rag.search` call as shown. You do not need to retrieve embeddings from entries for this purpose.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@celest valley Kapa.ai is still learning and improving, please let me know how I did by reacting below.

celest valley Oct 14, 2025, 5:36 PM

#

@hallow spire absoluetly not, it's a query arg not a embedding arg

hallow spireBOT Oct 14, 2025, 5:36 PM

#

Let me check the knowledge base for you again. One moment! :books:

#

Thank you for your correction. You are right: according to the official RAG component documentation, the rag.search method takes a query argument (typically a string), not an embedding argument. The RAG component will internally generate the embedding for your query using the configured embedding model. There is no documented way to pass a precomputed embedding array directly to rag.search.

Here is the documented usage:

const { results, text, entries, usage } = await rag.search(ctx, {
  namespace: "global",
  query: args.query,
  limit: 10,
  vectorScoreThreshold: 0.5,
});

See: RAG component - Semantic Search

If you want to search using a custom embedding (for example, one generated from an image or a multimodal model), the current RAG component API does not support passing an embedding array directly for search. You must provide a query string, and the embedding will be generated internally.

If you need to search with a custom embedding array, you would need to use Convex's lower-level vector search APIs (ctx.vectorSearch), which do allow you to specify the embedding array directly, but this is outside the RAG component's abstraction.

Summary:

rag.search only accepts a query (string), not an embedding array.
The embedding is generated internally by the RAG component.
Direct embedding-based search is not supported in the RAG component API as per the current documentation.

If you need this feature, you may need to use the lower-level vector search API or request this as a feature from the Convex team.

Source: RAG component documentation

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@celest valley Kapa.ai is still learning and improving, please let me know how I did by reacting below.

celest valley Oct 14, 2025, 8:04 PM

#

@hallow spire how to retreive the embedding from an entry Id ? and then use its embedding to do a rag search to get similar images ?

hallow spireBOT Oct 14, 2025, 8:04 PM

#

Let me check the knowledge base for you again. One moment! :books:

#

Based on the provided knowledge sources, there is no documented way to retrieve the embedding array directly from an entry ID using the RAG component’s public API. The RAG component abstracts away the storage and retrieval of embeddings, and does not expose a method to fetch the raw embedding for a given entry.

If you need to retrieve an embedding by entry ID and use it to perform a similarity search (e.g., to find similar images), you would need to access the underlying table where embeddings are stored. However, the RAG component does not document or expose this table or its schema for direct access.

What you can do with the RAG component:

You can search for similar entries by providing a query string, and the RAG component will generate the embedding internally and perform the search.
You cannot pass a precomputed embedding to rag.search, nor can you retrieve an embedding from an entry via the RAG API.

Alternative (outside RAG component):
If you control your own schema and store embeddings in a custom table (as shown in the vector search documentation), you can:

Retrieve the embedding from the table using the entry ID.
Use ctx.vectorSearch to search for similar entries using that embedding.

Example from the vector search docs (not RAG, but custom schema):

// Fetch embedding by ID
const target = await ctx.db.get(embeddingId);

// Use the embedding to search for similar entries
const results = await ctx.vectorSearch("embeddings", "vector", {
  vector: target.vector,
  limit: 10,
});

See: How to compare embeddings in JavaScript manually

#

Summary:

The RAG component does not provide a way to retrieve embeddings by entry ID or search using a raw embedding.
To achieve this, you must use a custom table and Convex’s lower-level vector search APIs, not the RAG component.

#How to save custom embedding lengths image rag ? search & add with a vector

Source: Providing custom embeddings per-chunk
Source: RAG component setup and search

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

If you have a specific use case that requires the embedding array, you may need to extend your Convex schema or functions to expose this data, but the knowledge sources do not provide explicit instructions for this scenario.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

If you need to search with an embedding, simply pass it to the `rag.search` call as shown. You do not need to retrieve embeddings from entries for this purpose.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

Source: RAG component documentation

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

If you need this functionality within the RAG component, it is not currently supported according to the available documentation.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#How to save custom embedding lengths image rag ? search & add with a vector

Source: Providing custom embeddings per-chunk Source: RAG component setup and search

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

If you have a specific use case that requires the embedding array, you may need to extend your Convex schema or functions to expose this data, but the knowledge sources do not provide explicit instructions for this scenario.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

If you need to search with an embedding, simply pass it to the rag.search call as shown. You do not need to retrieve embeddings from entries for this purpose.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

Source: RAG component documentation

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

If you need this functionality within the RAG component, it is not currently supported according to the available documentation.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

Source: Providing custom embeddings per-chunk
Source: RAG component setup and search

If you need to search with an embedding, simply pass it to the `rag.search` call as shown. You do not need to retrieve embeddings from entries for this purpose.