how to make the following more efficient is or(...) correct?

[last ember]

Hi @ashen pollen. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

---

Your current approach uses a filter with an or(...) clause to match multiple token addresses:

let query = ctx.db
  .query("tokens")
  .filter((q) =>
    q.or(
      ...tokenAddressesWithFees.map((address) =>
        q.eq(q.field("tokenAddress"), address)
      )
    )
  )

Is or(...) correct?

Yes, using q.or(...) is syntactically correct in Convex filters for combining multiple equality checks, as shown in the documentation:

const usersNamedAlexOrEmma = await ctx.db
  .query("users")
  .filter((q) =>
    q.or(q.eq(q.field("name"), "Alex"), q.eq(q.field("name"), "Emma")),
  )
  .collect();

(Filtering)

However, this approach is not efficient for large tables, because .filter (including with or) will scan the entire table and check each document, which can be slow and resource-intensive for large datasets. This is equivalent to an unindexed SQL WHERE clause and should be avoided for large or unbounded tables (Best Practices, Performance of TypeScript filters).

How to make it more efficient

Convex does not support an efficient "in" or "or" query using indexes. The recommended approach is to run a separate indexed query for each address and then merge the results. For example:

// Assuming you have an index on "tokenAddress"
const tokensWithFees = await Promise.all(
  tokenAddressesWithFees.map(address =>
    ctx.db
      .query("tokens")
      .withIndex("by_tokenAddress", q => q.eq("tokenAddress", address))
      .collect()
  )
);
const flatTokens = tokensWithFees.flat();

(No efficient “in” / “or”, Union)

This approach is much more efficient if you have an index on tokenAddress, as each query will only scan the relevant range.

Summary:

• Using or(...) is correct but not efficient for large tables.
• For efficiency, run separate indexed queries for each address and merge the results.
• Make sure you have an index on tokenAddress in your schema.

If you have a small number of addresses or a small table, your current approach is acceptable. For larger datasets, use the indexed approach above.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

@ashen pollen Kapa.ai is still learning and improving, please let me know how I did by reacting below.

[ashen pollen]

ok fix it for me.

// First get all token addresses that have fees from tokenMeteoraFees table
    const feeTokens = await ctx.db
      .query("fees")
      .withIndex("by_fee")
      .order("desc")
      .collect()

    const tokenAddressesWithFees = feeTokens.map((fee) => fee.tokenAddress)

    // Query tokens and filter by addresses that have fees
    let query = ctx.db
      .query("tokens")
      .filter((q) =>
        q.or(
          ...tokenAddressesWithFees.map((address) =>
            q.eq(q.field("tokenAddress"), address)
          )
        )
      )

make it efficient. use filtering or wat u propse

@last ember ok fix it for me.

// First get all token addresses that have fees from tokenMeteoraFees table
    const feeTokens = await ctx.db
      .query("fees")
      .withIndex("by_fee")
      .order("desc")
      .collect()

    const tokenAddressesWithFees = feeTokens.map((fee) => fee.tokenAddress)

    // Query tokens and filter by addresses that have fees
    let query = ctx.db
      .query("tokens")
      .filter((q) =>
        q.or(
          ...tokenAddressesWithFees.map((address) =>
            q.eq(q.field("tokenAddress"), address)
          )
        )
      )

make it efficient. use filtering or wat u propse

[last ember]

Let me check the knowledge base for you again. One moment! :books:

---

To make your query efficient in Convex, you should avoid using .filter with a large .or(...) clause, as this will scan the entire table and does not use indexes efficiently. Instead, run a separate indexed query for each token address and merge the results. Here’s how you can do it, assuming you have an index on tokenAddress (e.g., by_tokenAddress):

// First get all token addresses that have fees from the fees table
const feeTokens = await ctx.db
  .query("fees")
  .withIndex("by_fee")
  .order("desc")
  .collect();

const tokenAddressesWithFees = feeTokens.map((fee) => fee.tokenAddress);

// Efficiently query tokens by tokenAddress using the index
const tokensWithFeesArrays = await Promise.all(
  tokenAddressesWithFees.map(address =>
    ctx.db
      .query("tokens")
      .withIndex("by_tokenAddress", q => q.eq("tokenAddress", address))
      .collect()
  )
);

const tokensWithFees = tokensWithFeesArrays.flat();

This approach is recommended in Convex for "IN" or "OR" queries, as it leverages indexes for each address and avoids a full table scan. If you expect a very large number of addresses, you may want to consider batching or pagination, but for most use cases this pattern is efficient and scalable see: Union.

Note:

• Make sure you have an index on tokenAddress in your tokens table schema.
• This approach is much more efficient than using .filter with .or(...) for large tables.

If you need paginated results, you can use stream merging as described in the knowledge sources. Let me know if you need an example for that!

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: