Filter data based on any array of field values

[clever canyon]

I have a table of users and a dropdown for filtering them. Since this is a large dataset, I use a paginated query to manage the data efficiently. However, I want the filtering to happen at the query level on the server to improve performance.

For example, if I have a data table of emails and I want to apply a server-side filter to only retrieve user IDs matching the selected options (e.g., Bob, Jane, and Jenny), how can I achieve this using an array of filters?

[split jungle]

Hi @clever canyon. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

---

To filter data based on an array of field values in Convex while using pagination, you can use the filter helper from the "convex-helpers" package along with pagination. This approach allows you to perform server-side filtering efficiently. Here's how you can achieve this:

1. First, install the "convex-helpers" package if you haven't already:

npm install convex-helpers

2. Then, in your Convex query function, you can use the filter helper along with pagination like this:

import { filter } from "convex-helpers/server/filter";

export const filteredUsers = query({
  args: { 
    selectedUsers: v.array(v.string()),
    paginationOpts: paginationOptsValidator 
  },
  handler: async (ctx, args) => {
    return filter(
      ctx.db.query("users"),
      (user) => args.selectedUsers.includes(user.name)
    ).paginate(args.paginationOpts);
  },
});

This query function will filter the users based on the selectedUsers array and return paginated results. The filter helper allows you to use TypeScript/JavaScript filtering logic directly in your query, which is then applied efficiently on the server side.

3. On the client side, you can use the usePaginatedQuery hook to fetch and paginate the filtered results:

import { usePaginatedQuery } from "convex/react";

function UserList() {
  const selectedUsers = ["Bob", "Jane", "Jenny"];
  const { results, status, loadMore } = usePaginatedQuery(
    "filteredUsers",
    { selectedUsers, paginationOpts: { initialNumItems: 10 } }
  );

  // Render your list and pagination controls here
}

This approach allows you to efficiently filter and paginate your data on the server side, improving performance for large datasets.

For more details on using complex filters and pagination in Convex, you can refer to these resources:

Complex Filters in Convex
Help, my app is overreacting!

Remember that while this method is flexible and powerful, it may not be as efficient as using indexes for very large datasets. If you find that performance is still an issue, you might need to consider restructuring your data model or using more specific indexes.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

@clever canyon Kapa.ai is still learning and improving, please let me know how I did by reacting below.

[clever canyon]

@split jungle no but i just want to filter a field not a e.g. i have a table of deliveries and for each delivery i want to check if it was sent to one of the occupants selected in the filter