#Maximum call stack size exceeded on /store/products with large dataset

12 messages · Page 1 of 1 (latest)

vale charm
#

Hey everyone đź‘‹

I’m facing an issue when calling the /store/products endpoint.
My database contains around 160,000 products, and in the code there’s this line:

    const queryObject = remoteQueryObjectFromString({
        entryPoint: 'product',
        variables: {
            filters: req.filterableFields,
            ...req.queryConfig.pagination,
            ...context,
        },
        fields: req.queryConfig.fields,
    });

The problem is that req.filterableFields ends up including a filter on id with the full list of all product IDs in the database.
That massive filter causes a “Maximum call stack size exceeded” error.

Has anyone run into this issue before? Is there a known workaround or a better way to handle filtering by a very large set of IDs?

Any advice would be much appreciated 🙏

lament sun
#

are you passing the pagination parameters?

vale charm
#

Yes, I’m paginating.
Here I have more info:

With a small dataset (~240 products), queryObject looks like this (shortened):

{
  "__value": {
    "product": {
      "fields": ["id","title","subtitle","description","handle","is_giftcard","discountable","thumbnail","collection_id","type_id","weight","length","height","width","hs_code","origin_country","mid_code","material","created_at","updated_at"],
      "type": {"fields": ["*"]},
      "collection": {"fields": ["*"]},
      "options": {"fields": ["*"], "values": {"fields": ["*"]}},
      "tags": {"fields": ["*"]},
      "images": {"fields": ["*"]},
      "variants": {"fields": ["*"], "options": {"fields": ["*"]}},
      "__args": {
        "filters": {
          "id": ["prod_...", "prod_...", "… ~240 ids …"],
          "status": "published"
        },
        "skip": 0,
        "take": 50
      }
    }
  }
}

This produces a query like:

select count(*) as "count"
from "public"."product" as "p0"
where "p0"."deleted_at" is null
  and "p0"."status" = 'published'
  and "p0"."id" in ( ... large list of ids ... )
lament sun
#

are you calling graph.query yourself? Or are you getting an HTTP route? If you're doing it yourself try using the productModule.list method with skip and take instead, maybe it's correct.

#

But actually thinking about it now I dont think it's even possible in sql? Pagination is for the results of a query, but you are getting a stack exceeded error because you pass too much stuff as input, pagination wouldn't solve that and I don't think medusa can handle that

#

you could open an issue, the core team might have a better answer. But I would "paginate the input" by splitting the ids array and making N queries(as large as possible)

#

you could also try to see if you can somehow merge the query that creates all those ids with this one, so that the id array doesn't have to be created.

vale charm
#

@lament sun
I’m not calling graph.query directly — it’s actually a copy of the MedusaJS /store/products route. I had to duplicate it so I could add some additional custom logic.

From what I can see, the huge id array isn’t something I’m explicitly passing. It seems to be coming from Medusa’s middleware. In particular, the function maybeApplyLinkFilter looks like the one that’s adding all product IDs to the filter.

In my case I have a filter on the sales channel, and since there’s a link defined, Medusa automatically expands that into a list of product IDs and injects them into the query. That’s what ends up creating the giant id IN (...) clause.

So the stack overflow isn’t really about pagination of results, but about the way Medusa constructs the filter when links are involved.

lament sun
#

also 2.10.2 literally just release with full index support for all entities, and it's pretty much explained in the release notes why this problem happens: since sales_channels is linked from a different module, before the index module they had to fetch all the sc ids and make a second query.

vale charm
#

I will try this ! 🙂

vale charm