#`useInfiniteQuery` - indexed data? Or am I misunderstanding it entirely?

38 messages · Page 1 of 1 (latest)

potent jacinth
#

To note: I also want to be able to just look up any given item. I want it to be in an indexed form, so even if I'm on the 100th page w/pageSize of 25 and I want to get the 10th item, I don't want to have to calculate which page it should be in and do a data.pages[calculatedPage][id] - I'd greatly prefer just to do data[id]

drowsy yacht
#

when returning your data from queryFn you could transform your results into an object of id: item entries, then you can reduce all pages to an object by using

const allRecords = query.data.reduce((all, page) => ({ ...all, ...page.results }))

now you can access a record using allRecords[id]

#

as far as I got it, object destructuring is O(1), hopefully, it might be O(n) because we are destructuring all properties

potent jacinth
#

sorry but this is for a new var, allRecords, and doesn't work for my optimistic updater, does it?

As in, when I read the query cache again, it won't be updated there if I update it in allRecords like that.

#

And yeah object destructing is going to be o(log(n)) at the very least but to me looks like o(n) because .reduce must run on every member of query.data

drowsy yacht
#

o(n) where n is the number of pages

potent jacinth
#

Basically, I want it to be stored in the query cache, the entire reason I use tanstack query, as an object. I want it to receive information about what page it's on and to trust me that it's on that page, instead of having an array of pages.

o(n) where the n is the number of pages is actually the same as where it's the number of data, since it's the worst case scenario in this case and the worst case has one piece of data per page. o(n/2) = o(n) 🙂

drowsy yacht
#

another approach would be to, when fetching the records, you could add the page it is on in the record itself, then you wouldnt have to compute it

#

you might also be able to do this in a select but i am not sure

potent jacinth
#

yeah faq i think thats the best thing for it

#

i'll leave this question open incase it's possible to have the data.pages object be a map instead of an array.

brave sequoia
#

the data in the cache is always an array, and when you do updates, you have to work with that data structure. We can't store it any differently because there's no guarantee that every page returned from the queryFn is compatible with an object. For example, if we have queryFn: () => Promise.resolve(5), it wouldn't work.

t's o(n) instead of o(1).

as long as operations are linear (which they are), this really doesn't matter. Even if you have 10k pages, iterating over them to make an update of a user interaction is super fast. Yes, you can't do it 100.000 times per second, but optimistic updates aren't done that often. And if you have 10k pages, you need to iterate over them to render them anyway so the bottleneck is not the one place where you need to map over them to perform an update. There's no need to "optimize" for this at all 🙂

#

of course direct object access is a lot faster. But doing the looping over 10k items is like 0.495ms, while doing the direct object access is 0.0090ms. Half a millisecond is plenty fast for doing something once, on a user interaction.

potent jacinth
#

Sorry but being told that "looping over is fast enough" isn't exactly the answer I was looking for. And I may not be dealing with 10k items but may be more like a million in my case. This simply isn't good enough 😦

For example, if we have queryFn: () => Promise.resolve(5) it wouldn't work.

What wouldn't work? Way I see it, it would replace your entire object with the number 5. Yeah that breaks stuff and makes it delicate, but don't I at least get the choice?

It seems like arbitrary hardcoding really. I can do the array approach for now but I will be picking up a client that potentially gets 20, 50, 100k of these entities in and then I'm screwed.

#

Let me pivot and ask because statement that

Data in the cache is always an array

Just worried me - if I set queryClient.setQueryData(["myEntity", 123] and then I want to get that piece of data by passing in myEntity and 123 into the query key - is that direct access??

drowsy yacht
potent jacinth
#

Right, that doesnt answer my question exactly but it is good to know. Let me rephrase: is getQueryData direct access? And so, useQuery, when query keys are defined uniquely?

drowsy yacht
#

queryClient.getQueryData calls .get on the queryCache which calls .get on Map

brave sequoia
#

What wouldn't work? Way I see it, it would replace your entire object with the number 5. Yeah that breaks stuff and makes it delicate, but don't I at least get the choice?

The queryFn is called for every page, so if it just returns a number, the cache structure is:

{ pages: [5, 10, 25, ...], pageParams: [...] }

we can't easily make an object out of the Array 5/10/25. Sure, arrays are just objects with an index as key, so we could do:

{ 0: 5, 1: 10, 2: 25 }

but that is an array. If you know the index, you can also update at the index.

is getQueryData direct access?

yes, because the QueryCache is a Map. However, an infinite query is one cache entry that is chunked up into multiple pages. And those pages are an array.

And I may not be dealing with 10k items but may be more like a million in my case.

I'm not really following. The array we are talking about is the array of pages. Every call to fetchNextPage adds one page to the array. The structure inside the array (= what's on every page) is what you return from the queryFn. How would you get millions of pages?

#

This simply isn't good enough

Still thinking about how to reply to a statement like that 🙄 . Probably, not at all

potent jacinth
#

Sorry I misspoke with the "good enough"; i just meant access time/computational complexity for my purposes, not anyones work or anything

#

Ok i get it now, i think the only way to do it is Mr Ferret's suggestion of popping the page # on the object... or calculating the page the updated object is on, potentially.

#

I may be shoehorning in infinitequery here where it doesn't belong

#

Thx guys, i get it now i think 🙂 ) )

drowsy yacht
#

you'd anyway run into other kinds of issues with that amount of data if you keep it loaded on the client, assuming 10 numbers/booleans and 5 strings of 20 characters each per record, for 10.000.000 records you'd end up with ~2240Mb of data loaded on the client

#

wait, you said 1 million

potent jacinth
#

Nowhere near that big luckily, but i am expecting like 30mb which means i may not want it passed down in props the current way i do it.

drowsy yacht
#

remove a 0, 224mb, which is still quite a lot

#

passing it through props only passes the reference afaik, so you would not copy the data

potent jacinth
#

Yeah, 10 million is out of reach, and the entity you mentioned is a fair chungus-er than mine.

Speaking of zoomer terms, i think my conception for mapped pagination is just sussy beyond among us yo

I don't know what i was thinking. I may flesh out my idea further into some type defs?

potent jacinth
drowsy yacht
#

ah, i see

potent jacinth
#

I do a funny way of SSR where i just populate setQueryData and read it on the client. Currently dont even have pagination set up so interestingly I'm using react query - without the query :-))))))

potent jacinth
#

The docs aren't too too clear on it.

#

Like, I just have a basic paginated query. I pass in the page number and I can optionally pass in the page size. I cannot, for the life of me, find a simple implementation of this wtihout cursors or anything even mroe confusing. And I still can't figure out what pageparams is supposed to be after all this searching.

My question boils down to: How do I have a simple paginated dataset that can start at any page all-the-while I keep track of the page and can access it, E.G. if the first page I access is 15, I would need either a translation function to turn 15 into 0 or the array would need to allow for a [...empty x 14, Array(25)] situation; neither of which I seem to be able to do.

potent jacinth
#

The question that would help me most if answered is:

  • How am I supposed to keep track of what page I'm currently on and thus what page out of data.pages to display? Should I, outside of the useInfiniteQuery call, keep track of the page with state?
  • In this case, am I missing something as it pertains to the usage of pageParams? Do I even need it? Why is there a pageParam array? Is the exact amount of pageParams always equal to the exact amount of pages? Why is it like this in the first place, I think I'm just missing the point and going around in circles like an idiot really 😦