caching server-fetched data

[oak heath]

I'm used to apollo graphql, which has a very strong caching layer that takes results, determines their id and stores them in cache by id. Then, every time an object is retrieved, it updates the cache and then also all components that use that object.

I'm wondering how to do that in QwikCity with server$ calls. I imagine I should have a store that stores retrieved objects, and a hook that uses the context to retrieve objects. Right?

How can I do GC? is there a a way to determine if a signal/store object has no more listeners?

[hallow halo]

@oak heath I actually wish to use apollo in the future again with qwik. It's a nice abstraction layer.

Depending on you purpose, have you checked out dataloaders? They are actually a graphql thing, but we use them in our qwik environment. You can add a TTL on every single data type and they are a good interface for backend loads

[narrow crater]

i was thinking about implementing lru cache within my graphql layer that basically just fetches.. but i have too little requests to think about that yet.

[oak heath]

I'm actually in the thinking about things phase. Right now my server and client use graphql to communicate, which I find awesome for various reasons, but server$ calls are way better DX, and I think that with some care you can still have very maintainable code.

So I'm actually considering having an internal API only and dropping graphql (when I don't need to provide an API for others).

Imagine basically subscribing to objects in the client, and the server sending updates, and signals doing the heavy lifting on the client. You can do this with graphql too of course, but I think that it will be easier to write without those abstractions in the way.

[hallow halo]

@oak heath got it. Right now, server$ has a disadvantage. You can't abort it, and it's not 100% async. Not sure if manu has released worker$ yet, but we relied heavily on server$ and refactored it again.

[glossy lake]

I’m using Apollo and I had issue with its in memory caching 😦

[hallow halo]

Do you have custom cache policies? Doing Apollo in the backend is a complete different beast. I would put ttls on everything, because every user gets the cache of another user. With custom policies you can fail hard easy.

[oak heath]

users shouldn't get caches of other users, that means you're sharing apollo client instances between SSR runs

[oak heath]

What do you mean with it not being 100% async? And I haven't done any request aborting in apollo so far, so should I really worry about that?

[hallow halo]

If you don’t need aborting, than no.

I just wanted to point out , that it can be possible, in some cases even good, if the data is the same for all users.
I’m not 100% sure, but we were fetching product relations in the frontend and navigation was kinda waiting for the server$ to end.

Now when I think about, it was probably our fault.

[narrow crater]

Wouldn't setting stale-while-revalidate on that endpoint already help a lot with requests that fetch public data? I've been using this approach on a nextjs site and configured the LB in front of everything to cache according to SWR headers received, which helped a lot without having to bother with handling cache in the app itself.

I actually tried this earlier and it works on dev, as soon as express or fastify serves, i get response header issues tho. Looks like setting headers in a server$() causes anything other than the dev server to feel unhappy.

[oak heath]

You're mixing server$ with endpoint?

[narrow crater]

Isn't server$() essentially an internal endpoint, albeit a "transparent" one? The docs say "A server$() wraps a function and returns an async proxy to the function. On the server, the proxy function directly calls the wrapped function, and a HTTP endpoint is automatically created by the server$() function." I mean, based on that description and the availability of the request object etc it "feels" like an endpoint. Unless I am getting something completely wrong. Which of course is most likely 😉

[glossy lake]

Actually I was looking for examples of how to set up such cache policy for server side but couldn’t find anything. You are right also about users can get cache of other users. Do you have some best practices to share?

[glossy lake]

Do you suggest that I should create Apollo client instance on every request and let server$ or routeLoader$ handle the caching part?

[oak heath]

yes correct, each request should get its own apollo client. In server$ calls you'll also need to do this. You can perhaps put it on the request object itself so that if it's already there you don't create a new one

[glossy lake]

Thanks a lot!! Does it also mean that, we need to rely on server$ caching like we have discussion here?

[oak heath]

I don't think server$ does any caching at all? You should cache on the client side if you want to avoid server calls. Not sure actually

@hallow halo btw server$ supports abortSignals now https://qwik.builder.io/docs/server$/#server

[hallow halo]

This is why we used resource again. Turned on cachecontrol and the rest is up to the browser

[hallow halo]

😳😮

Qwik is getting better and better.

[glossy lake]

Would be nice if we have some examples in Qwik examples🙏 I was also thinking about thing that server$ can somehow stream the response at a later phase like you said @oak heath, but I didn’t manage to do that, especially when I wanted to do it in initial rendering

[oak heath]

@glossy lake the core team promised something like that before 1.0 but I don't think they ever got around to it. It would require the client to have JS enabled in any case.
AFAIK, during SSR all promises are awaited until completion.
You /can/ have server$ return an async iterator though, so you can get updates pushed to you

[hallow halo]

If you want to disable cache, you don't have to initialize the client on each request. You can also turn of cache:

 const defaultOptions: DefaultOptions = {
      watchQuery: {
        fetchPolicy: 'no-cache',
        errorPolicy: 'ignore'
      },
      query: {
        fetchPolicy: 'no-cache',
        errorPolicy: 'all'
      }
    };
const client = new ApolloClient({
      link,
      cache: new InMemoryCache(),
      defaultOptions
    });

This way, you are able to cache, if you need it.

[oak heath]

yes but I do want cache during SSR so it doesn't do the same query twice

I don't think there's much overhead in having a client per SSR. My link is the node app itself so I don't need to have TCP connections opened though. But even then it's safer to do a connection per SSR so you don't accidentally mix permissions

[hallow halo]

@oak heath sounds good. When I tested apollo the last time it wasn't working. How and when do you create the client? I'm setting up code-generation now and looking forward to extend graphql-codegen later.
I would love to have something like useUserQuery autogenerated.

[oak heath]

Right now I'm not using Apollo in Qwik, and I think it may stay that way because of server$ 🤔

[hallow halo]

I meant how do you create apollo. It's not serializable. So you create it in a plugin@apollo or use a function to get the instance?

[oak heath]

ah - in my React apps I create it when the SSR happens and then inject it into the app with Provider. I also had one app that had to do graphql requests itself as part of the resolvers (sort of proxying), and for that I added the client to the graphl context when the request comes in (although now I realize I should do that lazily since not all requests need the client)

[glossy lake]

Create and provide it to context would allow components to use it but if we want to use in request handlers, we always have to create an instance everytime right? And use sharedMap to reuse the instance if we want to share between request handlers?

there is one problem with Apollo client that I encountered was, build production fails because the package isn’t fully compatible with esm build target

[oak heath]

urgh. are you on latest version?

[glossy lake]

Yes, and my workaround was I need to explicitly define the import paths, there is an issue here https://github.com/apollographql/apollo-feature-requests/issues/287 Looking at Apollo client repo, they don’t seem to care about maintaining such thing so I plan to switch to urql

[hallow halo]

But you export it as singleton in your app? I have a factory function and it works even in production.

A big but. We only use it Server side.

Wait, let me test it again, maybe I talk BS 😄

[glossy lake]

currently I'm exporting it as singleton, but @oak heath recommended to create an instance per request handler, so singleton shouldn't be used. I read somewhere also that, use singleton might run into side effect so general recommendation I found online is avoiding singleton

[hallow halo]

singleton exports in qwik is not pretty

we did this with redis as well

and it failed

looks like you are right

it fails in the last part of the build process

so you imported * to fix it?

We tried urql as well. It has a weird peer dependency with react 😄

I feel like apollo and qwik are not friends 😄

Maybe I need to write a graphql-codegen plugin using fetch with useLocation

[glossy lake]

this is how I workaround it. I think giving uqrl a try and I believe it will be better. Apollo client nowadays is like, more advertising than actual open source focus. urql they said we only need the core package. The core package shouldn't depend on react

[hallow halo]

@glossy lake yes, I started hating the guys, the first time I used there getting-started in swift. In one app, I still don't get the generation included in build files 😄

I will also checklout urql

checkout this:
https://the-guild.dev/graphql/codegen/plugins/typescript/typescript-urql

this is sweet

[glossy lake]

that is beautiful ❤️

[glossy lake]

Btw, I saw this link in another help post https://github.com/vendure-ecommerce/storefront-qwik-starter probably useful resource. This one uses gql as well as plugin to generate codes like your link.

[hallow halo]

Yes I have the same setup, but I wanted to have something better. Qwik puts a spoke in my wheel with serialization:

export function useQuery<
  Data = any,
  Variables extends AnyVariables = AnyVariables
>(
  query: DocumentInput<Data, Variables>,
  variables: Variables,
  clientFactory: QRL<() => Promise<Client>>
)

This was my idea. If serialisazion would not be a problem you could just use it like this:

const test = usePrivateQuery(StaticPage, {
    path: 'service/agb',
});

And the variables would be inferred by the document

the privateQuery is a hook without the clientParameter

we have multiple graphql server and can't put them in federation, because there are public and private ones 😦

Apparently the whole graphql stack DocumentNodes etc are not serializable

puhh 🙃

[glossy lake]

if it's not serializable, does it mean that if we use the hook inside component, either the component will crash or the DocumentNodes is not lazy loaded together with the component?

[hallow halo]

It’s crying about the DocumentNodes if you use gql from the urql package. 🫨

Inside of qwik hooks

I will try again with server$ tomorrow.

[glossy lake]

I didn't get serialization error. One thing is not so clear to me yet is, should we also create different instance of graphql client even when we use inside component?

[hallow halo]

morning

you are not using useTask or useResource. My idea was that useQuery returns ResourceReturn to have automated loading states and you can use it everywhere in qwik

@glossy lake how do you work with the promise you are getting back?

[glossy lake]

Ah I used useResource$ too, I put the promise into Resource value and it didn’t throw any errors

[hallow halo]

and how you track variable changes?

this is my problem right now, tracking the query and variables 😄

I will checkout if I can make a string out of the document node somehow

[glossy lake]

Ah, if tracking the query, it will throw errors definitely, I wasn’t aware that you had track

Or we can use noSerialize, but I guess that is the last resort

Another idea is tracking query as string, which means we don’t use gql tag template but just track on the string query instead

[hallow halo]

but noSerialize is crying for me as well 😄

but now I have fun with our web firewall

I hate WAFs so much

[glossy lake]

It looks to me like, the concept of Qwik encourages us to only put simple thing like reactivity state inside components, everything else goes out, like api interaction goes to route loader or server$ 🤔

[hallow halo]

but even with server$ you have to load it somewhere

and if the input of your server$ is tracked in a useResource locally, you will have the same issues

For me everything works now, but I get the serialization error

I guess you can use a routeLoader and be fine, but for my matter, I can't use the routeLoader

or at least not now

ok with no serialize in a useResource the query is of course empty on change 😄

[glossy lake]

urql doc said we can use query/mutation as a normal string. If we use query/mutation as normal string, we can track right? track only has problem with when we convert query/mutation string to DocumentNode via gql

[hallow halo]

yes, but I wanted TypedDocumentNode. Otherwise, I can't infer return types. We need to be so type safe that it hurts 😉

[glossy lake]

@hallow halo - do we also need to create instance of graphql client whenever we use hooks? Were you successful in trying out typed document node with server$?

[hallow halo]

Right now I have a semi typed solution, but I had no time, we needed to deploy a new react app until tomorrow. How is it going for you? My plan is now to only use Apollo in the backend with just a normal function building the client in it and use routeLoaders

[glossy lake]

Yes I am also doing similarly. I don’t have the need of tracking query or variable change yet so route loaders or server$ are enough for me to continue. The only thing left is caching which I haven’t figured out yet. For now I will just do gql request without cache and figure out later

[narrow crater]

Hmm I think i got serverside in-memory cache working with typed document nodes, authenticated requests and token refresh with retry. Not sure if it would be sanctioned by the gods of qwik but it kind of works(tm). I can try to put together a gist if someone wants to checkout the code. it's using just regular undici fetch and node-cache and generated types courtesy of graphql-codegen. each query has it's own server$ function, so that's a bit of overhead, but it allows finegrained control over what should be cached and theoretically how long. plus, i need to augment some returnresults with secondary data pulled from somewhere else, so I need some sort of middleware approach, if it can be called that.

[glossy lake]

would you pls do that? I would really appreciate!

[narrow crater]

https://gist.github.com/genox/b482cb806df1ab1abf4f136cd1468619 - hope it helps. Let me know if you have questions.

i've had issues with components nested under <Resource/> components (state stuff) so I opted out of that and use signals to push data into components. There is probably a nicer way of doing it but so far it works. The added benefit is that the same cache mechanism can be used to cache other requests as well, like in my case, there is sanity.io content in addition to the whole Saleor API that is being consumed. And its lighthning fast once the cache is warm, really nice. Probably allows me to drop the whole nginx caching stuff that was my backup plan.

Ah and.. well, I am oldschool and run things in a docker container with express.

So I have no idea if this works on serverless deployments

[glossy lake]

I was reading about caching gql with cdn recently for serverless, not sure how to do it and I couldn’t find examples on google either. For me, it looks pretty advanced and I don’t have much experience with serverless so I’m not sure if caching in memory would work. I will try your solution on my project since I’m using serverless there

[hallow halo]

You can just point a cdn to routeLoaders and use cache control

[glossy lake]

I found this guide https://graphql.guide/background/cdn/ it seems like, the communication between cdn and backend api is done somewhere outside of frontend area

[glossy flax]

Hey, I've been thinking about this question too. I used apollo-graphql in the past as well and more recently tanstack-query. I think it's similar to Next.js' SWR as well.

Out of curiosity, what benefits do you see in using something like that with Qwik?

My understanding is that those solutions are all client-side fetch/server-state. They not only provide caches that help prevent over-fetching, but also things like loading states, optimistic updates, pagination, deduping requests and more.

But the key thing to consider here imo is that those are client-side fetch/server-state solutions. So I am wondering if these are actually desirable for SSR Qwik applications. Aren't you guys using SSR? Cause with route loaders, you get better performance (thanks to SSR), you don't have to worry about loading states, and it doesn't seem too hard to implement optimistic updates, debounced inputs, and pagination. On the otherhand I'm not sure about deduping requests, and more importantly, I'm not sure about caching. I've just discovered cache-control (https://qwik.builder.io/docs/caching/#caching-responses), but I'm not sure it can do everything that tanstack query can.

I'd love to get some feedback on this. Perhaps these client-side libraries aren't the right way to think about SSR state/cache. Perhaps that is why tanstack is now working on tanstack loaders and tanstack actions (https://tanstack.com/) ?

[narrow crater]

It depends on what exactly you want to improve with caching and in case of backend requests, wether they happen in the browser or the server. Another thing to consider is wether a cache should be available for all visitors. In my case, api/database requests are designed to be always serverside and cache is public (authenticated requests are excluded from caching altogether). So for me, an in memory cache on the server is just fine and isnt complicated to implement. stale-while-revalidate without a proxy in between that honors the Cache-Control http header accordingly will result in caching in the browser only, so every session needs to cache first and only then relies on locally cached data on subsequent requests. That itself is a problem if your content is short-lived or user generated. A cache using simple fetch() requests and in memory cache with keys hashed by query and variables has the added benefit of automatically deduping identical requests - so no deduping logic needed. In my case, it is also important to save on requests to the backend because that metric is directly translated into service cost, so it makes sense to have as little requests as possible.

On the other hand, if all your data is technically static anyways and you dont want to SSG, you can use something like nginx or caddy and set stale-while-revalidate on this proxy and cache there.

I tested a bit in the recent days and although this is only valid for my environment, it gives some idea: caching on nginx results in max request/transfer time of about 180ms, caching requests in memory is about 200ms, so the render cycle of qwik really isn't the bottleneck

that includes my ping which is about 20-40ms to the loadbalancer

i decided to only cache backend requests to APIs my app consumes in memory because it is simpler to manage (think flushing - you want that option too, hard to to on nginx without crutches). on that small project, i max out at under 1mb for a bit of json in memory but that is nothing compared to the expense of waiting for 10 requests to finish on two different backends onfor each page render cycle - depending on context. Without cache, rendering the html of the page easily pushes 500ms or more. And that is without transfer time of static assets.

An important thing to note about qwik caching (essentially setting Cache-Control headers on the response) is that qwik doesnt provide the facilities to cache those requests itself. Each SWR response is only cached in the client that requested it, unless - as i mentioned earlier - there is another instance in between that takes care of actually persisting those responses in accordance to the Cache-Control headers. At least from what i was able to discover. Otherwise, qwik would have to provide the measn to invalidate the caches.

That itself makes it pretty much useless to think of as "cache" by the common understanding of what we mean when we say we want to cache stuff

Like this I can assume that a browser always has fresh data and the only place where i need to worry about when updating content is the running qwik instance. On there, I provide an api route protected with a secret that acts as a webhook for the services that receive content updates and flushes the entire cache. this could be extended to only flush certain routes or queries for example, but that makes the cache key logic more complicated.. it's all a game of compromises..

In case of graphql or rest - a library like apollo or others only makes sense if the client is involved in caching. with Qwik being focused on explicitely doing as little as possible in the client, the only thing that makes sense is to cache serverside. it just adds overhead and let's say, if you generate types with grapqhl-codegen and write your own queries anyways, you might as well just use fetch() on the server.

[glossy flax]

Yeah I have the feeling that apollo-client, tanstack query or Next.js' SWR are not the right fit for SSR first apps like Qwik or even Remix and Next.js with the app router. But I'd love to read some counter arguments from people that think otherwise!

[glossy flax]

is this true with Qwik adapters?

From MDN (https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control):

The Cache-Control HTTP header field holds directives (instructions) — in both requests and responses — that control caching in browsers and shared caches (e.g. Proxies, CDNs).

So when deploying with the Vercel Edge adapter, and from what I've seen, I don't think that you need to setup anything like a reverse-proxy to make use of the cache control headers. Indeed, Vercel Edge is basically an improved CDN.
From Vercel (https://vercel.com/docs/concepts/edge-network/overview):

Our Edge Network is both a Content Delivery Network (CDN) and a globally distributed platform for running compute at the edge.

Don't other adapters like the Node adapter help with this problem?

[hallow halo]

you can add a plugin and set a default cache-header. Create a route [email protected] and implement only onRequest

I would not call qwik a ssr first framework. It's neither SPA or MPA (SSR). It's both

SSG is also possible

[glossy flax]

Qwik in SPA mode can still be used with route loaders, so it's not SPA !== SSR

But anyway 🙈

[tawny mist]

I have a working Qwik tanstack integration in my local machine. I would like to hear form you what are the kille features of this library

[glossy lake]

Does it include in memory cache @tawny mist? The main discussion we have here is about caching on server side and we are looking for some best practices in general to deal with fetching data in server, not particular to any libs

[glossy lake]

Btw I found this blog post which is quite useful https://blog.logrocket.com/http-caching-graphql/#:~:text=This solution exists only because,run on the client side.

[tawny mist]

Yep but I would like to implement the best possible solution with routeLoaders

Do you have any advise?

[glossy lake]

For now idk either 😅 still need to experiment and try out. There is solution of in memory cache and http cache control. The in memory cache already looks promising. I will try out http cache control to see how it goes. I think using routeLoaders or server$ is not really the concern and I’m more interested in how to cache properly.

[glossy lake]

One thing I just found via digging google was: if the app is deployed to a serverless env, we need external storage like Redis for persisted queries instead of in memory cache

[oak heath]

Depends, if you're fine with requesting something a few times, keeping in memory should be ok for edge, no?

[glossy lake]

I agree, probably external storage is needed for long persisted stuffs