Preferred Redis Setup?

[weary grove]

Any gotchas/reccomendations/opinons?

On my project I'm going to probably need to cache some responses on high traffic pages. I've never really done this kind of thing before on any project and could just use some feedback.

Things I've been torn between as a newb

1. Should I use upstash?
2. If I do use upstash do I connect with their http library or just node-redis?
3. If I don't use upstash what should I do instead?

[restive basalt]

Try cloudflare worker KV?
https://developers.cloudflare.com/workers/learning/how-kv-works/

100K requests/day are free and $5/month for 1M requests/day

Here's a demo. I used it recently with Remix.

https://github.com/abhagsain/read-changelogs

[weary grove]

This is an alternative to using fly.io right?

[restive basalt]

it's an alternative to redis.

But I'd say try cdn and browser caching first.

these are two awesome videos by ryan
https://www.youtube.com/watch?v=bfLFHp7Sbkg

https://www.youtube.com/watch?v=3XkU_DXcgl0&t=411s

[weary grove]

I saw those they are amazing

I think my biggest concern doing things that way is that I'm accidentally going to mess something up and the result will be a ton of people having to reset cookies just to see the updated content

[restive basalt]

Yeah, cache invalidation is hard.
But if you add short duration for the browser cache and longer for CDN then that should work just fine.

For CF Worker KV you can binge-watch this free 30m egghead course
https://egghead.io/courses/cache-supabase-data-at-the-edge-with-cloudflare-workers-and-kv-storage-883c7959
then you decide for yourself which route you want to with

[weary grove]

@restive basalt assuming I'm not ready to make the jump to cloudflare just yet I have a couple of questions

typically where does the CDN actually get controlled from? Like if I'm hooked up with Fly.io is that something I configure there?

[restive basalt]

It's controlled from the CDN provider that you use, AWS Cloudfront, Cloudflare or Bunny CDN etc.

High level overview is, you'd want your domain name to route to CDN first and then CDN will check for a given content if it already have a copy, if yes it will return the copy if not it will make a request to your remix server hosted on fly.io and then store it and return it.

Now based on your TTL on your Cache-Control header CDN will cache the resource for that duration.

"Cache-Control": `public, max-age=${BROWSER_CACHE_TTL}, s-maxage=${CDN_CACHE_TTL}`,

BROWSER_CACHE_TTL should be min value so users don't see the stale content.

And you can always invalidate cache from the CDN from their UI or API.

---

It's easy to do miss something with cache stuff so I'd say try it on a dummy project at first. Then use it with your prod.
Or if you don't feel comfortable yet use the Worker KV. Don't need to deal with all of these steps.

apologies if I over explained or under explained

[weary grove]

No this is perfect I really appreciate the explanation, it's been feeling like I have some gaps between the sandboxed "How do you optimize CDN / What are the caching strategies" and "How do all these things actually connect to each other"

so your advice is helping fill in those gaps a bunch

So what I need to do basically is instead of routing my domain name straight to my Fly app, I should probably route it to something like Cloudfront or another CDN first

then I can configure the stuff @elfin sleet talked about in those videos on whatever CDN I choose?

[restive basalt]

✅ Correct, but I'd suggest you to migrate your entire domain name to Cloudflare itself. (Yes, it's a thing, if you didn't know. You'd just need to switch 2 nameservers, create Cloudflare account, they will guide from there or let me know.

Then you can manage your DNS, caching, everything from Cloudflare, much better DX.

Later on if you decide to use Worker KV you can have a sub domain for your worker such as api.jonhigger.com

Oh, I read Cloudfront as Cloudflare. Please ignore the above msg.
but yes you're correct

[weary grove]

This is super helpful. I'm going to research if there's something similar on fly, not saying it's better or anything I'm just already on there and going to be live on a real product pretty soon here so I'm not sure it's the right time for me to switch

But I'm definitely floating the idea around

[restive basalt]

Sorry, I think you misunderstood maybe because of my poor wording

but after you move your domain to Cloudflare you'd still be using fly.io as your server.
Cloudflare isn't a server (although they have similar service to called Cloudflare Pages) but you don't have to use those.

Anyways, if you decide to use Cloudflare as a CDN, you'd still be moving your domain over so I don't know why I mentioned it in the first place.

[weary grove]

Either way I'm learning a ton from the conversation so I appreciate you

[restive basalt]

Happy to help 😊
Feel free to ask if you need any further info. I'm not an expert on this but we'll figure something out.

[weary grove]

@restive basalt can a redis cache be considered a type of CDN

I'm reading this fly.io article https://fly.io/blog/last-mile-redis/

and they kind of make it seem that way

[silver cairn]

Upstash is easy but doesn't work with BullMQ so I ended up rolling my own on fly

[weary grove]

@silver cairn if you were using upstash, would you choose to use their http library, or would you use something more like 'node-redis' or just the 'redis' package

upstash reccomends the http library for serverless, but since fly is edge and its not really serverless I'm not sure where it would fall on that reccomendation

[silver cairn]

I followed the lead from https://github.com/remix-run/remix/tree/main/examples/bullmq-task-queue so I'm using ioredis

[weary grove]

strange it's not showing up for me

OK I'll have to research the different packages a bit more in depth

[restive basalt]

I don't think so, Other than caching your page content, CDN can cache other resources such as css, js, images and even add compression.
So can't call Redis a CDN.

But if you're unsure should you choose CDN cache or API cache (redis) it really depends on what you're trying to optimise.

For eg, let's say you have blog page built with Remix and you're using a CMS such as Wordpress or Sanity. Once you have published a blog, it highly unlikely that you'll make any new changes other than a few edits. So it's best to cache your blog page with browser and CDN cache and not the cache API request for the CMS. (Although you can completely do it)

On the other hand let's say you maintain a hotel management SaaS. As you onboard a new hotel, as part of the onboarding you save, config for different rooms such as total no of rooms and other room info.
It's best to cache this config since it's something that's hardly going to change ever as a hotel will have a fixed set of rooms. (I work as a FE dev in such company that's why this eg lol)