Workers returning cache hit for 500 response without cache headers

[indigo juniper]

I have a remix application deployed to cloudflare pages. We recently had a scenario where some unexpected load was generated against our API which caused an outage. The specifics of this are still being investigated, but as this happened we found that the remix application took longer to recover than the API itself. From what we can see the 500 response from the worker as a result of the api outage had been cached and as such the cached response continued to be returned after the API recovered.

I have attached a screenshot of the response. We cannot see any specific details of the response that would indicate that it should be cached like a cache-control or expires header. We currently have a very rudimentary implementation of the cache api within our worker function which we suspect needs to be updated to avoid this. However I have been unable to find any documentation explaining the condition which is causing it to be cached making it difficult to know what changes exactly are required. Ideally we would just like our cache implementation to match the behaviour of cloudflare when not using workers. I'm not sure how practical that is though. I have attached our worker entry point for reference as well.

[fallen marsh]

Create a cache rule that bypasses cache

Or unproxy the CNAME record on your custom domain

You are double proxying Pages, causing issues like this

[indigo juniper]

Thanks for the response @fallen marsh. How am I double proxying? Our DNS is not managed by cloudflare so the domains are not be proxied via the CDN. This testing has been done directly on a pages.dev domain. Prior to the usage of the cache api within our function, we were adding Cache-Control headers to our responses from our application. It then seemed (due to a lack of cf-cache-status) headers that these were not having any impact. As far as I'm aware a worker runs before the cache and therefore any caching that is required must be done manually in the worker function as we have done.

As a workaround, we have also updated our cache logic to not put a response into cache if the status is > 500. With this change only, we are no longer seeing a cf-cache-status header on this response and the error condition does not persist past the API outage.

So I'm just trying to understand why the cache-headers within the response that is being cached(or lack-thereof) are not being respected. There are some default cache behaviours for http status codes but 500 does not seem to be included according to the documentation.

[fallen marsh]

That shouldn’t set cf-cache-status though