Gatsby + Payload CMS + Netlify: Missing Content After Publishing

[sterile sentinel]

Hey everyone! 👋

We’re running into an issue where publishing in Payload CMS sometimes results in a partial Gatsby build, causing pages or images to disappear. After some investigation, we found that clearing caches in both Payload and Gatsby in Netlify before rebuilding resolves the issue—but we’re trying to understand the root cause and find a proper fix.

The problems means everytime we publish anything our deploy breaks and we receive a busted production website until we clear all caches and rebuild.

---

Project Setup

• CMS Backend: Payload CMS (deployed on Netlify)
• Website Frontend: GatsbyJS (deployed on Netlify)
• Content Syncing: Using gatsby-source-payload to pull content from Payload CMS into Gatsby
• Storage: S3 storage plugin + RDS/Postgres
• Build Triggers:
  • Publishing in Payload CMS triggers a Gatsby rebuild.
  • Merging to main in our monorepo triggers a Payload and Gatsby rebuild.

---

Issue Description

• After a fresh, clean build of both payload and website, the website is fine—all content loads correctly.
• After publishing a page in Payload CMS, Gatsby does not fetch all content during the next rebuild.
• Missing elements include:
  • Some blog posts disappearing.
  • Some (or all) images not appearing.
  • Occasionally, full pages missing.
• There are no errors in logs

[jaunty burrow]

New Community-Help Thread Created!

Help is on the way! To mark it as solved, use the /solve command. In the meantime, here are some existing threads that may help you:

Documentation:

• Versions
• What is Payload? - Use Cases - Digital Asset Management
• What is Payload? - Use Cases
• Vercel Content Link - How it works
• Vercel Content Link - Getting Started

[sterile sentinel]

What We've Tried

1. Using the repopulate option in gatsby-source-payload → No effect.
2. Clearing Gatsby’s cache (gatsby clean) before rebuilding → Does not restore missing content.
3. Checking Payload’s API manually after publishing → The API still returns all content, meaning Gatsby should be able to fetch everything.
4. Clearing the Netlify cache & triggering a full rebuild → Fixes the issue, but only if Payload’s cache is also cleared.

Key takeaway: It feels like Payload CMS and Gatsby’s incremental build processes are involved, and if they get out of sync immediately after a publish.

It also isnt clear why only partial content is returned - for instance, if we publish a single new page, you'd expect an incremental build would produce just the 1 delta page.
But, we have 40 blog articles, and we might publish 1 new article, and the subsequent build will have only 9 articles built...

---

Questions for the Community

1. Does Payload CMS have a caching mechanism that could be serving stale or partial content during Gatsby’s build?
2. Is there a way to force gatsby-source-payload to always pull the entire dataset, rather than relying on an incremental response?
3. Has anyone experienced a similar issue with Payload’s API and Gatsby builds?
4. Any debugging tips to pinpoint whether the issue is on Payload’s side or Gatsby’s side?

We’d really appreciate any insights or guidance on this! 🚀

Thanks in advance! 🙏

Environment Details:
gatsby-source-payload version: 1.0.0
Gatsby version: 5.13.3
Payload CMS version: 3.3.0
Deployment: Netlify
Storage: S3 & RDS
Hosting: Netlify

[sterile sentinel]

Resolved: Payload CMS + Gatsby Cache Issues in Deploy Previews (Netlify)

Root Cause

• Gatsby’s cache tightly depends on Payload’s cache, and they must remain in sync for builds to work correctly.
• If a deploy preview of the website builds before its corresponding Payload instance is available, it may use an older cache state or fail due to missing data.
• Installing gatsby-source-payload-cms with ^1.0.0 sometimes pulled in 1.0.6, which introduced bugs.

How I Fixed It

1. Ensuring Deploy Previews Are Always Available & Synced

   • Every deploy preview of our website now waits for its corresponding Payload preview to be fully deployed and healthy before building.
   • This is handled via a pre-build health check script (wait-for-payload.sh), which pings PAYLOAD_URL until it returns a 200 OK response.

2. Making Sure Payload Deploy Previews Always Run

   • Even if the payload directory in the monorepo doesn’t change, Netlify should always deploy a fresh preview of Payload.
   • Updated netlify.toml to ensure that every Netlify deploy preview for Payload is forced to run.

3. Fixing repopulate Usage in Payload CMS

   • repopulate must be defined inside collections as a collection object, not at the root

4. Pinning the Correct Version of gatsby-source-payload-cms

   • ^1.0.0 sometimes installed 1.0.6, which had issues. The correct version for us was explicitly pinned to 1.0.0.

5. Strictly Isolating Build Environments

   • Each website instance now builds with its own corresponding Payload instance.
   • Local development uses its own local Payload instance.
   • Payload production should only ever be used to build website production.
   • This ensures that cache state alignment between Gatsby & Payload is never broken

Results

✅ No more cache mismatches between Payload and Gatsby.
✅ Deploy previews work reliably with the correct data.
✅ Production builds are fully isolated, preventing accidental data leaks.

---

Final Takeaway

If you are using Payload CMS with Gatsby on Netlify, make sure that:

• Your Payload deploy preview is available before your website builds and your website uses an isolated instance of Payload
• You always use the right Payload instance for the environment you’re building.
• You pin gatsby-source-payload-cms to a known stable version.
• Your cache state between Payload & Gatsby is always aligned.

Hope this helps someone else! 🚀

[slim viper]

hosting

[last anchor]

Great write up mate, appreciate the attention to detail.