Google Search Console Soft 404 issues (Vercel Nextjs 15)

[upbeat pulsar]

Hi everyone, I've been deaing with a major indexing issue using the latest versions of Nextjs (15.1.6), React 19.0.0 as well as the PayloadCMS integrated in my site hosted on Vercel (for reference https://finlywealth.com/).

This issue is practically impossible to reproduce on your own. I've tried almost everything you can think of, like the Chrome Canary with disabling javascript, changing user agent to Googlebot Phone, disabling caching, etc.

I've even bought a license to ScreamingFrog to crawl my entire site with the "googlebot" user agent, and so far everything works just fine. I ended up setting up Sentry on our Nextjs website to be able to narrow things down to what errors the googlebot crawler might be encountering, and pretty much it all comes down to this:

Error: Connection closed.
  at t(./node_modules/.pnpm/next@15.1.6_@babel+core@7.24.5_@opentelemetry+api@1.9.0_react-dom@19.0.0_react@19.0.0__react@19.0.0_sass@1.77.4/node_modules/next/dist/compiled/react-server-dom-webpack/cjs/react-server-dom-webpack-client.browser.production.js:1498:1)

Here's a publicly sharable link on Sentry:
https://finlywealth.sentry.io/share/issue/91884b73180244a7bfbfac76bf10f4c6/

At this point, I feel lost. I've been trying to figure out what I'm doing wrong on our website for 2 weeks now and starting to lose hope. The soft 404 issue is persistent on googlebot for most pages, even though a lot of these pages are static content. I'm using ISR and SSG for producing a lot of these pages, as well as integrated unstable_cache for DB calls via PayloadCMS to fetch data from our MongoDB Cloud.

I've even increased the maxDuration flag on all our pages to 60 as well as the settings on Vercel for functions durations to be 45. I've even boosted our CPU utilization to max on Vercel, hoping to get this resolved but still not successful.

Anyone have any debugging tips? Really anything to help at this point would be super appreciated.

[leaden frigate]

Post created!

🔎 This post has been indexed in our web forum and will be seen by search engines so other users can find it outside Discord

🕵️ Your user profile is private by default and won't be visible to users outside Discord, if you want to be visible in the web forum you can add the "Public Profile" role in id:customize

✅ You can mark a message as the answer for your post with Right click -> Apps -> Mark Solution
(if you don't see the option, try refreshing Discord with Ctrl + R)

[upbeat pulsar]

Here's another screenshot of the Sentry Googlebot issue:

[upbeat pulsar]

Google Search Console Soft 404 issues (Vercel Nextjs 15)

[idle gulch]

Hey, seems as not many have looked into this. we have the same issue(https://github.com/vercel/next.js/discussions/78288) and havn't find any solution for it

ping @light ruin