I'm compiling 20k pages, and it's only using a fraction of my CPU | Next.js | Page 1

spark prism Jun 13, 2023, 9:33 PM

#

I'm on the latest version of next.js (13.4?), and I've put together a benchmark to see how next.js would perform on my 20k page website.

I'm running next build, with the export option turned on. It only seems to be maybe a single CPU core. Is there a way I can speed this up further? I haven't added things like CSS or images to my test to optimize yet, so if there is no parallelization currently, I'm worried this will slow down too much.

I need static site generation. ISR isn't an option for me.

tender stratusBOT Jun 13, 2023, 9:33 PM

#

Post created!

🔎 This post has been indexed in our web forum and will be seen by search engines so other users can find it outside Discord

      🕵️ Your user profile is private by default and won't be visible to users outside Discord, if you want to be visible in the web forum you can add the "Public Profile" role in <id:customize>

      ✅ You can mark a message as the answer for your post with `Right click -> Apps -> Mark Solution`
      (if you don't see the option, try refreshing Discord with Ctrl + R)

brittle prawn Jun 13, 2023, 9:39 PM

#

https://github.com/vercel/next.js/blob/d27a4969f61bc1db2f6f56d7b0cf2b73cfa56552/packages/next/src/server/config-shared.ts#L174

GitHub

next.js/packages/next/src/server/config-shared.ts at d27a4969f61bc1...

The React Framework. Contribute to vercel/next.js development by creating an account on GitHub.

#

Check out these options

#

They aren’t documented

#

But there’s an option for experimental.cpus

#

But by default it should try to use all of them…

#

Maybe enable workerThreads

brittle prawn Jun 13, 2023, 10:02 PM

#

One more thing you could do

#

https://stackoverflow.com/questions/48387040/how-do-i-determine-the-correct-max-old-space-size-for-node-js

Stack Overflow

How do I determine the correct "max-old-space-size" for Node.js?

I'm having some trouble to understand how Node.js acts based on the parameter max-old-space-size.
In my case, for example, I'm running two t2.small AWS instances (2GB of RAM).
Not sure why, but I d...

#

Increase the heap size

#

So it utilizes more of your available memory

spark prism Jun 13, 2023, 10:09 PM

#

brittle prawn Maybe enable workerThreads

How can I enable worker threads, or choose settings for webpack?

brittle prawn Jun 13, 2023, 10:13 PM

#

Take a look at the code I linked

#

Those are all the options available in your next config

#

So you would add the corresponding option to your next config

#

To enable it

#

But I think your biggest win here will be increasing the heap size

timid dune Jun 13, 2023, 10:20 PM

#

I was wondering about how are you compiling 20k pages?

spark prism Jun 13, 2023, 10:25 PM

#

So right now, it's just relatively simple .tsx pages. We're working on generating tests that are more complicated.

We're evaluating moving everything over to next.js.

brittle prawn Jun 13, 2023, 10:38 PM

#

You mentioned that ISR isn't an optoin

#

does that include using the "fallback" strategy ?

#

https://nextjs.org/docs/pages/api-reference/functions/get-static-paths#fallback-true

Functions: getStaticPaths

API reference for getStaticPaths. Learn how to fetch data and generate static pages with getStaticPaths.

#

the idea is that you only build a subset of all your popular pages during build-time

#

and the remaining pages will be built on demand when a user navigates to them

#

and then the newly built paged will be served for subsequent requests

spark prism Jun 13, 2023, 10:46 PM

#

How can I choose what pages are built at buildtime?

brittle prawn Jun 13, 2023, 10:49 PM

#

by using the fallback option

#

the ones without fallback enabled will be built

#

you need to define a getStaticPaths function which returns a list of all the page paths

#I'm compiling 20k pages, and it's only using a fraction of my CPU