Dev performance

[broken warren]

We're moving our website over to astro soon (ish). Our current website has been up for about 12 years, and has about 230 blog posts. I "exported" all of them from wordpress into mdx files. They're mostly not very big, and once up and running, it's fine, but starting up the dev server takes some time, and running a build on github CI takes about 30 minutes. Livable, but wondering if there are some performance tips. Using AstroWind as the base theme (tailwind css), astro 2.4.5.

[cursive fulcrum]

Still waiting for an answer?

It looks like no-one has responded to your question yet. People might not be available right now or don’t know how to answer your question. Want an answer while you wait? Try asking our experimental bot in #1095492539085230272.

[grim drum]

Hey kpgalligan! Thanks for the benchmarks here. Off the top of my head, I know our docs team increases the Node memory limit for CI to avoid throttling which may help. 30 minutes definitely sounds high for 230 posts

[uneven quartz]

Curious if you’ve tried saving them as .md files instead. I’m guessing if they come from WordPress they don’t include component imports?

[broken warren]

30 minutes is definitely because it's GitHub Actions which can be quite slow. We actually have a self-hosted runner for our main work, which is crazy native mobile stuff that runs for hours. I'll move the website build over there is 30 minutes gets to be too long. The more involved issue is local dev, but for now I just moved "old" blog posts into a different source folder, and only add them to the full site when building on CI. Local dev doesn't need to have all of the blog posts available (they're only adding new blog posts, so posts from 2015 don't help much).

[broken warren]

"Import" is a stretch. I pulled the rendered HTML from the API, put it through an HTML to Markdown library, then parse that to do some "magic". The MDX is because we have components for our blog that I want to use. It would be difficult to move strictly to MD for that reason. Not really related to Wordpress specifically.

[uneven quartz]

Gotcha. Makes sense. 30 minutes is still very slow, so I’m curious why. We build the Astro Docs in ~6 minutes in a GH action and that has something like 2400 pages generated from at least 500 MDX files.

[misty lynx]

starting up the dev server takes some time

I had a similar situation that may be similar to yours, i documented it a while back:
#general message
#general message

Basically: I discovered that VS Code was taking a while to load for all my projects, Astro, Vue/Nuxt, etc. VS Code relies on a .cache file. Once I deleted it, the dev server was back to loading lightning quick and my slowness went away.

I hope this helps your slow dev server 👍

[broken warren]

Thanks, I'll give it a shot!

[broken warren]

I am curious if it's just something weird in the AstroWind template. Like if the page script does something computationally bad (big O comp sci stuff I mean). I haven't dug into that much. I don't quite have my head wrapped around how collections work yet. Querying yes, but the whole [...blog] style folder/file name thing. In any case, we're at a "good enough" point for our needs now. I'll dive into that when I get some time. I assume 230-ish posts is nothing for a production environment, so there's got to be something weird happening.

[uneven quartz]

Ah, one thing to check could be the Astro version. We’ve definitely been improving performance with each release, so might also be you’re on an older release?

[grim drum]

Seeing 2.4.5 in the original message 😕

[uneven quartz]

Ah, missed that — yup, then that’s not the issue!

[broken warren]

OK. I just watched the build while it ran, which should've been step one, and essentially it's using svg icons from https://tabler.io/docs/icons/react. That's cool, except while building it does some kind of "transforming" on all JS, and tabler icons is 4000+ JS files. It seems intent on processing all of them. I feel like that's a "tomorrow" kind of thing to keep digging into, but if anybody's got a quick answer, I'd be interested. I think I'll just grab the icons we're actually using and add them directly.

The actual processing of the mdx files was quite fast. There are other things that take time, but the icons seem to really drag.

[broken warren]

Well, need to run a few more times (some builds with "everything" were less, lowest 22 minutes), but very roughly, removing just that tabler-react library seems to cut everything about in half (or more). The AstroWind template uses tabler icons in astro files, but I added the react version for a couple react files we just moved over. 2 icons, to be exact. 15+ minutes for 2 whole icons. Anyway, the bulk now is images in assets (not the experimental one)

optimizing 187 images in batches of 2 
12:29:29 AM [build] Waiting for the @astrojs/image integration...
Completed in 524.60s.

Not sure I'm going to worry about that for now, though. Each post has a hero image in assets, so "optimizing" that would be a different effort, and a fair bit of the template uses them in the site itself.

[misty lynx]

that's really weird!

I use Astrowind on two sites, and it uses https://github.com/natemoo-re/astro-icon package 🤔 which gets its icons from https://icones.js.org/ 🤔

maybe i'm misunderstanding something here.

[daring elbow]

Thanks for figuring that out. Rollup (the JS bundler we use) really struggles with a lot of JS files. Vite (the higher-level bundler we use) is experimenting some stuff to optimize this later, so hopefully soon this can be fixed.

Or if you’d like to try, you can check out https://vitejs.dev/config/dep-optimization-options.html#optimizedeps-disabled (the warning box) which you can configure that in the Astro config’s vite option.

[broken warren]

It's because I added the react version of that on top, which adds a JS file for each icon. The JS files took all that time. We're only using 2 of the react icons, so it's a huge waste. I removed that library and will just figure out adding the icons directly, or converting the components using it to astro. We were porting from a docusaurus site, so everything was react. I got a little "pragmatic" (I.E. lazy)

[broken warren]

I think I'm just removing the library, but I'm curious to see if optimizeDeps.disabled does anything.

[broken warren]

Tried experimental assets, and that cut it in half again. First try was 7.5 minutes.

[misty lynx]

Big time savings 😄

[broken warren]

It's pretty wild. I added the svg icons directly, and the move to experimental assets seems to be working fine, so merging to main and calling that "done". Now just need to get the "content people" to actually figure out what the non-blog part of the site should say.