The compiler ran out of memory

[vast fog]

I'm working on a rather large document in the Typst web app, and after several pages I get the error ‘The compiler ran out of memory’. Some pages compile, but others don't, and I can't export the PDF because I get the same error.

Any way to solve it?

Thanks!

[zenith wyvern]

would seem weird that the compiler runs out of memory after just a few pages, any way you can share the document (only if not private ofc) or a similar small example?

other than that, if you have any show rules or so, try temporarily removing them and see if that resolves the problem, if so perhaps you have some kind of recursive show rule

[fresh sonnet]

IIUC the document is large and it does compile ("Some pages compile") successfully but then runs out of memory after rendering a few pages.

[vast fog]

Here is a link to a copy of the project (sorry, it's in Spanish, but I don't think it's relevant).

It's more than 100 slides using Polylux, maybe too many images, any way to optimize it?

(link deleted) (please, ask if you need it)

[zenith wyvern]

okay so the preview works, but PDF export fails

but once you try to export it, the preview then doesn't work anymore either

[nimble knot]

somehow my polylux projects always end up being way more expensive than "articles"

[zenith wyvern]

yeah it's possible the way polylux works is not great for memory 😄 but I'm not sure

[nimble knot]

I'm not sure whether it's polylux itself or a disproportionate usage of cetz/fletcher on slides

[vast fog]

I've tried commenting slides until it stops crashing (about halfway through the document).

So it seems. I'll have to compile it locally for now.

[fresh sonnet]

Yeah, compiling local is the easiest fix for now

[zenith wyvern]

(btw tho I think you with the link you shared others have write permissions. maybe you should delete the link and only share it with read, since this is a public server)

[fresh sonnet]

We really need to optimize memory usage soon-ish, it hasn't been a focus area so far.

[vast fog]

No problem, that's a copy of the original project.

I'll change it anyways

[fresh sonnet]

I don't think cetz/fletcher is actually used

[nimble knot]

ah yes

[vast fog]

No, it's not

[zenith wyvern]

is it possible that it somehow memoizes more stuff on the web version? @fresh sonnet

[fresh sonnet]

I saw your contact form submission. Since we're discussing it here now, I won't answer there anymore, just so you know.

[vast fog]

Fine, no problem 👍

[fresh sonnet]

why?

[zenith wyvern]

or is it just that the wasm memory limit is lower?

just wondering because I dont seem to have problems compiling it locally at all

[fresh sonnet]

yes, WASM is always limited to 4GB

it's 32 bit

that's why we can all perfectly reproduce it

4GB should be plenty if we wouldn't memoize everything left and right

[vast fog]

yep, local compiling works fine for me. Seems like a web app limitation ?

[zenith wyvern]

okay yeah so this is what I get if I run watch locally

but good that the document can be compiled at least 😄 haha

[fresh sonnet]

yeah, the web app uses some extra memory for rendering etc. so it is just enough for compile, but then it goes OOM.

[frosty flare]

Does comemo use compression at all? I assume a lot of what's being stored has massive overlaps.

Even when memoizing everything it seems like you could claw back some memory that way

[fresh sonnet]

It stores in-memory data structures, not serialized data, so it cannot transparently compress anything. There is probably quite a bit of duplication, but many things are also shared through reference-counted pointers.

[frosty flare]

I wonder if memory bandwidth is a bottleneck at all

https://crates.io/crates/sucds https://crates.io/crates/sux these seem interesting, but presumably not useful for typst

[burnt shard]

Unfortunately if it stores hashes, hash of arc of T is equal to hash of T

Doesn't it store hash to compare equality?

[fresh sonnet]

I'm not sure I understand what you mean. Why is it unfortunate?

[burnt shard]

Because it's storing the same hash again and again when I'd hope for compression?