jolly tendon
For details and more statistics, you may go to #cpu-clinic(Dec. 15th)
I encountered the same issue of heap memory leak. For me, 300 creeps, 32 colonies and 55 rooms visioned. I use ~35%heap after a halt, and after about 40k ticks the heap float around 68%-78%. Seems gc happpens quite frequently and causes more cpu. Sadly it won't drop below 68% any more.
leaden lagoon
i think the current going theory is that the heap fragments in some way gc cant handle, by tigga. not sure if he did some tests or it was inferred from data and reports
sonic carbon
wasnt there exposed method from node to trigger gc manually ?
i remember that this
if (global.gc) {
global.gc();
}
should be working, since the scrreps-mmo node process was launched with the --expose-gc flag
spiral furnace
There's some theory that teh heap gets fragmented in a way that gc can't recover all the space.
It certainly can happen in JS, it's just a question of whether it does in Screeps VMs.
Later versions of Node reduced the impact of fragmentation a great deal, so in the process of updating this sort of problem may just go away.
hallow sapphire
I've also confirmed this happening for a rust based bot
because its a rust bot i can easily re-create the entire wasm state & cleanup any bot data without resetting global but the heap doesn't shrink back down if i do that
so definitely something relating to the server having a memory leak or heap fragmenting, not something that my bot is leaking
the average rate for my bot on shard2 with ~180 rooms visible, 29 owned rooms and ~800 creeps is ~140 bytes per tick extra heap usage
not too bad, but over long global lengths times that is quite significant
considering its way worse for JS based bots the idea that its heap fragmentation is way more likely, since my heap usage went from 50% to 57% over the course of 350k ticks
jolly tendon
@worldly siren @strange hawk
strange hawk
We read all posts in issues channels, no need to ping us
leaden lagoon
fwiw i just refactored my caching to have it all in one centralized ledger and i confirmed that at any given time my own data on the heap is always less than 10% of the used heap space. all of mine that's left outside of that ledger are custom constants, which will take at most 1-5kb, if even that.
on shard 2 just now i have 470/650MB (70%) used, 45MB of which is my cached data.
jolly tendon
Sadly, after the node updated, the problem is still here. My heap started floating around 70% after about 30k ticks.
sonic carbon
that's normal though after all classes and stuff you put in heap settles
i do see that the cleanup is more aggresive and happens more often when looking at charts
i'd say it still happens, but cleanup is a lot better
jolly tendon
If age is 13k ticks, the heap is much better.
jolly tendon
And here's the heap of 8w ticks, about 80%
jolly tendon
Has anyone let your bot run 100k ticks? The heap problem is still there. Every time my heap usage reaches about 74%, it takes more than 400CPU to gc, and it goes back to 68%. Happens more than once a minute.
hallow sapphire
im fully convinced its just heap fragmentation since my rust bot is basically immune to the whole issue even though i have now have over 1200 creeps and a significant amount of vision / game objects loaded
that also indicates that its mostly just player code that is causing fragmentation cause the amount the game causes really isn't much somehow
last time it got to 144k ticks global and the heap got a whole 18mb bigger (2.3% of total heap)
jolly tendon
I thought it should be the work of v8.
My heap increases 35% (175MB) in the end, and I seldom create huge arrays.
hallow sapphire
huge long lasting arrays would make the GC/allocator very happy, its the many short lived objects with varying lifetimes and varying/growing sizes that cause issues
leaden lagoon
im fully convinced its just heap fragmentation since my rust bot is basically im...
creep count is really not telling anything about your heap situation. if you have 150-200+ rooms of vision on average and you dont have any issues though, that'd basically confirm that the js space is the issue
hallow sapphire
i've got 310 rooms visible at the moment, average seems to be around 325
heap usage is stable at around 60%
delicate cargo
could be worth checking how xxscreeps handles that
jolly tendon
huge long lasting arrays would make the GC/allocator very happy, its the many sh...
But that's how Screeps runs. Every tick lots of objects are created and then deprecated.