#[SOLVED] Build time increases with bigger sprites

1 messages · Page 1 of 1 (latest)

cosmic robin
#

This is a more "techie" question so I'm using our beloved kitchen this time.
We're making a high-res game. Recently our artists imported all their creations and compiling time went up from 8-10 sec to up to 60. Sprites folder size is 1.6 Gb.
Questions:

  1. Am I correct assuming that image compression takes most compiling time?
  2. If yes, why the heck doesn't GM cache all the compressed images and reuse them?
  3. What can be done in general to optimize compiling time? Don't wanna spend 3X of the planned time on developing my game
tacit eagleBOT
spice sierra
#

There might be more to this, but I'll just list out the general points.

  1. I don't think it's the most common factor in terms of slowdown, but I haven't strictly measured image compression times between the different modes (though someone may have).
  2. GameMaker does cache all texture pages it makes. It only usually needs to rebuild them whenever one of the sprites in a group get updated. And that's usually done on a per group basis.
  3. There's a couple things to consider that might be impacting you here, but GM should only be caching these once after building for the first time while the IDE is opened (or if there's been a previous GameMaker compile cache), but it could be:
  • Images that are larger than the target platforms texture page (GameMaker has to resize every single image down to fit onto the texture page)
  • A huge amount of sprites (The time it takes will increase, but usually it's a minimal impact)
#

That being said, GameMaker shouldn't be recreating them on every single compile, unless someone is constantly modifying sprites or there's no cache.

cosmic robin
spice sierra
#

I should also note that using the brush (clean build) will also remove all caches

#

If you are ever using that, I suggest instead trying to just clean code from Build -> Clean Code

image.png
gritty tartan
#

you could compare cached and uncached time by cleaning, running, then running again without changing anything

#

and timing it each time

spice sierra
#

Actually now that I think about it

#

I wrote something like this for a client I belieeeve lol

#

Well close

#

It just printed out the date time between each step

cosmic robin
#

Ok, I did it. The short answer: yes, it's actually caching stuff.

  1. Clean build took 123 seconds.
  2. Cached build took 32 seconds. Faster, yeah, but still.
  • Noticed that clean build shows elapsed time in the build progress bar. Repeated build with cache shows "reverted" clock starting from first build time
  1. Made a separate git branch and removed 1.5 Gb of assets (it doesn't break anything, it's only level overpaint sprites)
    Build's ready in 8 seconds
#

I guess 3 is my only option when have to tweak my code a lot.

#

And 30 seconds is still quite long. Why is it 4 times longer than empty project if it caches textures?

obtuse trellis
#

30 seconds is a long time (these days). how many assets are there?

#

how big are the level overpaint sprites?

#

what size texture pages are you using?

#

how many texture pages are being generated?

feral bay
#

1.5 Gb of assets
my project have 50MB and it builds in 10 seconds, I would say that only 3x longer for 30x bigger project isn't that bad... while GM caches textures, I suspect that time to traverse their list and checking if those texture pages are already "cached" might be still taking some time

#

what about this setting? I've noticed that setting it to 20+ makes things worse, and in case of 50+ it's even 5x slower

obraz.png
obtuse trellis
#

lmao

#

probably disk read bottlenecked

sturdy salmon
#

Does OP have HDD or SSD?

cosmic robin
sturdy salmon
#

If some assets are larger than texture page, then GM will resize then down to fit

#

I think GM will keep halving the image size until it fits in texture page.

spice sierra
#

That is a lot of texture pages

#

Also a lot of big textures

#

The vram size would be insane too

sturdy salmon
#

Yeah. Especially when in GPU they must be uncompressed (if you are not using specially compressed ones).
So one 4096x4096 image would take 64MB of VRAM

spice sierra
#

(With all 88 pages loaded at once, that’s 5.5GB)

#

But… Not the point

sturdy salmon
sturdy salmon
spice sierra
#

So there’s at least two things I can suggest here:

  1. look at properly fitting any textures that are bigger than 4096x4096. GameMaker is doing unnecessary work (as in, a complete waste of time and processing) that could be fixed manually by appropriately resizing said images
  2. If you haven’t looked at putting together texture groups, I’d do that now.
spice sierra
#

The remainder of those images being nearly the size of the texture page makes me wonder what kind of game this is

sturdy salmon
#

Yeah

#

For example, is there any way to modularize the images?

#

Or are they all just unique in way that can't be done

spice sierra
#

(FNAF games are a big example on this sort of stuff… Seriously)

sturdy salmon
cosmic robin
sturdy salmon
cosmic robin
sturdy salmon
#

Just for kicks, try enlarging texture page size to 8192x8192

cosmic robin
spice sierra
#

But also for memory management

cosmic robin
gritty tartan
#

If you never changed from the default texture page size of 2k then it'd be compressing stuff which takes a good chunk of time

#

and that same compression will also ruin the sprite and make it look like poo poo, ruining your artist's work haha

sturdy salmon
spice sierra
#

That’s not to say it’s a great thing

#

If you were targeting mobile phones, a lot of mobile phones usually have low vram (at or under 1gb, iirc). RAM is similar, likely your game could implode on those devices. But that’s assuming you were targeting that low lol

#

I’d at least consider unloading things that aren’t necessary

#

And loading things that you do need

#

I wrote something like this ages ago. A system that could mark sprites to be loaded whenever needed, and a couple defaults.

sturdy salmon
spice sierra
#

Defaults are always kept in memory, no exception. And any other sprites could be loaded or unloaded. Even had room groupings

cosmic robin
sturdy salmon
#

And just for another kicks, try 1024x1024 😂

cosmic robin
#

it takes ages to rebuild

sturdy salmon
#

Yeah, it practically needs to resize down all textures now 😂

spice sierra
#

Just wait after the first compile, it'll be all cached at least awesome

cosmic robin
spice sierra
#

But the idea is that any sprites I need, it'll load the group + the texture pages I specifically requested, as apart of preloading

#

And anything else that I haven't requested, gets unloaded

#

Even if it was already loaded in

#

Defaults are moreso requirements, ones that you cannot have unloaded under any circumstance

cosmic robin
spice sierra
#

Which also makes sense

#

Textures have to be moved from RAM to VRAM, and that does also involve decompressing them

#

Dynamic groups take an extra step

#

Disk -> RAM -> VRAM

cosmic robin
spice sierra
#

🙂

#

Actually they're already decompressed once they get to RAM

#

But still, memory balloons somewhere down the pipeline

#

Disk would be compressed, then when the game boots up (or dynamic group is loaded), it gets put into RAM.

#

And then whenever you need that texture page, it'll send it to VRAM

#

The smaller the data you need to send through all of those layers, the faster it'll be

sturdy salmon
#

Yeah,loading happens faster as there is less data to move in memory and process.

sturdy salmon
spice sierra
#

But you see this with a fallback sprite as well

#

So it does take as little as 1 to a couple frames as needed

cosmic robin
spice sierra
#

You're just sending a texture page

#

If it's not loaded into VRAM

#

And since a texture page of 1024x1024 is only 4MB, it's a lot better than 4096x4096, which is at 64MB

cosmic robin
spice sierra
#

Dynamic groups loading from disk, would be processing on a separate thread, wouldn't necessarily make that faster. But it would allow your game to continue running other logic, as well as helping manage memory

sturdy salmon
cosmic robin
cosmic robin
#

unless I messed up smth

cosmic robin
spice sierra
#

And actually even required replacing room_goto with a custom function

#

To give you an idea

# 
// Loads and keeps texture group in memory, no matter what.
AtlasDefault(groupname, ...) 

// Assigns group names to rooms, loading and unloading whenever loaded.
AtlasRoomAdd(roomId (array of room ids, or individual id), groupname, ...) 

// Same as above, except it adds it to all rooms matching prefix.
AtlasRoomAddByPrefix(stringPrefix, groupname, ....);

// Loads the texture groups, unloading anything that isn't a default texture group
AtlasLoad(groupname, ...);

// Same as above, except it keeps all currently loaded texture groups
AtlasForceLoad(groupname, ...);

// Returns whether Atlas has finished processing texture groups
AtlasIsReady();

// The next 3 functions are apart of a whole system

// Like room_goto, except goes to a specific room called rm_AtlasLoad.
AtlasRoomGoto(roomId)

// To be called in an instance that manages room loading, to be placed in rm_AtlasLoad.
AtlasLoadRoom();

// To be called after all texture pages are done loading, usually you pair this with AtlasIsReady()
AtlasRoomGotoFinalize();
#

Though of course in your case, a system like this would require further texture group separation

spice sierra
#

I was rereading the manual just now, they do apparently get sent straight to VRAM by default when a dynamic texture group is loaded

#

So that's what may actually be causing the speedup there too

spice sierra
#

Remember how a single 4096x4096 texture page took up 64MB? And you had 88?

#

1024x1024 is only 4MB, and even with 108 t pages

#

Your entire memory footprint has also shrunken

#

This is the uncompressed texture pages by the way, not PNGs in your project folder

#

Compared to 5.5GB, your texture pages here are only 432MB

#

Total

#

Calculating texture page memory size is as simple as width x height x 4 (4 represents RGBA)

#

And then you can multiply that by how many texture pages that takes up, and you get your byte equivalent

#

That's the major difference here

#

You haven't done anything wrong, just those texture pages are a lot smaller now

sturdy salmon
#

But as they are smaller, quality is worse

spice sierra
#

Yeah, and obviously we don't want that haha

#

But you can't really achieve a smaller texture page size without breaking up the assets, or having as you've mentioned, an asset library

sturdy salmon
#

Yup

cosmic robin
# spice sierra So that's what may actually be causing the speedup there too

Sorry but I still don't get it 😂

How exactly does smaller t pages improve loading time in your mind?

In pseudo code I wrote, I'm checking for the whole tex group. Which means if I have 2 Gb of tex data in a group I need to load these 2 Gb no matter which page size I use, right?

So the amount of data is the same or almost the same. Where does this speed up come from?

spice sierra
#

They’re 1024x1024 texture pages, 4MB each

#

Right now…

#

Reread my message

cosmic robin
#

and ugly look 😂

spice sierra
#

Not more compression

#

Your textures have all resized to fit on 1024x1024 texture pages

#

Those have quality loss because clearly you can’t just squish big images like 4000x4000 neatly into 1024x1024

#

So they will look back

cosmic robin
#

Ahhhhh

#

Got it

spice sierra
#

Basically, the compression hasn’t changed

#

What has changed is the size of your texture page, which in turn affects how much the images need to be sized down

#

And how much memory they take up overall

cosmic robin
spice sierra
#

All good

#

🫡

cosmic robin
#

Ok, it was fun and very educative. Feel 2x smarter now 🧠
Thank you everyone!
Love you guys ❤️ and ladies ❤️

#

Oh, no, one more question. Does anyone know what
Writing Chunk... SPRT size ... 0.00 MB
row in output log means?
Is it simply loading textures?
It "spends" the most building time on this row

spice sierra
#

That’ll be writing your sprite data

#

All of those sweet textures

#

Into the WAD/s

cosmic robin
#

The bottleneck in this case ig