#Saving very large dictionary in binary

11 messages · Page 1 of 1 (latest)

warm snow
#

Hi I have a dictionary called saved_tile which store the position as key and the tiles is ,atlas cord ,flip_h etc as it's data like this
{"tile_id": 31, "atlas_cord": (0, 0), "flip_h": false }, (-2764, 4108): { "tile_id": 31, "atlas_cord": (11, 9), "flip_h": false }, (-2763, 4111): { "tile_id": 31, "atlas_cord": (17, 6), "flip_h": false }, (-2764, 4114):

This dictionary is very very large and is growing as the player is exploring the world whi I save this dictionary which happens every 30 second there is a huge lag spike this is the func I use to save the var

func save_binary_file(data: Dictionary, path: String) -> void:
var file = FileAccess.open(path, FileAccess.WRITE)
if file:
file.store_var(data) # Save dictionary in binary format
file.close()
#print("Binary save successful:", path)
else:
print("Failed to save binary data to:", path)

Is there any different way of saving the var with less lag JSON lags a lot binary also stops the game for like a second

gloomy scroll
#

I don't know if an alternative way of saving would be faster, but have you considered to optimize what you save? Maybe not everything needs to be saved for example

#

Also, some games save a lot of data and because of that, they make saving manual as in those old pokemon games for example, and the console tends to 'freeze' for some seconds in order to save and the game even tells you that as a warning (a message sometimes says 'saving a lot of data, don't turn off the system' and the like)

rough kelp
#

games shows you often a "if this icon is playing do not close the game, game is saving"

#

its often question of miliseconds but you could hit the time window when its not finished and lose progress

tame void
#

You could move the saving to a separate thread, or turn it into a coroutine with the await keyword, to reduce the amount of lag it introduces

Not really going to be any way of speeding up the saving itself, not unless you figure out how to trim the data down or use an external library that can stream data

tired python
#

you could also change the way the save works, having the game add only the new data to the dictionary, and making a full save after the player quits the game

warm snow
tame void
#

you don't have to store the data as one big chunk. you can iterate over it in chunks of X

#

store it as a set of key-value pairs that you can reconstitute into a dictionary when you load