#Distrubuted Pumpkin

1 messages · Page 1 of 1 (latest)

midnight cloud
#

This is an idea for how pumpkin could add distrubuted support for enterprise deployments:

The idea is to be able to have multiple servers that just handle networking and chunk requests.

That way you could have a repository that stores worlds, and each world has a map of the chunks and what stage they are in.
Then you would have a world-gen image that would grab a task from the repository, do one stage, update the repo, and if no dependencies are required, continue. If there are requirements for the next stage, update the chunks priority and grab another task. (theoretically you could have a priority system with both world-gen and player pos priority).

This way, you can have the world-gen worker nodes with multiple replicas. And you could scale the world gen independently of the servers. And the servers share the world gen. So if only one of 10 servers is active, it takes up all the world gen.

Optionally, you could also have multiple servers/layers connected to one world, like WoW. So you could have one server for people in the americas, one in europe, and one in asia. All with the same world. (This would however make players invisible on each server, so would need (native?) support for layering).

The benefit of this is also that you can add big storage nodes for the repository, and tag them with an identifier so they only run the repo.
Same with fast processor servers/multicore cpus, tag them with worldgen.
This makes server hosting a breeze, and the server owners can scale the different parts independently as needed.

midnight cloud
#

The cross chunk gen really makes it hard though. We would have to mark the needed chunks as locked while using them for generating another chunk. Othewise, we could generate a stage, and then when we check so that the chunk is still the same as before the stage in the repo, we would have to invalidate that whole stage if a chunk around it crossed chunk boundaries.

reef idol
#

I believe Hyperion was working on something like the multi national thingy you described but for chunk gen? what benefit would making it distributed be?

midnight cloud
#

And sharing world gen pods between servers

reef idol
#

sharing worlds between servers is pretty cool though

midnight cloud
reef idol
#

enterprise? but why

midnight cloud
reef idol
#

fair, idea sounds cool enough to justify it

midnight cloud
#

you could also have something like crypto mining, except the hosting company pays you for each like 1000-10000 chunks generated, and it becomes a marketplace selling and buying world gen performance. That way you could have an even thinner pumpkin locally.

slender falcon
#

What about different servers for different world regions? Sort of like what folia does with threads. Once the world is mostly generated you don't tend to need a ton of distributed chunk gen right?

compact marsh
#

This could be useful for Kubernetes setups, where there can be multiple chunk generation servers at once

muted cypress
#

how much would this matter for anything other than anarchy servers though?