Cloud save architecture

[leaden monolith]

hey im looking into building a little multiplayer text-based RPG where the multiplayer functionality is pretty much just a database with shared data between clients.
Its inspired by those old browser games in the early 2010's

So, although it is a multiplayer game, there is no real time communication between clients i guess.

What I need help with is understanding some do's and don't of these tools. I am worried about cheating and general performance.

The way I understood the documentation, cloud code will be split in C# modules. The clients will open their unity game normally and it will connect to the backend, which is shared between all clients.

About data, both the client and the server will have the same datatypes, i.e. PlayerData, Items, Skills, etc...

Im order to avoid cheaters, i think wherever a player creates an account, a default PlayerData will be created and saved on the cloud. Then, as the player progresses, changes in the data will have to be propagated to the server. Im not thinking in sending the whole player data every time, but just what changes.
That is, a player equipped an item. Send that to the server. Raised an attribute point, send that to the server... And the server will do verifications on the data that's stored there. Has the player have the item its trying to equip? Does the player has attributes to spend?

In short, the server would never trust a client and just use the requests to know what is it that the player wants to do and then send the data for showing it to the player.

Would appreciate any and all insights anyone can give!

Thanks! @stiff elbow

[real lark]

You could potentially look into Firebase, namely the Realtime Database and Cloud Functions.
You can use cloud functions to execute logic on the cloud that affects your database, and that way the local clients function more or less purely in a "read-only" mode.

[bitter gazelle]

Basically, your server should always be your only source of truth. Your player will always just (as the name states) request a change on the playerdata. So as you said, equipping an item should be checked against the inventory and so on. It highly depends on what the playerdata actually stores. What does it make multiplayer and where could it be cheated. Also important to decide, if its breaking the game for others, IF they would cheat at some point.

[leaden monolith]

Sure, but thats the same thing as unity gaming services. Not sure how that would solve anything, its just switching tools

[leaden monolith]

I'm considering adding guilds and also some multiplayer fights to the game. PvP and PvE.

So, multiple players would be able to join together to beat some tough monsters. Loot would be exclusive for each player

[real lark]

Ah, suppose I misunderstood your question. It sounds to me like you have both the tools and the understanding of the architecture you want to target. If you run your logic server-side, you won't have a problem with cheaters.

[leaden monolith]

I appreciate your time brother!

[leaden monolith]

Btw, a doubt that just occurred to me. Can the server send the client some data without the client asking for it.
So, for instance, instead of the client making a get request, the server kind of makes a post or a put on a specific client?

[bitter gazelle]

Yes it can. Depends on your server but a lot can send state updates or notifications to groups/matchea/users

You can check out nakama for example

[stiff elbow]

Hi - Sorry for the delay getting to this, I'm not sure why discord didn't ping me when you pinged me. 🙂

I'm going to put together some code snippets for how I solved this.. but you're on the right track already so there's not a ton that's gonna be magic for you

I basically made two solution files - one for unity (because unity automatically regenerates the .csproj file and it's kind of finicky) and one for "everything else". Everything else for my game was a shared DLL, the server, the admin tools (a web based app), a Cosmos DB project (basically just an interface for the server to push/get data from whatever database you're gonna use), and a couple other small things (bot load testing, and "Telepathy" - a standalone networking framework that I like and would recommend):

The important bit is the ISG-Shared project - this is simply a bunch of data objects that unity and the server and the admin tools can use.. for example, a "player model":

There's a lot of annotations and shit on it, but it's basically just a big ass data file that can be serialized to JSON (using newtonsoft) or binary (using MessagePack) - binary is a bit better imho for sending across the internet

also note the Messages in the shared dll - this is how i'd structure it, to send data to/from your clients/server

A given message is wrapped inside an IsgMessage so that the server never crashes if a client tries to send a malformed or unknown message:

Then there's all the various message types that are things a player can do - usually these are pretty discrete and small..

So in a nutshell.. the player connects, gets the configuration of the game/world and their latest player file (after authentication, which is it's own nut to crack), and then just sends/receives messages as needed

The server is pretty straightforward - it just loops over all the messages, processes them, and does some "tick" stuff every so often

basically "while true, instance.poll()"

(poll is the network method that sleeps or returns incoming data/connections)

when messages come in, i parse them (with the handlers above - sent via events from "Telepathy"), validate them, hydrate them, and act on them

this is basically what you're asking - the way I validate which users are which is by connection ID - so like, I have a dictionary that maps connectionid to player guid.. during authentication, i look up their player guid (they claim to be) and verify their credentials match, then say "ok, you're validated as Katchi, and your connection id is 123".. whenever data comes in from connection 123, i know it's Katchi.

However - there's a lot of stuff a user can do on their side that we don't care to validate (say in an idle game, tapping on the screen once), or that networking would be too busy. Periodically the user can send their ENTIRE player object to the server to save:

when the server gets that message (btw, the server routes all the messages in a big ass switch statement):

we assume that the player object the player is sending is tampered with:

so as you can see - all of my API surfaces have "payloadPlayerModel" and "authenticatedPlayerModel" - the authenticated player model is the one that resides in server memory (and we know to be trusted and true) and the payload model is the one the player's sent us, claiming to be real

we validate it in a number of ways, and if it passes, then update the authenticated model in memory (and periodically in the database)

update player model looks like this:

it strips out hidden values from the player model (they won't get them anyway) and then applies those values to the player model:

So basically we can have an identical model (PlayerModel) defined in the shared DLL that unity can use .. and honestly, a player could even look at in-memory and TRY to edit, but the server just discards all the junk from a player model that we know is "secret" or untrusted, and applies the true values to the player.

On the server, then, in order to not create too much thrash on the database, since you obviously don't want to be reading/writing to the database hundreds of times per second.. we just hold all the player models in memory and write them all to the database if they're dirty.. and this amount is configurable since you obviously want to balance risk (the server crashed, everyone loses progress) with cost (databases can be expensive)

So then.. now that you've got the ability to send/receive an entire player model from a player.. you're free to make your networking messages a lot smaller since the player doesn't need to send the ENTIRE player model every time they do something tiny - like in the above example where a player buys an upgrade - it's a network packet that's like 4 bytes (instead of 2mb or whatever our player files were at the end of the project)

actually the example i sent you (the player buying an upgrade) wasn't a good one - since in that case the player has a number of things that they could buy the upgrade WITH - boosts, gold, promotional currency, etc, so for that example we actually have them send the entire model so we can see/validate their balances

but here's a better one - a player completed a reward video:

literally just a tiny string with which video they watched

or a player requests new "mission models" (missions are little quests in our game that players can randomly get as they travel through space):

player says "send me a list of missions with these guids", and the server says "ok, here you go, here's the content":

this is also how you can have a ton of content that you don't need to send to a player in order for them to start playing.. at the end of our game we probably had around 100mb of "content" - text based missions, cutscenes, levels, etc - it wasn't practical to either package it with the build or send it each and every time a client connected.. but the content was living, so it was changing daily

so we could send a list of guids and some high level information about something (missions, in the example above) and then wait for the player to request the content when they need it - say when a mission was loaded by the client and going to be presented to the user in the next 60 seconds

Validation looks like this - when a player wants to start a mission, for example, they require enough "crew" (we used to call them "primes" in the code) - so we take a "start mission request" and validate it on the server:

if it passes validation, then we update the player model and do all the business logic, then send that back to the player:

anyway, I know that's a lot of screenshotted code and some of it might be hard to read or make sense of, but hopefully that gets you on the right track.. feel free to ask anything you want.. this project never made money so we axed it after a couple years (sadly).. but the networking and server/database architecture was pretty solid.. i think we had zero "our-fault" crashes in 2 years.. many many people tried cheating, and aside from some autoclickers (which we had some fun protection against, if you're interested), as far as we know, no one succeeded

this was a multiplayer pvp game fwiw so cheating was front and center for us

[leaden monolith]

Hey @stiff elbow
Thank you so much for the in-dept explanation. I do indeed have some questions to make, and other may take a bit of chewing to actually come up.

First of all, it seems like your main RunAsync() function is some sort of gateway that routes the requests to different modules?
If thats is the case where does the DispatchMessageAsync() gets called?

Also, right now i get a pretty basic authentication system going on and it seems Unity Cloud code can have access to an interface "IExecutionContext"

[CloudCodeFunction("SayHello")]
    public string Hello(IExecutionContext context, string name)
    {

        return $"Hello, {name}! {context.AccessToken} with id : {context.PlayerId} ";
    }

which is giving me possibily correct values for playerId and access token whenever i call this function on the client.

is there any reason why i should not use this?

The calling of this function comes from here:

  private async void Start()
    {
        try
        {
            // Call the function within the module and provide the parameters we defined in there
            var module = new HelloWorldBindings(CloudCodeService.Instance);
            var result = await module.SayHello("World");

            Debug.Log(result);
        }
        catch (CloudCodeException exception)
        {
            Debug.LogException(exception);
        }
    }

which also makes me thing the whole switch statement might be unecessary for my case. Seem to me like unity is creating some routes under the hood whenever the backend compiles.

Also for actually managing content references i'm thinking of just using scriptable objects, creating some sort of ID system for it (probably using GUIDs on scriptable object awake method) and instead of saving a whole item object i can just save the ID of the scriptable object.

My idea is to design in a way to actually never reading the PlayerModel from the client. If i will manage or not we will see...

[stiff elbow]

I'm not super familiar with Unity Cloud but my "run async" is how the networking library I used (Telepathy) works. Basically you poll the socket and listen for a response - you do it async so the poll doesn't soak 100% of the cpu and "hang" while it waits up to 100ms

A lot of the other stuff is async because that's what the API surface for interacting with databases is moving towards.. and that's probably worth learning since there's a lot of benefits and reasons to use async

Unity doesn't use async "natively" - it can still be done but there's some interesting/weird effects (like everything actually happens on the main thread). If you're interested, google up UniTask for unity and try to wrap your head around that.. but if you haven't done async work before, start with generic C# async before trying to work with unity async since they're ... different-ish and you might not understand why something isn't working

but as far as the above - I assume that unity is handling the authentication internally (not sure how - probably some magic device id stuff?) and so you don't need to deal with it

however... the problem with that is.. because it's not "your" player id - you might have issues if players want to play from another device, change their device (upgrade their phone/computer) or whatever

also, from a bit of my experience with some of this tech from another industry (igaming) - sometimes if a person upgrades their computer substantially (new os, new cpu or motherboard) the magic player id will be a new one, meaning your server will think it's a brand new player and never be able to associate the old player with the new one

[leaden monolith]

Currently the authentication service from Unity handles a lot of different scenarios. I just implemented the username/password once, but on the documentation its possible to link it with google play, apple, facebook, etc..

[stiff elbow]

authentication and users are kind of a headache to be honest.. but if you want to roll your own, then you'll want to read up on how to authenticate (like, the process - first a handshake, then the user sends a userid, then you verify the userid is authentic, then they enter their password, you hash and salt the password and send it to the server, where it compares that against the (previously hashed and salted) password in the database

[leaden monolith]

Seems to me that it will make my life much easier

[stiff elbow]

so you basically never have a plaintext password but can 100% authenticate an individual from an email and password that they type on their end

yeah - i'd do the google auth

honestly big companies like google have the workflows infinitely better than you (or i) are going to do ourselves, but they're still a little complex to integrate

[leaden monolith]

yeah, no surprise for me there

I'm all for using tried and tested wheels intead of inventing my own

[stiff elbow]

steam auth is pretty good too - the user authenticates to steam (on their machine), gets a token from steam.exe, sends it to your server, then your server makes a call to steam and says "is token 12345 = the user katchi" and steam says yes/no and bam, your user is authenticated and you've never saved their email/password/etc

buuuuut the thing to remember is sometimes steam goes down so.. you can't auth during that time

[leaden monolith]

yeah yeah, there's a lot to think about

[stiff elbow]

re: never reading the player model from the client - i like that approach, but just bear in mind you will probably still need a message to send the entire player model (for debugging)

[leaden monolith]

I'm leaning towards giving the unity auth a try and see how it goes, i wont be saving the username or password to the database as that's something unity is doing under the hood.

[stiff elbow]

a big category of bugs that you'll need to wrestle with is desync bugs - the server/client think different things about the state of the universe - not even just related to the player, but the world

[leaden monolith]

ohh yeah, that's actually a good ide

[stiff elbow]

there's also work worth doing to debug those desync issues.. for example, when we were doing our "battle" mode (basically a turn based collectible card game sorta thing), there were lots and lots of abilities that impacted effects, board state, whatever

obviously you don't want to be sending the entire battle state back and forth each move (it was like, several hundred data points), but desyncs would basically wreck the game

my solution was to send the battle state at the beginning and end of a turn - like once every minute? - and comparing the entire thing against the version in memory, so you could easily see what you goofed up

you may need to do something similar for some/all of your data objects that you use - desync bugs are really nasty and you need a good strategy to find them consistently and proactively

[leaden monolith]

okay, got it (i mean, i didn't but im sure i'll remember your words)

[stiff elbow]

a "battle" is just a plain class but.. with many, many data points

if they are different, great, I can tell that easily but.. seeing what is different is a lot of work .. so it's good to just be thinking about that early on

[leaden monolith]

okay, thats solid advice

[stiff elbow]

the take home message is just .. make sure you're 100% clear on what is "in memory server state" and "in memory game state" and "persistent database state" - and make sure they're all clear to yourself

because just trying to mix them all magically together may work, but may be really expensive (super frequent and large database read/writes) or sluggish (super frequent and unnecessary network traffic)

i think that's what the unity cloud or unity servers kinda does - it just sends the entire game state back and forth .. there's probably some clever optimizations in there but .. might just be harder to diagnose later on since it's magic black box stuff

anyway, hope this was helpful

[leaden monolith]

i guess im going in an adventure here

[stiff elbow]

heh, no doubt 🙂

[leaden monolith]

yeah, you were super helpful dude

[stiff elbow]

feel free to save all of the above, read it later, and if you need help, reach out to me via email (i'll DM you) and if i'm able, happy to have a call or tell you more

[leaden monolith]

can't thank you enough. i'm sure ill be coming back here to read your answers a lot

holy, thank you so much mate

[stiff elbow]

i'd save it to a file or something - i think inactive threads get archived, but i don't know for sure