#Can I leverage celestenet to make a code mod that sends information to other clients?

100 messages · Page 1 of 1 (latest)

hot arch
#

sorry for resurrecting a thread but I have one more question - does the celestenet client do any kind of bundling or should i implement that kind of thing myself to help reduce network traffic? I do see the CelesteNetSendQueue class and CelesteNetConnection.GetQueue() which look like it does something like that but I want to confirm that so I don't accidentally make something that's too heavy on the network

scarlet sundial
#

it kinda does bundeling? it's mostly a mechanism to queue packets while the connection is busy sending previous ones, and implementations can send multiple packets in the same udp/tcp message

hot arch
# scarlet sundial it kinda does bundeling? it's mostly a mechanism to queue packets while the conn...

that makes sense. Is there some straightforward way I can send a packet in a way that it'll just always get held until the next regular update packet is sent and tack it onto that same message, even if there isnt one already in flight?

I'm looking into a new idea using this but a naive implementation of it would probably generate a metric ton of packets and just trying to figure out the best strategy for it

scarlet sundial
#

what do you want to do?

#

you can implement bundeling in your packet yourself, by just having it be an array of something you actually care about

hot arch
#

synced entities; but ssshhhhh it's a secret ;)

scarlet sundial
hot arch
#

if there isnt something i can use already in place

hot arch
scarlet sundial
#

anyway, you would then just have a list of that stuff in your module or something, and every update, you create one or multiple packets with the contents of the list, then clear it

#

you should be able to access the maximum packet size in a field somewhere, alternatively just hardcode a fixed const "max number of XYZ per packet"

scarlet sundial
#

almost nothing needs it (in fact the only thing implementing fragmentation manually is the avatar sync code), and it would create tons of additional complexity + potential for abuse (remember, mods can send arbitrary packets)

hot arch
#

that makes sense. Okay a simple bundling/message buffer shouldnt be hard to implement, thanks for the tips

frozen hill
#

Idk how much data you're actually trying to send, but you may want to consider this conversation I had with jade a while ago

Screenshot_20220820-095824_Discord.jpg
scarlet sundial
#

quick note though that celestenet has a concept of ticks, which run at a variable rate the server can adjust on the fly

#

that mechanism is intended to allow the server to downscale the per-tick load if needed

hot arch
#

<@&615240487518994462>
At a high level, I want to make a mod that sends information to a predefined group of other players with the mod installed. Players connected to a celestenet channel seems like a reasonable way to handle that since the infrastructure for sending information like that already exists there. The kind of information i'd like to send is relatively simple, stuff like file times or a signal for the receiver to handle in their installation of the mod. Everything I'd want to send could be serialized to a relatively short string if necessary but it would still be nice to support richer data types.

I'm digging through the celestenet code and it looks like I could make a class that inherits DataType and send it to the server, where it would be received as a DataUnparsed object along with the remaining information in a byte array; from what I'm seeing, if that's then repeated to other clients then it should be handlable by those clients listening for DataTypes with that DataID while being benign on clients that don't have the mod listening for it.

There's a couple things I'd like to learn:

  1. At a high level, is this something that the celestenet team would be comfortable permitting in the first place? I understand this kind of thing comes with security considerations and I am willing to work with y'all to keep it airtight.
  2. Without making any changes to celestenet code, can I send a DataType (received as a DataUnparsed on the server) that will get rebroadcast to other clients connected to the same channel?
  3. What else do I need to know going into this?

Other thoughts:

  • To avoid unnecessary network traffic and other considerations, i do not want this to work in the global level; it should only work at the channel level. I'd also be okay with restricting it to max out at like 16 clients or something like that
  • If security necessitates only allowing this to be done with information serialized into a string, I'm okay with that. It would be nice to be have support for bigger data types but I understand that makes security much more difficult.
  • I'm willing to take this conversation to DMs if it makes more sense to discuss the details in a less visible space

Sorry for the wall of text. I think this would open the door to doing some absolutely incredible things with the game and I'd like to help open up some new horizons.

frozen hill
#
  1. Yes, mods already do this, such as Madhunt
  2. Yes
  3. you may want to take a look at existing mods that do this. The only ones I know are Madhunt, Madeline Party (my unreleased mod, but the code is on github), and Brokemia Helper (for the CelesteNet flag controller entity)

Addressing your other thoughts:

  • Depending on how you define your datatype, CelesteNet will automatically limit it to the channel
  • You can use arbitrary data types
#

@hot arch

hot arch
frozen hill
#

Also Guneline, not sure if that is open source

scarlet sundial
#

Yeah, the Unparsed feature only exists so that other mods can use CelesteNet as well

hot arch
#

incredible. I am so glad y'all are a step and a half ahead of me on this

scarlet sundial
#

what data do you want to broadcast?

#

because if i know that i could help you pick out the right set of metatypes to properly broadcast it

hot arch
#

I'm looking into making a helper for future BTA contests or other head-to-head races/competitions. So at a minimum:

  • File time at the start and end of a match (or possibly more frequently, like at every room transition)
  • anti-cheat information like whether a player uses debug screen
  • information about the format of a match that's starting
  • a flag for players to "ready up"
  • a signal from the host to start a match that would then cause readied players to be launched into the chosen map

I have a lot of other ideas as well, but these would be the core components. I am very comfortable in my coding skills but of course welcome any and all domain-specific guidance.

scarlet sundial
#

i doubt you would ever be able to get proper anti cheat working, because in the end you still have to trust the client

#

i faced the same problem with madhunt

#

other than that, do you want your data to kinda act like it's "bound" to the player, and you want to retrieve it based on the player as a key (and want the server to automatically rebroadcast it when switching channels/joining the server), or do you want it to simply blindly broadcast your packet without the server helping you out with the channel switching logic?

hot arch
#

For sure, I just want basic information that could be used to verify exceptional circumstances like a crash & reload

#

But it wouldn't be hard i think to just report on what everest-based mods are installed as a preliminary measure

hot arch
# scarlet sundial other than that, do you want your data to kinda act like it's "bound" to the pla...

I'm sure what you mean exactly by bound to the player, but knowing who sent the message is critical.

The ideal infrastructure would be something like this: people connected to a channel can select a Role. Different messages would ideally only go to players in the channel with one of a certain set of Roles depending on the message type, but if the solution is to just rebroadcast all of them to everyone and then filter out the irrelevant ones, that's also fine by me.

scarlet sundial
#

i meant like that the server is maintaing a kind of "dictionary" between players and your custom values, which is automatically kept in sync across connects / channel moves

#

the alternative is blind broadcasting to everyone in the same channel

hot arch
#

The former would be a cool way to implement it, but I think just the simplicity of a blind broadcast would be more than worth it

scarlet sundial
#

you would have to deal with channel switching and other edge cases on your own then

#

madhunt uses a combination of both approaches

#

two packets ("start" and "stop") which are blindly broadcast, and one bound packet tracking each player's current state

hot arch
#

I was assuming joining a cnet channel would be a prerequisite to any of this working, and that the messages would not ever need (or want) to go across channels or to global. If that's the case, would it still be necessary to track channel changes?

#

In short: everyone* in the channel want to receive the message, and nobody who isn't in the channel would want to receive it

scarlet sundial
#

yeah, but what if someone disconnects, or switches channels during a match?

hot arch
#

Then we're talking about a network failure or a game crash. I'm open to suggestions on how to handle those

#

If someone chooses to switch channels in the middle of a match that's on them lol and there isn't anything here that would have a decent risk if it missed some intermediate updates

hot arch
#

ok so yesterday I did some experimenting and I have something put together that seems like it should work. Once I connect to cnet and run the debug command, it's instantiating the datatype object and calling sendandhandle, and the local handler fires. I got a friend online and tried testing that it actually sends and receives over the network but when he tried installing what I had so far, it seems the mod silently failed to load on his end (he was not able to use the debug command at all - command not found error message), but there's no information whatsoever in the log.txt about a failure. Interestingly, it also did not seem to recognize the dependency on CelesteNet.Client even though I verified multiple times it's there in Everest.yaml.

This is my first time writing a code mod that has binary dependencies on other code mods, and I'd like advice on 2 things:

  1. Is this a common issue in this type of mod? What issues should I be looking out for?
  2. How should I structure the references in VS to CelesteNet.Client.dll and CelesteNet.Shared.dll so that I don't have to redistribute them along with my mod (or is redistributing the only option)?
scarlet sundial
#

it should work without redistributing them, maybe the mod's in your friend's blacklist?

hot arch
#

Checked that, it was not blacklisted. His log.txt also said it was registered successfully (interestingly, it did so before registering celestenet and did not delay it). It's almost like it did not even read Everest.yaml

scarlet sundial
#

how did you layout your mod?

#

i suspect one of thosr packaging errors everyone falls victim to at least once

hot arch
#

[Mod dir]

  • Everest.yaml
  • Code/mybuiltcode.dll
#

That's the whole mod

#

(So far anyway)

frozen hill
#

everest.yaml should be lowercase

scarlet sundial
#

zip files are case sensitive

hot arch
#

Crap

#

Thatll do it

#

Sure would be nice if either zips stopped being case sensitive or directories would start being case sensitive...

scarlet sundial
#

blame windows for this .-.

hot arch
#

Darn you, Bill Gates!

scarlet sundial
#

even ntfs is case sensitive, windows just explicitly adds case insensitivity as a """feature"""

hot arch
hot arch
#

(Inconsequential post to stop the thread from archiving)

frozen hill
#

Is there still something you needed or are you just keeping it open for potential future questions?

hot arch
#

Just keeping it open for questions since I haven't yet been able to test that what I have so far actually works.

frozen hill
#

Alright cool

#

I can turn up the time to archive if you'd like

hot arch
#

I do have 1 immediate question though: if the internet connection is shaky, how confident can I be that messages are actually sent and received? Or do I need to implement confirmation messages for the critical ones?

hot arch
frozen hill
#

Bumped it to 1 week now

#

I unfortunately can't also answer your question, all I know is it seems pretty consistent anecdotally

scarlet sundial
#

those are transmitted over udp, while all others are via tcp, and as such can be assumed to be both ordered and reliable

hot arch
#

Ok that makes sense, thanks!

scarlet sundial
#

they may be delayed though

hot arch
#

As is anything over a network

hot arch
#

ok so here's where I'm at and I'm not sure how to troubleshoot. I'm done working on it for tonight but will probably pick it back up again tomorrow.
Right now I'm just trying to send a simple test packet containing just a string. Almost all the important pieces I can see based on looking at Madhunt code seem to be present; the only obvious thing from that that I haven't implemented is GenerateMeta/FixupMeta on the DataType; but I don't see why those would need to be overridden for just sending a string that isn't attributed to a particular player. Maybe that's the issue though? Does the server only know how to send stuff to other clients if it gets metadata?

What I'm seeing is that I call SendAndHandle, it seems to send, the handler fires on the game that sent it, but it's not received by the other person on celestenet who helped me test. For the test we were both connected & joined the same channel and could see each other so we can rule out the obvious answer that we werent connected. I am not sure how to troubleshoot at this point.

scarlet sundial
#

Does the server only know how to send stuff to other clients if it gets metadata?
Indeed, thats why there is metadata at all laugheline

hot arch
#

Ah

#

What the heck do I need to send then, nothing about the metadata madhunt was sending looked like it was related to that

scarlet sundial
#

the private player state meta type is what you want, it's used for sending stuff to all other clients in the same channel

#

if you add the bound ref type, the server does all the dict synchronization for you (you can get it per player using .Data.GetBoundRef)

hot arch
#

Oh weird, the name "private" left me assuming that was strictly client-side metadata, like a state container of sorts. Ok that's good to know; i'm not enturely sure i underatand the role of the boundref and and dict yet, but, baby steps.

scarlet sundial
#

it binds data to a player, and the server retransmits the last packet for new players (in the channel)

hot arch
#

ok, a few questions here:

  1. Is there a function on the celestenet client available to get a list of all players connected to the current channel, or do I have to listen for connect / disconnect events and maintain my own collection? (please tell me the former is true...)
  2. What's the best way to identify a player that'll be the same across all clients connected to the channel? is the uint ID from DataPlayerInfo going to be persistent enough to use as a global identifier, or do I need to implement something more complicated?
#
  1. When nesting a DataPlayerInfo in another DataType, are there any side effects to calling writeall / readall instead of just write/read? Is the presence of that additional metadata something I need to think or care about?
frozen hill
#

I can answer 2. The ID is in fact unique. I can't remember if it's unique to a connection or to an account (or connection for guests)

scarlet sundial
hot arch
#

hmmmm unfortunately with the cnet server on lockdown I can't test how guest connections behave; for now I'll probably check out how non-discord players' IDs work, and potentially have some kind of rebinding function where other clients remember who has been connected and match on discord name or something to update stuff if the uid changed on reconnect. Or i can just forget about it and say you're effed if you lose connection...

scarlet sundial
#

player ids are actually session ids, and are unique per connection

#

in fact celestenet offers no mechanism to identify players over the span of multiple connections

hot arch
#

ok so that's something i'll have to implement myself if i really need it then.
to clarify, you're saying that if you just toggle the setting to disconnect then reconnect you're going to get a new uid?

scarlet sundial
#

yeah

hot arch
#

ok I'm still stuck on #1. The closest thing I could find to a list of connected players is the collection of Ghosts, but that does not seem to do the job because it only gets players in the same level as you. I'm trying to find all the events I should listen to to track it myself but I'm not catching everything yet:

  • ChannelMove fires on the person moving's machine always, so that's good
  • ChannelMove fires for other players only when they leave your channel; not when joining your channel
  • I have not yet found the events that fire when a remote player connects to celestenet and is already in the channel
  • I have not yet found the event that fires when a remote player disconnects from celestenet

The second and third could be remedied by implementing new messages to force everything to sync up, but I don't think the last one has an elegant remedy.

scarlet sundial
#

it would be best if you engineered your protocol to not rely on exact player lists, otherwise there'll likely be a lot of edge cases / race conditions

#

madhunt's protocal was especially designed to never require strict synchronization / awarness of others, and tbh it was kinda hard sometimes (especially during sychronization)

#

also it is possible to get a list of all players, which you can then use to check certain attributes of them

scarlet sundial