bevy_renet

[autumn canopy]

rustc 1.75.0 (82e1608df 2023-12-21)

so a little newer but hasn't changed in a little while

I can try to roll back

[hollow karma]

ill upgrade, been wanting to anyway

i doubt this is it tho like you said, that's from 2023

ya seems fine

[autumn canopy]

Yeah, I appreciate the help. I want to just uninstall everything and start fresh VSCode setup but I would like to figure out what is going on to learn....

[hollow karma]

yeah these are the worst problems

C:\Users\D\.cargo\registry\src\index.crates.io-6f17d22bba15001f\renet-0.0.14\src\transport\client.rs:87:12

has this been cleaned out?

[autumn canopy]

let me go look.

nope...

[hollow karma]

(no idea what happens if you delete it)

[autumn canopy]

lmao

lets find out

I am deleting the whole .cargo folder.

Going hard core lmao

[hollow karma]

DO IT LIVE

[autumn canopy]

ok deleting the whole folder, not a good idea. Deleting the registry data now. 😛

[hollow karma]

oh ya that sounds gross, I'd just start by deleting that one renet folder

[autumn canopy]

ok, building again. Lets see.

Well son of a gun... It worked.

So apparently those registry folders can get corrupt sometimes. Good to know. Nice find and thanks for that, didn't think about that.

and cargo clean doesn't remove them.

[hollow karma]

odd, but great to know

[autumn canopy]

Thanks so much man, it was nice to have someone to work with on figuring this out and you saved me some headache.

[hollow karma]

no worries! cheers

[autumn canopy]

Hopefully one day I can return the favor. If you ever run into an issue, ping me. We might get lucky and I can help. 😄

[hollow karma]

hahah noted!

[normal bay]

@eternal crater I made a PR to remove the Steam transport so we can get a release: https://github.com/lucaspoffo/renet/pull/144

I will also look into refactoring renetcode a bit so more of it can be used by the webtransport transport (which should be maintained in a new crate, either by @fleet crescent, myself, or some other enterprising soul).

IMO the in-memory transport should be maintained in renet since it is very lightweight and useful for writing tests: https://github.com/lucaspoffo/renet/pull/117.

[near juniper]

I agree with this, I don't think you should have to get a 3rd party crate to get reproducible testing

[bleak token]

I also agree, but you don't have to remove it from the repo. Just remove the dependency. Looks like the author prefer to keep his renet-related crates in one repository.

[normal bay]

I thought the point was it doesn't work without the dependency?

[normal bay]

New PR to make the netcode transports agnostic to the underlying packet source: https://github.com/lucaspoffo/renet/pull/145

With this, renet would have two levels of abstraction. At a high level the RenetClient/RenetServer provide a uniform surface for user code. Then at a lower level the NetcodeServerTransport/NetcodeClientTransport abstract over a source of unreliable packets.

@eternal crater @bleak token @weary moat @prime imp @feral crane pinging all of you who have been interested in this kind of abstraction.

[bleak token]

Just to clarify, not the dependency on steamwork-rs, butrenet_steam dependency from renet. I.e. just make renet_steam independent, like other transport crates, just keep it in the repo.
But it just my assumption on what the author prefers.

[bleak token]

I like it!
I suggested doing something similar to the author a long time ago.

I don't think that @Periwink is interesting, he have its own implementation.

[prime imp]

I just saw this, mb for late response. My opinion is that it's a good start, but I'm not super sure about some parts:

• This locks renet into using netcode, with no opportunity to swap out the protocol
• Parts of netcode are not necessary on some transports, like you've already said webtransport already encrypts packets, but it goes further: steamworks doesn't require fragmentation, channel transport doesn't require fragmentation or ordering or anything. IMO this is an unacceptable tradeoff, and different transports should be able to use only the features they need to function - no "double-layering".
• I assume, because this uses netcode, that this works exclusively with [u8]s. I'm not a fan of this, and in aeronet I take the approach of instead having a user-defined message type, which may be converted to/from [u8]s if needed by the transport. This:
  • makes it so serialization is not required for channel transports
  • makes the API surface clearer (imo), since you know exactly what type of message is sent on atransport

[normal bay]

This would only be used if you want netcode, it basically makes the netcode transports agnostic to the source of data packets.

If you don't want netcode, you write your own transports (that are compatible with RenetClient/RenetServer) as usual. That is necessary for steam which is too far removed.

[prime imp]

I see, yeah this makes sense then. I still think that having a single "set of protocol components" would be a useful feature, i.e. take the reliable+ordering of netcode and put it on top of steamworks.

[near juniper]

That's already how it works, I believe. The transport layer just promises to try to send packets of bits from the source to the target, and then the RenetClient/RenetServer implement everything on top of that

[prime imp]

yeah I just mean that these promises could be provided by renet, as higher-level pluggable components, so that e.g. someone who wants to implement a steamworks transport doesn't have to code fragmentation themselves, they can just use the renet code for it

[normal bay]

RenetClient/RenetServer don't include the core security bits of netcode, which are all implemented in renetcode.

[bleak token]

It's not what you meant, but imagine ECS-based transport library, it would be cool :)

[prime imp]

hmm, how would that work?

each packet is an entity?

[bleak token]

Just a crazy idea

[bleak token]

Yeah, imagine that

Or transport is an entity

protocol*

[prime imp]

personally I think it's a bad idea in the context of an ECS, specifically because the packet doesn't really represent anything important. it's like treating your game save file as an entity; what's the point? what can you do with it?

the purpose of packets/save files/etc. is to be used to populate the world, rather than exist in it

[bleak token]

Yeah, it's not efficient, just a fun idea which occured to me.

I mean you mentioned "components" and an association came up :)

[prime imp]

hmm that's true

maybe "layers" is a better term for it

like layers of the TCP/IP stack

[near juniper]

Oh, I thought renet did that internally, oops 😅

[normal bay]

Ah, this PR would make it super easy to define your own 'packet control' layer for artificial packet loss/latency/etc.

[near juniper]

How does that PR work for the in-memory case? What would you use as the address and stuff when doing in-memory things that don't have actual sockets?

[normal bay]

You would just use an empty address. Renet doesn't actually use the address for anything other than mapping, but with in-memory you will only have one client.

From renet's point of view, the SocketAddr is just an identifier.

[near juniper]

You can have multiple clients with the in-memory transport layer

I guess you can just invent random SocketAddr ss you go though

Just put localhost:1 and increment the port from there

[normal bay]

The in-memory server transport will internally have a map of senders/receivers for each client. You can overload the clients' SocketAddrs to record their index. That's also what the WebTransport server transport would need to do.

[glad sonnet]

i often get error "send channel 2 with error: reliable channel memory usage was exausted" and disconnected. i saw the default channel buffer is 5 MB, what if use Vec as channel buffer and dynamic allocate if needed?

[eternal crater]

Hey, you can increase the memory usage of the channel in the config. But usually, if you are getting this error, you might be sending to much network data. Or you are not pooling the messages every frame and they are acumulating. The channels have a fixed memory too make sure clients/server don't exaust their memory usage.

[glad sonnet]

thank you, i am making a 3d Voxel Game, the error occours when sending ChunkData of voxel terrain.

message pooling sounds good but it might should be implement by the library instead of myself?

at the end, i just increase the channel config from 5 MB -> 20 MB and 'solved'

---

actually i have another question: i think it's useful if a custom disconnect-reason string could be send to client if server disconnect a client.

e.g. when i implement "Login", i assume i should use "user_data" to provide UserId and PasswordDigest, then if the password is wrong or the account is already logged in, a custom message should be send while disconnect.

[eternal crater]

No, the user_data should not be used for that.

When generating a token, you need to make sure the user is already authenticated so that in the server you can assume the user_data is valid/safe.
For this you would need login/auth service, and you only generate the token for the client when he is authenticated and you can send the serialized ConnectToken to the client safely/secured.

The user_data is encrypted, so that only the auth/login service and the server can use it. The client will not be able to tamper/read it

[fleet crescent]

Steamworks.rs made a release

[mortal robin]

Hey y'all, I'm new to Bevy Renet and I'm trying to create a server browser similar to Minecraft. It just needs to go across a list of IPs and ask them for a LobbyData struct that has stuff like player count, map, etc.
I feel like opening a NetcodeClientTransport and closing it for each server doesn't make much sense but I don't really know what else to do, any advice?

[hollow karma]

I'm not sure I quite understand what you mean, to be clear, is there a master server involved? I think you will need one

Like, where are clients getting the list of IPs in the first place?

IMO a way to do this is have a master server, and when a new server goes up, it posts to that master server. "Hey I'm here, and here's my IP address, my server name, my active map, etc". Then clients ask the master server for this info, they can even do it over HTTP

That being said, if you want to display ping in the server browser, the clients will need to likely open a NetcodeClientTransport and then close it.

I have more thoughts, but I'll wait till you reply to see where you're at with the above

[mortal robin]

In Minecraft, you add servers by adding their IP to the list and it'll display info about them, I don't think there's a master server. That said though, this is a project simply to learn Bevy stuff and it's probably more useful to have a master server and create a community browser experience similar to older Source games like TF2

Which would connect to a master server and display all the servers it knows of

I suppose the better way to do that is to connect with a NetcodeClientTransport to a master server?

[near juniper]

You can either do it through renet, or you could just make an https endpoint and do a normal GET that returns the list of known servers and their most recent status

[normal bay]

netcode is designed for game code, not general-purpose networking. I'd recommend something like HTTP or websockets for talking to the master server (my websocket crates: bevy_simplenet, bevy_simplenet_events).

[mortal robin]

Makes sense, I'll look into these options, ty!

[scarlet cradle]

Would crate author consider adding a broadcast for a specific list of clients? also all except a list

[normal bay]

bevy_replicon has this: SendMode

[bleak token]

It mirrors Renet's.
I think he asking about adding something like BroadcastExceptList and DirectList.

I think it's not a good idea since it will require allocation.
It's better to use a loop.

[scarlet cradle]

can a protocol be described without a huge enum with renet?

[prime imp]

you could use your own u8 at the start as a discriminator, then the rest of the bytes can be used as the packet payload

[scarlet cradle]

I'm expecting hundreds of message types in some channels, doing like the example is feeling pretty cumersome

I probably need a u16 message id

are there any examples of doing it this way I could reference, not sure exactly what you are suggesting I do, the issue I'm having is more dealing with it on the bevy side, ie. handling hundreds of cases in an event reader

if let Client::Message(0) { else if let .... ... ... ...

[bleak token]

You can also define separate channels for different messages.

[scarlet cradle]

this is considered already

there are still hundreds of messages in some channels

or will be

[bleak token]

I mean create a separate channel for each message.

Or do you mean that you reached the limit of 255 channels and squished some messages into one channel?

[scarlet cradle]

I guess I could do that, I'll take a look at the code later and see how much of an issue I'll have patching it to suit my whims

on, I was also concerned about the boilerplate of doing that, and thought it may not be a good thing to try

but I'll look into that

thanks for your input btw, appreciated

[bleak token]

I would suggest to take a look at bevy_replicon then. It's built on top of renet and it automates this. You register a message and it will be available as Bevy event.

[scarlet cradle]

and yes I would probably hit the 255 limit on channels doing that

[bleak token]

Do you have that much logic?

Like 255 different totally different things?

[scarlet cradle]

it's planned to be yes

I have about 100 client messages right now

for in game messages

and more for non-game stuff

[bleak token]

If you synchronize things, like components, you can use a single channel for it.

[scarlet cradle]

I don't sync any bevy components

it's all lock-step and hand-rolled

[bleak token]

Ah, I see. You can replicate data from server by utilizing a single channel, it's much easier then defining hundreds of messages.
And keep other channels only for the game logic.

[scarlet cradle]

besides a few road bumps, renet is seeming like my best option still, so there's that at least =)

[bleak token]

I mean I imagine that you have messages like SpawnTank, DespawnTank, etc.

[scarlet cradle]

I want this protocol to not rely on bevy tho

that's a design choice

I may switch engines someday

[bleak token]

Ah, I see, then I don't think that you can avoid the boilerplate.

[scarlet cradle]

it is what it is, thanks again =)

[bleak token]

But just to clarify, the idea is not Bevy-specific.
If you will switch the engine, it will most likely work as I described: replication from server to client and messages only for game logic, like "Get this quest", etc.

[scarlet cradle]

oh, there's a lot of filtering that would need to be done on the client list so, I need to make rooms that frequenctly update the clients in the server room. many events are only sent to client if they pass some predicate, including spatial and logical seperations of game ares, ex. only units in the same zone that pass a predicate get a message such as spawn a villain, reveal map data in radius

i do want to broadcast to rooms tho

where possible

I'll see what it ends up looking like, just ported my server code working on the client now

[scarlet cradle]

well took about 1.5 days to port, everything working as expected, virtually no pain points

[atomic elk]

this is gonna sound dumb but how do I stop a server once I've created it?

I'm assuming RenetServer resource in Replicon is the same as the one in this package? Like Replicon just builds off this crate so they should be the same right?

at least, it's listed as a dependent in bevy replicon

does the server just stop after I remove the transport and server resource?

yeah it looks like that's what happens. at least I can no longer connect to the server after I remove those resources

still not entirely sure if I'm disposing of it properly

[bleak token]

Correct.

[bleak token]

Yep, you need to just remove the resource.

[atomic elk]

sweet thanks!

[scarlet cradle]

One more question, has anyone done long-lived server processes with renet? just curious if I'd run into issues if a server ran for a few months.

[near juniper]

You should never run a server for a few months anyway

[scarlet cradle]

ya, that's an outer-bound for sure

I can see a month tho

most of my stuff would be under a day as far as game instance go, chat and few other backend server I'm implementing as a long-living unix daemons in headless bevy app

something akin to Battle.net like services is the idea here

I will shard most things ofc, just looking for gotchas in the extremes

[hollow karma]

Not exactly the extreme you're looking for but I did leave my server and client running for a week by accident and when I came back home I ran around the level a bit and didn't notice any issues.

[scarlet cradle]

that's a good sign at tbe least

[hollow karma]

What is user_data and how do I access it on the server?

I figure I can put a username in there or something

[normal bay]

Just some raw bytes in the connect token that are readable on the server (but not the client, they are encrypted).

[hollow karma]

ok that's what I figured, I just don't know where to access it on the server

[normal bay]

Server event when client connects iirc

[scarlet cradle]

i'm probably just blind, but where do you find connecting/connected client address

[bleak token]

https://docs.rs/renet/0.0.14/renet/transport/struct.NetcodeServerTransport.html#method.client_addr

Was a wrong link, corrected.

[scarlet cradle]

ah thanks, I was looking in the wrong place for it =p

[bleak token]

@eternal crater Bevy 0.13 has been released, could you check https://github.com/lucaspoffo/renet/pull/147 which updates renet?

We tested it as part of bevy_replicon (https://github.com/projectharmonia/bevy_replicon/pull/198), all tests and examples work.

[eternal crater]

Thanks, already merged, gonna test and fix clippy. Gonna release new version today

[normal bay]

@eternal crater any chance of getting #134 in?

[eternal crater]

Just released the new version :0

[normal bay]

Lol np, I think it would be fine in a 0.1 release too, or the next major. Not a big deal just pretty annoying.

Thanks for releasing btw :) lots of people on the edge of their seats for renet haha

[eternal crater]

BTW, why you are having this problem? Why not remove the RenetClient when the disconnect happens

[normal bay]

Also now that you're here, what are your thoughts on #145?

[normal bay]

I leave the renet client in there until ready to reconnect. It keeps things simple.

Afaik there is no advantage to updating a disconnected client, so disabling it on disconnect is a small quality-of-life boost to the API. And more explicitly aligns with the actual behavior.

[eternal crater]

yeah, it seems fine

[eternal crater]

I haven't checked it fully, but using traits for injecting sockets in the netcode transport was a thing I tried before.
I tried for the first steam implementation (now I just don't use netcode for it, only steam with renet).

I prefer to manually implement a new transport instead of impl a trait. Most of the time the traits need tweaking when adding new stuff, and then we have blank impl for some functions, only half ones are used by some.

Also some people don't need netcode when making a new transport, and this just adds complexity to the netcode implementation.

[normal bay]

I agree with all your caveats about when a trait is not useful/inconvenient. However it is very useful for:

• Sticking netcode on top of different transports. Right now there is no reasonable solution for using netcode in the browser other than completely forking renetcode, which IMO is a big problem. The webtransport PR is essentially useless for games without something like netcode on top (regardless of whatever extremely dubious ideas people have about using raw unreliable packets).
• Using netcode in unit tests and benchmarks with an in-memory transport.
• Layering network effects between the socket and netcode for programmatic testing, e.g. inserting latency/jitter/packet loss. If renet had a third-party crate with a cross-platform network-effect instrument, that would be huge.

[near juniper]

Is the steam transport released now?

[eternal crater]

Soon,
BTW, does anyone want to test the renet_steam pretty quickly. Just need to get the main branch of renet_steam and run the example to sse if everything works. Before I had a notebook to test, don't want to download a VM just for this quick test :0

[eternal crater]

Just release the renet_steam 0.0.1 using the latest version of steamworks-rs
Didn't had a lot of testing, but seems to work fine. Any bugs/errors reports are welcome
Need to setup a VM for easier testing

[normal bay]

New PR updating renetcode so encryption is optional: https://github.com/lucaspoffo/renet/pull/149. If the webtransport PR #107 rebases onto this PR, then we will have easy netcode in the browser.

[normal bay]

New PR to support servers with multiple data sources (sockets): https://github.com/lucaspoffo/renet/pull/150. This PR would make it easy to have both native and WASM clients connecting to the same server, and make it possible for clients to switch between native and browser mid-game (assuming you have a reconnect framework).

[ember flame]

Hi! I am writing a multiplayer game and noticed that there is a flood of small packets (26 bytes payload) between the server and the client, in my case it is several thousand, probably because my server runs with MinimalPlugins, which does not limit the framerate, which ended up sending 3000-7000 packets per second. Are these the keepalive packets? Is there any way to tune the frequency of those?

I noticed the same packets in the renet demo_bevy are sent at about 250 packets per second.

[scarlet cradle]

2024-02-25T17:54:08.850129Z ERROR log: Failed to process packet: no private key was found for this address spams until the client drops the connection ^ the above seems to be the cause why I get this flooding too =)

that's a previously connected client attempting to send to a new server process

it's repeating due to the above issue

[ember flame]

I don't have the error message and the packets do not get sent before I start the fresh client, so I am pretty sure it is not the cause. I am testing everything on my localhost

I was able to reduce the flooding by introducing a delay to the ScheduleRunnerPlugin (which also helped the CPU usage)

[scarlet cradle]

the error message is tangential to the flooding to be clear

the error only happens if you connect to a server, restart the server process while connected

server rejects the packets in this case, but due to the flooding I get hundreds of these messages

[ember flame]

Got it, sorry misunderstood initially 😅

[scarlet cradle]

I'm going to look into what is happening with send-rate later, maybe there's a config option or something I missed for that

[final beacon]

I think the typical thing to do is to run server at a fixed timestep

[scarlet cradle]

it's still an issue with a fixed timestep

just less of an issue

no need for per-frame heartbeat if that's what it is

[scarlet cradle]

I think what can be done here is adding a new field for heartbeat, also server never should respond to a heartbeat IMO, just track that the client sends one at around the expected delta

also I've fouind I needed to patch bevy_renet to use Time<Fixed> and FixedUpdate schedules

there's a branch or two I noticed could be removed with some slight refactoring, not a huge win but something

https://github.com/AxiomaticSemantics/renet/blob/master/renetcode/src/client.rs#L342-L372 the matches! macro call can be removed and the last_packet_send_time can be updated after the self.state match since that would return, no point in checking things twice

[bleak token]

@eternal crater could you take a look at this PR?
It's a one-liner: https://github.com/lucaspoffo/renet/pull/152

[eternal crater]

Merged 👍

[eternal crater]

Its not actually a heartbeat, it's the ack packet sent every frame. We send every frame for simplicity and easier RTT calculation. Its a tradeoff to send every frame, most common cases for games this is what we want to do since there is mostly no idle time in the connection.

And since we send every frame, running at really high framerate really pumps up the cost for this packet.

[scarlet cradle]

ya, I figured that out after =)

[scarlet cradle]

ya, I'm going to have to go with a custom protocol which is fine, I'll add it as a new crate alongside renetcode

[scarlet cradle]

https://github.com/AxiomaticSemantics/renet/commit/0286cf44300dee4fe02a6b9ad275dc78e4784466 I noticed a byte can potentially be saved in renet's Packet, unsure if upstream would want, someone can pick it out if they want

would upstream be down for allowing for a codec and transport trait end-users can implement?

I did manage to integrate an alternate protocol into renet and bevy_renet without too much patching but it was still somewhat intrusive to the higher level crates

[pure bramble]

i bind the client socket to 0.0.0.0:0, the server socket to 0.0.0.0:PORT, and connect the client to CLOUD_PUBLIC_IP:PORT
the server on cloud is not getting anything from my client (everything worked when testing locally)
(i did not forward the port or setup a proxy or do anything extra on the cloud)
(yes i always double check the public ip of the cloud VM since it changes)

in my client

in my server

(i try to connect to the cloud with cargo run -p client online)

[hollow karma]

Is it containerized?

BTW you can do this instead SocketAddr::new(IpAddr::V4(Ipv4Addr::UNSPECIFIED), port);

How is PORT getting there? Is the port correct?

You are also putting 0.0.0.0:PORT as the public address, I wonder if that needs to be the actual IP of the VM

Although, mine is like that too and my friend can connect to my PC, but maybe there's something funky going on with that when it's in a VM, internal vs external

I'm working on my cloud impl right now so I can post here if I find anything

[pure bramble]

thanks, the PORT is just a constant in a shared lib

[pure bramble]

hmm.. im using Google Cloud Platform

[pure bramble]

not sure what you mean by that

[hollow karma]

Like is it in a Docker container

[pure bramble]

nono, just a linux vm running a server executable

also something confusing, there is a difference between "public" IP from what i can see when i curl ipconfig.io in the VM, and what GCP states as the "external" IP

they are different, i think i tried both, and in both cases the cloud server isnt receiving anything from my local client

[hollow karma]

So you tried binding to 0.0.0.0, and set the public address to the external ?

[pure bramble]

in the client yes

i will try both again just to make sure

[hollow karma]

I would try that on the server, I think you always want to bind to 0.0.0.0, but you may want the public_addresses array to contain whatever GCP states as the external ip

[near juniper]

The way I have it set up is:
server:
public_address: <actual address>:<actual port>
socket: 0.0.0.0:<actual port>

client:
server_address: <actual address>:<actual port>
socket: 0.0.0.0:0

Looking at your code your public address on the server is wrong

[hollow karma]

yeah

thats gotta be it

and in this case the actual_address is the external address

[pure bramble]

yep trying it now

[pure bramble]

the server still cannot get anything from the client.. 🤔

client, binds to 0.0.0.0:0 and connects to EXTERNAL_IP:PORT

server, binds to 0.0.0.0:PORT and public_addresses is EXTERNAL_IP:PORT

[normal bay]

Optimization PR (fairly small impact): https://github.com/lucaspoffo/renet/pull/154

[pure bramble]

It works when using localhost locally, but i cant get it to work on the cloud, do you guys think its not an issue in my code, but rather the cloud configuration? Im using a GCP VM on default settings

[eternal crater]

Might be a firewall configuration problem

[pure bramble]

oh i didnt think of that, i will investigate

[pure bramble]

YES

thank youuuu!!!!!

SOLVED ✅

[hollow karma]

Awesome

[pure bramble]

do you just just register your server network code to Update, or do something extra/special?
my server is pretty slow on a super basic game with only a few clients, or frequent messages
it could be because im sending too many client messages over ReliableOrdered i guess
but i wonder if server could run recursively, without waiting for a bevy time step or something (i dont really understand this but yea..)
(my server is on MinimalPlugins, and all of my transfer is Strings which i parse (will use bincode soon))

[pure bramble]

everything is smooth as butter now,
started using serde and bincode for data transfer, and i optimised the amount of assetserver loads

[hollow glade]

Anyone had luck or have an example of running a renet server in a docker container?

I can’t seem to connect to a locally running container, despite exposing and mounting the UDP socket to my host. Not sure if it’s a socket/renet config or host network issue.

Or possibly an iptables issue within the container? I’m using the Rust slim buster as my base..

[hollow karma]

yes:

# STAGE1: Build the binary
FROM rust:bullseye as builder

# Install build dependencies
RUN apt-get update && \
    apt-get install -y libasound2-dev libudev-dev

# Create a new empty shell project
WORKDIR /quantsum

# Copy over the Cargo.toml files to the shell project
COPY Cargo.toml Cargo.lock ./

# Build and cache the dependencies
RUN mkdir src && echo "fn main() {}" > src/main.rs
RUN cargo fetch
RUN cargo build --release
RUN rm src/main.rs

# Copy the actual code files and build the application
COPY src ./src/
COPY assets ./assets/
# Update the file date
RUN touch src/main.rs
RUN cargo build --release

# STAGE2: create a slim image with the compiled binary
FROM debian:bullseye as runner

EXPOSE 7777/udp

# Copy the binary from the builder stage
WORKDIR /quantsum
COPY --from=builder /quantsum/target/release/quantsum quantsum
COPY --from=builder /quantsum/assets assets
RUN apt-get update && \
    apt-get install -y libasound2-dev libudev-dev
CMD [ "./quantsum", "dedicated-server" ]

fn insert_server(
    server_has_player: bool,
    mut commands: Commands,
    mut gamemode_controller: ResMut<GameModeController>,
    game_mode: Option<GameMode>,
    map: Option<String>,
    port: u16,
) {
    let server_addr = SocketAddr::new(IpAddr::V4(Ipv4Addr::UNSPECIFIED), port);
    if let Ok(socket) = UdpSocket::bind(server_addr) {
        let server_config = ServerConfig {
            current_time: SystemTime::now()
                .duration_since(SystemTime::UNIX_EPOCH)
                .unwrap(),
            max_clients: 64,
            protocol_id: PROTOCOL_ID,
            public_addresses: vec![server_addr],
            authentication: ServerAuthentication::Unsecure,
        };

        if let Ok(transport) = NetcodeServerTransport::new(server_config, socket) {
            commands.insert_resource(ids::IdPooler::new(server_has_player));
            commands.insert_resource(player::LocalPlayer {
                client_id: ClientId::from_raw(0),
                ..default()
            });
            commands.insert_resource(RenetServer::new(crate::net::connection_config()));
            commands.insert_resource(transport);

            let map = if let Some(map) = map {
                map
            } else {
                "m4".to_string()
            };

            let game_mode = if let Some(game_mode) = game_mode {
                game_mode
            } else {
                GameMode::Coop
            };

            gamemode_controller.set_map_and_gamemode(&map, game_mode);
        }
    }
}```

[hollow glade]

Thank you! Only obvious difference I see is the use of an “unspecified” server addr but I think that is equivalent to 0.0.0.0:0. Will take a close look in a bit. I may try a different base image to see if it makes a difference.

What does your client side look like as far as connecting to a local container?

[hollow karma]

Port is the external port

                let client_id = utils::timestamp_millis_since_epoch();

                let authentication = ClientAuthentication::Unsecure {
                    server_addr: SocketAddr::new(client.ip, client.port),
                    client_id: client_id,
                    user_data: None,
                    protocol_id: PROTOCOL_ID,
                };

                let socket =
                    UdpSocket::bind(SocketAddr::new(IpAddr::V4(Ipv4Addr::UNSPECIFIED), 0)).unwrap();

                let current_time = SystemTime::now()
                    .duration_since(SystemTime::UNIX_EPOCH)
                    .unwrap();

                if let Ok(transport) =
                    NetcodeClientTransport::new(current_time, authentication, socket)
                {
                    commands.insert_resource(ids::IdPooler::new(false));
                    commands.insert_resource(client.clone());
                    commands.insert_resource(player::LocalPlayer {
                        client_id: ClientId::from_raw(client_id),
                        ..default()
                    });
                    commands.insert_resource(RenetClient::new(crate::net::connection_config()));
                    commands.insert_resource(transport);
                }

[hollow glade]

External port being the 7777 you have exposed and I assume mounted to the host?

[hollow karma]

yep

[hollow glade]

Thanks for the examples, got it working!

[heady root]

Hello - I feel like I'm missing something really obvious, but trying to run any of the renet steam examples gives me:
thread 'main' panicked at renet_steam\examples\echo.rs:147:84: called Result::unwrap()on anErr value: InvalidHandle

I have Steam open, I'm on Windows.

Thank you!

[eternal crater]

What command you used to run the example?

[heady root]

• cargo r --example echo server
• copied the id from here: SteamInternal_SetMinidumpSteamID: Caching Steam ID: <THIS_STEAM_ID> [API loaded no]
• cargo r --example echo client <ID>

[heady root]

Happens for the demo_bevy example as well

[eternal crater]

Does this happens with the server or client. Are you running both in the same machine?

[heady root]

Server starts and works, client fails SteamClientTransport::new(...) call and panics on unwrap. Running on the same machine.

Can it not run on the same machine this way? 🤔

[eternal crater]

No, you can have only one app running with steam. Need to add it to the README. You need another machine (could use a VM) with another steam account running

[heady root]

Oh, looking at the example .webp I really count not tell

[eternal crater]

If you want you can try connecting to my server just to check:
cargo r --example echo client **********

[heady root]

Thanks - I managed with my laptop and a 2nd account as you said

[eternal crater]

Nice

Yeah, if you have any problem you can ping me, renet_steam is new, so bug reports are welcomed, also any problems with the API

[heady root]

Thank you, much appreciated

[hollow glade]

What’s the right way to ensure a client is connected to a server before proceeding into systems that send messages? I’m trying to work out a “server selection” menu and want to confirm I’m actually connected before asking for map information, for example.

[normal bay]

There is the client_connected run condition for the client. Or ServerEvent on the server.

[hollow glade]

It looks like that run condition evaluates to true even when the connection status is "connecting", which isn't actually connected yet..

hmm actually I was wrong, it does wait for RenetConnectionStatus::Connected let me try rejiggering some things

[bleak token]

@eternal crater May I ask you to take a look at this issue before the next release? https://github.com/lucaspoffo/renet/issues/153
It should be quite simple to add. Needed for bevy_replicon_renet to work with IDs in a transport-independent way.

[prime imp]

I think the idea of a client id is very transport specific

not all transports will use a u64 client id

I use a slotmap key for example, another might use a SocketAddr (but i wouldnt recommend this), etc.

[bleak token]

But I think it could be represented as u64, right?

[prime imp]

currently for replicon integration I use uh

1 sec

#[derive(Derivative, Resource)]
#[derivative(
    Debug(bound = "T::ClientKey: Debug"),
    Clone(bound = "T::ClientKey: Clone")
)]
pub struct ClientKeys<P: TransportProtocol, T: ServerTransport<P>> {
    id_map: BiHashMap<T::ClientKey, ClientId, ahash::RandomState, ahash::RandomState>,
    next_id: ClientId,
}

T::ClientKey being the transport's underlying client id type, and ClientId from replicon

[bleak token]

Got it, makes sense.

[prime imp]

so the client key can't safely be represented as a u64, but it can be mapped like this

[bleak token]

It would be great to have something like this in renet as well...

[prime imp]

wdym exactly

the associated type for client keys?

or a mapping between ClientId and a T?

[bleak token]

Mapping, yes

[prime imp]

ehh

just use a bimap::BiHashMap

[bleak token]

I mean in get an ID from renet, I have to interact with transport which is not good. For each new transport I have to set client id.

The only reason why client ID is optional is because I want to support switching transports. If you disable netcode, the ID won't be set. And user need either to set it manually or write an integration for my integration with renet 😅

[prime imp]

yeah it gets complicated when trying to use types from different crates

imo the best solution would have been an associated client id type, but its sorta too late for that

so the best you can do is just map between their client ID and your client ID type

[bleak token]

You mean generics?

[prime imp]

yea

trait Transport {
  type ClientId: Send + Sync + ...;
}

[bleak token]

This would require generics all over the place.

[prime imp]

that's true

but i haven't found it to be a dealbreaker for my code

and usually i dont like excessive generics

all of my event types have 2 type params, 1 for protocol, 1 for transport

[bleak token]

I think mapping IDs are totally fine. Especially when mappings are simple 1 to 1 without a lookup.

[prime imp]

yeah the fact that it's 1 to 1 helps a lot

makes it not a nightmare to deal with

just a tiny bit of overhead

or make it a u128 and let people pack whatever data they want in that 😈

[bleak token]

Is your ID bigger then u64?

[prime imp]

pub struct KeyData {
    idx: u32,
    version: NonZeroU32,
}

its same size as u64 actually

but you cant guarantee that ofc

because of rustc

[bleak token]

The size will always be 64 bit.

[prime imp]

technically it's not guaranteed because of padding and whatever

if it was repr(C) it would be tho

[bleak token]

Yes, but you don't need to do it.

Just create a new u64 and bitshift your data into it.

[prime imp]

hmmmmm true

[bleak token]

And you will get rid of the mapping.

[prime imp]

im not a fan of relying on slotmap's KeyData layout though

since i dont have control over slotmap

actually

[bleak token]

Good, I think like it would be applicable in this case.

But the discussion goes beyond renet :)

[prime imp]

oh right, I can't actually rely on this, because T::ClientKey might not be from a slotmap

[prime imp]

shit i thought we were in #networking

mb

[bleak token]

You probably could create a trait and implement it using as_ffi for key from slotmap.

[prime imp]

that forces all users of a transport to have a key which can be converted to/from a u64

so the replicon integration only works for transports whose client keys can be converted to/from u64s

and i would much rather just use a mapping than an encoding like this

[bleak token]

I don't think that it's a hard limitation, but it's up to you.

[prime imp]

well if the hashmap ever pops up in a profiler, then maybe it would be reason to switch over

for now i gotta get the actual ordering working 😭😭😭

[eternal crater]

As Boris said, the client_id is transport specific. Adding to the RenetClient will only be used for the getter of it, it's not necessary for the renet protocol, only for the transport part.

@bleak token How is the ID used in the Client renet integration?

[bleak token]

Yes, I know that it's not used by renet protocol, this is why I suggesting it only as a getter for convenience.
People quite often want to get current client ID for their game logic and they have to do it in transport-dependent way.

[scarlet cradle]

speaking of replacing things with a hashmap, I did exactly that with connection tokens

kinda had to since I set 64k max clients

seemed like the code was trying to act like a hashmap that avoided allocations anyways, if allocation amortization was the reason for this it's a non-issue as far as I can tell

[eternal crater]

Yeah, I need to do a refactor pass on the renetcode crate. There is some cleanup that can be done, somethings are like that because It was ported from C.

[scarlet cradle]

aye, fair, just an observation i made while refactoring things

[bleak token]

@eternal crater and in replicon integration I set it like this: https://github.com/projectharmonia/bevy_replicon/blob/master/bevy_replicon_renet/src/lib.rs#L206
But for other transports people need to set it manually.

[zealous wigeon]

hi! for some reason my server drops clients randomly after a few minutes with a transport layer error. it does that regardless of whether they are running on the same machine or a dedicated server/client. i had similar symptoms when i forgot calculate the delta time for server update. any ideas what it could be?

[hollow karma]

hmmm just off the top of my head the first thing I'd do is ensure I'm handling errors everywhere I interact with Renet, and print out that error

are you doing that?

oh, wait, what's the error?

[zealous wigeon]

oh, not an actual error, just a client disconnect event with the reason "connection terminated by the transport layer"

[hollow karma]

any chance you can post some code or too complicated?

[zealous wigeon]

i'll try..

https://gist.github.com/xyaon/8570c07cbad29d8fe4785a0f9f73cdcb

one sec, ill add the client code too

client_sync just reads, and player_movement just sends one input message

does the bevy renet client need any updating like the regular one that i'm missing or is that automatic?

[hollow karma]

automatic

[zealous wigeon]

also, in wireshark, i could see that the client was still sending packets after the server stopped responding, and it didn't throw any errors

[hollow karma]

I'm not too familiar with just renet, only bevy_renet, but I honestly don't see where the problem could be

How often is the client sending to the server?

hopefully Poffo sees this

[zealous wigeon]

it only sends messages on input, but there is constant traffic back and forth even without that

[scarlet cradle]

renet sends acks every frame is what you're seeing there

[zealous wigeon]

that was my assumption, also, i misremembered, the input data is sent every fixed update, regardles in input state

oh, and when multiple clients are connected, only one is dropped at a time

[eternal crater]

You can get the transport error from the emitted event NetcodeTransportError:
transport_error: EventReader<NetcodeTransportError>

This will help debug the problem

[eternal crater]

@zealous wigeon

[zealous wigeon]

unfortunately the server doesn't run inside bevy

i guess i could try to insert some debug statements inside the bevy server, i just didn't really want to touch crate code

i have that event reader on the client, but it doesn't report any errors

[eternal crater]

Ok, I took a look at the code, my guess it's because the server is running in a loop at high frequency.
Try adding a sleep inside the loop:
std::tread::sleep(Duration::from_millis(16));

[zealous wigeon]

sure, I'll try that

i saw that delay in examples, but wrote mine under the assumption that it would have enough clients, didn't really think of how i would test it tho

[near juniper]

Add in some intelligent sleep, that sleeps for a while if it's not under high enough load

Like framepacing

[zealous wigeon]

yeah, i was thinking of that too

but i'm still not sure if this solves the problem

i don't have a reliable way to reproduce it, just waiting until it happens

unfortunately it didn't :{

one of the three clients was disconnected without any error reported on its side

[zealous wigeon]

i tried printing every message, and for some reason it does seem to be stable. maybe 16ms just wasn't enough

[zealous wigeon]

i may have misread something, the client stops responding before the server disconnects it.

[zealous wigeon]

the problem seems to be with the client. the system just stops running, but i don't see any errors and the app doesn't freeze

[eternal crater]

You can try enabling the logs on the client side, it should print something or emit an event with the error

Maybe its the client_id that is generating repeated values if running the clients at the same time.

[zealous wigeon]

i tried logging both messages as well as rtt and packet loss. it would eventually stop printing for like 20 secs and then resume printing the same rtt as before and an increasing rate of packet loss. the first thing that system did was print the logs, so it stopping would mean that either it, or some other system was blocking something.

it does print transport errors when for example i stop the server, but not here.

it's a debug build with editor_pls, so there might be some interference

Oh, good news. i had done those previous test without the thread sleep, because that seemed to improve it and i was trying to reproduce it to see the logs.

if delta_time.as_millis() < 32 { thread::sleep(Duration::from_millis(32) - delta_time); };

with this, it has been running fine with 3 clients for half an hour

thanks for the help :}

[hollow karma]

Is it possible to query a server without connecting to it? My use-case is calculating ping for a server browser.

[hollow karma]

I can think of a way of doing it but curious if this is built in

[hollow glade]

@hollow karma I’d be curious what you came up with if you can share

[normal bay]

@fleet crescent are you around? I'm struggling to run the webtransport client example.

[fleet crescent]

Hey yeah, so the pkg file was created from the rust example when you build it

[normal bay]

Built how? Does it need an index.html?

I ran this but it's rejecting the files no matter what I do:

cargo build --target wasm32-unknown-unknown --release \
&& wasm-bindgen --no-typescript --out-name echo_client_wt --out-dir pkg --target web ../../target/wasm32-unknown-unknown/release/echo_client_wt.wasm

[fleet crescent]

It needs a index.html yeah, but i mean you need to build the lib. The path is: renet_webtransport/examples. It was either just a Cargo build or a command from the wasm doc page, but it was a one liner for sure and not something crazy like that

Ahhh I think it was wasm-pack build

[normal bay]

I'll try that out thanks!

[normal bay]

Cool it actually runs :) now to debug all the stuff I modified...

I forked renet and added a bunch of stuff, including code from your PR: https://github.com/UkoeHB/renet2/tree/main

Still WIP but will announce it once it actually works.

[fleet crescent]

Wuhu good to hear 🙂 Yeah debugging is quite annoying in wasm

[frail wren]

what is the diffrence between NetcodeServerTransport and RenetServer?

[bleak token]

NetcodeServerTransport is a transport, you interact with it when you create server.
RenetServer is for transport-agnostic way to interact with the server, you use it after server creation.

[normal bay]

@fleet crescent the dream is alive

[fleet crescent]

Wow that's amazing! Thank you and thanks for bringing this PR to an end with security! 🙂

[normal bay]

I need to do a bunch of polish, and clean up the commit history, before it's really presentable. But it works! Woohoo

[fleet crescent]

Yeah thats great! ^^

[bleak pecan]

i have made compiling client and server but when i run server then client i dont see connected message in the server console

server:

fn main() {
    let mut app = App::new();
    app.add_plugins(RenetServerPlugin);
    
    let server = RenetServer::new(ConnectionConfig::default());
    app.insert_resource(server);

// Transport layer setup
app.add_plugins(NetcodeServerPlugin);
let server_addr = "127.0.0.1:5000".parse().unwrap();
let socket = UdpSocket::bind(server_addr).unwrap();
let server_config = ServerConfig {
    current_time: SystemTime::now().duration_since(SystemTime::UNIX_EPOCH).unwrap(),
    max_clients: 64, 
    protocol_id: 0,
    public_addresses: vec![server_addr], 
    authentication: ServerAuthentication::Unsecure
};
let transport = NetcodeServerTransport::new(server_config, socket).unwrap();
app.insert_resource(transport);
app.add_systems(Startup, handle_events_system);
}

fn handle_events_system(mut server_events: EventReader<ServerEvent>) {
    for event in server_events.read() {
        match event {
            ServerEvent::ClientConnected { client_id } => {
                println!("Client {client_id} connected");
            }
            ServerEvent::ClientDisconnected { client_id, reason } => {
                println!("Client {client_id} disconnected: {reason}");
            }
        }
    }
}

client:

    let mut app = App::new();
    app.insert_resource(AmbientLight {
            color: Color::WHITE,
            brightness: 10000.0,
        });
        app.insert_resource(ClearColor(Color::hex("D4F5F5").unwrap()));
        app.insert_resource(RapierConfiguration::default());
        app.add_plugins(DefaultPlugins);
        app.add_plugins(RapierPhysicsPlugin::<NoUserData>::default());

        app.add_plugins(RapierDebugRenderPlugin::default());
        app.add_plugins(FpsControllerPlugin);
        app.add_plugins(
            WorldInspectorPlugin::default().run_if(input_toggle_active(true, KeyCode::KeyL)),
        );
        app.insert_resource(Initem("default".to_string()));
        app.insert_resource(HandItem(100));
        app.insert_resource(AOItems(2));
        app.add_systems(Startup, setup);
        app.add_plugins(GameUI);
        app.add_plugins(UserInv);
        app.add_systems(
            Update,
            (manage_cursor, scene_colliders, respawn, find_interactable,upd_handitem_vis),
        );
        app.add_plugins(RenetClientPlugin);
        let client = RenetClient::new(ConnectionConfig::default());
        
        // Setup the transport layer
        // app.add_plugins(NetcodeClientPlugin);

        let authentication = ClientAuthentication::Unsecure {
            server_addr: "127.0.0.1:5000".parse().unwrap(),
            client_id: 0,
            user_data: None,
            protocol_id: 0,
        };
        let socket = UdpSocket::bind("127.0.0.1:0").unwrap();
        let current_time = SystemTime::now().duration_since(SystemTime::UNIX_EPOCH).unwrap();
        let mut transport = NetcodeClientTransport::new(current_time, authentication, socket).unwrap();

        app.insert_resource(transport);



        app.insert_resource(client);
        app.run();

any help is appreciated

[eternal crater]

I think the problem is at the server, when adding the handle_events_system, it should be:
app.add_systems(Update, handle_events_system);

[bleak pecan]

i have changed it to update but still nothing, should the server use app.run(); ?

[hollow glade]

Yes

[bleak pecan]

when i add the run it fails to run because of Resource requested by bevy_renet::RenetServerPlugin::update_system does not exist: bevy_time::time::Time

[eternal crater]

You need to add the DefaultPlugins in the server also

[bleak pecan]

i have done that but it still doesnt connect, no messages of client connecting in the server

[hardy cape]

Hello! I am trying to connect to my server w/ my external ip. I have verified the correct IP is being encoded into the ClientAuthentication::Unsecure enum when trying to connect. However, when I try to connect no connection can be established with the server, and nothing gets logged in the server's console. I pinged my external IP's UDP port and it did hit the server (I got an error message about too small of a packet in my server's console), so I know it is possible to connect to the server using that IP.
Client log when trying to connect:

 INFO cosmos_client::netty::connect: Connecting to [myip]:1337
ERROR log: Failed to update client: client has no more servers to connect    

And nothing gets printed into the server console.

The code to create the ClientAuthentication enum (in client):

let auth = ClientAuthentication::Unsecure {
    client_id,
    protocol_id: PROTOCOL_ID,
    server_addr,
    user_data: Some(token),
};

info!("Connecting to {server_addr}");

NetcodeClientTransport::new(current_time, auth, socket).unwrap()

Any clue why my client can't reach the server?

For context, netcat -v -u -z [myip] 1337 does reach the server, and the client is able to connect if my local IP address is used instead.

[eternal crater]

Can you show the server setup/configuration?

[hardy cape]

let public_addr = format!("{addr}:{port}").parse().unwrap();

let socket = UdpSocket::bind(format!("0.0.0.0:{port}")).unwrap();
socket.set_nonblocking(true).expect("Cannot set non-blocking mode!");

let current_time = SystemTime::now().duration_since(SystemTime::UNIX_EPOCH).unwrap();

let server_config = ServerConfig {
    max_clients: 20,
    protocol_id: PROTOCOL_ID,
    public_addresses: vec![public_addr],
    current_time,
    authentication: ServerAuthentication::Unsecure,
};

let transport = NetcodeServerTransport::new(server_config, socket).unwrap();
let server = RenetServer::new(connection_config());

[normal bay]

You are binding to 0.0.0.0. Your public_addr needs to be socket.local_addr().

[hardy cape]

Just tried this and I get the same result

[normal bay]

Best guess is whatever you're doing with server_addr on the client is not the right address.

[hardy cape]

This is the NetcodeClientTransport creation logic I have in the client:

let addr = format!("{host}:{port}"); // [myip]:1337
let server_addr = addr
        .parse()
        .unwrap_or_else(|e| panic!("Error creating IP address for {host}:{port}. Error: {e:?}"));

let socket = UdpSocket::bind("0.0.0.0:0").unwrap();

socket.set_nonblocking(true).expect("Unable to make UDP non-blocking!");

let current_time = SystemTime::now().duration_since(UNIX_EPOCH).unwrap();
let client_id = current_time.as_millis() as u64;

let name = "CoolPlayer";

let mut token = [0; 256];

// Bincode because this is stored un a u8, with a fixed length of 256
let serialized_name = bincode::serialize(&name).expect("Unable to serialize name");
for (i, byte) in serialized_name.iter().enumerate() {
    token[i] = *byte;
}

let auth = ClientAuthentication::Unsecure {
    client_id,
    protocol_id: PROTOCOL_ID,
    server_addr,
    user_data: Some(token),
};

info!("Connecting to {server_addr}"); // this does print "Connecting to [myip]:1337"

NetcodeClientTransport::new(current_time, auth, socket).unwrap()

I don't see anything out of the ordinary here, or that would otherwise restrict connections

[normal bay]

Did you explicitly verify that server_address matches the socket.local_addr on your server?

[hardy cape]

I tried it with let public_addr = socket.local_addr().unwrap(); in the server, and I got the same result - works on local IP but not external IP.

[eternal crater]

In the server:
let public_addr = format!("{addr}:{port}").parse().unwrap();
addr is the public ip?

[hardy cape]

Yes

[normal bay]

That's not what I asked. I asked if you printed out socket.local_addr() on the server, and printed out server_address on the client, and they are the same value.

[hardy cape]

Ah my bad, the client prints [myip]:1337, and the server is the wildcard ip: 0.0.0.0:1337

[eternal crater]

Can you try accessing the server with another computer?
This could be a harpin NAT problem that could be from firewall/router

Prob not since you can ping it using the netcat, but never know

[hardy cape]

I'll check on my laptop, but it'll take a sec to compile it

[hardy cape]

looks like my laptop on the same network can't connect even using the local IP. I don't think it's a firewall/router issue, because I ran an http server on the same port and tried to connect on my laptop, and that worked fine, even with the non-local ip. I ran tcpdump -ni any port 1337 and am not getting anything when I try to connect to the server w/ my laptop. I do get stuff logged when I connect to the http server via my laptop. Could the client be rejecting the outgoing connection?

[normal bay]

socket.local_addr() will not print the wildcard... it will print the bound address.

[hardy cape]

I bound 0.0.0.0 to it

let socket = UdpSocket::bind(format!("0.0.0.0:{port}")).unwrap();
info!("Server Local Addr: {:?}", socket.local_addr()); // -> 2024-07-03T23:46:09.606856Z  INFO cosmos_server::init::init_server: Server Local Addr: Ok(0.0.0.0:1337)

socket.set_nonblocking(true).expect("Cannot set non-blocking mode!");

let public_addr = socket.local_addr().unwrap();

let current_time = SystemTime::now().duration_since(SystemTime::UNIX_EPOCH).unwrap();

let server_config = ServerConfig {
    max_clients: 20,
    protocol_id: PROTOCOL_ID,
    public_addresses: vec![public_addr],
    current_time,
    authentication: ServerAuthentication::Unsecure,
};

let transport = NetcodeServerTransport::new(server_config, socket).unwrap();
let server = RenetServer::new(connection_config());

app.insert_resource(ServerLobby::default())
    .insert_resource(NetworkTick(0))
    .insert_resource(ClientTicks::default())
    .insert_resource(server)
    .insert_resource(transport);

info!("Public address: {public_addr}"); // -> 2024-07-03T23:46:09.606955Z  INFO cosmos_server::init::init_server: Public address: 0.0.0.0:1337

[normal bay]

Hmm I was mistaken then. Someone else will have to help, I'm at a loss here.

[hardy cape]

All good, I appreciate the help

[bleak token]

@eternal crater any chance you could consider adding a simple getter for ClientId for RenetClient?
Because right I can automatically set it only for NetcodeClientTransport:
https://github.com/projectharmonia/bevy_replicon_renet/blob/dfd1a584e83ddff6a2b453a8948f89f850530ea9/src/lib.rs#L210

For all other transports I need to ask users to set it manually :(

[normal bay]

RenetClient doesn't know the client id

Would require changing the constructor

[bleak token]

Yes, that's what I asking to provide.
Like we have is_connected information duplicated from the transport.

[eternal crater]

My problem with this, is that I would need to pass it to the constructor just for the getter. The client does not use it

[bleak token]

Maybe you could wrap it into Option and set to Some after the transport is set up?

[near juniper]

Maybe change the connected state enum to have a Connected(ClientId), so that when the transport changes the status to connected it has to provide a ClientId, and then just have a getter that returns an Option (which is then guaranteed to always be Some if it's connected)

[bleak token]

Makes total sense, that's how we doing it in bevy_replicon.

[bleak token]

@eternal crater Could you draft a new release for Bevy 0.14?
There is PR for it: https://github.com/lucaspoffo/renet/pull/161

[glad sonnet]

yes please, I'm just waiting for the renet update so I can update to 0.14 my sandbox game which is a bit sad

[eternal crater]

Just released a new version with the bevy update. If there is any problem you can ping me or open an issue 👍 @bleak token @glad sonnet

[grim orchid]

Is there some timing consideration to be aware of on server side?

I have a basic move system (forward/backward) that moves my player way slower than expected (like 5px/second instead of 90px/second).

I copy pasted the exact same move logic on client side and it works perfectly.

fn handle_move_events(
    mut move_events: EventReader<FromClient<Move>>,
    mut players_query: Query<(&mut Transform, &PlayerMarker)>,
    time: Res<Time>
) {
    for FromClient { client_id, event } in move_events.read() {
        'inner: for (mut transform, player) in &mut players_query {
            if player.client_id == *client_id {
                let (direction, mut speed) = match event.move_direction {
                    MoveDirection::Forward => (transform.up(), 3.),
                    MoveDirection::Backward => (transform.down(), 1.5)
                };

                if event.is_running { speed *= 2. }

                transform.translation += direction.normalize() * ONE_METER * speed * time.delta_seconds();
                break 'inner;
            }
        }
    }
}

Here are the two behavior, on client(1st) & server(2nd) side

[grim orchid]

Wake up revelation! not tested yet, still in the bed

Because it's a continuous move, my problem is probably this:
While the Move message goes to the server and the new position is returned to the client, multiple frames occurs on the client side.

Because I rely on the delta time, the move is correctly calculated but only for this server frame. When the new position get back to the client, probably 5 or 6 frames were executed and ignored on the client side.

The solution are:

• calculate the move not on the delta time but on the delta from when the last message was sent back to the client (question included: does renet have some kind of registry for the events I could rely on?)
• translate the entity on both the client and the server, my move message should include the current position and the direction, the server would only be a validator and calculate if my character could have done this move from the server-known position to the client-known position, if it's ok, the server only returns a validation, if not, the server returns the last possible position.

[bleak token]

Correct. But this question is more suited for #1090432346907492443 channel.
Also if your game is fast-paced, you will need client-side prediction and input queuing.

[grim orchid]

Wasn't sure for the channel yeah.

[vocal aspen]

thanks @normal bay (linking from [here](#1090432346907492443 message))! maybe dumb question -- how do i reduce the tick rate? do i need to copy the plugin code to my own implementation? i didn't see a setting when i looked through, but maybe i missed it

[normal bay]

You can control bevy's tick rate with bevy::app::ScheduleRunnerPlugin::run_loop for headless apps

[vocal aspen]

oh of course, overall tick rate, not something renet specific. thanks!

[normal bay]

Opened an issue for this https://github.com/UkoeHB/renet2/issues/20. Not sure when I'll have time to properly look at it.

[vocal aspen]

i appreciate it!

[vocal aspen]

i'm new to the code, and this might be an intentional part of the netcode protocol, but:

it seems like the "empty" payload being sent is a Packet::Ack.

whenever a packet is sent pending_acks is set to sequence..sequence + 1 here. that range will never return false for is_empty here.

because of this, there is always a new Packet::Ack to send, even when all packets have actually been acked.

would that make sense at all?

[vocal aspen]

i’ll move it to the issue 🙂

[normal bay]

Sounds plausible! I assume this is either a bug or meant as a keep-alive by netcode.

[eternal crater]

Yeah it's intentional, if no packets are being sent, both will be exchanging acks between them.

I have made draft that adds a delay for this to reduce this workload and add a delay to acks:
https://github.com/lucaspoffo/renet/pull/156

the ack can ackt as a keep alive as you think @normal bay, this PR seems to work pretty well, but never finished it and tested completed, I think the rrt calculation were being messed up since before, there was never a delay in acks.

[normal bay]

Ah got it!

[eternal crater]

Just added a comment explaining a bit about it if you ever intend to use it

[eternal crater]

BTW, renet ack system is pretty similar to the QUIC one. Ack ranges and how it serialized comes from there:
https://www.rfc-editor.org/rfc/rfc9000.html#name-packetization-and-reliabili

[vocal aspen]

interesting! if nothing else this made me realize i was running my server with an unbounded tick rate, so thank you 🙂

[bleak token]

@eternal crater I see that you updated the crate to 0.15, could you draft a new release on crates.io?

[eternal crater]

Yeah I will try this weekend or monday to make a release

BTW, this will be a 1.0 release, the library seems pretty stable for me and the API should not change much after the release

If anyone who is using renet wants to showcase it, I can put a link/image in the 1.0 release (would be nice).
Some people I know that are using: @slim torrent @odd palm @bleak token

[odd palm]

Hello 👋 . Btw I am not using bevy_renet, but just renet. And yeah it would be nice if you put a link to my game in there https://crystalrealmsgame.com/ :)

[bleak token]

By any chance you could also address this one?

https://github.com/lucaspoffo/renet/issues/153

Just a getter, similar to https://github.com/lucaspoffo/renet/pull/164

[slim torrent]

You can put https://gunbug.xyz/ (you can also download the image from that page and use however you want)

[bleak token]

This looks sick!

[odd palm]

Thanks!

[bleak token]

Added screenshots to my game's GitHub page: https://github.com/projectharmonia/project_harmonia

The game uses https://github.com/projectharmonia/bevy_replicon, an I/O-agnostic library for replication, and https://github.com/projectharmonia/bevy_replicon_renet, which integrates Renet as its messaging backend. Not sure if it's relevant, but decided to share 🙂

[rare warren]

Hi all, does anyone have any information what happens if I want to make a server without Bevy as a separate project? on the renet library, but bevy_renet will send packets to regular renet, will something break?

[normal bay]

bevy_renet is a thin wrapper over renet, I recommend checking the implementation.

[hollow karma]

yeah like koe suggested, I dont see why it wouldn't work. Just match whatever settings bevy_renet uses

[wispy ferry]

does anyone know what 2025-03-13T22:21:42.548331Z ERROR renetcode::server: Failed to process packet: no private key was found for this address means?

[hollow karma]

I may have discovered a silent failure with Renet's Unreliable messaging

So, I'm resending inputs 3 times to the server, over an Unreliable channel. I was not getting any logs saying packets were dropped, or any Renet logs at all.

What I was experiencing was the server receiving them extremely delayed, like several seconds.

How I was sending them:

• 1 message per input
• resend each frame
• at 128hz
• this means several messages being sent out per frame

Workaround/Solution:

• bundled them in an array and send them in a single message, meaning only 1 message per frame
• this completely solved the problem

[eternal crater]

Can you make a small repo of this? I was testing with the echo example and was not able to replicate, was sending 500 messages per frame at 128 hz, and all were received correctly without delay.

[hollow karma]

kk will do

Must be something on my end though, if that's the case

[eternal crater]

The messages could be silently being dropped here: https://github.com/lucaspoffo/renet/blob/aafeee251f8d15fac102c8bebf0cba6ff8c0804e/renet/src/channel/unreliable.rs#L60

You could add a debug log to check it

[hollow karma]

awesome thank you

[eternal crater]

I might actually add a real log::warn there 🤔

[bleak token]

@eternal crater looks like you forgot to release a new version of the bevy_renet on crates.io

[hardy cape]

Has anyone else used renet's steam feature and gotten this error on the server when sending a lot of data?
2025-07-27T21:11:51.753766Z ERROR renet_steam::server: Failed flush message for <my steam id>: limit exceeded

I'm trying to figure out a way to send a large quantity of data without getting this error message spammed, which causes tons of packets to be dropped and having to be re-sent.

Eventually all the data makes it, but it takes a while for everything to get through. When I was just using netcode, this was not an issue. Renet shrinks messages into 1200 byte packets, and I tried reducing the size to 1000 just to make sure the size wasn't a problem. It seems the quantity of packets being sent is the issue, but I'm not sure if there is a way to fix that 🤔

[bleak token]

Yes, I think the problem is the quantity.
There is nothing I could suggest, except try a different library. Or try to compress your data.

[hardy cape]

I think the fix that works best so far is reducing the amount of data sent per tick to a small number of KB and increasing the packet non-acknowledged resend time. This isn't an amazing solution, but seems to work well enough for now

[eternal crater]

I released the new version of bevy_renet 2.0, with support for bevy 0.16 and the newest steamworks version 0.12.2:
https://github.com/lucaspoffo/renet/releases/tag/1.1.0

[bronze cedar]

is there a standard way of shutting down the server? (while notifying all the clients instead of them getting timed out)

re-starting the server by reconstructing the resources makes clients unable to connect

[bleak token]

Released bevy_replicon_renet targeting the bevy_renet 2.0 release.

[eternal crater]

You can call transport.disconnect_all(&mut server); this will send the disconnect packets directly

[bronze cedar]

yes but what about this? #1038137656107864084 message

[eternal crater]

What transport you are using?

Are you recreating the clients also?

[bronze cedar]

netcode

[bronze cedar]

no

i just destroy the resources and construct new ones

[eternal crater]

You need to recreate the client if it was already connected to the server. If new clients can't connect, then might be a config/connection_token error.

[bronze cedar]

is there a way to notify a client that server is running at a different protocol id than the client?

[bronze cedar]

btw this was my fault i was saving player names on connect and removing them on disconnect (so every player would have an unique name) but i wasn't clearing the hashmap that contained those names when shutting the server down

[eternal crater]

If the protocol_id is different from the server/client the client will get an error when trying to connect: InvalidProtocolID.

This is useful when you change the game version and wanna make sure different versions of the game don't ever connect

[bronze cedar]

are you talking about NetcodeClientTransport::new() Result<>?, i just unwrap() that and there's no panic when connecting to a server with a different protocol id, its just treated as if there is no server and just gets timed out after a while

[eternal crater]

Yeah, the client will timeout if the protocol differs. Otherwise, it is pretty much up to the matchmaking/server coordinator/lobby system/.... to handle the different versions.

[bronze cedar]

is there a way to make it not timeout and tell the client the reason for not being able to connect?

[eternal crater]

Not really, this implementation follows this standard: https://github.com/mas-bandwidth/netcode/blob/main/STANDARD.md
That tells to ignore packets with different protocol_id. So the client will just timeout. But you should handle it before creating the ConnectToken, and return a error to the user.

Example: when the user searchs for a match, and the client it is not updated, the matchmaking should return an error and request a game restart/update to the client

This could be applied to the lobby system/ and other stuff

[bronze cedar]

what if there is no lobby system? is there no way to tell netcode not to ignore the packets with a different protocol id when there's just a client and a server?

[eternal crater]

No 🙁 .Netcode is built with in mind of something middle man that manages who generates the ConnectToken. This could be matchmaking/lobby/..

You would need to do the check yourself in some way.

Wouldn't worry about it in early development

[bronze cedar]

i guess i could do something like a query port, isn't that what some steam games do?

[eternal crater]

yeah, like a status check. Not sure what some steam games do.

I guess they would use a metadata in the lobby with the game version.

Or steam handle it automatically for you. Would need to check

[tough sandal]

hello kind people, i have a question about the renet examples, and more precisely about the demo with the pills shooting spheres on client side. everything works nicely, on local or over network. however on client input when holding a movement key down, the pill gets noticeable microtwitches. those microtwitches only appear on the client side window initiating the input. i suppose im not the first to notice this and its not something “on my machine”. im digging and poking around to find a solution, but i wonder if someone knows a fix or a direction to take. i didnt try playing with lerping in the camera follow, or maybe it needs chaining of player input and camera follow when adding systems but that didnt help so far.

help or direction appreciated, so far having great fun playing with it🙏✨

(edit: running on through xorg server native winit but probably not related)

[normal bay]

The examples are very basic, there is no client side prediction or rollback. You are seeing the result of variant latency.

[tough sandal]

Thank you for getting back at me. Thankfully the examples are basic, makes it great to poke around and try to wrap my head around whats happening!
So I take "variant latency" as you mean the server is trying to be authoritative and is sending back a transform to my own client, which overrides my own current transform "naively".
Ideal case would be if client does graceful reconciliation not exposing the timing mismatch. I assume working around a fixed server tickrate would also help smooth that out, which I guess isnt handled here.
i'll try to probe and confirm if thats the case (i guess try and filter out my own client id from incoming commands, verifying with my own eyes that's the case).
Did some googling and came across a really cool blog post https://gabrielgambetta.com/client-side-prediction-server-reconciliation.html#server-reconciliation
Thanks again ✨

[glossy dust]

I have an error that I don't understand:

Send + Sync resource renet_steam::server::SteamServerTransport initialized as non_send. It may have been inserted via World::insert_non_send_resource by accident. Try using World::insert_resource instead.

I explicitly insert the transport with world.insert_resource().

Also,

fn test<T>()
where
    T: Send + Sync + 'static,
{
    println!("Hello");
}

test::<SteamServerTransport>();

compiles, so I'm not sure what the issue is.

[stiff cradle]

i'm also having this problem, really confusing

[bleak token]

I updated renet to 0.18:
https://github.com/lucaspoffo/renet/pull/193

The author asked me to change the API slightly to avoid Renet depending directly on Bevy. This is because he released Renet 1.0, and since Bevy updates quite often, he have to bump the major version every time.

To address this, I wrapped renet types into newtypes inside bevy_renet.

I would appreciate user feedback on this.

[bleak token]

Drafted a new release of the bevy_replicon_renet with bevy 0.18 and bevy_renet 4.0.

renet no longer depends on bevy_ecs, so bevy_renet now wrap all types.
If you migrate, you need to change the imports from renet to bevy_renet (or to bevy_replicon_renet which re-exports renet). See the changes in examples in the PR above.