#bevy_replicon

1 messages Β· Page 8 of 1

dire aurora
#

Which is why you can just track the last received tick for any message separately

spring raptor
#

Could you elaborate?

#

Under comparsion I meant using %, not just compare u16 to u32 of course :D

#

But if last re-constructed tick was received longer then u16 / 2 ago, it will break.

dire aurora
#

You can network the u16 version, but then turn it into a u32 by just checking what the latest of any received type is

#

Could even include events (if those even get a tick)

#

That way it would be essentially impossible to get an 8 minute gap

#

Like if you send 0 packets for 8 minutes your connection won't even be alive anymore

spring raptor
#

I.e. it's two different ticks, one is event's tick, another is last received init tick.

#

I think I get what you meant, but I can't track separately for each message. I need to compare last received init tick with event's tick.

dire aurora
#

You don't track it separately for each message, you track 1 tick for any message received. If you receive an update that's 2 ticks newer, it goes up by 2

#

That value is used to convert all the u16 values to a full tick so you don't need to worry about wrapping anymore

spring raptor
#

Ah, I get it.

#

You are right

#

Thanks!
Maybe I even implement it for this release.

spring raptor
#

No, better with events rework, will postpone for the next release

#

Waiting for your feedback on ticks and I will draft a new release

dire aurora
#

Ah, you mean the single system thing?

spring raptor
#

Yeah, otherwise this tick that I compare to other ticks will be present in user-defined systems, not convenient. I better rework events first and then introduce the optimization.

lapis parcel
#

What happens if I spawn an entity on the server, get its ID, and immediately send a packet out?

#

Does the client receive an entity ID it doesn't know about? what does it do with it?

spring raptor
#

You can't receive an invalid ID

lapis parcel
#

Isn't there a possibility to receive a message with a client ID before it has been sync'd to the client?

spring raptor
#

On server? No, connection handing happens before sending.

lapis parcel
#

So no race conditions here?

image.png
#

spawn an entity, immediately send a packet with the entity id

#

For some reason, I dont ever get that packet on the client

#

wait I may be dumb

spring raptor
#

Ah, you are not on the latest master...

#

We have a better logging on master

#

Could you try the latest master and enable logging?

glacial ridge
lapis parcel
#

No, my mistake was not wrapping it in ToClients<>

lapis parcel
#

However, for some reason I thin Im getting the wrong mapped entity ID back

#

oh, there is a special add_mapped_client_event

#

I wonder if that could be automatic...

grave yarrow
#

@dire aurora honestly, what if we did just use FixedUpdate as the basis for rolling back/resimulating

#

systems in FixedUpdate should already expect to be ran more than once a frame

#

and will generally be worried about simulation

#

the rollback already needs to be outside of it in RunFixedMainLoop I believe

spring raptor
grave yarrow
#

the only thing I can think of more is maybe Time?

#

nvm, that should still stay the consistent tick rate...

dire aurora
#

You could add run conditions, but that seems a bit sketchy, ideally systems wouldn't be able to observe the difference between real simulations and resimulations πŸ€”

grave yarrow
#

and then finalize them at the start of a FixedUpdate tick

#

or well a game tick

dire aurora
grave yarrow
#

true

dire aurora
#

One potential solution would be to say that needs to go in FixedPreUpdate/FixedPostUpdate, and we only resimulat FixedUpdate πŸ€”

#

Would still need an extra schedule for resimulation stuff but that could be provided by the rollback crate

grave yarrow
#

well, I'm thinking all of FixedMain should be re-ran, since Pre/Post are still useful for gameplay

#

First/Last... less so, but consistency

#

@spring raptor RepliconTick not being constructable is a little bit annoying for rollback

dire aurora
#

That's already fixed in a PR ... Also you can ::default() then .increment_by() as a hack πŸ˜‚

grave yarrow
#

lol

#

@dire aurora are you thinking the entire simulation schedule is just in FixedUpdate then?

dire aurora
#

That's essentially how my simulation works, tho ofc I just run the simulation schedule in FixedUpdate πŸ˜‚

grave yarrow
#

Honestly, I think I see inputs as the same as other components to rollback

#

the only difference being they are "client authoritative" components

spring raptor
dire aurora
grave yarrow
#

ah true

#

I've also realized you kind of need 2 separate snapshots or at least a marker

#

1 for confirmed components from the server, 1 for predicted

#

ideally you didn't get the server ticks out of order, but could cause issues if you receive a more recent message and then an older message after that and overwrite the recent message

dire aurora
#

Yes, rollback is an absolute nightmare ... I have all of this stuff working, but I still need to do correct removal and despawning (which I'm hesitant to put more effort into considering how the Disabled marker PR went)

#

#1211045598854127636 message
This is with full support for out-of-order and missing messages, you can see just how out-of-order things are about halway trough the video when enemies show up (they aren't predicted yet)

#

Oh in case you're wondering how it can still roll back accurately that way: If a tick is not confirmed (nor a newer one without changed data) you load the predicted value, then before every resimulated frame after the first you check if there is authoritative (so from the server) data for that tick for that component, and load that

grave yarrow
#

ya, that's how I was thinking about it

#

I think I'll just wait for you to publish your rollback stuff and work on some character controller stuff for now

#

maybe bug people about the DefaultQueryFilters stuff

#

the more I think about networking the more I feel like it'd be fairly valuable to just have all networked components in the same place in memory

#

I'm also kind of feeling like reference counted entities might be so much better for despawning things

#

but maybe thats too extreme of a solution

dire aurora
dire aurora
#

Not using the type in rollback yet, but the changes look fine and my app works on this branch. I did ofc have to change a few things and finally remove that RepliconTick on the client side (because calling it ServerTick makes it a lot more obvious just how wrong that is)

spring raptor
#

I guess with could patch things later...

dire aurora
#

Hmmmm ... Did we add a "this was the oldest and newest tick received this frame" feature yet? πŸ€”

dire aurora
spring raptor
dire aurora
#

Hmmm, I guess with how simple Confirmed is now it would be fairly doable ... Guess I'll try that for now and see if it causes any unexpectedly long mispredictions later πŸ€”

spring raptor
#

But maybe it will be slow for you to iterate over all entities? πŸ€”

dire aurora
#

Shouldn't be a large issue, atm it's mostly that I'm not sure what needs to trigger a rollback

#

It's obvious that changes to authoritative data should trigger a rollback, but I'm not sure about events or updates to non-rollback entities πŸ€”

spring raptor
#

Got it, then yes, better to experiment with Confrirmed for now.
But I'm open to provide the necessary API if implementing it outside of replicon is slow or impossible.

#

I would rename ServerInitTick into ReceivedTicks (much better name) and add minimum and maximum ticks.

dire aurora
#

Anyway I think we're almost ready for a release, I'm pretty much at the point where I'm more blocked by bevy changes than bevy_replicon changes πŸ˜‚

spring raptor
#

Okay, I will draft a new release and if something will be needed - we add it later.
I usually do releases often. This one is exception because I wanted to flesh out the API for rollback + groups.

dire aurora
#

Frequent releases is good, that way a lot of the changes can be done outside of bevy releases ... It's really annoying when a new bevy release shows up and you have to fix every bevy change + 3 months of changes to every third-party crate you use πŸ˜‚

spring raptor
#

Especially last 4-5 releases.

grave yarrow
#

tbh idk if more releases would help here

#

I think there is more just internal pr merging issues and backlogs on that

dire aurora
#

I think the main issue is that releases get arbitrarily big because they get delayed last moment and people start cramming in all features they can πŸ˜‚

#

Last bevy release would've been tiny, it only had primitives as a real big feature, except somehow while writing the blog people merged 5 large features

spring raptor
# grave yarrow tbh idk if more releases would help here

My problem is that I have to deal with all the breakage at once. It's quite overwhelming πŸ˜… Especially when you discover a few bugs like in the previous release.
I hope that the mentioned new approach with release candidate will help.

dire aurora
#

I've hit at least one 0.x.0 bug the past 3 releases πŸ˜‚

spring raptor
#

@dire aurora one more thing!
Not sure about ComponentWorldMapper and ComponentMapper... Maybe left their implementation to users?
They aren't used inside replicon and users may try to use them on already mapped components.
The use case is very niche and users can implement them just in a few lines of code.

#

I just trying to keep things minimal, but provide the necessary API to implement anything needed on top.

dire aurora
#

The one potential risk there is that people might forget to add Replicated on spawned entities, tho I'm not sure what the actual impact of that is πŸ€”

#

Also I think the only tests on the server mapping functions are from those types right?

spring raptor
spring raptor
dire aurora
spring raptor
dire aurora
#

Yea that's the one I was talking about

#

I mainly suggested them because replicon had them before the Context types. Not sure if many people used them tho πŸ€”

spring raptor
#

Yeah, I just doubt that they were used outside of deserialization functions before...

dire aurora
#

Yea and ideally we'd have a better solution for the usecases where it would be used today ... Tho I feel like that's gonna need observers and possibly relations πŸ€”

spring raptor
dire aurora
#

Links wouldn't help much on this release since it's gonna be 0.14 at earliest when they might work for others (assuming DefaultQueryFilters gets merged, and I finish up some of the other crates anytime soon)

spring raptor
#

Okay!

dire aurora
# dire aurora Yea and ideally we'd have a better solution for the usecases where it would be u...

I think the earliest we could get this in a nicer state would require observers (hopefully in 0.14) and we add some feature to replicon to force certain entities to be grouped, and that could then later be replaced with a way to mark certain relations to require grouping them ... Tho the latter doesn't have the issue where you could encounter the entities twice, since you could just ignore those archetypes πŸ€”

spring raptor
dire aurora
#

Yea grouping is kind of the key feature for situations like this. If you can get invalid state by having two entities be at different timestamps the only solution is to either group them, or send all changes in the world as a single packet ... And the latter is not ideal when your world needs more packets than just the grouped entities alone do

spring raptor
#

One last question. Ho you have a Mastodon/Pleroma account?
I usually duplicate release announcements there and I think that it would be nice to mention you as well.

dire aurora
#

I don't ... Discord is the closest I get to social media πŸ˜‚

spring raptor
dire aurora
#

Yea once I get to the point where I can market my game I'll definitely need to work on some social media presence πŸ˜‚

#

I'll need some actual graphics first tho, because those capsules get old fast

#

Hopefully once I get to a point where I can market the game and do playtests I'll at least have all the networking stuff in a state where no one ever needs to complain about it. Not having significant networking issues is one of the top goals of my game, since networking problems are annoying for users, introduce limitations to what the game's logic can and can't do, and restricts people to playing on regional servers, making the game look dead outside of "rush hours"

spring raptor
spring raptor
dire aurora
#

I think these should work πŸ€”

image.png
distant shore
#

Is there a path/reccommended way to replicate the dont_replicate_with functionality?

spring raptor
distant shore
#

Because if not then I probably didnt need that feature in the first place

spring raptor
#

Also hierarchy is not replicated by default, unless you use SyncParent component.
It replicates the children-parent relation.

distant shore
spring raptor
distant shore
#

Perfect thanks for helping me out!

glacial ridge
#

Hi @spring raptor I'm working on https://github.com/projectharmonia/bevy_replicon/issues/245 and making some progress now. However Bevy lacking API like https://discordapp.com/channels/691052431525675048/742569353878437978/1239902003862179931 in order to complete the API. I'm probably not understand the internal of bevy and unsafe Rust enough to send a PR. Can you help with this?

GitHub
Use a single system for events Β· Issue #245 Β· projectharmonia/bevy_...

Instead of registering multiple systems, we can use a single system by using custom functions (similar to what we do for components). This will make the public API simpler and more efficient. Bevy ...

#

Here is what I have now, pretty messy for now

struct ClientEventFns {
    event_component_id: ComponentId,
    channel_component_id: ComponentId,
    from_client_component_id: ComponentId,

    send: fn(&mut RepliconClient, Ptr, Ptr),
    resend_locally: fn(Ptr, Ptr),
}

impl ClientEventFns {
    fn new<T: Event + Serialize + Debug>(
        event_component_id: ComponentId,
        channel_component_id: ComponentId,
        from_client_component_id: ComponentId,
    ) -> Self {
        Self {
            event_component_id,
            channel_component_id,
            from_client_component_id,
            send: send::<T>,
            resend_locally: resend_locally::<T>,
        }
    }
}

fn send<T: Event + Serialize + Debug>(client: &mut RepliconClient, events: Ptr, channel_id: Ptr) {
    unsafe {
        let events = events.deref::<Events<T>>();
        let channel_id = channel_id.deref::<ClientEventChannel<T>>();
        events.get_reader().read(events).for_each(|event| {
            let message = DefaultOptions::new()
                .serialize(&event)
                .expect("mapped client event should be serializable");

            info!("Sending event: {}", std::any::type_name::<T>());
            client.send(*channel_id, message);
        });
    };
}

fn resend_locally<T: Event + Serialize + Debug>(events: Ptr, from_client: Ptr) {
    unsafe {
        let mut events = events.deref::<Events<T>>();

        for event in events.get_reader().read(&events) {
            info!("Resending event: {}", std::any::type_name::<T>());
        }
    }
}
# 
#[derive(Resource, Default)]
struct ClientEventRegistry {
    events: Vec<ClientEventFns>,
}

impl ClientEventRegistry {
    fn register_event<T: Event + Serialize + Debug>(
        &mut self,
        event_component_id: ComponentId,
        channel_component_id: ComponentId,
        from_client_component_id: ComponentId,
    ) {
        self.events.push(ClientEventFns::new::<T>(
            event_component_id,
            channel_component_id,
            from_client_component_id,
        ));
    }
}

fn send_system(world: &mut World) {
    world.resource_scope(|world, registry: Mut<ClientEventRegistry>| {
        world.resource_scope(|world, mut client: Mut<RepliconClient>| {
            for event in &registry.events {
                let Some(untyped_event) = world.get_resource_by_id(event.event_component_id) else {
                    continue;
                };

                let Some(channel) = world.get_resource_by_id(event.channel_component_id) else {
                    continue;
                };

                (event.send)(&mut client, untyped_event, channel);
            }
        })
    })
}
fn resend_locally_system(world: &mut World) {
    world.resource_scope(|world, registry: Mut<ClientEventRegistry>| {
        for event in &registry.events {
            let Some(event_id) = world.get_resource_by_id(event.event_component_id) else {
                continue;
            };
            let Some(from_client_id) = world.get_resource_by_id(event.from_client_component_id)
            else {
                continue;
            };
            (event.resend_locally)(event_id, from_client_id)
        }
    })
}
# 

pub struct NetWorkEventPlugin;

impl Plugin for NetWorkEventPlugin {
    fn build(&self, app: &mut App) {
        app.init_resource::<ClientEventRegistry>().add_systems(
            PostUpdate,
            (
                send_system.run_if(client_connected),
                resend_locally_system
                    .run_if(has_authority)
                    .chain()
                    .in_set(ClientSet::Send),
            ),
        );
    }
}
dire aurora
#

What are these channel and from client component ids?

glacial ridge
glacial ridge
#

I can push to Github if you want to play with it

spring raptor
dire aurora
#

Yea ClientEventChannel<T> could just be removed since the registry would be the only place that needs it πŸ€”

spring raptor
dire aurora
#

If that channel no longer exists I think there should be no double borrow problems in either of the functions anymore πŸ€”

spring raptor
#

And instead of passing Ptrs into a function, I would just extract it from the world... The API will be safer.

#

I.e. the signature will be like this:

    send: fn(&mut World, &mut RepliconClient),
    resend_locally: fn(&mut World),
spring raptor
glacial ridge
#

yeah, but we still need mutable access to the Events<FromClient<T>

spring raptor
#

If you need a reference to anything else in the world

#

So I suggesting to just pass World to functions.

glacial ridge
spring raptor
#

We will polish the API later when we will see full picture.

glacial ridge
#

ok I will try that tomorrow, have to sleep now.

spring raptor
#

If you get stuck, I will help you finish the PR.

dire aurora
#

If my app pauses for a while (bevy does this when the app is on another desktop for some reason) it crashes after it resumes because the Confirmed::contains_any ranges get too big πŸ€”

bevy_replicon-0.25.0/src/client/confirmed.rs:72:21:
attempt to shift left with overflow
#

Not a major bug since networked apps ideally don't pause in the background, but we probably don't want it to crash either

spring raptor
spring raptor
#

I will draft a patch release after merge

distant shore
#

Is replicon still intended to be used for games where a client is also a server?

#

I'm updating a project from 0.9 -> 0.25 and before it treated my server like a client and worked fine but now its not working and the readme/getting started isnt super clear

spring raptor
spring raptor
distant shore
#

Yeah replicon updates fast lol. Is has_authority still intended to be used for a listen server?

#

has_authority isnt running for me anymore but as far as I can see my setup matches the tictactoe plugin

#

hmm, maybe it is sometimes. I'll keep debugging on my side

spring raptor
spring raptor
#

Also replicon resources always exist, unlike resources from renet.

dire aurora
glacial ridge
#

hi @spring raptor I finish the implementation for client now. Things work really well. However we lose ability to use custom send and receive system. My plan is add an enum to ClientEventFns

enum SystemType {
  BuiltIn
  Custom(Box<impl IntoSystem>)
}

when the SystemType is custom we run the system using one shot system API. Is that fine for you?

dire aurora
#

If you want to use a oneshot system you'd probably want to accept a SystemId instead of a boxed system πŸ€”

glacial ridge
spring raptor
glacial ridge
#

and seem more performance

spring raptor
#

I will help you to update the docs

#

The idea is to make API similar to components, but without markers / commands complexity.

#

Just a simple way to register event and optinally pass functions

glacial ridge
#

So let go with that. I will update PR soon

spring raptor
# glacial ridge So let go with that. I will update PR soon

One more thing. In the PR I mentioned that we probably want to have 3 channels for each event type. I think I rushed a bit with this suggestion.
Having a separate channel for each event is better for bandwith. When messaging backend sends a message, it includes the channel ID into it. And if we use only 3 channels for each event type, we will also need to include event ID.
So let's keep separate channel for each event. That's basically what your PR is doing right now.

glacial ridge
#

yep, this PR only merge system into 1, the channel still the same as before.

#

I think the PR is ready now. For the server event, I will send a seperate PR

dire aurora
spring raptor
#

I think its better do in a single because having different API for server and client events would be strange...

spring raptor
dire aurora
spring raptor
#

That's unfortunate. But let's consider channels as a separate PR to keep this one simple.

spring raptor
spring raptor
#

@glacial ridge left a review with more concrete suggestions about the API.

spring raptor
glacial ridge
spring raptor
dire aurora
glacial ridge
#

The example I added just an easyway to test the feature lol, have no idea to make it more like real world use case

spring raptor
spring raptor
glacial ridge
glacial ridge
glacial ridge
spring raptor
#

running rustdoc tests consumes a lot of ram and if you run it in parallel, it can easily eat all RAM

glacial ridge
#

yeah that might be the problem, it also often hang when build bevy from scratch and I open too many other App. I think I need to download more RAM πŸ˜…

spring raptor
#

I would probably use SerializeEventFn<T> name to distinguish it from components. And since we have two functions (serialize and deserialize), I would create an abstraction, similar to RuleFns. You can copy-pase the code, just change signatures and methods.

#

Feel free to ask follow-up questions if anything is uhnclear.

glacial ridge
#

So the NetworkEventFns should be something like

struct NetworkEventFns {
    channel_id: u8,
    send: unsafe fn(),
    resend_locally: unsafe fn(),
    receive: unsafe fn(),
    reset: unsafe fn(),
    serialize_fn: unsafe fn(),
}
glacial ridge
#

This stuff

fn add_client_event_with<T: Event + Serialize>(
        &mut self,
        channel: impl Into<RepliconChannel>,
        serialize_fn: SerializeFn<T>,
        receive_fn: ReceiveFn,
    ) -> &mut Self {}
spring raptor
spring raptor
glacial ridge
#

@spring raptor Do you want to serialize_fn return Option<Bytes> instead? Same for the dereceive_fn to return Option<T>. Because they can be failed to ser/der and we just ignore if they return None.

spring raptor
#

Just take a look at already existing SerializeFn inside the repo

#

We trying to make a simillar API :)

glacial ridge
#

I update the PR, please check the latest commit if I understand correctly your idea.

spring raptor
#

I.e. you won't need to pass World to receive and send.

glacial ridge
spring raptor
#

Right now you pass world and for each event type obtain RepliconClient and AppTypeRegistry. I suggest to obtain them inside your systems and just pass to send/receive

spring raptor
glacial ridge
#

I need to reolsve Events<T>

#

For the server event, the serialize and deserilize seem need to take &mut Bytes, is it ok to make the api for the server different from the client? I can not make it the same interface.

spring raptor
spring raptor
glacial ridge
spring raptor
glacial ridge
glacial ridge
spring raptor
spring raptor
#

It will return ComponentId which you can store in the event. Then in systems you just get the component by this ID.

#

It's a bit hard PR for a new contributor. If you want, I can finish the work for you.

glacial ridge
#

yeah please, feel free to push in

spring raptor
#

Okay :) I will be able to start working on it in a few hours.

glacial ridge
spring raptor
dire aurora
#

Unless you spam a ton of events the difference would be hard to even measure tho πŸ€”

#

The main overhead with events tends to be the number of systems and early on in the lifecycle of the app maybe the buffers getting expanded a few times

spring raptor
fluid moth
#

Hi! I am writing a client-server game and when I switch the server visibility_policy from All to Blacklist, only some of the entities are replicated (even though I haven't changed any visibility yet) and sometimes this expect fails: https://github.com/projectharmonia/bevy_replicon/blob/v0.25.1/src/server.rs#L385
I'll try to debug what is happening but would appreciate any pointers

GitHub
bevy_replicon/src/server.rs at v0.25.1 Β· projectharmonia/bevy_repli...

Server-authoritative networking crate for the Bevy game engine. - projectharmonia/bevy_replicon

spring raptor
fluid moth
fluid moth
fluid moth
spring raptor
spring raptor
#

@fluid moth released a new version with the fix.

fluid moth
spring raptor
ionic jackal
#

Hello guys, I've started a project with bevy-renet, however I found out that It's more about message delivery, and doesn't natively integrate with ECS very well, so there's a need to make a lot of boilerplate code (entity mapping, message collection for further event distribution between systems etc.).

My question: Does replicon aim for better ecs integration? Is it easier to use with bevy?

dire aurora
spring raptor
ionic jackal
#

Thank you for the replies

spring raptor
glacial ridge
willow osprey
#

i lost momentum with my bevy stuff, inc. timewarp lib for rollback. trying to get back into it and update to latest replicon api changes. noticed some chat about rollback further up - are you writing something for rollbacks that plays nicely with replicon, @dire aurora πŸ‘€ ?

dire aurora
#

Yea, I started writing a no compromise rollback implementation that actually plays nicely with all the extra issues created by client-server and using authoritative replication rather than purely deterministic replication

willow osprey
#

how far have you got?

#

(also did you look at how lightyear does it? i just started looking at that too)

dire aurora
#

A large portion of it works now, but it still requires a bevy fork for disabling entities

willow osprey
dire aurora
#

To handle the spawn/despawn edges yea

#

All the other approaches I looked at would break in my game because I have fun things like status effects being entities that aren't directly replicated

#

Hadling jitter was the biggest pain, but it works now, here's a comparison between predicted entities (the spheres) and unpredicted entities (the enemies) #1211045598854127636 message

willow osprey
#

cool, looking good. lmk if you want me to try it out – curious about your approach. i found testing rollback stuff to be rather awkward

dire aurora
#

Yea rollback is a pain to debug. I just ended up writing unit tests for every edge case I could think of

willow osprey
#

for me it's mostly tests which are a result of observing weird behaviour in my game, eventually leading to isolating wtf happened in a test

dire aurora
#

Yea that's essentially what I had too "There's something weird going on but I don't understand what", and then I wrote tests for everything related to the histories ... Still need to write tests for a few of the other things tho, but at least those are simple to debug, like it rolling back to the correct frame and running the simulation the correct number of times

spring raptor
willow osprey
#

thanks πŸ™‚ yeah, i was previously deserializing by writing to my history component. api for those callbacks all changed, but i noticed a similar thing in the docs so will try and fully absorb the docs before i pester you

#

were the api changes for rollback/custom serializers etc in pursuit of a specific goal or to support some new use-case? or just general tidying?

spring raptor
# willow osprey were the api changes for rollback/custom serializers etc in pursuit of a specifi...

Previously you just override deserialization that also function as a writing and that's where you wrote into your history component.

But it's quite limiting. You may want to predict one entity and interpolate another. Yes, you could have a check like if entity.contains<History>() {}, but it will be quite slow.
Also it's impossible to have a custom serialization and prediction/interpolation, user need to manually merge hist deserializaiton logic with a crate-provided rollback logic.

Now we have a different API. First, ser/de and writing are separated. You can override ser/de and have your custom writing for prediciton completely independent. When you register a replicated component, you assign ser/de.
And to overwrite writing (insert data into your history component, for example), you can create a marker, assign a writing function to it and select which components it will affect. We cheaply check entity markers and call appropriate writing functions for each components. You can also overwrite the default writing function that called when no markers are registered.

#

So the idea is similar to what it was before, just a bit more flexible.

#

Huge credit to @dire aurora for all of this :)

willow osprey
#

thanks for the explanation, sounds very sensible

distant shore
#

Can I get pointed to how/where replicon decides which replicated entities to send to a client? Does it use change detection or something?

#

I'm running into a really weird issue where a entity replicates normally 90% of the time but when a certain field changes it doesnt

echo lion
distant shore
#

Ok I think somehow remove_component of an unaffiliated component is breaking replication for the frame that it occurs on. I don't know enough about Replicons internals to experiment with it to be completely sure but... If I have commands.entity(entity).remove_component::<RandomComponent> call in my system then it won't send the other components on the entity but if I remove that call then it works

echo lion
distant shore
#

yeah

echo lion
#

Could be a bug around archetype changes... Can you make a minimum reproducible? Or link to a commit with the issue?

distant shore
echo lion
dire aurora
#

If it's removals it's probably other things like inserts too ... Could try making a system that removes a component if it exists, and inserts it if it doesn't, and then see if it ever replicates correctly anymore πŸ€”

spring raptor
#

I have tests for a change with despawn and insertion, but not with removal πŸ˜…
And looks I can even reproduce it... Let me try to debug

distant shore
spring raptor
distant shore
#

Mmmm, that would make sense with why it was weird and also why after I changed it it was intermittently working. Maybe also try multiple removals? Glad its for once not me doing something wrong lol

spring raptor
dire aurora
spring raptor
#

I figured it out. We collect removals first (because its serialized first) and I bump last entity tick if we find a removal. And after it we collect changes and the code thinks that all updates are already received because the entity tick is higher.

spring raptor
#

The easiest way to solve it would be to move collecting changes first, but it will break the case where we have a removal and insertion at the same tick πŸ˜…

echo lion
#

I am reviewing #262 now.

spring raptor
ionic jackal
#

Sorry for stupid question, but why RepiconTick exists? Why don't we just use u32?

grave yarrow
#

but internally it is a wrapping u32

ionic jackal
#

Makes sense

spring raptor
spring raptor
distant shore
distant shore
spring raptor
ionic jackal
#

Does replicon checks if component was changed before sending it? Or does it send all component data every tick?

I looked into source code, and it does check for changes (ComponentTicks struct contains "changed" field, which according to docs is updated on mutable dereference) but as my rust skills are still not very good, i'd like to be sure.

spring raptor
ionic jackal
#

Thanks, very cool to hear!

spring raptor
#

@willow osprey what is your plan about timewarp? Are you planning to abandon it and use lightyear for your game?

willow osprey
#

just investigating the competition at the moment πŸ™‚ i decided i need to make a stripped down client/server demo to test rollback in a more useful way, so i want to write a small sandbox to play around. seeing how things are done elsewhere before i dive back in

spring raptor
#

Ah, got it!
Wanted to ask to know if I should keep your crate in the readme.

echo lion
#

@spring raptor PRs approved

spring raptor
#

@distant shore new patch version available on crates.io

distant shore
echo lion
#

I think we should rename Confirmed to ConfirmHistory or LastConfirmed. Working on updating bevy_replicon_repair and got a little confused.

spring raptor
willow osprey
#

is there a way to use replicon with wasm now?

dire aurora
dire aurora
#

Speaking of Confirmed, did I mention we should make a getter for mask? πŸ€”

#

Somewhere in my code I transmute it to grab the mask

spring raptor
spring raptor
dire aurora
#

That was the case until I added the logic to check for newly confirmed ticks

spring raptor
willow osprey
#

thanks, hadn't seen xwt yet πŸ‘€

spring raptor
echo lion
echo lion
willow osprey
#

nice πŸ™‚ also sensible to be able to disable built in encryption as soon as browsers get involved, since they are already encrypting.

spring raptor
spring raptor
#

@dire aurora Have you thought about publishing your rollback crate and adding it to the readme as well? πŸ€”
You can just make it depend on the latest Bevy release in Cargo.toml and just mention in the readme that [patch.crates-io] section with your fork is required.

It would be at least something πŸ˜…bevy_replicon_snap doesn't have prediction and bevy_timewarp doesn't have a direct integration with replicon.

dire aurora
#

I'd need to do something about the weird Tick situation first, and I'd probably want DefaultQueryFilters to at least be merged, so people don't start writing code for something that might never get merged πŸ€”

#

In fact if it gets merged I need to first refactor some things so I actually use it instead of the older Disabled PR πŸ˜‚

spring raptor
# dire aurora I'd need to do something about the weird `Tick` situation first, and I'd probabl...

About Tick you mean the mask getter? I planning to draft a new release today or tomorrow, just want to squeeze a few more refactor PRs into it πŸ˜…

Regarding DefaultQueryFilters, I think this is an implementation detail. I mean even if Bevy decide to deny DefaultQueryFilters, I pretty sure that they decide on some other way to achieve it and in this case and you will just need to refactor the code (again, like from Disabled to DefaultQueryFilters). But it won't affect users.

dire aurora
#

No I mean the Tick vs RepliconTick thing, I still haven't fixed that yet πŸ˜…

#

It would still be use to use RepliconTick, since it's on the client, but the plugin could probably get a generic arg for Resource + Into<RepliconTick> ... That way Tick can live in the user's code until we have a official bevy solution for this

#

Also Disabled vs DefaultQueryFilters vs whatever potential other solution would affect users, since they aren't allowed to despawn entities. If you despawn entities rolling back will always produce some weird incorrect state, especially with things like status effect entities or short lived things like projectiles

spring raptor
dire aurora
#

If bevy decides on another solution, the way in which users avoid despawning entities changes, especially if the goal isn't just to leave those entities around forever but still despawn them when they fall out of history (the rollback crate would obviously need to provide some sort of API for this kind of thing)

spring raptor
granite hill
#

Hey guys :),
Currently trying to improve the client-side prediction API in bevy_replicon_snap and I could use some input.

To handle prediction two systems per Event <-> Component pair are needed:

  • A server-only system that just runs some logic: A and directly applies any mutation to the relevant component
  • A client-side system that runs/replays the same logic A multiple times from the latest state snapshot it received from the server

I am currently solving this using generic systems and the Predict<E,T> trait to define the common logic:

/// This trait defines how an event will mutate a given component
/// and is required for prediction.
pub trait Predict<E: Event, T>
where
    Self: Component + Interpolate,
{
    fn apply_event(&mut self, event: &E, delta_time: f32, context: &T);
}

pub fn server_update_system<
    E: Event,
    T: Component,
    C: Component + Interpolate + Predict<E, T>,
>(...

pub fn predicted_update_system<
    E: Event,
    T: Component,
    C: Component + Interpolate + Predict<E, T>,
>(...

which can be registered by a library user with this app extension:

app.predict_event_for_component::<MoveDirection, MovementSystemContext, PlayerPosition>()

However this feels kind of clunky since you need to have a system that pre-collects all needed data for the calculation into a context component,
it would be much nicer to just be able to pass in a system into the extension function instead.
Do you have any Idea how I could implement this or any other Ideas on how to improve the API?
For details the current main is up to date with this: https://github.com/Bendzae/bevy_replicon_snap

GitHub
GitHub - Bendzae/bevy_replicon_snap: High-level networking library ...

High-level networking library that extends the bevy_replicon library to allow snapshot interpolation and client-side prediction - Bendzae/bevy_replicon_snap

spring raptor
granite hill
spring raptor
granite hill
spring raptor
#

But I dont understand a few things:

  • Looks like you use Update and predict components separately, but why? You probably want to run all systems in FixedUpdate and replay each buffer at the same time...
  • Looks like on client you apply event directly, but I don't see when you re-apply all events logic after receiving a value from server...
#

@granite hill ^

#

it would be much nicer to just be able to pass in a system into the extension function instead.
You mean that you want to run a system instead of writing function? You can trigger a one-shot system using Bevy's Commands.

granite hill
# spring raptor But I dont understand a few things: - Looks like you use `Update` and predict co...
  1. Not a 100% sure I understand what you mean by that, I don't see a way to update different component types on the same system with my current (generics based) implementation
  2. The reapplication happens in this function, the corrected_component is the latest update from the server, and starting with that the newer inputs are replayed in the loop after that:
// Client prediction implementation
pub fn predicted_update_system<
    E: Event + Clone,
    T: Component,
    C: Component + Interpolate + Predict<E, T> + Clone,
>(
    mut q_predicted_players: Query<
        (&mut C, &SnapshotBuffer<C>, &Confirmed, &T),
        (With<Predicted>, Without<Interpolated>),
    >,
    mut local_events: EventReader<E>,
    mut event_history: ResMut<PredictedEventHistory<E>>,
    time: Res<Time>,
) {
    // Apply all pending inputs to latest snapshot
    for (mut component, snapshot_buffer, confirmed, context) in q_predicted_players.iter_mut() {
        // Append the latest input event
        for event in local_events.read() {
            event_history.insert(
                event.clone(),
                confirmed.last_tick().get(),
                time.delta_seconds(),
            );
        }

        let mut corrected_component = snapshot_buffer.latest_snapshot();
        for event_snapshot in event_history.predict(snapshot_buffer.latest_snapshot_tick()) {
            corrected_component.apply_event(
                &event_snapshot.value,
                event_snapshot.delta_time,
                context,
            );
        }
        *component = corrected_component;
    }
}
granite hill
spring raptor
granite hill
glacial ridge
#

How do I initialize ParentSync component when the field is private?

dire aurora
#

Isn't there a plugin for that?

spring raptor
#

Probably the description is a bit confusing, I should remove some implementation details from it...

spring raptor
dire aurora
#

I mean there's already a ParentSyncPlugin ... I don't load it in my app at least πŸ€”

spring raptor
#

Yep, it can be disabled completely.

dire aurora
#

We could maybe go a step further and make some feature flags to disable things people don't use ... I'll probably make a PR at some point to make some more things optional, mostly so I don't accidentally mix up client/server things between my apps ... Ofc the default would still want most of these enabled

#

ParentSync, the scene stuff and having both client/server at once seem like pretty common usecases, it's just that they could be optional

spring raptor
#

Makes sense to me.

spring raptor
spring raptor
spring raptor
glacial ridge
sharp roost
spring raptor
#

My initial thought was that since we usually only provide core functionality, things like these usually are separate plugins.
But these are simple one-file modules that are quite useful, so moving them into a separate crate doesn't make much sense, only make the maintenance harder for me.
Moving them under a feature enabled by default is an option, but yes, they don't pull other dependencies and it's just two small files (scene.rs and parent_sync.rs), it won't save compile time a lot...

dire aurora
#

The scene stuff would be a bigger issue since it does enable a bevy feature iirc

spring raptor
dire aurora
#

Why is FromClient in client::events when it's the server that needs it? πŸ€”

spring raptor
dire aurora
#

The ClientEventPlugin vs ServerEventPlugin distinction also feels weird. This wouldn't play nicely with adding features that will actually avoid adding pointless client/server system to the opposite app πŸ€”

#

Things seem to work fine after updating (well after I fixed the renames and got the plugins right after 5 wrong tries), except I now get flooded by these warnings:

WARN bevy_replicon::server::events::event_data: discarded 1 client events due to a disconnect
``` I might be able to fix some of them but should these really be sent for broadcasts? πŸ€”
spring raptor
spring raptor
dire aurora
#

I started the server, connected, then closed the client, which disconnects it after a while, then prints these messages until the server closes because there are no players

spring raptor
dire aurora
#

I didn't get these messages in the previous release

spring raptor
#

Found the root of the issue. Caused by a copy-paste, working on a fix.

dire aurora
#

Looks like that fixes the warnings

spring raptor
#

Thanks, I checked it myself, of course, but wanted to be sure.
I will draft a patch release after the merge.

spring raptor
spring raptor
#

No, it requires moving some public items.
Will do in the next release. I think drafting releases too often will disrupt plugin authors. So maybe in the next week or so.

dire aurora
#

Maybe next release will be all about adding features, the feature release that lets you make replicon have less features by disabling them πŸ˜‚

spring raptor
#

Published a small patch release with doc updates and the server event reset fix.

glacial ridge
#

is the component change dection of replicated component base on client or server? For example, when the client connected to the server that already has some entities and the client will receive relication, will this query Query<(), Added<Component>> run on client match those entites?

spring raptor
#

Are you working on reconnects?

glacial ridge
echo lion
#

All my crates are updated to bevy_replicon v0.26 now.

candid eagle
#

am i right in thinking that rooms are implemented now but not covered in the docs?

#

i got engulfed by work stuff for the last couple of months but i noticed that the rooms PR/issue got closed

dire aurora
#

There are no rooms, but visibility has been implemented

spring raptor
glacial ridge
#

is it possible to change ClientId after connect?

#

I'm trying to add mock client, I also want to act as the mock as well.

spring raptor
spring raptor
echo lion
spring raptor
echo lion
stone moss
#

i have a bunch of components like this to describe units:

#[derive(Component, Debug)]
pub struct Health {
    pub current_health: f32,
    pub max_health: f32,
}

how can i make the bundle that represents a player's stats replicated? i would rather not call app.replicate::<Health>() etc for every component

#

currently spawning like this on the server but seeing nothing on the client using bevy-inspector-egui:

pub fn spawn_test_unit_system(
    mut commands: Commands,
    mut meshes: ResMut<Assets<Mesh>>,
    mut materials: ResMut<Assets<StandardMaterial>>,
) {
    commands.spawn((
        TestUnitBundle {
            health: Health {
                current_health: 100.0,
                max_health: 100.0,
            },
            combat_stats: CombatStats {
                base_attack_damage: 10.0,
                current_attack_damage: 10.0,
                base_attack_speed: 1.0,
                attack_speed: 1.0,
                attack_range: 1.0,
            },
            movement_stats: MovementStats {
                base_speed: 1.0,
                current_speed: 1.0,
            },
        },
        PbrBundle {
            mesh: meshes.add(Capsule3d::new(1.0, 4.0)),
            material: materials.add(Color::WHITE),
            transform: Transform::from_xyz(0.0, 0.0, 0.0),
            ..Default::default()
        },
        Replicated,
    ));
}
spring raptor
spring raptor
stone moss
#

might be nice to have this as a native feature though

stone moss
spring raptor
grim mortar
grim glacier
#

Does replicon have a Resource containing the ClientId? I find I have to constantly go into the client transport and ClientId::new(transport.client_id().get()) and wonder if there's a better way.

#

The use case would be e.g. to add a leafwing_input_manager InputMap to an entity whenever a Player(ClientId) component is added where its ClientId == Res<ClientId>

iron flare
grim glacier
#

Okay, I ended up doing this as well. Naming was annoying though, I didn't want to have another ClientId besides the one from bevy_replicon and the separate ClientId from the actual transport, so now it's a Res<ClientIdResource> 😟

dire aurora
spring raptor
grim glacier
#

Wow that's wonderful thank you both! πŸ™‚

echo lion
echo lion
#

@spring raptor can you do v0.26.2 for the recent small merges?

spring raptor
spring raptor
echo lion
#

Nice thanks :)

spring raptor
#

This change in 0.14 from world field to getters hits hard, we have 400+ usages of it πŸ˜…

#

Because sometimes I need world() and sometimes world_mut().

Luckily I use Neovim, it's quite convenient for such tasks πŸ˜„

glacial ridge
#

You can use my branch, I do a lot find and replace though lol

spring raptor
#

I used it as a reference, but decided to do the replace from scratch since a lot of code changed since then and I wanted to completely replace all occurences, even docs.

spring raptor
#

@echo lion thanks for the review and sorry for a few last minute changes πŸ˜…

#

Thinking about releasing an RC version that supports 0.14.0-rc.1. I can't release bevy_replicon_renet, but other messaging won't be blocked by replicon.

spring raptor
#

Drafted a release

#

@echo lion you are still blocked by h3?
I remember that you considered integrating nevy.

echo lion
spring raptor
candid eagle
#

assuming no pre-existing code to work around, what would you guys argue is the best way to handle lobby code (tracking players joining/leaving, syncing the player list with colours, readiness to start the game etc)
Resources are the most practical for accessing from other systems, however then you have to deal with replication and message passing yourself
One invisible entity per player with components for that kind of info also sense, especially regarding replication but then detecting players joining/leaving becomes a hassel (if you have to use entity change detection)

#

i'm curious to hear how you guys implemented it

spring raptor
candid eagle
#

Yeah, thanks!

long folio
#

I have configured a max tick rate of 60, and even without any replicated entities and with a single client connected, the client receives about 75kB/s, and it seems to climb over time. After 2 minutes its at 95kB/s.

I use bevy_replicon_renet::renet::RenetClient::bytes_received_per_sec to measure.

Any ideas on what could be wrong?

dire aurora
#

I think if you enable trace logging you can see what is being sent/received

#

I think when I checked last time replicon sent out more packets than necessary, and in addition renet didn't make any attempt at grouping them, which wouldn't help with bandwidth either πŸ€”

spring raptor
spring raptor
long folio
# spring raptor Try running with `RUST_LOG=bevy_replicon=trace <cargo command>`. Maybe you have ...

Doesn't seem like it, or at least this is the only log I get repeated on the server

2024-06-08T16:31:37.055536Z TRACE bevy_replicon::server: incremented ResMut(ServerTick(RepliconTick(762)))
2024-06-08T16:31:37.055633Z TRACE bevy_replicon::server::replication_messages: no init data to send for ClientId(8450268747470345538)
2024-06-08T16:31:37.055689Z TRACE bevy_replicon::server::replication_messages: no updates to send for ClientId(8450268747470345538)
#

Removing any replicated components (as in not registering them for replication) and bundles (from bundlicon) has no effect either

dire aurora
#

What do you get on the client side?

long folio
#

Other than a log confirming connection, nothing

spring raptor
long folio
#

Was double checking with client diagnostics, and yup, nothing. Renet is sending tons of packets through, avg of 3002/s going of off log timestamps only

spring raptor
#

Because looks like we don't do anything at replicon side with it.

#

You can even try renet alone, without replicon

#

There are a bunch of examples in the renet repo

long folio
#

Logs are sadly only "received/sent" packet. Thanks for the help through, now I know its renet.

spring raptor
long folio
#

πŸ‘ will do

echo lion
spring raptor
#

Why the demo works as expected? πŸ€”

echo lion
spring raptor
#

@long folio how many replicon-registered events do you have?

long folio
#

I have not registered any events, commenting out all app.replicate::<T>() has no effect either on received bytes.

spring raptor
#

Let me try to add more logging πŸ˜…

What version are you using? 0.26.x?

long folio
#

0.26.0

#

Ill test with 0.26.2 too

#

No difference with 0.26.2

spring raptor
long folio
#

TRACE bevy_replicon::server::replicon_server: received 0 messages over channel 0
INFO bevy_replicon::client::replicon_client: sending 0 bytes over channel 0

Only getting these, both only ever on channel 0.

Client never sent anything.
Server received 1 message 150 times over 1.5~ seconds. (and received 0, 8000~ times over the same 1.5 seconds)

spring raptor
#

This is suspicious:

sending 0 bytes over channel 0

Why do we send 0 bytes?
Looks like we found a bug.

#

Let me double check the logic

long folio
spring raptor
#

Found it! It's a bug πŸ˜… I forgot to check if there are any acks after recent major refactoring.

spring raptor
#

@long folio Done!
Please, try the latest commit in send-receive-logging.

vivid quartz
#

In the example in the docs, why is receive_events the one with the run condition? Seems like if you're sending events from server to client, you should only send the message if has_authority

image.png
#

Seems like the client won't receive events

spring raptor
vivid quartz
#

Ok, just making sure I'm understanding. Thanks!

dire aurora
#

I thought we already fixed that one, someone was confused about that exact same example before iirc πŸ€”

long folio
#

25-30kB/s with a single entity and no registered components for replication through, but maybe that's unavoidable overhead?

spring raptor
spring raptor
echo lion
spring raptor
#

@long folio could you send me full logs? Just in case.

echo lion
#

Unless you are running at 600hz lol.

long folio
spring raptor
long folio
#

I should mention, the numbers are from RenetClient and not ClientStats. Through I do not use renet directly to send/receive anything

#

Having 1000 replicated entities has basically no change on the amount of data sent, if that helps

spring raptor
#

From renet

long folio
#

I can send you my project, or hop into a vc, if you want and have any debugging ideas

spring raptor
#

In the meanwhile, marking the PR as ready for review.
There was a bug on our side. @dire aurora maybe renet groups messages between channels, the issue you saw could be caused by this bug 😒

spring raptor
#

@vivid quartz ^

dire aurora
vivid quartz
spring raptor
spring raptor
#

@echo lion What do you think about keeping rc in a separate branch? I think it may provide a better experience since users may try to download the repository and they won't be able to run examples due to renet.

echo lion
spring raptor
#

Swapped branches. Now master tracks the latest release.

spring raptor
#

You can try applying this patch on the latest master to see the mentioned results.

πŸ“Ž print.patch
spring raptor
dire aurora
#

Tho realistically it doesn't really matter if a packet is 5 bytes or 200 bytes, tiny packets are awful for troughput

long folio
# spring raptor I measured the `simple_box` example from the replicon repo and I having ~420 byt...

On latest master, with the applied patch, a single client, and no movement I measure 998-1007 bytes/s up and down.
Measuring in my own project, using the master branch of bevy_replicon I measure 55000-80000 bytes/s (using the same function as the patch). This is without registering any components for replication, and with a single replicated client. Changing the max tick rate through TickPolicy on the server has no effect either.

I'll try to narrow it down later today when I'm home.

spring raptor
spring raptor
# long folio On latest master, with the applied patch, a single client, and no movement I mea...

I think you see different results on the example due to time variation.
But in your project results are strange...

Here is how I debugged it. Clone renet repo and add a local folder as a patch.
Then inspect these parts:

  1. https://github.com/lucaspoffo/renet/blob/b22876cb018523c8cb6a02614277f5dc9b5f1580/renet/src/remote_connection.rs#L503-L515 - here we read bytes from channel. packets variable after this block should have zero len when there are no changes.
  2. https://github.com/lucaspoffo/renet/blob/b22876cb018523c8cb6a02614277f5dc9b5f1580/renet/src/remote_connection.rs#L517-L524 - here renet adds his ack. packets variable should have len equal to 1.
  3. https://github.com/lucaspoffo/renet/blob/b22876cb018523c8cb6a02614277f5dc9b5f1580/renet/src/remote_connection.rs#L611 - here is where the number of bytes are calculated. bytes_sent should be equal to 7.
long folio
# spring raptor I think you see different results on the example due to time variation. But in y...

Seems like renet is sending a single packet each frame in PostUpdate. Starts with 6 bytes and grows to 9 over a few seconds. Given not much is happening on the server right now, it runs at 7000-9000 fps. Which at 9 bytes amounts to 63000-81000 bytes per sec, around what I am seeing. Also makes sense why it was fluctuating quite a bit, due to other stuff happening on my pc.
Now I guess the question is why renet is sending these packets.

echo lion
#

6 -> 9 is probably a varint growing

echo lion
long folio
echo lion
#

I haven’t studied that part of the design yet, so not sure what solution is needed exactly.

#
  • An app tick rate is probably better then renet timers because otherwise you will introduce latency if renet and replicon get out of sync.
grim glacier
#

Is there a Wireshark protocol parser for replicon + renet (maybe + netcode) by any chance? Anything that lets me read into "Data (1078 bytes)" πŸ™‚

spring raptor
spring raptor
# grim glacier Is there a Wireshark protocol parser for replicon + renet (maybe + netcode) by a...

Do you want to read what's inside each message?
The format is very simple. There are two kind of messages.

First kind contains reliable data (like removals, insertions, etc.), we call it "init message". Second kind contains component updates, we call it "update message".

Init message header is the current replicon tick. Update message header consists of two ticks and u16 index.

Each message body consists of arrays. First comes the array len and then its data.
1 array contains entity mappings. Each element consists of 2 entities
2 array contains despawns. Each element consist of 1 entity.
3 array contains removals. Each element consist of an entity and an inner array with size and component IDs as its element.
4 array contains changes. Like 3, but inner array element consist of ID and component data.
Sizes serialized as u16. Entities serialized using this optimized function: https://github.com/projectharmonia/bevy_replicon/blob/434837b34c6616bc0edee44e99c9bd224bd3e32a/src/server/replication_messages.rs?plain=1#L680

But I think that there may be additional bytes from the used messaging backend.

GitHub
bevy_replicon/src/server/replication_messages.rs at 434837b34c6616b...

Server-authoritative networking crate for the Bevy game engine. - projectharmonia/bevy_replicon

grim glacier
#

Thanks! I'll try to write a parser for Wireshark based on this & see how well that works out πŸ™‚

#

Simpler question -- in which use cases should I use NetcodeServerTransport::disconnect_all vs RenetServer::disconnect_all ?

#

It seems both disconnect the client, but the client sees different DisconnectReason (Transport vs DisconnectedByServer). I'm not sure what the semantics of these cases are though

echo lion
vivid quartz
#

Looks like the visibility API only has entity granularity. Is there a way to conditionally replicate components to clients?

dire aurora
#

Not yet, but there has been discussion about the idea of creating VisibilityLayers for components

#

I'm not sure if that would be blocked on anything, it might be possible for someone to implement it. The basic idea is that besides just storing the visibility per entity per client you can store the visibility layers (a bitmask just like in most physics engines), and when registering rules you also register which layers it is on ... There are some nuances that might be hard to get right tho, like you might want replication group A when the visibility doesn't match and replication group B when it does match. Those issues could be solved with existing things like priority, but it might be difficult when processing the archetype cache into some per-layer format

vivid quartz
#

Hmmmmmmmmmm ok so picture a visibility system like FTL has, where visibility on hostile ships depends on the upgrade level of your sensors subsystem. Maybe the way to do it would be to spawn an entity with progressively more intel about each ship, but then only make the relevant ones visible to all clients? Is there an easier way to do what I'm thinking?

#

To clarify, I would spawn an entity with basic information (damage level), then an intermediate with more info (crew locations), then a maximum with even more (weapon charge levels), all per ship.
Then I would make each visible to a client based on that client's sensor level, so if a client's sensors are level 2, we make the basic and intermediate visible to that client

dire aurora
#

In a world with only entity visibility you'd have to make them separate entities yes

#

Or you could just do what I do: Ignore the problem until visibility layers are added πŸ€”

#

With visibility layers you would simply look at the sensor level of the client when deciding the visibility layers of entities

vivid quartz
#

Next question: I'm getting NetcodeServerTransport does not impl Resource. bevy_replicon_renet pulls in bevy_renet, which pulls in renet with bevy feature enabled. Doesn't that mean pulling in replicon renet should be enough to get the impl Resource?

#

Oh, do I need the renet_transport feature?

#

Nope, that's on by default...

spring raptor
#

I just checked in my game just in case - it works.

vivid quartz
#

Oh, pebcak

#

the trait Resource is not implemented for Result<NetcodeServerTransport, ...> 😩😩😩

#

It's things like this that make me want to be a goose farmer instead of a software developer

spring raptor
vivid quartz
#

True

#

But also geese are great

vivid quartz
#

One more question... I can't get the examples to connect over local network, I keep getting "Can't assign requested address". Is this something obvious I'm doing wrong?

spring raptor
#

Try running first on the same machine, but use your public IP instead of the default 127.0.0.1

vivid quartz
#

Which is what I'd assume would work without hole punching/nat traversal

spring raptor
vivid quartz
spring raptor
vivid quartz
#

Will do

vivid quartz
#

I guess that's probably all right or else it wouldn't work locally 🀷

spring raptor
grim glacier
vivid quartz
grim glacier
#

On the server side:
let public_addr = SocketAddr::new(Ipv4Addr::UNSPECIFIED.into(), port);

#

let socket = UdpSocket::bind(public_addr)?;

grim glacier
dire aurora
#

This example code does look kind of sus, the client is binding to the same IP as the server

#

Which is definitely not correct, unless both are running on the same system

#

The only way I could get connecting to work in a stable way over the internet is with this cursed function:

pub fn try_connect(target_addr: &str) -> Option<UdpSocket> {
    for bind_addr in ["::0:0", "0.0.0.0:0"] {
        let socket = UdpSocket::bind(bind_addr).unwrap();
        for addr in target_addr.to_socket_addrs().unwrap() {
            info!("Trying to connect to {:?} from {:?}", addr, socket.local_addr());
            if socket.connect(addr).is_ok() {
                return Some(socket);
            }
        }
    }
    None
}
#

The addresses are about the same as using UNSPECIFIED for both IPv4 and IPv6 I guess? πŸ€”

vivid quartz
#

Unspecified is 0.0.0.0 iiuc

dire aurora
#

Both of those are what rust to be considers unspecified

assert_eq!(IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)).is_unspecified(), true);
assert_eq!(IpAddr::V6(Ipv6Addr::new(0, 0, 0, 0, 0, 0, 0, 0)).is_unspecified(), true);
vivid quartz
#

I switched the client to bind to 0.0.0.0:0 and... that seems to have done something. At least, it's connecting properly. What I'm more worried about is that that's like an unsafe thing to do or something

dire aurora
#

With UDP anyone can kind of send you packets anyway, tho normally you'd use .connect(server_addr) so you only ever receive packets from the server's address

vivid quartz
#

With UdpSocket::bind(0.0.0.0:0) will I get every udp packet coming to the machine?

#

That's certainly what that seems to be saying

dire aurora
#

Every packet sent to the port it just picked

#

From every interface you can bind to

vivid quartz
#

πŸ€”

#

Hmmmm

#

I think I'm missing context. Gonna go read about udp

spring raptor
#

I copied the renet example which runs only locally.

#

Let me put a fix.

#

@vivid quartz There are 2 places that needs to be fixed. I will open a PR in a moment.

dire aurora
#

Then there's a lot of magic rules in firewalls that allow you to send packets back and forth despite at least one of the sides probably being behind NAT and not having that port forwarded

vivid quartz
dire aurora
#

Those rules basically work on the concept that if you send a packet over UDP, you can receive a reply to that same address for a limited time

dire aurora
vivid quartz
vivid quartz
spring raptor
dire aurora
#

Rust also allows you to call connect which makes it so that you can only receive packets from the address you connected to

#

In theory it would be possible to check the network interfaces, see which one allows you to connect to the server, then listen only on that one. But you'd need to use extra crates for it so that's probably not worth it πŸ€”

vivid quartz
vivid quartz
#

Or

#

Wait

vivid quartz
vivid quartz
dire aurora
#

Well if it's a server that's not behind a router 0.0.0.0 would bind to it's public IP too, but in that case that's probably what you want, otherwise people can't connect

vivid quartz
#

Ahhhhhhhhhhhh ok

#

Thank you all for the help! Very much appreciated. Network is still an area of exploration for me so I'm glad to have help figuring things out

spring raptor
#

You are welcome!

candid eagle
#

under what circumstances should we be using renet::ClientID as opposed to just handling the raw u64

#

it seems like a largely redundant distinction

vivid quartz
candid eagle
#

fair enough

#

so for our purposes we just use ClientId?

#

i ask because ClientId doesn't seem to implement Reflect so you can't put it in a replicated component

echo lion
candid eagle
#

is there any feasible way to send a client event from the same system where the client's connection is established?

#

trying to avoid a separate scheduled system to send my player hello event (username etc) so i don't have what amounts to a one-shot event with a run condition evaluating to false every update

echo lion
candid eagle
echo lion
candid eagle
#

are you 'connected' as soon as you finish inserting the client and transport resources or does the connection span multiple game ticks?

#

i.e. if the last thing i do in the system where the renetclient, transport etc are initialised is push a client event to the eventwriter (assuming that the client connection was initialised using direct world access)

#

or do i have no choice but to cache the event in a component on a temporary entity and have another system with a run condition process it later

echo lion
echo lion
candid eagle
#

ah my bad, i thought NetcodeClientTransport::new blocked until after the handshake

echo lion
candid eagle
echo lion
candid eagle
echo lion
candid eagle
#

is it possible for the replicon handshake to fail but renet still fires ServerEvent::ClientConnected?

#

cause i just noticed i already have a system for dealing with the renet connected event

candid eagle
echo lion
echo lion
candid eagle
#

hmm, if i used a custom channel and sent it just before i add the renet client as a resource, would the event be guarunteed to have arrived at the server by the time ServerEvent::ClientConnected fires?

echo lion
candid eagle
#

aight

#

out of curiosity, is there a particular reason that replicon drops events sent before connection but renet doesn't?

#

cause i can't think of a situation where you'd be sending events preemptively but would want them to be dropped

echo lion
candid eagle
#

makes sense

candid eagle
#

hmm, did you guys intend to remove std::fmt::Display from bevy_replicon::core::ClientId?

#

just updated from 0.12 to 0.13 and that was one of the breaking changes apparently

candid eagle
#

hmm, last time i read the docs this wasn't here, does that mean that in bevy 0.13 i can't have the Replicon client and server loaded on the same bevy app?
I'm not sure how this is supposed to work with LAN hosting, where one instance typically acts as both client and server

image.png
#

in 0.12.X i had that working fine, did something change that will irreparably break it now or was I just always violating best practices?

spring raptor
# echo lion Hmm I checked again and looks like messages can be submitted to renet clients wh...

Ah, I remember that we decided to clean events on reconnect in order to mimic renet: https://github.com/projectharmonia/bevy_replicon/pull/163#issuecomment-1885148968
We should change it then. I think that the use case brought by @candid eagle is valid.

GitHub
Clean up events more rigorously by UkoeHB Β· Pull Request #163 Β· pro...

Moves ClientSet::ResetEvents before renet updates, to ensure events are cleared immediately before connecting.
ClientSet::ResetEvents runs every tick while waiting to connect to ensure the tick whe...

spring raptor
spring raptor
candid eagle
spring raptor
candid eagle
#

cool

#

currently trying to figure out why my clients arent connecting anymore

#

the renet client connected event isn't firing on the server either πŸ€”

spring raptor
candid eagle
spring raptor
candid eagle
#

pretty sure this is the cause but if i remove it i get no logs cause at some point last year i disabled the Log Plugin

image.png
#

i tried setting replicon to trace there too but still nothing from replicon

spring raptor
#

@candid eagle you mentioned that you migrating from an old version... Do you have bevy_replicon_renet as a dep?

candid eagle
spring raptor
candid eagle
#

ohhh

#

yeah i missed that in the changes to the example code

#

yep that fixed it

#

thanks @spring raptor!

#

i take it the replication loop mentioned in the article looks like a LOT of entities getting added with the ConfirmHistory component attached?

#

i accidentally ran my old client-server code :P

vivid quartz
#

Is there a simple way to replicate resources to clients?

echo lion
spring raptor
spring raptor
#

For now just spawn an entity and assign the needed data as components to it.

#

Or use events

#

Most of the time you don't want to replicate resources. For example, if a resource is an Vec - you will send the whole Vec each time.

dire aurora
#

I mean writing an event that's sent when a resource is changed is trivially simple

#

But yea replicating resources would only make sense if we first had delta compression probably

vivid quartz
dire aurora
spring raptor
vivid quartz
#

Hmm is there a way for a client to check its id?

#

Or do I have to send it a message myself?

dire aurora
#

If the transport sets it, it can be found on RepliconClient (using .id()) once the client is in the Connected state

grim glacier
#

What should I call when I want to stop a server (before returning to main menu), given that I might want to start another server again, or become a client, in the future? For now I'm getting ResMut<RenetServer> and calling .disconnect_all(). I think I also need to remove the RenetServer and its transport's resources? Any other steps I missed?

dire aurora
#

Calling disconnect and getting rid of the RenetClient/Server and transport seems to have been the right solution for me

#

Tho there are some issues where you need at least 1 frame before being connected again, otherwise replicon won't pick that disconnect up correctly and might leave some state around

verbal cave
#

Trying to set up bevy_replicon with bevy_replicon_renet in a server-client architecture (no single player here).
According to the displaying the connection states, my connection looks ok but events seems not to be sent.

I strictly followed the code from here: https://docs.rs/bevy_replicon/latest/bevy_replicon/#network-events

Both server and client events looks muted.

spring raptor
#

You should see a connection

verbal cave
#

Or does it? I thought that all of these message was the server events, but it's written "received 0 message"

#

wait a minute thinksmart

#

Ok, I receive the server event, it works better when you don't forget to register the server system -_-

Verifying the other side...

#

Well no, the client to server events doesn't seem to be ok:

// Client side
app.add_plugins((RepliconPlugins, RepliconRenetClientPlugin));
app.add_client_event::<DummyClientEvent>(ChannelKind::Ordered);
app.add_systems(Update, send_events);

fn send_events(mut dummy_events: EventWriter<DummyClientEvent>) {
  dummy_events.send_default();
}

// Server side
app.add_plugins((RepliconPlugins, RepliconRenetServerPlugin));
app.add_systems(Update, (receive_events, handle_events_system).chain());

fn receive_events(mut dummy_events: EventReader<FromClient<DummyClientEvent>>) {
  for FromClient { client_id, event } in dummy_events.read() {
    info!("received event {event:?} from {client_id:?}");
  }
}

fn handle_events_system(mut server_events: EventReader<ServerEvent>) {
  for event in server_events.read() {
    match event {
      ServerEvent::ClientConnected { client_id } => {
        println!("Client {client_id} connected");
      }
      ServerEvent::ClientDisconnected { client_id, reason } => {
        println!("Client {client_id} disconnected: {reason}");
      }
    }
  }
}
#

All I get is the client connection etablishment

image.png
#

The trace from the client side show me that it sent the events correctly

2024-06-13T22:21:53.133721Z TRACE bevy_replicon::client::events::event_data: sending event `mmopixel::events::DummyClientEvent`
2024-06-13T22:21:53.133745Z TRACE bevy_replicon::client::replicon_client: sending 0 bytes over channel 2
2024-06-13T22:21:53.135991Z TRACE bevy_replicon::client::replicon_client: received 0 message(s) from channel 0
2024-06-13T22:21:53.136049Z TRACE bevy_replicon::client::replicon_client: received 0 message(s) from channel 2
2024-06-13T22:21:53.136641Z TRACE bevy_replicon::client::events::event_data: sending event `mmopixel::events::DummyClientEvent`
spring raptor
#

From the quick start guide: ... The event must be registered on both the client and the server in the same order.

verbal cave
#

Hmm, I would love that it was that, but it's just my copy/paste mess

spring raptor
#

Ah, okay :)

verbal cave
#

(I didn't want to blow the channel with a huge code)

spring raptor
#

Ah, so it doesn't work for you for some reason?

verbal cave
#

Yeah, really not sure why.

Should the server trace log works as it does on the client side?

spring raptor
#

Sure

verbal cave
#

I have a really clear trace log on client but can't make it work with the same command

spring raptor
#

Enable it on server and show me the output

spring raptor
verbal cave
#

In fact it's already enabled

image.png
#

Nothing is logged, maybe all of that is due to some deeper bevy config? Like I only run the MinimalPlugin on the server side

#

Ok, logging work with the DefaultPlugins (nice to know that it doesn't work with the MinimalPlugins)

#

And the log are fine, the server receives the events...

2024-06-13T22:33:41.186740Z TRACE bevy_replicon::client::events::event_data: applying event `mmopixel::events::DummyClientEvent` from `ClientId(1718318018071)`
2024-06-13T22:33:41.186750Z TRACE bevy_replicon::client::events::event_data: applying event `mmopixel::events::DummyClientEvent` from `ClientId(1718318018071)`
2024-06-13T22:33:41.186758Z TRACE bevy_replicon::client::events::event_data: applying event `mmopixel::events::DummyClientEvent` from `ClientId(1718318018071)`
2024-06-13T22:33:41.186765Z TRACE bevy_replicon::client::events::event_data: applying event `mmopixel::events::DummyClientEvent` from `ClientId(1718318018071)`
2024-06-13T22:33:41.186772Z TRACE bevy_replicon::client::events::event_data: applying event `mmopixel::events::DummyClientEvent` from `ClientId(1718318018071)`
2024-06-13T22:33:41.186779Z TRACE bevy_replicon::client::events::event_data: applying event `mmopixel::events::DummyClientEvent` from `ClientId(1718318018071)`
spring raptor
spring raptor
verbal cave
#

Yeah, everything works with the DefaultPlugins

spring raptor
verbal cave
#

Hmm, let me verify something

spring raptor
#

Logging on server is quite useful.

verbal cave
#

Ok, so the LogPlugin is also responsible for the prinln! macro, I didn't know

#

info! works without LogPlugin, but not println!

But anyway, MinimalPlugins + LogPlugin is what I need, thanks!

#

Hope I'll do something awesome with all that stuff, thanks for the work on the crate πŸ™‚

spring raptor
spring raptor
verbal cave
# spring raptor Did you manage to find the problem?

With the MinmalPlugins + LogPlugin, I don't have any problem anymore. And I need to fix some of my words: println! works without LogPlugin but info! doesn't

What caused me trouble is that in the crate documentation, info! is used. Not sure when but I modified this by a println for the connection/disconnection message but not for the receive_events message. So I received the "connection etablished" message but not the events message which mislead me into "damn I can't make it works"

#

Maybe changing the documentation to use println would avoid some future timeloss with this missing LogPlugin. I'd be happy to do a PR to modify the documentation to thank you ;p

spring raptor
spring raptor
verbal cave
#

This is for sure a wise decision.

vivid quartz
#

Here's an attempt at "generic just works" resource replication. It seems to work with my limited testing, feel free to give feedback/point out possible issues!

πŸ“Ž replicate_resource.rs
dire aurora
#

Looks like that would definitely "just work", not the most efficient approach like shatur already mentioned, but there's lot of room for hacky solutions early in the development of anything networked

#

Until your game gets to the internet bandwidth usually doesn't even matter

vivid quartz
#

Yeah, I anticipate blowing this away at some point in favor of a first party solution

#

But glad it doesn't look obviously wrong at least πŸ‘

grim glacier
dire aurora
#

You'd need to call disconnect_all before PostUpdate, then remove the server/transport the after that or the next frame I think πŸ€”

grim glacier
#

Digging through the source it seems disconnect_all() doesn't send the actual socket right away, it waits for a NetcodeServerPlugin system that runs in PreUpdate if(resources exist(RenetServer, NetcodeServerTransport), to find all the disconnected clients to send the disconnect packets

grim glacier
#

It would be nice if there were some magical RenetServer::stop(&mut self) that would handle sending the disconnects + removing itself and its transport afterwards, all in the right order, and I'd just have a system .run_if(server_just_stopped) to handle the result 😢

spring raptor
#

Updated the crate to the latest Bevy rc. It's just a version bump in Cargo.toml

candid eagle
#

If you've got an app with the Server plugins loaded acting as a lan host, can you reuse your systems with client event writers on that server and have them be recieved properly or does it not work like that?

spring raptor
candid eagle
#

Sweet

vivid quartz
#

This is me right now

image.png
#

I've been trying to get my replicon server running in an Azure container for like 3 days

vivid quartz
#

Omg

#

It works

#

Going to bed now finally

dire aurora
#

Like at least I only had to upload a binary to my VPS and run it πŸ˜‚

vivid quartz
dire aurora
#

3s ping? That's insane πŸ‘€

#

I have about 14-16ms ping to my game server. Would've been lower if my VPS was in the netherlands instead of germany ... but hetzner has been a much better option than ones that offer locations in the netherlands

vivid quartz
dire aurora
#

https://www.hetzner.com/cloud/ The cheapest AMD version ... With a whole 2 cores and 2GB RAM ... Actually runs my game surprisingly well ... And it's not even using both of those cores πŸ˜‚

vivid quartz
#

Also looks like the ones with 4gb are cheaper?

echo lion
#

Neat, they even let you save money by running ipv6

dire aurora
sharp roost
sharp roost
vivid quartz
sharp roost
#

Ah, I think you should look into something like serverless then

sharp roost
#

That's usually paid per time unit of computation you use (and/or the amount of data you transfer)

vivid quartz
#

Something like container instances? I tried something like that and it didn't give me near the performance I needed

#

Also I don't mind having a vps running full time in the future, I just like that I can just shut the thing down when I'm not going to be working on the networking for a month or two

sharp roost
#

The most basic VPS is only like $5/Β£5€5 tbf, it's not too much

vivid quartz
#

Yep I spooled one up already, it's working great πŸ™‚

echo lion
sharp roost
#

Oh, I thought was a statement you invented eather than repeated πŸ˜‚

dire aurora
#

Or rather by default you pay more to have one πŸ˜‚

echo lion
spring raptor
#

@dire aurora reorganized events as we discussed: https://github.com/projectharmonia/bevy_replicon/pull/292
I usually don't make functional changes when Bevy updates, but it's just a few imports change for end users, so it should be fine...

GitHub
Rework events organization by Shatur Β· Pull Request #292 Β· projecth...

Swap receive system between ServerEventsPlugin and ClientEventsPlugin to properly separate what what runs on client or server.
Move ServerEventsPlugin::reset logic inside ClientEventsPlugin::reset ...

spring raptor
spring raptor
#

What if we start to depend on renet directly?
bevy_renet is just 2 simple files, we can just copy the needed logic.

This will allow us to release quicker. And we can have proper rc releases.

#

I talking about bevy_replicon_renet of course.

dire aurora
#

That might work yea ... You'd need to use renet without the bevy feature and put it in your own resource tho πŸ€”

spring raptor
#

Ah, there is renet feature...

#

I forgot about it

#

Won't be nice then

#

It can be hidden from users by creating structs with the same name and providing Deref, but it's hacky and could confuse people

echo lion
dire aurora
#

Alternatively you could fork renet, migrate it yourself and call it renet3 πŸ˜‚

spring raptor
spring raptor
#

@echo lion @dire aurora opened: https://github.com/projectharmonia/bevy_replicon/pull/299
I think it's nice overall. But I not sure about using structs with the same name. After update users will confusing errors like RenetServer doesn't implement Resource. And they will have to change the import struct.

GitHub
Depend on `renet` instead of `bevy_renet` by Shatur Β· Pull Request ...

renet is not actively maintained and bevy_renet is often behind a Bevy version.
But since bevy_renet is very small (2 files), I decided to bundle it and depend on renet directly. The only downside ...

#

What do you think?

spring raptor
#

Maybe just use different names?
Like ServerEvent -> RenetEvent.
But not sure what to do with RenetServer/RenetClient.

#

But this could cause even more confusion. I probably more inclined to use the same names and export them to prelude...

echo lion
#

@spring raptor if you do a cargo patch will it prevent publishing to crates?

spring raptor
#

Also forgot to mention, but I will apply the client condition in a separate PR, wanted to isolate it.

echo lion
echo lion
#

Honestly I'm not sure what to do. If only there was no orphan rule..

spring raptor
#

So you don't think it's a good solution ?

echo lion
#

It's just a little janky. As long as someone doesn't include the Renet{Client,Server}Plugins in their app the compiler/crashing should help them migrate.

echo lion
#

Ah yeah the ServerEvent can be re-exported as RenetServerEvent.

spring raptor
echo lion
spring raptor
#

I mean bevy feature on directly included renet crate

candid eagle
#

Hey Shatur, quick question, is there any way for a server to send a client event and have it added to its own client event queues?
I'm handling player input with a client system that gets input from leafwing and sends it with an event to the server for processing, but i want to have headed server (lan host) instances which can run this system as well. But when the event gets sent from the server it doesn't make it to the listeners. Do i have to get rid of the FromClient<> on my listener systemparameters?

spring raptor
# candid eagle Hey Shatur, quick question, is there any way for a server to send a client event...

Hi! By headed server (lan host) do you mean listen server? Where server is also a client.
Then all events should be available on listen server just like on a client. So if you send event ToClients<T> from server, it's not only avaiable as T on clients, but on the listen server as well. Similar for client events: you just send T on listen server and it appears locally as FromClient with ID == SERVER_ID.

candid eagle
#

Hmm, is setup any different for listen servers compared to normal servers (other than loading the client related game systems as well)

spring raptor
#

After experimenting with direct renet wrapping I summarized my thoughts on the problem in https://github.com/projectharmonia/bevy_replicon/issues/300

I think that 3(.1) is the best one, but I would appreciate any opinions on this topic.

GitHub
Faster releases Β· Issue #300 Β· projectharmonia/bevy_replicon

The problem renet is great, but the author is not very active and its examples depend on other third-party crates. Because of it, the crate could be behind a Bevy version. It's not a problem on...

candid eagle
#

hmm, is client_just_connected() ever true on a server?

spring raptor
#

Maybe worth considering renaming to is_listen_server

candid eagle
spring raptor
#

Now will be located in a separate repo.
Nothing else will change. This will allow us to draft new releases without waiting for renet compatibility with Bevy or disabling CI for this crate.

spring raptor
#

Drafted a release for the new RC.

plush sparrow
#

How is replicon doing for syncing thousands of entities? I am thinking about whether I need determinism for my coop TD or can just brute force it :X

spring raptor
spring raptor
#

@plush sparrow oh, sorry, I meant to write ΞΌs, not ns. I typed from my phone.

low minnow
#

Is something like bevy_replicon::server::VisibilityPolicy only available for whole entitites? I want to use it to only update entities that are in render distance. But I also want to only update an entities components for some clients. Imagine characters like in Rimworld with a whole bunch of stats and traits. In my game a player "owns" multiple characters like that and I want only that client to be able to see those stats, among other things. Is that something that bevy_replicon already supports or do I have to build something myself? Great crate btw, for my other needs it just works

spring raptor
low minnow
spring raptor
low minnow
spring raptor
# low minnow Sure, I could give it a try

Great!

I started writing a quick code walkthrough, but realized that we probably need to talk about the API first πŸ˜…
The API was originally suggested by @dire aurora. The idea is to have component access levels via bitmasks like in physics engine. User define their meaning. Some examples:

  • "Owner", "Party member", "Allied", "Neutral", "Hostile"
  • "Very close", "Close", "Far away", "Very far away", "Extremely far away"
  • "Friend", "Guild Member", "Alliance member", "Stranger"
    To achieve this, we assign a mask to each component. Like "send this component only to owner and party members".
    Now we need to assign this layers clients. I think that we need to turn hashsets for blacklists/whitelists into hashmaps that point to masks. So users enable/disable visibility for an entity and optionally set a mask for it.

Does it make sense?
Should this API be per component or per component group (i.e. per replication rule)?

low minnow
#

Haven't written Trait Extensions yet, but I'll figure it out

spring raptor
low minnow
spring raptor
low minnow
#

0.14 looks like it's just around the corner πŸ™‚

spring raptor
#

We have two kind of messages for replication:

  1. Init message. This message is for reliable channel and contains all insertions, removals, despawns, etc.
  2. Update message. This message is for unreliable channel and contains all component updates.
    This separation is needed for bandwidth optimization. For component updates client care only about the last value, so if it fails to arrive, we always send the last value.
    Both messages serialized sequentially and manually (also for optimization).
    It's just a quick summary to give you the context, not related to the feature directly.

Here is the system that sends the data (note that I use rc branch): https://github.com/projectharmonia/bevy_replicon/blob/aefac55d099ddf26807a81e02ccf295e4f11f319/src/server.rs#L217
The system collects data into messages and sends them. Let's omit removals for now and focus on component changes only first.
It's called here: https://github.com/projectharmonia/bevy_replicon/blob/aefac55d099ddf26807a81e02ccf295e4f11f319/src/server.rs#L243
We iterate over all archetypes, then over entities and then over components. If you're not familiar, an archetype in Bevy stores all the entities for each set of components. Example: an archetype with all entities that have (A, B, C).
Here we cache visibility for the entity: https://github.com/projectharmonia/bevy_replicon/blob/aefac55d099ddf26807a81e02ccf295e4f11f319/src/server.rs#L340
And we use it later to detect if we need to send the entity or not.

#

You don't have to understand all the details, I just giving you the context :)

#

Here is what needs to be changed.

  1. Add the ability to register a mask per component or per component group. Depends on what we decide on the API.
  2. Add the ability to set mask for an entity inside ClientVisibility.
  3. When we cache archetypes, also store masks from 1 inside ReplicatedComponent (here: https://github.com/projectharmonia/bevy_replicon/blob/aefac55d099ddf26807a81e02ccf295e4f11f319/src/server/replicated_archetypes.rs#L84).
  4. When we cache visibility (https://github.com/projectharmonia/bevy_replicon/blob/aefac55d099ddf26807a81e02ccf295e4f11f319/src/server.rs#L340), we should also cache the mask (since we switched from hashset to hashmap)
  5. Here we should additionally check the mask: https://github.com/projectharmonia/bevy_replicon/blob/aefac55d099ddf26807a81e02ccf295e4f11f319/src/server.rs#L376
    That's it :)
#

Feel free to ask as many questions as needed.

low minnow
#

Yeah I have a question for 2.: That mask just for the component visibility or you want me to also implement masks for entity visibility?

spring raptor
low minnow
spring raptor
spring raptor
low minnow
#

You thinking about having a mask for each client and component, so each client has a map of masks for each component or something? Maybe it's flexible enough when you can update the masks

spring raptor
#

Let me edit how implement it...

spring raptor
#

@low minnow edited the API description here (again): #1090432346907492443 message
And the proposed implementation here: #1090432346907492443 message

As you can see, there is one open question about the granularity. But I think that the idea in general is clear.
Could you open an issue with the text I wrote and wait for the response from @echo lion and @dire aurora. I would like to hear their opinions.
I just currently at work right now, so I can't open an issue myself πŸ˜…

low minnow
spring raptor
#

@low minnow thanks!
For removals we will need additional logic, but we will back to it later... Maybe it worth removing "needs to be changed" at all, I probably started thinking about it too early πŸ€”

In the meanwhile I continue thinking about it. How to track changes in masks? Maybe also store the previous mask internally. So instead of the proposed hashmap with entity -> mask, we probably want something like entity -> (mask, mask).

echo lion
#

Hmm I will think about it and leave a reply on the issue.

spring raptor
#

Do we need to keep RC versions in the changelog and Bevy compatibility table? πŸ€”

echo lion
#

Probably don't need them in the table once the full version is out.

ember flower
#

Sorry for a potentially very dumb question, but assuming the ClientId of a unsecured renet connection is the same, does the replication happen automatically upon re-connection? Or it has to be some kind of re-initialzation function?

  • [x] make sure the server actually spawns the WantReplicated component received from ClientEvent
  • [x] make sure it is not a mapping issue
  • [x] make sure it is visible
spring raptor
ember flower
spring raptor
#

There is also trace with more diagnostics, but you probably want debug.

ember flower
#

In case someone also runs into a similarly shaped problem, in my case, I forgot to include the Replicate component with the bundle of thing I wanted replicated. It is silly, but my mental model was that app.replicate<Flashcard> was sufficient and it will ear mark all flashcards for duplication.

But the doc is clear on that

The component will be replicated if its entity contains the Replicated marker component```
spring raptor
#

@dire aurora @echo lion quick remainder about this issue: https://github.com/projectharmonia/bevy_replicon/issues/304
Could you write your opinions about it? I just want to make sure that I understand @dire aurora suggestion correctly and @echo lion also agree with this.

GitHub
Per Component Visibility Β· Issue #304 Β· projectharmonia/bevy_replic...

I'd like the ability to not only be able to define which entities are visible to specific clients (possible with the bevy_replicon::server::VisibilityPolicy) but also be able to define what com...

dire aurora
spring raptor
# dire aurora The functional description looks correct, idk about the implementation details t...

I planning to write the implementation details later once we agree on the API.
I mentioned masks and lists to clarify that each mask will be assigned per entity and per client. Am I understand your suggestion right?
Regardless blacklist, you mean that it would be more consistent if we ignore components that are set in a mask? For example, if we use blacklists and for entity X we have mask that means "Guild Member", we need to hide all components that can see a "Guild member"?

dire aurora
#

You set a mask per entity per client yes. I think I would just avoid the whole blacklist thing, it makes sense to say "this client is a guild member of this entity", but it's really weird to say "this client is everything except a guild member"

#

I also feel like generally the way things should work is: Allow/deny is set as a "default policy", then for each entity you can specify allow/deny/inherit for entity visibility. Meanwhile component visibility would be entirely separate from that besides being stored in the same per-client per-entity list

spring raptor
echo lion
#

I think per-component visibility is not needed for high-volume replicated entities, so it should be ok to be a little heavy-handed in the implementation.

echo lion
#

@spring raptor #engine-dev message might be the bug you encountered in the example.

spring raptor
spring raptor
dire aurora
#

I think generally per-component visibility is not what we want. It wouldn't be fun to maintain the systems for this, and you lose the ability to do things like "Send this format if A, or this other format if B"

spring raptor
dire aurora
#

The part of it being hard to maintain is mostly because you write your visibility logic based on groupings that would've matched layers, not components. You'd need to mark every component as visible or not visible from some centralized place, which is really annoying dependency wise. The alternative would be to split it up and do it seprately everywhere, or make yet another registration system to handle it for you.
As for the other part, groups and priority allow you to change how things are replicated, and having visibility based on groups would mean you can change the format for some clients but not others, your allies for example might get a few more details on your skill's stats than your enemies

spring raptor
dire aurora
#

So what's the best solution we have no to the bevy_renet not being migrated yet problem? πŸ€”

spring raptor
dire aurora
spring raptor
spring raptor
#

@low minnow I have outlined a complete description and implementation details for component visibility in this message: https://github.com/projectharmonia/bevy_replicon/issues/304#issuecomment-2217522473
I think it's pretty complicated for a beginner, so I think it would be best for me to implement it :D I'm quite busy right now, but I will put it on my TODO list.
If anyone want to try to implement it - I can't stop you and will definitely help or answer any questions about the implementation.

GitHub
Per Component Visibility Β· Issue #304 Β· projectharmonia/bevy_replic...

I'd like the ability to not only be able to define which entities are visible to specific clients (possible with the bevy_replicon::server::VisibilityPolicy) but also be able to define what com...

dire aurora
#

Gotta love how confusing the errors from bevy get when it secretly uses two different crates ... Guess it's a good thing we didn't do that using renet directly hack πŸ˜‚

spring raptor
#

@echo lion highly suggest to avoid depending on bevy_* crates. It makes it very inconvenient to patch :(
And it's something that many people do.

dire aurora
#

Usually the only ones I'd expect to patch are bevy_app, bevy_ecs, bevy_reflect, and bevy_math, since those are usable standalone

echo lion
#

Hmm what's the problem exactly? I haven't had to patch anything so not sure what the issue is.

#

bevy_replicon_renet2 has a dep on bevy with default-features = false, and bevy_renet2 has deps on app/ecs/time/window subcrates.

dire aurora
#

Overlap a few crates that use different subcrates and you end up with a list that contains every bevy crate ... And of course the errors are usually pretty abstract

echo lion
#

I kind of prefer having minimal deps especially for headless servers

dire aurora
#

Ah yes, that matches, the ones I had to add were bevy_app and bevy_time

#

I wonder what deps you actually save by not using bevy directly πŸ€”

#

I guess you can drop a few things bevy doesn't make optional because they're super common or not all that heavy ... log and hierarchy for example ... I mean I don't actually use hierarchy in my server so I guess that one is sort of reasonable actually

echo lion
#

Looks like I can remove that bevy_window dependency. *Done

spring raptor
dire aurora
#

I wonder if there's any plans on the cargo side to fix cases like this ... Maybe namespaces could be relevant somehow πŸ€”

echo lion
#

It's definitely not ideal but I think the reduced build footprint is higher priority unfortunately.

spring raptor
spring raptor
wary wolf
#

@spring raptor would you consider having this system run in a lifecycle schedule (pre/update/post) with .run_if(resource_added::<RepliconClient>) rather than on Startup? https://github.com/projectharmonia/bevy_replicon/blob/master/src/client.rs#L54 it would be nice for my application to insert ClientPlugin and decide for itself when to enable has_authority for example by removing/reinserting RepliconClient alongside e.g. RenetClient. (if there is a workaround you had in mind let me know!)

spring raptor
wary wolf
#

ah - client.disconnected handles it. my bad

#

misread the function sig

spring raptor
#

It's inserted automatically by ClientPlugin

vivid quartz
#

Is there a replicon crate out there for async request/response?

#

Or should I make it tonight?

spring raptor
dire aurora
#

If you want it to be async as in "no frame-based delays" the transports are really going to fight you πŸ€”

vivid quartz
vivid quartz
# spring raptor Not that I'm aware of. We usually use events for it. But yes, you can create you...

Yeah I'd just build it around events for sure. The thing is I run into scenarios where I send an event and then want a response, so my thought was just to add a thin wrapper where you could register a request/response, which would register two channels and assign a monotonically increasing counter so you could tell which response corresponds to which request. Then you slap an async API on top and you can just request(MyRequest).await. Tbh feels a bit roundabout to build something like that when replicon itself is built on top of something with arguably better support for that but 🀷

dire aurora
vivid quartz
spring raptor
vivid quartz
#

I'm actually really liking how the API is turning out:

fn open_pod_bay_doors(mut requests: RequestHandler<OpenPodBayDoors>) {
    requests.handle_requests(|_request| Err(MutinyError));
}
spring raptor
#

Not sure if I understand what is happening

echo lion
vivid quartz
raw idol
#

in RepliconChannel there is a resend_time. by default it's a Duration::ZERO I believe. what is the best way to figure out what to set this value to?

#

I'm thinking that the resend time should probably vary with the connection RTT rather than being a fixed value

raw idol
#

6.2.1. Computing PTO

spring raptor
spring raptor
raw idol
#

is there any benefit to having different channels have different resend times?

spring raptor
#

You can use bigger resend time for non-sensetive data.
In renet you can set it like this, so I just allow to configure it.

raw idol
#

I see

#

a lot of the replicon api was inspired by renet right>

spring raptor
spring raptor
#

@dire aurora 2 weeks ago I mentioned the filter PR in #ecs-dev to get attention, and your last comment was even upvoted by Alice, but I'm afraid it will be stuck for a while.

Have you considered releasing a version with entity disabling under a feature? Users who need it, can activate it and patch Bevy.
I think the community will benefit from your work. No other crate with rollback handles entity disabling and I think it's a situation where "the perfect is the enemy of the good" πŸ˜…
If your crate gains some popularity, I think it may draw additional attention to the problem.

dire aurora
#

Hmmm, that might be possible, if I feature flag disabling it would at least be buildable and possible to make a crate out of it πŸ€”

#

Currently I think it's probably stuck on fixing that unsoundness with sparse filters

#

Which is luckily no longer a thing specific to the PR but existing undefined behavior that can be triggered trough safe APIs

#

It's pretty hard to do rollback correctly without entity disabling or some ugly workaround tho. That would require essentially never spawning new entities in the simulation

spring raptor
dire aurora
#

If this is solved correctly I should only need to rebase πŸ€”

spring raptor
#

Never mind, systems still can be triggered

#

Yeah, it's hard.

dire aurora
#

You also can't really do any despawns in general, since all the references to entities break

spring raptor
#

Yep

#

And I think that's what other crates do right now.
So I would just mention this in the readme to make people aware about the Bevy limitation.

#

And other similar crates as well*

dire aurora
#

I'd imagine the only crate that might handle spawning/despawning correctly would be ggrs πŸ€”

#

It's so easy to fall into the trap of just not handling it because some games don't use it yet, and then once you need it it's not supported so you just work around it

spring raptor
#

Exactly, most people even don't know about this issue.

spring raptor
dire aurora
#

You delay despawns until they fall out of history, and accept the weird side effects of spawning

#

And avoid creating X as Entities patterns, since those will introduce more spawns/despawns

spring raptor
#

Ah, interesting

dire aurora
#

I also need to make a better API for passing in the ticks, both the input queue stuff and rollback still rely on the tick for bundlication iirc, but with the current API it should probably just be a resource passed to the plugin when adding it

spring raptor
spring raptor
spring raptor
spring raptor
#

Just published a new version of bevy_replicon_renet that supports the latest versions of replicon and renet.

granite hill
spring raptor
dire aurora
#

Shouldn't interpolation only be used in a client in the first place? πŸ€”

#

Tho in that case it would probably still need to disable default features

raw idol
#

if I want to have different connection states or phases in replicon, what's the solution to that? for example, an initial setup phase where a client sends its username and config to the server, then the connection transitions to playing phase, and actual component replication etc. is performed

spring raptor
spring raptor
raw idol
spring raptor
# raw idol a game

You can control it with entity visibility.
I assume that what you planning to replicate is based on what the client can see.

raw idol
#

is that possible?

dire aurora
#

Ah, I used to have that in bundlication but I'm pretty sure replicon doesn't have that. There's no "start replication now" signal

#

But you can work around it with visibility ofc

raw idol
#

that feels really hacky to me

#

I might have to write this at the transport layer itself

#

but then I would have to renegotiate lanes aaaughhh

spring raptor
raw idol
#

the client needs to send over stuff like its username, authentication info, etc

#

and any preferences it has