#ETA for a netcode utility package without the ghost stuff ?

1 messages · Page 1 of 1 (latest)

steep sandal
#

Like If i'm to build my own system I would still need to be able to send commands & stuff like that on the network. is there an epurated version of netcode for entities available somwhere ?

balmy quartz
#

Netcode without ghosts is just the transport package

north crescent
#

Netcode is based on that

steep sandal
#

yeah but there is no implementation of commands like the RPC commands are doing

north crescent
#

you'll have to make your own

#

I don't like the way rpcs work in netcode

#

you need structural changes

#

so I just wrote my own which is reading from dynamic buffer of bytes

steep sandal
#

Do you have pointers on that ?
cause it's been two days i'm struggling on that topic ^^

#

Is it something you can share ?

north crescent
#

no

steep sandal
#

FML ... ^^"

north crescent
#
  1. make some ReadRpc , WriteRpc dynamic buffers with only one field of byte
#
  1. make generic method that writes your rpc structs into it
#
  1. make a generic method that is able to read from those buffers rpc of your generic type
#

that's how mine works

#

so it looks like rpcBuffer.WriteRpc(new SomeRpcStruct{ someField = 123 } );

#

and reading looks like bool hasRpc = rpcBuffer.TryReadRpc(out SomeRpcStruct rpcData);

steep sandal
#

how do you know what type of rpc is being sent ?

north crescent
#

I don't

#

I just check buffer for it

#

if it has one

#

internally that generic method just goes over all rpcs in a buffer

#

reads their header

#

compares typeHash to generic one

#

typeHash is obtained via BurstRuntime utility method

steep sandal
#

typeHash -> gotcha you have a type hash to identify what data is coming

north crescent
#

yeah I use type hashes in headers

#

when I write data

steep sandal
#

lemme check that method

north crescent
#

over network though I send stableTypeHash

#

to identify rpc type

#

it's a long, but it's a ready thing

#

you can get it from entities type manager

steep sandal
#

TypeManager.GetTypeInfo().StableTypeHash);

#

this ?

#

and other question when you talk about dynamicBuffer You mean the type in entities ? or Something else ?

north crescent
north crescent
#

Technically it doesn't matter where you push rpc

#

I just prefer to have this buffer on connection entitu

#

These buffers*

steep sandal
#

wtf it's a pain in the ass to do something that is scalable on that side ...

#

my god

balmy quartz
#

writing your own networking is a pretty big undertaking

steep sandal
#

that the thing ... I don't want to

#

If I could usesomething like netcode for entities With the basics already implementated that would be perfect

steep sandal
#

@north crescent coming back to you, thanks for the hash idea, it's funny cause when digging into the source code for netcode for entities

Turns out that's the same method they are using with their IRpc commands

#

and it work like a charm

#

a new question arise tho. what is your solution doing differently than RPC in netcode for entities ?

cause you said you are using dynamicbuffer of bytes, but that's the same thing they do with a few extra steps

north crescent
#

they expose it as components

#

I don't like that

#

basically caps everything to main thread

#

my implementation is fully threaded

#

and will scale as long as you have free thread

steep sandal
#

Okey, make sense.

Cause i've been looking into their OutgoingRpcDataStreamBuffer and IncomingRpcDataStreamBuffer and i was wondering what was diferrent.

Theirs is per connections, your is , per commands ?

north crescent
#

per connection

#

each command is polled via generic method

#

so basically you can have 50 jobs reading same buffer

#

with different methods like buf.TryRead<MyRpcType>

#

it's just that their buffers are internal

#

instead they want you to rely on entities getting created per RPC

steep sandal
#

yeah, you create an entitie with a rpc command attached to it, then a RpcSystem is gathering & sending them on the network , right ?

#

Your implementation don't rely on these said entities

north crescent
#

smth like that, i don't remember details tbf

#

yeah

#

in my case you just query that buffer and call generic method buf.Write<MyRpctype>

#

we have this buffer cloned on actual "character" entity of player

#

so it's easy to actually look it up

steep sandal
#

Do that mean you have 1 buffer per command type ?

north crescent
#

and then all buffer internals just gets copied to connection entity

north crescent
#

I just write different bytes to it

#

and magically it gets sent to client

#

😅

steep sandal
#

Okey yeah , because you always start a message by the hash of the command you always know what packet is coming next

north crescent
#

kind of, I write hash + length

#

so next header is known

#

next header's location*

steep sandal
#

Cleaver

#

Do you have codegen too to create the boiler plate for the commands ?

#

or is it an another methods you are using ?

#

Like for automatic registration & stuff

north crescent
#

it's generic

#

there's no boilerplate

#

it's a combination of reflection and generics, no codegen

steep sandal
#

I was thinking , instead of relying on their codegen for my command I could just have an initial system looking in the assembly for stuff implementing a INetworkCommand Interface and registring it in a hasmap somewhere

north crescent
#

yeah, that's exactly what I do

steep sandal
#

Okey that would implify a lot

north crescent
#

you can even make it "constant"

#

and staticly available

#

like TypeManager

#

just need to look how TypeManager functions

#

to make it burst compatible

steep sandal
#

Yeah, I was thinking to turn it into a shared static DB

#

That would be perfect

north crescent
#

just don't forget that it needs very special cleanup treatment

#

so do look at how TypeManager disposes it's own native collections on domain reload

steep sandal
#

I'll look into it.

#

Last question and the biggest hurdle i'm facing currently.

How to handle the automatic reception / sending of these commands ?

Like as I stated before netcodeEntities use a lot of functions & systems to take care of that. I feel like it's overcomplicated.

What does your pipeline look like ?

north crescent
#

just have a system that queriees those buffers and sends them?

#

I didn't look at netcode of entities

#

so it was simple to me

#

😅

steep sandal
#

probably shot myself in the foot by looking into that first

#

FML

#

Alright so to get my thoughts wrapped up.

I'm thinking :

public interface INetworkCommand : IComponentData { }
ISerializableCommand where T : struct, INetworkCommand {
public void Serialize(ref DataStreamWriter writer, in T data);
public void Deserialize(ref DataStreamReader reader,ref  T data);
}

This would be the base structure.

Then I would have a System taking care of the registration with assembly searching & stuff ( easy to do )

This would create a list of :

public struct NetworkCommandData : IComparable<NetworkCommandData>{
public ulong TypeHash;
public NetworkPortableFunctionPointer<CreateDelegate> Deserialize;
public NetworkPortableFunctionPointer<CreateDelegate> Serialize;
public int CompareTo(NetworkCommandData other){}
}

Holding delegates for both serialization & deserialization pointers

public struct NetworkPortableFunctionPointer<T> where T : Delegate
{
public NetworkPortableFunctionPointer(T executeDelegate) { Ptr = BurstCompiler.CompileFunctionPointer(executeDelegate); }
public readonly FunctionPointer<T> Ptr;
}

just a copy of netcode wrapper for function pointer, very handy.
The collection would look something like this :

public struct NetworkCommandCollection
{
public NativeParallelHashMap<ulong, int> TypeHashToIndex;
public NativeList<NetworkCommandData> NetworkCommands;
}

With create delegate functions & stuff that goes with it.

The command collecting system will then put that as a DB in a shared static for later usage.

Then the question is how to manage the buffers to write & read from ... & how to actually generically read and write on

Make sense so far ?

north crescent
#

unless you literally expose it (which is a bad thing)

#

generics only exist on high level

#

when you are making a packet, you only have bytes at this point

#

so you just need to figure how to convert those bytes into networkable bytes

#

and then vice versa

steep sandal
north crescent
#

stable hash is only used in networked packet

steep sandal
#

and that hy there is a hashmap in the first place ^^ to match these bytes to their correct command ?

north crescent
#

outside of network packet

#

I use burst type hash

#

hashmap is to serialize/deserialize into/from network packets

#

your write buffer goal is to just let user write commands easily to it

#

your read buffer goal is to just let user read commands easily from it

#

the rest is handled internally

steep sandal
# 
DynamicBuffer<byte> buffer = new DynamicBuffer<byte>();
NativeArray<byte> bufferConverted = new(buffer.Length, Allocator.Temp);
writer.WriteBytes(bufferConverted);

buffer being where you write the commands

And you send the buffer with something like that I suppose ?

writer being DataStreamWriter writer

#

I might be overthinking at this point xD

#

And you get the bytes from

int streamLength = stream.Length;
NativeArray<byte> streamData = new(streamLength, Allocator.Temp);
stream.ReadBytes(streamData);

& then read from streamData

#

seeing that it's a copy you can read that multiples time , gather all the command in the stream and read them in a parrallel jobs if needed

north crescent
#

they cannot be networked

steep sandal
#

raaaaah so confusing

#

is there a specific reason as of why you don't use the same hash EVERYWHERE ?