#archived-networking | Official Unity Discord | Page 51

gleaming prawn Oct 15, 2019, 2:00 PM

#

ByteStream is just the stream...:)

weak plinth Oct 15, 2019, 2:00 PM

#

It doesn't do bitpacking mind you

gleaming prawn Oct 15, 2019, 2:00 PM

#

Which is the right approach, BTW

weak plinth Oct 15, 2019, 2:01 PM

#

It's just called bytestream

foggy vale Oct 15, 2019, 2:01 PM

#

what about the packing?

#

unpacking?

weak plinth Oct 15, 2019, 2:01 PM

#

bitpacking only saves bandwidth

#

At the cost of cpu

foggy vale Oct 15, 2019, 2:01 PM

#

I was reading and searching... I found out a guy using this but it seems to be crap to me... not sure:

#

https://github.com/Dgut/unnyhog/blob/master/Assets/Files/UnnyNetPacker.cs

GitHub

Dgut/unnyhog

Contribute to Dgut/unnyhog development by creating an account on GitHub.

gleaming prawn Oct 15, 2019, 2:01 PM

#

And on top of a byte stream (like this one from Complex), you'll need to handle the higher level things yourself...

weak plinth Oct 15, 2019, 2:02 PM

#

Yes

#

The thing I sent is literally only for converting primitives (and strings) to memory

#

Formatting the data to and from is for your own implementation

gleaming prawn Oct 15, 2019, 2:02 PM

#

We use both byte and bit streams, with and without packing, depending on the case.

foggy vale Oct 15, 2019, 2:02 PM

#

I understood there are 3 layers.... the stream... the packing and the handlers for the CMD right? I'm trying to solve the third because I'm planning to use libraries dealing with the streaming and packing

weak plinth Oct 15, 2019, 2:03 PM

#

I'm currently writing some IL weaver that automatically adds read and write functions to whatever objects you want to serialize

#

But I'm still working on that

#

Uses generics/interfaces without virt function calls ❤

#

The stream of data comes from your transport layer

foggy vale Oct 15, 2019, 2:04 PM

#

could you suggest a good library for packing and unpacking packets?

weak plinth Oct 15, 2019, 2:04 PM

#

And it shouldn't really be a stream anymore at that point

#

But rather usable packets

foggy vale Oct 15, 2019, 2:04 PM

#

yeah.... let's call them packets then

weak plinth Oct 15, 2019, 2:04 PM

#

IE, segments of memory

#

You convert those to your actual structured objects (preferably structs)

#

And pass those down to your entities that need their data

gleaming prawn Oct 15, 2019, 2:05 PM

#

preferably structs
Memory aligned ones, and stored in sequential buffers preferably (write your own heap allocator)...

foggy vale Oct 15, 2019, 2:05 PM

#

stilll it is needed an unpacking library that does the parsing to my struct/interface type

weak plinth Oct 15, 2019, 2:06 PM

#

Not that difficult to make

#

Either hand write it

gleaming prawn Oct 15, 2019, 2:06 PM

#

Or use code gen

weak plinth Oct 15, 2019, 2:06 PM

#

Or, if you are exclusively using blittable structs, use marshal.offset

#

Storethe required data at startup using reflection

#

And you can easily restructure the data

gleaming prawn Oct 15, 2019, 2:07 PM

#

I think that was a very good question TBH. Funny there isn't a good library that does all that...

weak plinth Oct 15, 2019, 2:07 PM

#

Soon TM

gleaming prawn Oct 15, 2019, 2:07 PM

#

Properly fast.

foggy vale Oct 15, 2019, 2:08 PM

#

thank you ... any suggestiong for the unpacked packer library (parsing bytes to types?)

gleaming prawn Oct 15, 2019, 2:08 PM

#

Will you you adhere to DOTS standards or will it be a plain C# thing @weak plinth ?

weak plinth Oct 15, 2019, 2:08 PM

#

If you don't need bitpacking you can use mine. Probably as fast as it gets in C#

#

@gleaming prawn I haven't worked with DOTS, so you'll have to explain that one

foggy vale Oct 15, 2019, 2:09 PM

#

I'm not sure DOTS would make things faster here

gleaming prawn Oct 15, 2019, 2:09 PM

#

I know.

weak plinth Oct 15, 2019, 2:09 PM

#

The idea as far as it does is that you get a delegate that you hand your object and writer off to

gleaming prawn Oct 15, 2019, 2:09 PM

#

I assume then it will just work over any set of struct types just fine.

weak plinth Oct 15, 2019, 2:09 PM

#

And it writes the entire object to the buffer you supplied

gleaming prawn Oct 15, 2019, 2:10 PM

#

Not requiring the C# subset that burst compiler uses on Unity, etc

weak plinth Oct 15, 2019, 2:10 PM

#

Delegate is a dynamic method

gleaming prawn Oct 15, 2019, 2:10 PM

#

ok

weak plinth Oct 15, 2019, 2:10 PM

#

Probably faster using memory offsets

#

But mine works with classes and non-blittble structs

gleaming prawn Oct 15, 2019, 2:11 PM

#

ok... we work with blittable, memory aligned, and custom allocator even, so we can do as fast as it can possible be I guess.

weak plinth Oct 15, 2019, 2:11 PM

#

If you only use blittable, just use memory offsets

#

memory alignment doesn't matter for serialization

gleaming prawn Oct 15, 2019, 2:12 PM

#

From my colleague @graceful zephyr , if you are interested @foggy vale :

#

https://github.com/fholm/UnsafeCollections/tree/master

weak plinth Oct 15, 2019, 2:12 PM

#

Oh is he a colleague of yours?

gleaming prawn Oct 15, 2019, 2:12 PM

#

For blittable types, these unsafe collections are faster than the original .net counterparts.

#

Yes

weak plinth Oct 15, 2019, 2:12 PM

#

Small world I guess 🙂

gleaming prawn Oct 15, 2019, 2:12 PM

#

Of course

foggy vale Oct 15, 2019, 2:12 PM

#

mmmm thank you but I'm not sure I want to push it so much! 😉

weak plinth Oct 15, 2019, 2:13 PM

#

It's best to use only blittable structs, unmanaged

foggy vale Oct 15, 2019, 2:13 PM

#

I want to write clean and optimize but I don't need this criticality level

weak plinth Oct 15, 2019, 2:13 PM

#

That's going to be the fastest for sure, and saves you that GC alloc

gleaming prawn Oct 15, 2019, 2:14 PM

#

There are these two very very important reasons:

absolutely blazing fast performance (for several reasons, including cache hits, less indirections/lookups);
No GC allocs (and infamous GC spikes).

#

That's why in our case blittable/memory aligned/custom allocated is mandatory...

weak plinth Oct 15, 2019, 2:15 PM

#

I still need to grab some custom allocator

gleaming prawn Oct 15, 2019, 2:15 PM

#

Notice again wbonx, GC spikes would hurt you much more than the array vs switch difference

weak plinth Oct 15, 2019, 2:15 PM

#

Luckily I've abstracted allocation away everywhere

gleaming prawn Oct 15, 2019, 2:16 PM

#

Having 0 runtime allocations from gameplay loops is our #1 mandatory rule (not an optimization later, design rule).

foggy vale Oct 15, 2019, 2:16 PM

#

I'm gettting convinced about using a switch

gleaming prawn Oct 15, 2019, 2:17 PM

#

Careful with the network transport layer in that case... Example...

weak plinth Oct 15, 2019, 2:17 PM

#

I agree with that rule

gleaming prawn Oct 15, 2019, 2:17 PM

#

Say you use Complex's bytestream, all good, right?

#

Then if the transport layer copies the array somewhere to batch the network data safely, that's probably memory alloc.

#

That will be left to the GC (depending on the transport layer lib you use - I don't know how the new one behaves in this case).

#

But that's also very important.

weak plinth Oct 15, 2019, 2:19 PM

#

Use a transport that accepts pointers

#

And use the unsafe writer/readers

foggy vale Oct 15, 2019, 2:19 PM

#

that depends from the net libray... I'm using the unity new one

weak plinth Oct 15, 2019, 2:20 PM

#

You can use ENet, that's pretty good

#

I don't know about the Unity one

gleaming prawn Oct 15, 2019, 2:20 PM

#

Yeah, I haven't worked with it yet either.

foggy vale Oct 15, 2019, 2:20 PM

#

they are all quite similar at the end... I'm ok so far with it

weak plinth Oct 15, 2019, 2:20 PM

#

Does the Unity transport deal with IntPtr?

#

Or Byte[]?

gleaming prawn Oct 15, 2019, 2:21 PM

#

Just check the allocs... Contrary to what some people said, I haven't heard bad things about it actually (the new transport thing).

foggy vale Oct 15, 2019, 2:21 PM

#

I wasa cheking the byte stream lib... interesting.,.. but I need something to unpack on the other end now

weak plinth Oct 15, 2019, 2:21 PM

#

unpack on the other end?

foggy vale Oct 15, 2019, 2:21 PM

#

deserialize

weak plinth Oct 15, 2019, 2:21 PM

#

There's a reader and a writer

gleaming prawn Oct 15, 2019, 2:22 PM

#

this how our Serializers look like (an example:)

weak plinth Oct 15, 2019, 2:22 PM

#

Aren't you using the memory offset as well then Erick?

#

Since I'm sure Freddie uses that

foggy vale Oct 15, 2019, 2:23 PM

#

there will always be a batch of the buffer copied somewhere else... that's by definition in almost all the network libraies

weak plinth Oct 15, 2019, 2:23 PM

#

Depends if it's mamanged or unmanaged

gleaming prawn Oct 15, 2019, 2:23 PM

#

  partial class RuntimePlayer
  {

    public Int32 Data;

    partial void SerializeUserData(BitStream stream)
    {
      stream.Serialize(ref Data);
    }
  }

#

Bitstream has writing/reading modes.

foggy vale Oct 15, 2019, 2:23 PM

#

nice

gleaming prawn Oct 15, 2019, 2:23 PM

#

I guess Complex's has something similar

weak plinth Oct 15, 2019, 2:23 PM

#

I've split them up in a reader and a writer

#

Instead of both functionalities in one

gleaming prawn Oct 15, 2019, 2:24 PM

#

We actually use two instances, yes

foggy vale Oct 15, 2019, 2:24 PM

#

I like it... had already seen your lib and it was on the top of my choice

gleaming prawn Oct 15, 2019, 2:24 PM

#

But this functions is called passing either one depending if this is send/recv

#

Of course you can use two functions (pack/unpack), or check stream.IsWriting?, etc

weak plinth Oct 15, 2019, 2:25 PM

#

Maybe a silly question, but isn't the ref in this case adding overhead?

foggy vale Oct 15, 2019, 2:25 PM

#

could you help me out with the usage of it on the reading side?

weak plinth Oct 15, 2019, 2:25 PM

#

You need to copy 8 bytes instead of 4 no?

gleaming prawn Oct 15, 2019, 2:25 PM

#

Passing as ref is one of the convenient syntax sugars ours has.

#

could you help me out with the usage of it on the reading side?
This function handles both.

#

If stream is in writing mode, if uses the REF to set the data from the stream. Otherwise, it reads from data and sets on stream buffer.

weak plinth Oct 15, 2019, 2:26 PM

#

But ref passes the address no?

#

Instead of copying the value

gleaming prawn Oct 15, 2019, 2:26 PM

#

You need to copy 8 bytes instead of 4 no?
Yes... But in our experience this difference is minimal.

#

We do have the other syntax sugar, let me write here.

foggy vale Oct 15, 2019, 2:27 PM

#

should bea actually faster with the ref

weak plinth Oct 15, 2019, 2:27 PM

#

Yeah I understand that. Just making sure

#

I decided not to add ref, since the largest value is 8 bytes anyway

#

In my case

gleaming prawn Oct 15, 2019, 2:27 PM

#

This also works:

#

  partial class RuntimePlayer
  {

    public Int32 Data;

    partial void SerializeUserData(BitStream stream)
    {
      if (stream.Writing)
      {
        stream.WriteInt(Data);
      } else
      {
        Data = stream.ReadInt();
      }
    }
  }

weak plinth Oct 15, 2019, 2:27 PM

#

But I am using ref (optionally) with the new serializer I'm writing

gleaming prawn Oct 15, 2019, 2:28 PM

#

Which is probably similar to what yours do, correct Complex?

#

Same if you split (de)serialization over two functions.

weak plinth Oct 15, 2019, 2:28 PM

#

In my case you'd need a DeserializeUserData function

#

And pass the reader there

gleaming prawn Oct 15, 2019, 2:28 PM

#

For the new lib, because you'll add the extentions, correct?

foggy vale Oct 15, 2019, 2:29 PM

#

are you sure u need that if statement ?

weak plinth Oct 15, 2019, 2:29 PM

#

I'm probably leaving the new lib pretty damn open

foggy vale Oct 15, 2019, 2:29 PM

#

wouldn't be faster making the If implicit by passing two different ref type?

weak plinth Oct 15, 2019, 2:29 PM

#

I was thinking about allowing users to add their own serialization functions based on type

#

The entire IL is added on first run anyway

#

IE you'd have something like
AddFunction<T>(Methodinfi func)

gleaming prawn Oct 15, 2019, 2:30 PM

#

wouldn't be faster making the If implicit by passing two different ref type?
It's syntax sugar

#

Some people prefer one over the other.

foggy vale Oct 15, 2019, 2:30 PM

#

can be sugary anyway without the IF 😄

weak plinth Oct 15, 2019, 2:30 PM

#

And I was thinking about adding attributes as well, for bitpacking or something. Not sure yet

#

First experimenting on how to get reader and writer functions with IL

gleaming prawn Oct 15, 2019, 2:31 PM

#

@weak plinth for a serialization lib... Hmmm... Attributes

#

Make sure you don't need anything that would lead to runtime checks...

#

I'd rather go for code generation.

#

A generator that spills blazing fast code...

foggy vale Oct 15, 2019, 2:31 PM

#

I guess your library can't automaticly parse all types to a buffer right? I.E. I need to call a single method to get a single read and assign it to my struct right?

gleaming prawn Oct 15, 2019, 2:31 PM

#

And then, yes... Attributes are useful AF.

weak plinth Oct 15, 2019, 2:32 PM

#

There's runtime checks anyway.
You can't just throw any type down the serializer

#

If it's not known, it can't be serialized

#

IE non blittable types

#

But this is all on first startup

gleaming prawn Oct 15, 2019, 2:32 PM

#

I mean, but at SOME point there will be a compiler that will work over this.

#

If you embed the method to the type, you can as well code generate that method.

#

So in runtime, you only call it with the stream/buffer

weak plinth Oct 15, 2019, 2:33 PM

#

Startup runs, generates all the IL and spits out delegates that you can use, very crude

#

Will probably abstract a lot of that

foggy vale Oct 15, 2019, 2:33 PM

#

true... I mean it takes just longer to implement

gleaming prawn Oct 15, 2019, 2:33 PM

#

oh. ok

weak plinth Oct 15, 2019, 2:33 PM

#

I mean, there's codegen\

foggy vale Oct 15, 2019, 2:33 PM

#

I was looking for something in line with the library I posted

gleaming prawn Oct 15, 2019, 2:33 PM

#

you're generating IL on the fly?

weak plinth Oct 15, 2019, 2:33 PM

#

Or just IL gen/weaving

#

Yes

foggy vale Oct 15, 2019, 2:33 PM

#

you chekced this?

#

https://github.com/Dgut/unnyhog/blob/master/Assets/Files/UnnyNetPacker.cs

GitHub

Dgut/unnyhog

Contribute to Dgut/unnyhog development by creating an account on GitHub.

weak plinth Oct 15, 2019, 2:33 PM

#

Well, only once on the fly

gleaming prawn Oct 15, 2019, 2:33 PM

#

How does that goes with IL2CPP?

weak plinth Oct 15, 2019, 2:34 PM

#

I don't know the details, but I know more libraries use IL weaving

gleaming prawn Oct 15, 2019, 2:34 PM

#

I did wbonx... It's minimal, right?

weak plinth Oct 15, 2019, 2:34 PM

#

Unity uses it itself afaik

#

Only problem I might have. I'm using Reflection.Emit, which isn't technically supported for .net standard 2 (2.1+)

#

But you can ref it

foggy vale Oct 15, 2019, 2:34 PM

#

I think is qute not ptimized

weak plinth Oct 15, 2019, 2:34 PM

#

No idea yet what the comp of Unity is going to be

foggy vale Oct 15, 2019, 2:34 PM

#

not sure I should go for it

weak plinth Oct 15, 2019, 2:35 PM

#

If you codegen yourself, blittable structs, directly to writer/reader funcs

#

That's going to be faster than what I'm making

gleaming prawn Oct 15, 2019, 2:35 PM

#

That packer is storing enums for the type before each data... Not really needed if you know what you'll throw into the stream.

weak plinth Oct 15, 2019, 2:35 PM

#

Since delegates, again means virt calls

#

But the writer/reader calls are not virt

#

It's still in early stages, haven't had much time

foggy vale Oct 15, 2019, 2:36 PM

#

@gleaming prawn that's why it is a generic lib... but the implementation is odd... using the array resize and array of objects

weak plinth Oct 15, 2019, 2:37 PM

#

Yeah, stay away from that lmao

gleaming prawn Oct 15, 2019, 2:37 PM

#

🙂

weak plinth Oct 15, 2019, 2:37 PM

#

Not to sell my own product, but if you don't care about bitpacking, my lib is probably one of the best options out there

#

If you do care about bitpacking, I bet Erick has some nice classes for you

gleaming prawn Oct 15, 2019, 2:38 PM

#

Looks very very incomplete

foggy vale Oct 15, 2019, 2:38 PM

#

bitpacking means a parser of a package ?

weak plinth Oct 15, 2019, 2:38 PM

#

No, bitpacking means you fill your buffer with only the bits you need

#

So writing a bool takes 1 bit instead of 8

#

And if you have an int that uses only max 10k, you can use like 6 bits instead of 32

foggy vale Oct 15, 2019, 2:39 PM

#

problem is that it was used by a guy writing very interesting code... that's why I got confused.. this is the linki:

#

https://github.com/alerdenisov/llapi-example/blob/master/assets-folder/Code/Networking/NetworkController.cs

GitHub

alerdenisov/llapi-example

LLAPI Networking sample (Top Down Shooter). Contribute to alerdenisov/llapi-example development by creating an account on GitHub.

weak plinth Oct 15, 2019, 2:39 PM

#

But that requires bit shifting

#

And yes, that class you've send @foggy vale looks like crap

#

The code looks like an absolute mess to begin with

#

And he boxes all primitives

foggy vale Oct 15, 2019, 2:40 PM

#

since you checked the first unpacker... could you tell me why he uses an Objects[] ??

weak plinth Oct 15, 2019, 2:40 PM

#

Because he has no idea what he's doing

#

That's why

gleaming prawn Oct 15, 2019, 2:40 PM

#

Complex's has a lot MORe than that one...

foggy vale Oct 15, 2019, 2:40 PM

#

@weak plinth you talking about the first lib right ?

weak plinth Oct 15, 2019, 2:40 PM

#

UnnyNetPacker

#

That one

foggy vale Oct 15, 2019, 2:40 PM

#

yeah looks like crap

#

why he used the Object[] ?

weak plinth Oct 15, 2019, 2:41 PM

#

You can serialize any primitive with mine, or write an extension method for anything else

gleaming prawn Oct 15, 2019, 2:41 PM

#

BTW @weak plinth , this:

        public static void Write<T>(byte[] dest, int offset, T value) where T : unmanaged
        {
            fixed (byte* ptr = &dest[offset])
            { *(T*)ptr = value; }
        }

#

Do you know that "fixed" cast becomes a function stack on IL?

weak plinth Oct 15, 2019, 2:41 PM

#

Yes

#

One of the downsides of using managed memory

gleaming prawn Oct 15, 2019, 2:42 PM

#

In quantum, we IL-patch to make that a straight cast, with NO stack.

#

in release mode...

foggy vale Oct 15, 2019, 2:42 PM

#

guys since now we know each other since 1h... could you tell me what IL stands for ?

weak plinth Oct 15, 2019, 2:42 PM

#

I recommend anyone that if they need perf, they use the unmanaged writer

gleaming prawn Oct 15, 2019, 2:42 PM

#

Intermediate Language

#

The .net assembly intermediate language (that goes into .net compiled DLLs)

jade glacier Oct 15, 2019, 2:42 PM

#

@gleaming prawn Nice to see some more Exit faces in this chan

foggy vale Oct 15, 2019, 2:43 PM

#

Exit ?!

weak plinth Oct 15, 2019, 2:43 PM

#

Oh hey 👋

#

@jade glacier is writing a serializer as well

#

He actually inspired me to start working on one

gleaming prawn Oct 15, 2019, 2:43 PM

#

Yes, Exit Games (I'm one of the core developers of Photon Quantum)

foggy vale Oct 15, 2019, 2:43 PM

#

Emotitron I'm testing serializers... give me yours! 😉

jade glacier Oct 15, 2019, 2:43 PM

#

I am talking with Tobi in a PM at the moment, doing work with Exit on PUN stuff

gleaming prawn Oct 15, 2019, 2:44 PM

#

Tobi, I can hear his voice over the next room now...

foggy vale Oct 15, 2019, 2:44 PM

#

@gleaming prawn I guessed by your nick... saw a very bad use of your first implementations in some games

gleaming prawn Oct 15, 2019, 2:45 PM

#

Which implementations?

#

There are a few quantum games out there now, they are pretty good TBH. But it's always a work in progress, we learn every day (and the games themselves are not built by us).

#

But our internal stuff evolves as we go.

jade glacier Oct 15, 2019, 2:47 PM

#

@foggy vale
https://github.com/emotitron/BitpackingTools
https://github.com/emotitron/TransformCrusher

Though the bitpacking tools are not a wrapper like most serializers, so it loses a small amount of speed not maintaining its own position value.

foggy vale Oct 15, 2019, 2:47 PM

#

@gleaming prawn For example in Distrust they used photon... an early implementation I guess

gleaming prawn Oct 15, 2019, 2:47 PM

#

Distrust is not using Quantum AFAIK.

foggy vale Oct 15, 2019, 2:47 PM

#

went veeeery wrong

jade glacier Oct 15, 2019, 2:47 PM

#

Photon isn't quantum

gleaming prawn Oct 15, 2019, 2:47 PM

#

What is the team behind it?

jade glacier Oct 15, 2019, 2:47 PM

#

Exit owns all the Photon products... Quantum is one of them

foggy vale Oct 15, 2019, 2:47 PM

#

ah photom is not your software?!

#

*n

gleaming prawn Oct 15, 2019, 2:47 PM

#

Photon has three main Unity products (PUN, which is free, Bolt which is now free, and Quantum).

foggy vale Oct 15, 2019, 2:48 PM

#

we are talking about 5 years ago... wasn't this way

gleaming prawn Oct 15, 2019, 2:48 PM

#

PUN = mimics the original unity networking tool, it's quite popular, and there are hundreds of thousands of developers using it, very little quality control from us;
Bolt = complete state transfer (free for 20 ccu, so worth a try for FPS, survivals, etc);
Quantum = fast deterministic predict rollback

foggy vale Oct 15, 2019, 2:48 PM

#

5 years ago

gleaming prawn Oct 15, 2019, 2:49 PM

#

Probably either PUN or even the lower level transport layer directly (photon realtime)

jade glacier Oct 15, 2019, 2:49 PM

#

Quantum is pretty immune to "bad implementations"

gleaming prawn Oct 15, 2019, 2:49 PM

#

Quantum was released january last year.

foggy vale Oct 15, 2019, 2:49 PM

#

they just messed up something... it gave photon very bad advertisement

#

you should avoid this in future

gleaming prawn Oct 15, 2019, 2:49 PM

#

Not 100% immune @jade glacier but I'd say it helps you go in the right direction.

#

I wouldn't claim it is immune.

jade glacier Oct 15, 2019, 2:50 PM

#

By that I mean either it fails completely or it syncs your inputs I would assume?

gleaming prawn Oct 15, 2019, 2:50 PM

#

you should avoid this in future
Not really easy when the product is free for everyone...

foggy vale Oct 15, 2019, 2:50 PM

#

yes I understand

gleaming prawn Oct 15, 2019, 2:50 PM

#

Same as Unity... They can't avoid bad "publicity" if someone uses it the wrong way.

jade glacier Oct 15, 2019, 2:51 PM

#

How well stuff extrapolates I suppose falls into the implementation category

gleaming prawn Oct 15, 2019, 2:51 PM

#

By that I mean either it fails completely or it syncs your inputs I would assume?
There's more to it...

foggy vale Oct 15, 2019, 2:51 PM

#

@jade glacier will read your code.. thanks for the link!! 😉

gleaming prawn Oct 15, 2019, 2:51 PM

#

You can fail to write a very fast implementation, it may happen...

jade glacier Oct 15, 2019, 2:51 PM

#

Interesting

gleaming prawn Oct 15, 2019, 2:51 PM

#

BUt I'm quite happy with the overall quality of some implementations.

jade glacier Oct 15, 2019, 2:52 PM

#

From what I gathered, the main issue would just be creating a simulation that introduces difficult to extrapolate entities - resulting in some REALLY ugly resims

foggy vale Oct 15, 2019, 2:52 PM

#

Thank you guys for the help! ... leaving for a while

gleaming prawn Oct 15, 2019, 2:52 PM

#

AND there are a few very interesting titles being done now, under NDA.

foggy vale Oct 15, 2019, 2:52 PM

#

@gleaming prawn looking forward to see them!

jade glacier Oct 15, 2019, 2:53 PM

#

The couple titles fholm pointed out looked very solid

gleaming prawn Oct 15, 2019, 2:53 PM

#

It's a bit more nuanced @jade glacier . I'm a bit more humble in that. I'm very proud of what we are doing, yes... But I tend to be very down to earth when checking each case.

foggy vale Oct 15, 2019, 2:53 PM

#

@gleaming prawn you can mitigate by having a guy checking the forums for people having connection problems.... at the time that's how I learned about photomo and distrust

gleaming prawn Oct 15, 2019, 2:53 PM

#

We do that wbonx

foggy vale Oct 15, 2019, 2:53 PM

#

*sorry for the typos I'm dislessic

gleaming prawn Oct 15, 2019, 2:54 PM

#

We have even crawlers checking that, so we get alerts (in case we go for the rescue attempt)

foggy vale Oct 15, 2019, 2:54 PM

#

Good job then! 😉

gleaming prawn Oct 15, 2019, 2:54 PM

#

But not always possible to check everything, or sometimes it's just difficult to fix due to being a very inexperienced team, etc.

jade glacier Oct 15, 2019, 2:54 PM

#

PUN has the same issue Unity has - anyone can use it - and as such anyone can release a hot pile of garbage and have it associated with you

foggy vale Oct 15, 2019, 2:55 PM

#

true

gleaming prawn Oct 15, 2019, 2:55 PM

#

We've had people just trying to send "RPCs" with 1GB of data... In realtime.

#

And expecting that to just transfer immediately with almost no lag and all...:)

jade glacier Oct 15, 2019, 2:55 PM

#

Quantum you get their team, so the quality standard I assume is going to be typically much higher?

foggy vale Oct 15, 2019, 2:55 PM

#

then just give a different name to the two products... that's pretty basic business... otherwise you affect the brand

jade glacier Oct 15, 2019, 2:55 PM

#

I can't imagine any team is just using Quantum without making use of support?

gleaming prawn Oct 15, 2019, 2:56 PM

#

We've been in this for 15 years... Not really my area. But I have to say we're doing pretty well actually.

foggy vale Oct 15, 2019, 2:56 PM

#

ti takes a couple of rounds on Capitalism 2 plus to learn that 😉

gleaming prawn Oct 15, 2019, 2:56 PM

#

We're getting a lot more experienced teams now (specially with Quantum)

foggy vale Oct 15, 2019, 2:57 PM

#

You know... many people can write a good library... few people can write a library that users can't screw up 😄

gleaming prawn Oct 15, 2019, 2:57 PM

#

Currently, it has very little effect random forum complaints, honestly (but we still care and support anyone, of course).

jade glacier Oct 15, 2019, 2:57 PM

#

I'm looking forward to getting an NDA in place with Exit, just so I can hear what is going on with Q

gleaming prawn Oct 15, 2019, 2:57 PM

#

I'd say it's very very difficult, if not impossible to write something impossible to screw up

foggy vale Oct 15, 2019, 2:57 PM

#

I have no idea... I only remember years ago this distress thing and maybe a couple of more minor

jade glacier Oct 15, 2019, 2:58 PM

#

There is no net lib in Unityland that doesn't get negative forum hate.

#

Networking is hard. Trolls are harder.

gleaming prawn Oct 15, 2019, 2:58 PM

#

There's nothing on the internet that doesn't.

foggy vale Oct 15, 2019, 2:58 PM

#

again... thanks a lot for the help!! will go back at wok now

gleaming prawn Oct 15, 2019, 2:58 PM

#

yw

jade glacier Oct 15, 2019, 2:59 PM

#

aight, back to making codey bits. lates.

weak plinth Oct 15, 2019, 3:00 PM

#

Erick, you might know this. I'm working as a Java full stack dev atm. But it's pretty damn boring. Does a company like yours accept new people unfamiliar in game-dev, that have to work remotely 40hrs a week?

jade glacier Oct 15, 2019, 3:01 PM

#

Oh @weak plinth My final solution was to do some CodeGen. I generate an extension class for any class with the IPackable attribute.... and that has all of the serialization stuff pregen'd. Marshal was fragile, and reflection was slow.

gleaming prawn Oct 15, 2019, 3:01 PM

#

We're always looking for hardcore programmers, although it's a long process normally.

#

remote work is not a problem.

weak plinth Oct 15, 2019, 3:01 PM

#

What would this long process be? Lots of interviews and such?

gleaming prawn Oct 15, 2019, 3:02 PM

#

Where are you based? I assume somewhere in Europe due to timezone?

jade glacier Oct 15, 2019, 3:02 PM

#

Getting paperwork from Chrstiof mainly 😛

weak plinth Oct 15, 2019, 3:02 PM

#

Netherlands

gleaming prawn Oct 15, 2019, 3:02 PM

#

Lots of interviews and such?
Nah, just getting to know people better, seeing cool stuff (in game networking, or in our specific case hardcore cross platform determinism)

#

I started back in 2015, when I wrote a very very simple tutorial for PUN.

weak plinth Oct 15, 2019, 3:03 PM

#

So a portfolio?

gleaming prawn Oct 15, 2019, 3:03 PM

#

Very very simple, but the way I handled documentation and support caught the attention of management.

#

So a portfolio?
Nah, not formal like that... lets just say we'd not hire in the black...

#

It's like a pipeline... We have people that used to be customer now in our team (after they left their original studios), for example.

#

We used to have an interesting entry point...

weak plinth Oct 15, 2019, 3:05 PM

#

@jade glacier I'm using IL weaving. Basically means the serialize/deserialize functions of the objects are delegates

gleaming prawn Oct 15, 2019, 3:05 PM

#

In the past 2 years, we hired several small teams/developers to write samples to quantum.

jade glacier Oct 15, 2019, 3:05 PM

#

yeah, same result I think.

#

@weak plinth

gleaming prawn Oct 15, 2019, 3:05 PM

#

BTW, maybe this is not the appropriate channel to this dicussion @weak plinth .

#

You can PM me if you want.

#

Lets stick to netcode here.

jade glacier Oct 15, 2019, 3:06 PM

#

I am just rather than modifying a file with weaver, I am generating an extension for the class @weak plinth

#

but it also is just mapping delegates @weak plinth

weak plinth Oct 15, 2019, 3:06 PM

#

I'm not modifying a file. Reflection.Emit offers dynamic methods

jade glacier Oct 15, 2019, 3:06 PM

#

ah, yeah sounds like different path to a similar outcome

weak plinth Oct 15, 2019, 3:06 PM

#

You can create methods on runtime, throw IL in there, and reference them as a delegate

#

I didn't like the Mono.Cecil idea of loading assemblies, modifying them and saving them again

#

I looked at some examples (Like Mirror's weaver), but it's such a spaghetti code

gleaming prawn Oct 15, 2019, 3:07 PM

#

Complex, if this doesn't work with IL2CPP?

#

What is the solution?

weak plinth Oct 15, 2019, 3:07 PM

#

Mono.Cecil probably

gleaming prawn Oct 15, 2019, 3:07 PM

#

There's no point in Mono builds...

#

Performance is ABISMAL.

#

So IL modifications should be done before build.

weak plinth Oct 15, 2019, 3:08 PM

#

Fair point

jade glacier Oct 15, 2019, 3:08 PM

#

I am hoping that the codegen will spare me from having to chase that dragon

weak plinth Oct 15, 2019, 3:08 PM

#

In that case no IL weaving is going to work

gleaming prawn Oct 15, 2019, 3:08 PM

#

We don't even bother with Mono anymore now that IL2CPP is getting stable.

weak plinth Oct 15, 2019, 3:08 PM

#

Codegen brings a whole bunch of other problems

gleaming prawn Oct 15, 2019, 3:08 PM

#

I mean, mono builds.

jade glacier Oct 15, 2019, 3:09 PM

#

its all a giant pain in the ass

#

which is why this stuff doesn't exist 🙂

weak plinth Oct 15, 2019, 3:09 PM

#

I used codegen before for exactly this. But all the build exceptions drove me insane

#

Well, do it by hand

gleaming prawn Oct 15, 2019, 3:09 PM

#

We've been using code gen since day 1.

weak plinth Oct 15, 2019, 3:09 PM

#

Like probably everyone does lmao

gleaming prawn Oct 15, 2019, 3:10 PM

#

We code generate A LOT in quantum... It's a first class feature.

weak plinth Oct 15, 2019, 3:10 PM

#

Build time?

gleaming prawn Oct 15, 2019, 3:10 PM

#

Yes

#

Pre-build steps

#

Parsing our own DSL files.

#

To avoid developers having to deal with memory alignment, etc

#

(although we still let you do if you want to)

#

So you can mix and match as much as you want.

#

But most devs just 100% define game state data (entites, components, types) with our DSL.

weak plinth Oct 15, 2019, 3:11 PM

#

Well, you can use memory offsets still

#

For IL2CPP

gleaming prawn Oct 15, 2019, 3:11 PM

#

Yes

jade glacier Oct 15, 2019, 3:11 PM

#

I tried to avoid codegen for the longest time, but Marshal + OffsetOf + classes = hellzone

weak plinth Oct 15, 2019, 3:11 PM

#

IF you only use blittable structs

gleaming prawn Oct 15, 2019, 3:11 PM

#

Blittable structs is the ONLY solution for what we do anyway...

#

We use classes in just a couple of APIs.

jade glacier Oct 15, 2019, 3:12 PM

#

GCHandle and IL2CPP were not friends

gleaming prawn Oct 15, 2019, 3:12 PM

#

No GCHandle for us..:)

jade glacier Oct 15, 2019, 3:12 PM

#

yeah, for Quantum, the path is clear on this stuff

weak plinth Oct 15, 2019, 3:12 PM

#

How the hell do you use strings though?

jade glacier Oct 15, 2019, 3:12 PM

#

I live in the world of "Users want to stick vars in a monoB and have them magically do stuff"

gleaming prawn Oct 15, 2019, 3:13 PM

#

Why the hell do you NEED strings for gameplay simulation?

#

🙂

weak plinth Oct 15, 2019, 3:13 PM

#

Chat/Player names, something like that

#

But it's not sim

gleaming prawn Oct 15, 2019, 3:13 PM

#

We only use strings in a few places (what we call the Asset DB - your game balancing sheet of your own custom types - classes)

jade glacier Oct 15, 2019, 3:13 PM

#

Using your player name as the seed for your shotgun pattern... duh. @gleaming prawn

gleaming prawn Oct 15, 2019, 3:13 PM

#

and what we call events (to communicate simulation stuff to the view layer - unity)

#

Using your player name as the seed for your shotgun pattern
You can do that, lol...

weak plinth Oct 15, 2019, 3:14 PM

#

How would you send your player name to someone else?

jade glacier Oct 15, 2019, 3:14 PM

#

player name isn't sim really

weak plinth Oct 15, 2019, 3:14 PM

#

I know

gleaming prawn Oct 15, 2019, 3:14 PM

#

We have a few entry points for polymorphism in these asset classes which mean, the rollbackable game state is 100% comprised of blittable data.

#

How would you send your player name to someone else?
You can do that, but that's not SIM, as said.

#

AND, you can use your own collections, data and make it rollbackable by implementing our partial entry point for a frame "deep copy".

#

Our internal, code generated game state (from the DSL), is a memory aligned buffer from our own custom allocators, so we can essentially "deep copy" with a few native memcpy calls...

#

That's as fast as it can be...

weak plinth Oct 15, 2019, 3:16 PM

#

So basically, sim should literally only be blittable, ever

gleaming prawn Oct 15, 2019, 3:16 PM

#

Ideally yes...

#

And this is not quantum specific...

#

This is a GOOD design principle in general.

weak plinth Oct 15, 2019, 3:17 PM

#

I know

#

I'm not talking about anything quantum

#

Just in general

gleaming prawn Oct 15, 2019, 3:17 PM

#

This is what DOTS is all about.. It's pretty much the lessons you get from "Game Engine Architecture" by Jason Greggory, or what experienced studios use internally (check videos by Overwatch team, etc)

#

We just use the same principles (of course the OO conveniences exist in a few APIs).

weak plinth Oct 15, 2019, 3:18 PM

#

Sounds like I'm going to have to make a serializer for both mono and IL2CPP

gleaming prawn Oct 15, 2019, 3:18 PM

#

But a lot of what we do internally looks a lot more like C than C#.

#

I'd say if you want this to be used by experienced devs on games from now on, yes @weak plinth .

#

There's no point in making something for Unity now without working on IL2CPP.

#

IMHO. I think Unity guys would same the same.

#

IL2CPP does quite a good job, TBH

weak plinth Oct 15, 2019, 3:19 PM

#

Very likely. But there's a lot of mono people still around

#

And I'm one of those 😂

#

I haven''t worked with DOTS at all

gleaming prawn Oct 15, 2019, 3:20 PM

#

We compared performance of some of our math stuff (manually inlined) with:
Mono
.Net Framework (we can run ouside Unity)
.Net Core
IL2CPP
Full Native C with all optimizations on.

stiff ridge Oct 15, 2019, 3:20 PM

#

If you don't need DOTS, then don't use it. Unless you want to learn this specifically.

gleaming prawn Oct 15, 2019, 3:20 PM

#

Yes, IL2CPP is not specific to DOTS...

weak plinth Oct 15, 2019, 3:21 PM

#

Good to know

gleaming prawn Oct 15, 2019, 3:21 PM

#

No matter what you use in Unity, IL2CPP is the default runtime NOW.

weak plinth Oct 15, 2019, 3:21 PM

#

So, you click an option somewhere to build to cpp?

#

Is that it?

gleaming prawn Oct 15, 2019, 3:21 PM

#

In the stuff we tested it's between 4-10x faster than mono builds.

#

Yes

#

That's been the default for IOS and consoles since a while.

#

Now it's the recommended runtime for all platforms.

weak plinth Oct 15, 2019, 3:22 PM

#

What happens when you use something non cpp on that setting?

#

Like, IL weaving

gleaming prawn Oct 15, 2019, 3:22 PM

#

it breaks.

weak plinth Oct 15, 2019, 3:22 PM

#

It will only break at runtime though? lmao

gleaming prawn Oct 15, 2019, 3:22 PM

#

You should not use something like that at runtime.

#

I think it fails to build,

#

IL2CPP would see the non valid APIs being used.

#

There's no IL anymore after IL2CPP.

#

we don't use anything IL2CPP doesn't support of course, it would not make sense.

#

So I haven't tested much.

#

Maybe somebody else knows... @stiff ridge nice to see you around..::)

jade glacier Oct 15, 2019, 3:24 PM

#

Biggest headache for me with codegen is trying to deal with timing it. Like if I codegen in an extension class for ClassA ... the the dev deletes ClassA - there is a split second the log shows errors while my post compile callback handling deletes my extensions.

#

Chasing Unity's compiler is a pain in the ass

gleaming prawn Oct 15, 2019, 3:25 PM

#

We compile outside of Unity.

jade glacier Oct 15, 2019, 3:25 PM

#

Yeah, you guys live in that lovely ivory tower 😉

weak plinth Oct 15, 2019, 3:25 PM

#

IL weaving is a beautiful tool. Shame it doesn't work in Unity then though...

#

Had big hopes

jade glacier Oct 15, 2019, 3:26 PM

#

I have to muck around with the common folk in MonoB components 😛

gleaming prawn Oct 15, 2019, 3:26 PM

#

I wouldn't describe it like that.

jade glacier Oct 15, 2019, 3:26 PM

#

It seems every path leads to some ugly choices

gleaming prawn Oct 15, 2019, 3:26 PM

#

MonoB components
I have a tale about that...

#

a few actually.

weak plinth Oct 15, 2019, 3:26 PM

#

I really don't want to bother with codegen anymore. I've had my fill last time, and that's been more than enough

gleaming prawn Oct 15, 2019, 3:27 PM

#

Summary: beautifully convenient, but you need something else down the line for more complex stuff.

jade glacier Oct 15, 2019, 3:27 PM

#

For this stuff I just make a template and fill in the field/reflection mappings... and call it done

weak plinth Oct 15, 2019, 3:27 PM

#

I did as well, T4's

gleaming prawn Oct 15, 2019, 3:27 PM

#

Interesting Complex... We're actually quite happy we chose this path.

weak plinth Oct 15, 2019, 3:27 PM

#

Made code gen for packet handling basically

jade glacier Oct 15, 2019, 3:27 PM

#

I didn't T4, did something a little more personal

weak plinth Oct 15, 2019, 3:27 PM

#

I just hated making it

gleaming prawn Oct 15, 2019, 3:27 PM

#

And AFAIK our customers as well. It's a strength in our tools.

jade glacier Oct 15, 2019, 3:28 PM

#

since I wanted to be able to create a working template

gleaming prawn Oct 15, 2019, 3:28 PM

#

But we have our own (Fredrik's) AST parser and compiler, which gives us full control.

weak plinth Oct 15, 2019, 3:28 PM

#

Maybe T4 was just a horrible tool for the job. Maye that's why I didn't like it 🙂

gleaming prawn Oct 15, 2019, 3:28 PM

#

Maybe...

#

We don't use it...

#

Although we've recommended it for people who wanted to go even further down that rabbit hole.

jade glacier Oct 15, 2019, 3:30 PM

#

https://hatebin.com/kbpblwjqzh

This steaming pile is my template... all the ugly though just gets deleted. Its just there so I can write working code to use to make my codegen code from.

#

The main benefit just being I can literally code an example of resulting code to scratchpad what I am doing. Other than that - its a total hack horseshit move.

gleaming prawn Oct 15, 2019, 3:34 PM

#

this is what a Quantum DSL file looks like (for the developer, sample "entity" type):

entity Character3D {
    use Transform3D;
    use DynamicBody3D;
    use CharacterController3D;
    use Prefab;
}

jade glacier Oct 15, 2019, 3:34 PM

#

nice and tidy

gleaming prawn Oct 15, 2019, 3:34 PM

#

And this...

#

  [StructLayout(LayoutKind.Explicit)]
  public unsafe partial struct Character3D {
    public const Int32 SIZE = 344;
    public const Int32 ALIGNMENT = 8;
    [FieldOffset(32)]
    public CharacterController3D CharacterController3D;
    [FieldOffset(112)]
    public DynamicBody3D DynamicBody3D;
    [FieldOffset(24)]
    public Prefab Prefab;
    [FieldOffset(56)]
    public Transform3D Transform3D;
    [FieldOffset(0)]
    internal Entity _entity;
    public EntityRef EntityRef {
      get {
        return _entity._ref;
      }
    }
    public Int32 EntityIndex {
      get {
        return _entity._ref._index;
      }
    }
    public EntityFlags Flags {
      get {
        return _entity._flags;
      }
    }
    public override Int32 GetHashCode() {
      unchecked { 
        var hash = 271;
        hash = hash * 31 + CharacterController3D.GetHashCode();
        hash = hash * 31 + DynamicBody3D.GetHashCode();
        hash = hash * 31 + Prefab.GetHashCode();
        hash = hash * 31 + Transform3D.GetHashCode();
        hash = hash * 31 + _entity.GetHashCode();
        return hash;
      }
    }
    public static void Serialize(void* ptr, FrameSerializer serializer) {
        var p = (Character3D*)ptr;
        CharacterController3D.Serialize(&p->CharacterController3D, serializer);
        DynamicBody3D.Serialize(&p->DynamicBody3D, serializer);
        Prefab.Serialize(&p->Prefab, serializer);
        Transform3D.Serialize(&p->Transform3D, serializer);
        Entity.Serialize(&p->_entity, serializer);
    }
  }

#

Is the code generated struct...

#

I omitted some stuff because it was bigger than allowed.

#

There's a lot more we generate. It's very very convenient for the game developer in the end... Save Games, Snapshot based reconnects, replays (including full match and killcam), all this comes from two basic design principles:

deterministic predict rollback
code generation (Imagine just having to write serializers/packers by hand for all your types, just that).

jade glacier Oct 15, 2019, 3:38 PM

#

My existence is writing serializers/packers by hand 🙂

gleaming prawn Oct 15, 2019, 3:38 PM

#

So you're afraid of loosing your job? 🙂

#

Kidding

#

Codegen for your own benefit... So you can be as efficient as 20 yous.

jade glacier Oct 15, 2019, 3:39 PM

#

It more is for this kind of stuff, so its not REALLY hand writing, its writing the things that automate the packing, so somewhere in between.

📎 unknown.png

gleaming prawn Oct 15, 2019, 3:39 PM

#

Ok

#

Oh, you are the guy who's doing that.

#

Got it now

jade glacier Oct 15, 2019, 3:39 PM

#

But like the serialization code for packing down a Mechanim is all kinds of fun

gleaming prawn Oct 15, 2019, 3:40 PM

#

O(my)

jade glacier Oct 15, 2019, 3:40 PM

#

Lots if inline bits that indicate what follows that indicates the packing that will be used and such

#

But that still is just once, and its all just based on parameters. I am not hand coding that over and over or anything

gleaming prawn Oct 15, 2019, 3:41 PM

#

Transform crusher, this looks a bit like Bolt's approach for compression of fields.

jade glacier Oct 15, 2019, 3:41 PM

#

Its pretty common now yeah, range based floats

gleaming prawn Oct 15, 2019, 3:42 PM

#

yep

jade glacier Oct 15, 2019, 3:42 PM

#

half floats is on that dropdown too

#

That's the default, so it "just works", but range based is better

gleaming prawn Oct 15, 2019, 3:42 PM

#

We don't use floats, so we deal only with other types compression.

#

In quantum, ofc

#

Bolt does

jade glacier Oct 15, 2019, 3:44 PM

#

yeah, I assume you employ less inline packing and more entire byte[] (or ulong[]) packing?

#

Or do you have per field packing?

#

Knowing fholm... the answer is "both"

gleaming prawn Oct 15, 2019, 3:45 PM

#

yes, lol

#

But we don't do this prematurely

#

Specially because bandwidth in quantum is rarely an issue.

jade glacier Oct 15, 2019, 3:45 PM

#

You are already dealing with small syncs yeah

gleaming prawn Oct 15, 2019, 3:46 PM

#

Unless you create mammoth input data structs or have unusually high player counts.

#

But still important to be smart with META data (more important than the input struct itself)

jade glacier Oct 15, 2019, 3:46 PM

#

you are only going to get so much reduction, but I can see schemes for that, like putting all of the inputs that tend to stay at zero at the far left

#

and putting all the high velocity inputs on the right, etc

gleaming prawn Oct 15, 2019, 3:46 PM

#

We spent a lot more time designing and optimizing that for the several use cases than focusing on compressing the input data itself.

#

And this is a good advice IMHO for anyone writing HL netcode solutions:

take care of your meta as much as your data...:)

jade glacier Oct 15, 2019, 3:47 PM

#

Good luck preaching that to most of these networking channels

#

nearly all convos are "which is best" and which transport benchmarks the best... very little discussion of how to reduce your game to a simulation that consumes inputs and produces states.

#

So there is no meta, other than "which of these libs can I have the most connections to and cram the most data through"

gleaming prawn Oct 15, 2019, 4:00 PM

#

That message is for a specific target (people who write some of these libs, but probably I don't need to say any of that)

robust yacht Oct 15, 2019, 6:59 PM

#

Does anyone know how to create a fortnite style lobby

weak plinth Oct 15, 2019, 8:42 PM

#

Either load a new scene when the game starts

#

Or easier yet, make some secluded area away from the map and throw everyone in there

#

That's all it is

jade wharf Oct 16, 2019, 3:07 AM

#

@jade glacier just curious, but do you know why serialization seems to be so tedious to deal with for IL2CPP?

jade glacier Oct 16, 2019, 3:14 AM

#

You mean related to what we were talking about earlier?

#

I think that was in reference to IL not being usable, so weaving and reflection.emit and such were out

#

For mean its tedious because I have opted for codegen, but that creates a whole bunch of icky race conditions with the Unity compiler, play and build timings.

#

@jade wharf

jade wharf Oct 16, 2019, 3:16 AM

#

Huh, strange

#

I wonder what kind of IL couldnt be translated by a computer if a human could understand it

jade glacier Oct 16, 2019, 3:17 AM

#

Not sure what is up with that, totally out of my wheelhouse

weak plinth Oct 16, 2019, 7:57 AM

#

Hi, anyone familiar with PlayFab here for a quick discussion ?

weak plinth Oct 16, 2019, 8:23 AM

#

@jade wharf That's not the point

#

If you compile with IL2CPP, it's native

#

That's why it's called IL TO C++

#

You can't then, after you are running a C++ application, inject IL into itself

#

Since it's no longer running IL

#

https://docs.unity3d.com/Manual/IL2CPP-HowItWorks.html

#

As soon as you pass the step of C++ code

#

It's over and done with using IL on runtime

#

So, then there's codegen or reflection left for serialization. One is tedious to make and one is slow as hell to use

weak plinth Oct 16, 2019, 10:03 AM

#

@gleaming prawn Do you use bool anywhere or exclusively a blittable version?

gleaming prawn Oct 16, 2019, 10:05 AM

#

We use Boolean inside game state, yes

#

Have to check what is the alignment for that inside the custom allocator.

#

But for network sends we use a bitstream, so we actually pack as 1 bit.

weak plinth Oct 16, 2019, 10:07 AM

#

I figured that last bit

#

So your structs contain bools at places?

#

I wonder if it can cause problems for me, but I don't think so

gleaming prawn Oct 16, 2019, 10:39 AM

#

Should not... We use everywhere, of course.

weak plinth Oct 16, 2019, 1:54 PM

#

@jade glacier How were you doing the serializing of unmanaged types?

#

I can't seem to find a way to "bind" methods to fieldinfo's

#

Not unless I box values or something

#

I think I'm completely going the wrong direction with this 🤔

jade glacier Oct 16, 2019, 2:15 PM

#

its a long annoying chain at the moment

#

Basically I am constantly checking for changes to which classes/structs have my packing attribute - and then I generate an extension class for that class, that has all of the fields that will be serialized wired up to a generated serialization method.

#

Then since I can't hardwire that, that generated code has a runtime RunOnAwake initialization, that adds itself to a dictionary

weak plinth Oct 16, 2019, 2:18 PM

#

I can't seem to use generics.

#

I've wasted a few hours on this

#

I think I need WriteInt, WriteByte, etc

jade glacier Oct 16, 2019, 2:18 PM

#

yeah, I gave up on generics as well after hours of trying to make them happen

weak plinth Oct 16, 2019, 2:18 PM

#

I mean, it's childsplay with IL

#

But I can't do that :/

jade glacier Oct 16, 2019, 2:18 PM

#

I have interfaces that indicate supported primitives for each packing attribute

#

So any packing attribute I write, will have those non-generic handlers

weak plinth Oct 16, 2019, 2:19 PM

#

I tried just using Write<T>

jade glacier Oct 16, 2019, 2:19 PM

#

If course

weak plinth Oct 16, 2019, 2:19 PM

#

But you can't use this, unless you use dynamicinvoke

#

And I don't know the performance implications of that

jade glacier Oct 16, 2019, 2:19 PM

#

but its a long chain, and trying to get that all into a nice delegate faceplants somewhere

weak plinth Oct 16, 2019, 2:19 PM

#

Not even mentioning it only takes objects

jade glacier Oct 16, 2019, 2:20 PM

#

My last performance ding is that I have to cast from Object to my Class type

weak plinth Oct 16, 2019, 2:20 PM

#

Isn't casting extremely cheap?

#

It's 1 IL operation

jade glacier Oct 16, 2019, 2:21 PM

#

yeah, just its there and its annoying me that I can't find a way for it not to be 🙂

#

But otherwise, once its all running its just a delegate for the pack/unpack on the entire class

weak plinth Oct 16, 2019, 2:21 PM

#

If you are using codegen, anything should be possible

#

You should be able to just use generic delegates there

jade glacier Oct 16, 2019, 2:21 PM

#

and everything in that is normal code

#

The issue is I am not touching the actual class

weak plinth Oct 16, 2019, 2:22 PM

#

I know

jade glacier Oct 16, 2019, 2:22 PM

#

and you cant make extensions into delegates

#

So right now I have to produce a

{
    var packable = obj as GarbageStart;
    myIntPacker(ref packable.myInt, buffer, ref bitposition);
    mybytePacker(ref packable.mybyte, buffer, ref bitposition);
            
}```

Kind of thing for each class

#

I haven't sorted out the way to avoid making that an object, since I am putting all of these delegates into a dictionary

#

There may be a way, I am just a mediocre C# coder and my knowledge of what can be done with delegates, anonymous and all of that is pretty limited.

#

So right now that cast makes this step work, and lets me get on to the stuff I do know, which is making all of wackadoo bitpacking attributes

weak plinth Oct 16, 2019, 2:30 PM

#

But you have to assume the type has public members?

jade glacier Oct 16, 2019, 2:30 PM

#

yeah, it can only pack public fields

#

if that is what you mean

weak plinth Oct 16, 2019, 2:30 PM

#

That's harsh

jade glacier Oct 16, 2019, 2:31 PM

#

Its that or modifying the class itself as far as I can tell

weak plinth Oct 16, 2019, 2:31 PM

#

I know this might be a bit late at this point

#

But maybe there is a way to inject IL before compilation?

jade glacier Oct 16, 2019, 2:32 PM

#

Maybe, all of this is getting a bit lower level than I like to spend my time in. I am a UX and design guy. I am only doing any of this coding just to get me to that. So most of this is pretty much at my knowledge limits.

weak plinth Oct 16, 2019, 2:33 PM

#

Do mind that people will have to break encapsulation to make the serializer work

jade glacier Oct 16, 2019, 2:35 PM

#

Sure, but not as much minding that I won't finish this otherwise

#

This is networking for the masses, not ivory tower coding

#

The only non-IL way around that is to either modify the actual class.cs as far as I can see, or to have to make use of GetValue and SetValue

#

All of those aren't acceptable to me, so right now forcing public for networked fields seems like the lesser evil

weak plinth Oct 16, 2019, 2:37 PM

#

If I can be completely honest

#

Maybe it's a good idea to just have people "deal with it"

jade glacier Oct 16, 2019, 2:37 PM

#

You werent going to be?

weak plinth Oct 16, 2019, 2:37 PM

#

If someone wants to serialize classes per say for network usage, they are stuck with mono

#

And you can use IL

#

And if someone wants actual performance in IL2CPP, they can only use structs anyway

jade glacier Oct 16, 2019, 2:38 PM

#

You just exchanged forcing people down a path from one thing to another

#

None of the paths are perfect, and I just don't care that much. I have one objective and that is getting properly networked components into users hands

#

There may be a really good IL solution in all of this, but I don't touch IL and I don't like chasing rabbits

weak plinth Oct 16, 2019, 2:39 PM

#

There might be, if you can, ahread of compilation, modify the assembly

jade glacier Oct 16, 2019, 2:40 PM

#

I can always revisit that layer later if I decide I want to clean it up and have the time

weak plinth Oct 16, 2019, 2:40 PM

#

That's the only thing I can think of

jade glacier Oct 16, 2019, 2:40 PM

#

If you come up with the final ideal answer to all of this, I will love to know about it

weak plinth Oct 16, 2019, 2:40 PM

#

https://docs.unity3d.com/Manual/ScriptingRestrictions.html

jade glacier Oct 16, 2019, 2:40 PM

#

you are better equipped to find that answer

weak plinth Oct 16, 2019, 2:40 PM

#

Anything listed as AoT can't use Reflection.Emit

jade glacier Oct 16, 2019, 2:42 PM

#

Part of me would love to dive down that rabbit hole. The practical side of me knows there are a dozen guys like you working this problem right now - and my skills are suited to be focused on the UX layer and component layer.

weak plinth Oct 16, 2019, 2:42 PM

#

Also good to know that you can't use generic virtual methods

#

Since I am using those currently

jade glacier Oct 16, 2019, 2:42 PM

#

So unless someone comes up with the right answer and tells me, I have to move on

#

public class SampleClass
{
    [PackInt(0, 100)]
    public int intA1;
    [Pack]
    public byte byteA1;
}```

This is workable

weak plinth Oct 16, 2019, 2:49 PM

#

Nvm it's a bit more complicated than that

jade glacier Oct 16, 2019, 2:52 PM

#

If you do solve this riddle, be sure to let me know 🙂 I may need to revisit my choices.

weak plinth Oct 16, 2019, 2:55 PM

#

Alright

#

It should be possible, but with a lot of hassle

#

You can use mono cecil to edit and save (back to disk) a dll

#

You can then run this code that does this during building, by using the buildpipeline

jade glacier Oct 16, 2019, 2:56 PM

#

Does it handle itself correctly when you hit play?

#

The one major drawback right now with what I have is that if a player deletes a file that I had extended, the log shows errors for a second while my post compile code deletes the now invalid files

#

and I can't find any way to detect, delete and recompile before that error hits the log

#

Takes about this long to clear

📎 2019-10-13_22-01-36.mp4

weak plinth Oct 16, 2019, 2:59 PM

#

It should handle itself correctly if done right

#

Since you are essentially building the modified assembly with all of the delegates and such you created

jade glacier Oct 16, 2019, 2:59 PM

#

This is with codgen on the fly, rather than build time. Build time I couldn't get it to see the files if I made them with codegen on PreBuild callbacks

foggy vale Oct 16, 2019, 2:59 PM

#

hello guys, can I ask a very simple question in parallel? Don't want to interrupt

weak plinth Oct 16, 2019, 3:00 PM

#

I might take a look at this

#

Mono Cecil is pretty horrible to use since there's pretty much 0 docs

jade glacier Oct 16, 2019, 3:01 PM

#

I see why Unet chose it, the alternatives are a nut scrape

weak plinth Oct 16, 2019, 3:01 PM

#

Well, there are not really any (good) alternatives 😂

foggy vale Oct 16, 2019, 3:01 PM

#

I was wondering what is the most efficient way to define a univoque ID of an object shared between server and clients, so that I.E. when a transform.position change package is exchanged the new position can be assigned to the right object

jade glacier Oct 16, 2019, 3:02 PM

#

I just need passable and doesn't utterly break, which is where I think I am right now, so moving on for now to making my attributes. @weak plinth

#

its an instance I assume? @foggy vale

weak plinth Oct 16, 2019, 3:02 PM

#

I kindof want to make an IL Unity and non-Unity version

#

The IL should be exactly the same

foggy vale Oct 16, 2019, 3:03 PM

#

I guess at the beginning I have on the server an instantiate on the server that generate instance and tell the client to do the same giving it an ID. this objects are then stored on a sorted list?

#

@jade glacier yes

jade glacier Oct 16, 2019, 3:03 PM

#

Is this with a particular library?

foggy vale Oct 16, 2019, 3:03 PM

#

using yours 😉

jade glacier Oct 16, 2019, 3:03 PM

#

Because just spawning does that

#

Oooh, well for SNS you are going to need to spawn normally

#

my stuff rides on top of PhotonView, NetworkIdentiy and NetworkedIdentity

foggy vale Oct 16, 2019, 3:04 PM

#

poit is shall I use the native ID? isn't it heavy? Shall I then just instantiate on the server and share the unique ID with the clients?

#

your packers run on any library I guess

#

I'm using the new Unity library

#

it is like llapi

#

and your libs just to serialize packing transforms

jade glacier Oct 16, 2019, 3:05 PM

#

yeah, they don't try to fight the existing libraries - it makes use of the netobj of the HLAPI being used

foggy vale Oct 16, 2019, 3:05 PM

#

no Hlapi

#

Using the new unity llapi

jade glacier Oct 16, 2019, 3:06 PM

#

My stuff won't "just work" for that, since there is nothing there to just work on @foggy vale

#

but you can call my OnSerialize methods

foggy vale Oct 16, 2019, 3:06 PM

#

that's what I'm doing

jade glacier Oct 16, 2019, 3:06 PM

#

Its not throwing a wall of errors not having a recognized net lib?

#

The animatorsync specifically is worth the work you are doing, stripping out all of the net lib specifics

#

and just making use of the serialize/deserialize

foggy vale Oct 16, 2019, 3:07 PM

#

question is just about what kind of ID shall I use, how to share it (i guess in a serialized struct passed on the connection... and how to store the ID, how is it more efficient, if they should be a sorted dictionary with as a value the ref to the instance

jade glacier Oct 16, 2019, 3:07 PM

#

Writing that animatorSync yourself... do not recommend that.

#

That is kind of a bigger question, and for the new MP I am not fully equipped to give you the right answer

#

because you are basically making your own MLAPI

foggy vale Oct 16, 2019, 3:08 PM

#

I don't need to synch any anim

#

yes right

#

but I need only to synch maybe a transform.... rest I don't care

jade glacier Oct 16, 2019, 3:09 PM

#

yeah, you aren't going to be able to even think abut that part until you have your methods in place for spawning numbered objects

#

and yeah, the server is responsible for that

foggy vale Oct 16, 2019, 3:09 PM

#

exactly... could you suggest an implemettation for that?

jade glacier Oct 16, 2019, 3:09 PM

#

It will pick an unused ID, and then tell everyone to spawn a prefab and give it that ID

#

you have to make your own version of NetworkIdentity basically

foggy vale Oct 16, 2019, 3:09 PM

#

right

jade glacier Oct 16, 2019, 3:09 PM

#

They all generally agree on that

#

PhotonView, NetId and NetworkedId all do that

foggy vale Oct 16, 2019, 3:10 PM

#

what type of Id shall I use?

jade glacier Oct 16, 2019, 3:10 PM

#

how many objects do you expect to max out at?

foggy vale Oct 16, 2019, 3:10 PM

#

and what's the most efficient way to store thew references to the isntancesd

#

?

jade glacier Oct 16, 2019, 3:10 PM

#

you will just have a dictionary

foggy vale Oct 16, 2019, 3:10 PM

#

1000

jade glacier Oct 16, 2019, 3:11 PM

#

likeDict<ushort, MyNetID> spawnedObjects

foggy vale Oct 16, 2019, 3:11 PM

#

shall the ID be an integer... to be large enough but small enough to be in each packet? shall the dictionary be slowing things too much because of the lookup?

jade glacier Oct 16, 2019, 3:12 PM

#

You can internally just us an int for cleanliness

#

but for networking you can bitpack or cast that down if you know the upper possible value

#

like 0-1000 you can cast down to a ushort

#

send 16 bits rather than 32

#

or you can use PackedBytes and not worry about it

foggy vale Oct 16, 2019, 3:13 PM

#

but that's not really scalable.... 1000 is now much if you delete and create new instance and so on

jade glacier Oct 16, 2019, 3:13 PM

#

just keep your values on the low end when possible so it can pack out the unused bytes

#

That is up to you, if you want your server code to aggressively recycle ids

foggy vale Oct 16, 2019, 3:13 PM

#

what kind of list would you use? a dictionary isnt too slow?

jade glacier Oct 16, 2019, 3:13 PM

#

dicts are quick

#

the quickest is an array, but then you have to allocate a MyNetId[1239239139] up front... no good

#

Just use a Dict, that's what it is there for 🙂

foggy vale Oct 16, 2019, 3:14 PM

#

a dict would do a search as big asthe dictionary for each packet would be crazy

jade glacier Oct 16, 2019, 3:14 PM

#

how many objects do you think you will have in each packet there?

#

because if you have enough objects in a packet to crush your cpu with dict lookups... you are going to bring down the internet

foggy vale Oct 16, 2019, 3:15 PM

#

don't know yet.... was thinking to keep it open and scalablke

jade glacier Oct 16, 2019, 3:15 PM

#

yeah, just use a dict

#

you are going to paralyze yourself before you start here

foggy vale Oct 16, 2019, 3:16 PM

#

Ok... so I instantiate on the server... give an ID and an int, then tell the clients to instantiate and send the same ID with the call ?

#

it is not about killing the CPU or the bandwidth, I'm just trying to find the best solution/cheapest solution

jade glacier Oct 16, 2019, 3:17 PM

#

yeah, pretty much

#

the server and all clients instantiate the indicated prefab and assign the same id to your NetId component

foggy vale Oct 16, 2019, 3:18 PM

#

do you have experience with this? using your own MLIB to synch and assign custom ID?

jade glacier Oct 16, 2019, 3:18 PM

#

Dicts exist as the cheapest for looking things up by an index

foggy vale Oct 16, 2019, 3:18 PM

#

are ou talking about indexed dictionary?

jade glacier Oct 16, 2019, 3:18 PM

#

I don't write them, but all of the HLAPIs do this

foggy vale Oct 16, 2019, 3:19 PM

#

it is more complicate since when you kill an instance you have the indexed/sorted dictionary that has to deal with that

jade glacier Oct 16, 2019, 3:20 PM

#

That is all just later detail. For starters you just pull the next number.. have the server keep a counter of last assigned netid value

#

and increment it by one when you spawn

#

later you will want to replace that with something that keeps track of released IDs, so you can reuse them

#

Basically keep a queue or list of relased ids, and pull from that first rather than incrementing

#

that keeps your netid.value low - which makes it more compresable

#

but again... for later

#

To start, just have the server increment the assigned ID by one each spawn... done

foggy vale Oct 16, 2019, 3:22 PM

#

and the ID in your example would be like the key of my dictionary and the instance ref would be the value?

weak plinth Oct 16, 2019, 3:23 PM

#

I guess you could use an array as well instead of a dictionary and use a ringbuffer (or queue) to buffer free id's

#

Uses more memory, but guarantees no resizing

foggy vale Oct 16, 2019, 3:23 PM

#

array aren't static and expensive to resize?

weak plinth Oct 16, 2019, 3:23 PM

#

Every collection uses arrays

foggy vale Oct 16, 2019, 3:23 PM

#

or you mean just making a huge one and that's it?

#

true

weak plinth Oct 16, 2019, 3:24 PM

#

If you are going to have, say max 1024 networked objects

#

That's 1024 x 64 bits

#

8kb of memory used yay

#

And 1024 networked objects is already quite a lot

jade glacier Oct 16, 2019, 3:25 PM

#

you have lots of options

foggy vale Oct 16, 2019, 3:25 PM

#

so let's say... I make my 10K array... instantiate on the server... assign an int ID, send an instantiate msg with the int ID on the client and make the client add instantiate and assign the ref in the array at the same position right?

weak plinth Oct 16, 2019, 3:25 PM

#

Normally, they should be in sync

jade glacier Oct 16, 2019, 3:25 PM

#

main thing is don't paralyze yourself debating which collection to use. Just use the one that is easy. Refactor later if you decide it was a bad choice.

#

its all going to be a very contained bit of code, easy to refactor

weak plinth Oct 16, 2019, 3:26 PM

#

If you are using the same collection of objects on server and client (which I assume you will)

foggy vale Oct 16, 2019, 3:26 PM

#

it is critical to decide certain things early.... like the size of the ID variable... can't refactor that lateer

weak plinth Oct 16, 2019, 3:26 PM

#

Any add/remove commands will deal with exactly the same data and id's

foggy vale Oct 16, 2019, 3:26 PM

#

@weak plinth yes

jade glacier Oct 16, 2019, 3:26 PM

#

Some things yes, this isn't one of those critical things

#

All that matters at this point is that the server produce an unused ID

weak plinth Oct 16, 2019, 3:26 PM

#

If you add on the server on an empty collection, it will get id 0

jade glacier Oct 16, 2019, 3:27 PM

#

Later you can get fancy with your handling for recycling

weak plinth Oct 16, 2019, 3:27 PM

#

And if you send the client add

#

It will also give you this 0 as an id

#

Same for remove, or any value

foggy vale Oct 16, 2019, 3:27 PM

#

right

jade glacier Oct 16, 2019, 3:27 PM

#

I can't stop you, but these channels are full of dudes over engineering stuff for years - never producing any actual game code. Go the path of least resistance and get to a working state.. then evaluate.

weak plinth Oct 16, 2019, 3:27 PM

#

And you can use a fixed queue for recycling id's

foggy vale Oct 16, 2019, 3:28 PM

#

but what when I.E. things are reverted.... an onject has a collision and I need to know it's index?

weak plinth Oct 16, 2019, 3:28 PM

#

Its ID is its index in the array

foggy vale Oct 16, 2019, 3:28 PM

#

with an array I have to cycle through all the index to find out the index of the ref collliding

jade glacier Oct 16, 2019, 3:28 PM

#

I mean, this convo for example... nearing 20 minutes just trying to pick a container type... that is cripplingly slow. Just pick one and go.

foggy vale Oct 16, 2019, 3:29 PM

#

@jade glacier you are right... but I could waste time later by not thinking now

weak plinth Oct 16, 2019, 3:29 PM

#

You are not making something where dictionary vs array is going to matter

#

Trust me 🙂

foggy vale Oct 16, 2019, 3:29 PM

#

it is also about the ID type or how to handle it... not jsut about the the container

jade glacier Oct 16, 2019, 3:29 PM

#

Its going to be a long road though if you jawbone everything this hard

weak plinth Oct 16, 2019, 3:30 PM

#

If you want to go lazy mode 5 milion

#

Throw an int or even an long as a dictionary key

#

Just only increment

#

No need for recycling

#

You arent going to make 4.2 billion objects anyway

jade glacier Oct 16, 2019, 3:30 PM

#

You can refactor that down once you have a better idea what you actually need

#

The only reason for recycling is to be kind to the BytePacker

weak plinth Oct 16, 2019, 3:30 PM

#

Just keep an id counter next to the dictionary that's a static value

jade glacier Oct 16, 2019, 3:30 PM

#

but that is a later problem

weak plinth Oct 16, 2019, 3:30 PM

#

Meh

#

I'd call that microops as well

jade glacier Oct 16, 2019, 3:31 PM

#

yeah, this all is

#

too early

weak plinth Oct 16, 2019, 3:31 PM

#

4 bytes vs, what, maybe 2?

foggy vale Oct 16, 2019, 3:31 PM

#

on a network for each packet.... that's something 😉

weak plinth Oct 16, 2019, 3:31 PM

#

Just using 4 bytes at all times to define an ID isn't going to be bad

jade glacier Oct 16, 2019, 3:31 PM

#

Then revisit it later

weak plinth Oct 16, 2019, 3:31 PM

#

And that allows you to run through 4.2b objects

#

Which I assume you aren't going to do

jade glacier Oct 16, 2019, 3:31 PM

#

You want a clean working prototype to optimize

#

Not a 4 weeks in and still be debating how to best make your NetObj

foggy vale Oct 16, 2019, 3:32 PM

#

Okkey, so your advice is to use an int in any container

#

thanks 😉

weak plinth Oct 16, 2019, 3:32 PM

#

It would take you 48.8 days of recycling 1.000 objects per second

#

To cap out your integer

foggy vale Oct 16, 2019, 3:32 PM

#

I thought there were better implementations... like using thre unity ID generator and chopping that down

jade glacier Oct 16, 2019, 3:32 PM

#

yup, get to a working state using the most generic methods possible

weak plinth Oct 16, 2019, 3:32 PM

#

Probably not going to happen

jade glacier Oct 16, 2019, 3:33 PM

#

There are a zillion compression schemes and refactors

#

don't try to guess them yet

weak plinth Oct 16, 2019, 3:33 PM

#

As I said yesterday, serialization takes much more cpu than message handling for instance

#

Dictionary access is crazy fast

foggy vale Oct 16, 2019, 3:34 PM

#

again each implementation has problems downstream... the difference is try to predict them ... IMMO

weak plinth Oct 16, 2019, 3:34 PM

#

And then spending a month on 5% of your entire product?

foggy vale Oct 16, 2019, 3:35 PM

#

but you are right, maybe this is not going to be a problem later

weak plinth Oct 16, 2019, 3:35 PM

#

Not worth it

#

There's a reason almost every game botched up somewhere

#

People run out of money and time

foggy vale Oct 16, 2019, 3:35 PM

#

😦

weak plinth Oct 16, 2019, 3:35 PM

#

And good enough is an actual thing

foggy vale Oct 16, 2019, 3:35 PM

#

public static Func<DataStreamReader.Context, DataStreamReader, DataStreamReader.Context>[] _handlers = { MSG_Transform };

jade glacier Oct 16, 2019, 3:35 PM

#

It is THE thing in networking

weak plinth Oct 16, 2019, 3:35 PM

#

It's in every case of software, and probably engineering

#

If it meets the requirements 🆗

#

Time to ship it

foggy vale Oct 16, 2019, 3:36 PM

#

You are right maybe it is not a good quality to try to make things perfect and good looking... not sure

jade glacier Oct 16, 2019, 3:36 PM

#

I would back up and start by creating your NetIdentity class that is going to be the heart of all of this

weak plinth Oct 16, 2019, 3:37 PM

#

There's also a difference between end-user products and frameworks or libs

#

In the latter, I'd argue that performance is much more important

jade glacier Oct 16, 2019, 3:37 PM

#

being an asset maker, I can attest to the pain in the assness of not knowing all of the use cases, and having to account for them

foggy vale Oct 16, 2019, 3:37 PM

#

netidentity class is going to be ID and owner as a component on the instance, not much more I guess

jade glacier Oct 16, 2019, 3:38 PM

#

It will start as that

#

and grow

#

it is your primary conduit for all serialization, ownership, authority etc

foggy vale Oct 16, 2019, 3:39 PM

#

@weak plinth you are right... but good enduse stuff is made with reusable code that has the same quality of libs and assets

jade glacier Oct 16, 2019, 3:39 PM

#

You have SNS right ? @foggy vale

foggy vale Oct 16, 2019, 3:39 PM

#

sns?

jade glacier Oct 16, 2019, 3:39 PM

#

SimpleNetworkSync?

#

You were using my stuff you said, but not sure what stuff

foggy vale Oct 16, 2019, 3:39 PM

#

no I'm synching by myself

jade glacier Oct 16, 2019, 3:39 PM

#

oh, nm then

foggy vale Oct 16, 2019, 3:39 PM

#

just the transofrm cracker

jade glacier Oct 16, 2019, 3:39 PM

#

Oooh, nm then

#

that doesn't have any of the NetMaster update manager stuff

foggy vale Oct 16, 2019, 3:40 PM

#

if you have any reference I should look at just post me a link

#

I'm eigher to read this days

flint swan Oct 16, 2019, 3:41 PM

#

Question: Is it better to start to learn about the networking side and then code a game or do a single player game and convert it at some point to multiplayer (or any pointers to a starting point of how i can eventually achieve a working multiplayer are welcome )

foggy vale Oct 16, 2019, 3:41 PM

#

depends on what u know

#

to convert a complicate game to multiplayer later it is like to write it from scratch

jade glacier Oct 16, 2019, 3:42 PM

#

I personally would start with some of the basic HLAPI's tutorials, just to see the starting point - but be aware many of them do things very wrong

#

but it will give you the language you need to do better

flint swan Oct 16, 2019, 3:43 PM

#

ok - i'm kind of lost as it seems unity is between 2 different networking methods and want to get a basic understanding of what is the best one to use and where to start

foggy vale Oct 16, 2019, 3:43 PM

#

actually never go low if you can... takes forever to use low level libs

#

for the network at least

flint swan Oct 16, 2019, 3:44 PM

#

i've created basic games in the past and want to try a multiplayer game - starting points? The manual or are there any working examples that do it properly that i can disect?

foggy vale Oct 16, 2019, 3:45 PM

#

just lean from any Llapi documentation... the new unity low level lib is very similar

jade glacier Oct 16, 2019, 3:45 PM

#

PUN2/Mirror/Forge/Bolt all have tutorials that are worth poking at for starters

foggy vale Oct 16, 2019, 3:45 PM

#

or if you are beginning to code go for HLapi to understand things

flint swan Oct 16, 2019, 3:46 PM

#

ok and Hlapi is where unity is at or where its heading?

jade glacier Oct 16, 2019, 3:46 PM

#

The risk of any HLAPI is starting to think that how they do things is the right way, which isn't the case

foggy vale Oct 16, 2019, 3:46 PM

#

@jade glacier @weak plinth thanks guys for the help. I take a break now

jade glacier Oct 16, 2019, 3:47 PM

#

off to breakfast here, lates

flint swan Oct 16, 2019, 3:47 PM

#

ty for the tut links Emotitron

vernal relic Oct 16, 2019, 6:06 PM

#

Hi guys, I feel really stupid right now, could you help me with a very simple issue? I have just begun the PUN 2 basics tutorial, but something is wrong. When I call PhotonNetwork.JoinRandomRoom() I never get any message that the connection was successful or unsuccessful

#

https://hatebin.com/dkufsnbqtk

#

Thats all of it.

#

A button calls Connect() which calls JoinRandomRoom() or ConnectUsingSettings() if the client isnt connected. The connection to the master works fine, but in OnConnectedToMaster() it should call JoinRandomRoom(). Photon should then call back either OnJoinedRoom() or OnJoinRandomFailed() but nothing happens.

#

📎 unknown.png

#

Thats all I get

weak plinth Oct 16, 2019, 6:20 PM

#

@jade glacier https://stackoverflow.com/questions/53785910/avoiding-the-overhead-of-c-sharp-virtual-calls

Stack Overflow

Avoiding the overhead of C# virtual calls

I have a few heavily optimized math functions that take 1-2 nanoseconds to complete. These functions are called hundreds of millions of times per second, so call overhead is a concern, despite the

jade glacier Oct 16, 2019, 6:22 PM

#

I don't make my calls using the interfaces if that is what that is about? That is why I go after the delegates for my code gen methods rather than abstract into interfaces for that. If I am reading that right at a glance. @weak plinth

weak plinth Oct 16, 2019, 6:23 PM

#

No, just sending you this in general

#

If you are or going to use some interface to allow api stuff

jade glacier Oct 16, 2019, 6:23 PM

#

yeah, for hot paths I try to avoid interfaces, but I do have a bunch in place just because it makes the coding sane

#

but down the line I may replace them all with cached delegates. Or not.

#

This software is likely going to become the property of Exit with its main use being PUN2... and outlandish numbers of net objects are just not a thing in that environment.

#

So sanity of code will trump microop

weak plinth Oct 16, 2019, 6:24 PM

#

Definitely 😄

jade glacier Oct 16, 2019, 6:37 PM

#

@weak plinth Do you have any experience with trying to get codegen to happen right at build time?

#

I could solve a lot of my problems if I could make that work, but as it has been, generating those files in the PreBuild hasn't been getting them added to the project... so I am missing some way to get them compiled and included in time.

#

AssetDatabase.Refresh();```
aren't making it happen

#

Or I should say they happen, the files exist and are compiled in the editor after the build... but the build is missing them.

weak plinth Oct 16, 2019, 6:41 PM

#

Maybe with the buildpipeline also?

#

There should be a way to run code just before building

jade glacier Oct 16, 2019, 6:43 PM

#

I think I tried that, but will try again since I have lost track of all of the places I have tried to stick it

#

It runs, I see the log in the right order

#

but the build doesn't get it

weak plinth Oct 16, 2019, 6:43 PM

#

I haven't worked with that yet. Will have to look at it once I fiddle about with cecil

#

sharplab keeps giving me virtual calls

#

Even if I do what that stackoverflow link suggests

#

There is a possibility that the JIT compiles it away

#

But seeing JIT doesn't work on that site for me

#


If thisType is a reference type (as opposed to a value type) then ptr is dereferenced and passed as the 'this' pointer to the callvirt of method.

If thisType is a value type and thisType implements method then ptr is passed unmodified as the 'this' pointer to a call method instruction, for the implementation of method by thisType.

If thisType is a value type and thisType does not implement method then ptr is dereferenced, boxed, and passed as the 'this' pointer to the callvirt method instruction.```

#

So in case of a value type, it's a normal method call instead of a virtual

#

Good to know

fading apex Oct 16, 2019, 7:17 PM

#

for some reason the marked code is only executed on the host and not the clients. why is that ?

📎 unknown.png

jade glacier Oct 16, 2019, 7:20 PM

#

You are calling it from where? @fading apex

#

nm, you must be calling it on the host for that to fire at all

#

at a glance that should work, might need to see the whole file

fading apex Oct 16, 2019, 7:24 PM

#

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Networking;

public class LootController : NetworkBehaviour
{

    public List<Lootable> drops = new List<Lootable>();
    public GameObject crateDestrcutionParticle;
    public float crateHealth;

    private GameObject lootObject;

    void OnCollisionEnter2D(Collision2D collision)
    {
        if (collision.gameObject.CompareTag("Bullet")){
            crateHealth -= collision.gameObject.GetComponent<Bullet>().GetDamage();
            if (crateHealth <= 0){
                CmdDestroyed();
            }
        }
    }

    [Command]
    void CmdDestroyed() {
        Lootable loot = drops[Random.Range(0, drops.Capacity)];
        RpcSpawnParticle();
        if (loot.GetChance() >= Random.Range(0.0f, 1.0f)){
            lootObject = loot.GetLoot();
            RpcSpawnLoot(loot);
        }
    }

    [ClientRpc]
    void RpcSpawnParticle() {
        Instantiate(crateDestrcutionParticle, transform.position, Quaternion.identity);
        Destroy(this.gameObject);
    }

    [ClientRpc]
    void RpcSpawnLoot(Lootable loot){
        Instantiate(lootObject, transform.position, Quaternion.identity);
    }

}

#

that would be the whole file

#

its not very advanced/clean code but it works
and its the first time i am making multiplayer and only my second serious unity project

jade glacier Oct 16, 2019, 7:26 PM

#

You have a debug.LogError in that RPC to make sure everything is happening where you think?

#

I'm not seeing a lot of checks to make sure those commands and rpcs aren't really being checked to make sure they are coming from an actual authority

#

You need to debug that back, and make sure for starters that that Command is firing on the server as expected, the work forward from there.

fading apex Oct 16, 2019, 7:29 PM

#

ok ill try that thx

gray pond Oct 16, 2019, 7:42 PM

#

@fading apex What are those squiggle lines griping about?

#

oh it's UNet - nvm me

weak plinth Oct 16, 2019, 10:42 PM

#

Actually @jade glacier I'm not sure if I'm going to make that serializer anytime soon

#

It's going to cost me a whole lot of time

#

And I'm literally getting nothing of any game done

#

I'm just making libraries at this point

#

And I shouldn't be needing anything more than what I can do with Wobes' solution. Since I want my packets to be blittable exclusively (at least the ones for game state)

#

And I just know it's going to take me a whole bunch of time just figuring out how to do with Cecil what I can already do with reflection.emit

#

For something I probably won't be using anyway...

#

I'll finish the blittable serializer and get going again (hopefuly)

jade glacier Oct 16, 2019, 10:44 PM

#

this little side project has cost me 4 days already... its definitely not a hit it and quit it LOL

#

Yeah, all of the guys have libraries you can steal to get up and running

#

and you can be a champ and just PR theirs

weak plinth Oct 16, 2019, 10:45 PM

#

I spend about 1 day making serialization of non-nested classes/structs with reflection.emit

#

It was interesting working with IL

#

I've learned something

#

So it's all good for me to not continue with it

#

I'll throw the blittableserializer on git once it's finished

#

Using FSE's bitbuffer maybe

#

I'm going to go with attributes (optionally) as well like you are

#

To define max size

#

Because I know I will actually be using that a lot for my own projects

jade glacier Oct 16, 2019, 10:49 PM

#

I would, this mid to low level stuff isn't interesting to anyone... so unless you have a product you are producing with it - I would put a pin in it and just refactor it next year when you are bored some weekend.

weak plinth Oct 16, 2019, 11:29 PM

#

Well, it would be blazing fast serialization to raw bytes

#

I'm not sure if someone offers that for c# out there

#

Most serialization uses some custom format with overhead etc

amber trench Oct 17, 2019, 12:02 AM

#

@weak plinth you can use Cap'n'Proto, which is effectively an arena allocator

#

you can edit its C# client to support mutable memory, it's just not recommended, but it will work

#

authoring your own arena allocator is essentially reinventing DOTS

#

cap'n'proto has the benefit of having a cross-platform representation, which DOTS does not

#

the java and C++ mutations are fine, but i haven't tried it with C#

#

people just reinvent arena allocators all day

#

i don't know why it's so poorly taught / documented

#

its just very difficult to author something that uses region-based memory correctly

#

i think there are better approaches, e.g., use an embedded Lua runtime on both your client and your server, and synchronize the lua memory region

#

but even roblox, which had an opportunity to do this, still requires you to declare what gets replicated

#

so it's not clear if it's possible to make stuff any simpler than Cap'n'Proto, i.e., you will have to declare, somewhere, what memory should be treated specially. whether that's an attribute or a type definition file or a call to a method, it doesn't matter, that work will be done somewhere

amber trench Oct 17, 2019, 12:32 AM

#

i think you'd like it

weak plinth Oct 17, 2019, 8:44 AM

#

Sounds convoluted to me, at least for network messaging

#

Simply using a Bitstream is much more efficient than what I'm reading on their website

#

And what I've been talking about has nothing to do with region based memory management

#

https://capnproto.org/encoding.html
Their encoding has a ton of overhead

#

I don't need any of this

#

They are using 8 bytes just to store some information about a struct?
If I use a bitstream, I need exactly 0 bytes to store information about a struct

#

Maybe 1 bit per field to determine if it should be updated or not

karmic void Oct 17, 2019, 8:57 AM

#

Hello! Anyone knows other solution like SpatialOS but completely Open Source?

#

Foer example: https://lucidscape.com/

gleaming prawn Oct 17, 2019, 9:48 AM

#

I don't know of anything open source in that topic, no. I suspect the new company/product Coherence (announced a couple of months ago), which has the involvement of Unity Founder David Hagelson might be doing something similar (maybe better as well?).

#

My personal view is that theses are super cool and neat, but in reality (at least SpacialOS looked like) are extremely expensive from a business perspective (even open source, the server costs may skyrocket, IMHO).

#

But that's all I know (sorry).

karmic void Oct 17, 2019, 3:26 PM

#

@gleaming prawn Thank you very much!!

amber trench Oct 17, 2019, 9:47 PM

#

@weak plinth i think the idea is that your cap'n'proto message is also your game state, you mutate it as needed, when you need to transmit all or part of it it is already in a buffer for you

weak plinth Oct 17, 2019, 9:48 PM

#

But it's senseless to use this for network messasing

#

Just pass blittable structs around

#

Smaller and faster

amber trench Oct 17, 2019, 9:50 PM

#

maybe i'm missing something, but isn't an array of bytes and a blittable struct sort of the same thing?

weak plinth Oct 17, 2019, 9:50 PM

#

Definitely missing something

#

Structs are formatted (structured) data

#

An array of bytes of just data

#

It's pretty useless without interpretation

amber trench Oct 17, 2019, 9:51 PM

#

i think cap'n'proto's whole idea is that the interpretation is "free" in the sense that it requires no copying

weak plinth Oct 17, 2019, 9:51 PM

#

Definitely not

amber trench Oct 17, 2019, 9:51 PM

#

so once you receive a buffer that represents a cap'n'proto message (really a collection of objects), you can get at those objects without copying into your runtime

weak plinth Oct 17, 2019, 9:52 PM

#

That's not how it works

#

Everytime you read data from your buffer, you are copying

#

If you don't format it, but rather format it on the go

amber trench Oct 17, 2019, 9:52 PM

#

so why do they say on their site,
"This benchmark is, of course, unfair. It is only measuring the time to encode and decode a message in memory. Cap’n Proto gets a perfect score because there is no encoding/decoding step."

#

what could they mean by that

weak plinth Oct 17, 2019, 9:53 PM

#

Because encoding is not the same as copying memory?

#

Google encoding, it's something completely different

amber trench Oct 17, 2019, 9:53 PM

#

what could they mean by decoding being free

weak plinth Oct 17, 2019, 9:53 PM

#

They don't decode

#

So it's free

#

Because it's not there

amber trench Oct 17, 2019, 9:53 PM

#

so how could that work

#

i receive a byte buffer, and i cast it into a struct, for example

#

in C, for example

#

did i copy?

weak plinth Oct 17, 2019, 9:53 PM

#

Yes

amber trench Oct 17, 2019, 9:54 PM

#

maybe the more appropriate term woudl be a union

#

i have a type that is a union of a byte buffer and a bunch of ordered, name fields

#

is that a copy?

weak plinth Oct 17, 2019, 9:54 PM

#

Still yes

amber trench Oct 17, 2019, 9:54 PM

#

is there some term for

weak plinth Oct 17, 2019, 9:54 PM

#

Since your network layer doesn't recieve unioned structs

amber trench Oct 17, 2019, 9:54 PM

#

i have an array of bytes

#

hmm...

weak plinth Oct 17, 2019, 9:54 PM

#

You have to copy it into your unioned struct first

amber trench Oct 17, 2019, 9:54 PM

#

i'm aware you can't avoid copying from your network stack

weak plinth Oct 17, 2019, 9:54 PM

#

But don't ever do that

amber trench Oct 17, 2019, 9:54 PM

#

into your runtime

weak plinth Oct 17, 2019, 9:54 PM

#

Because it's bad

amber trench Oct 17, 2019, 9:55 PM

#

but once i have the byte buffer from my network stack

#

can't i just interpret it, in a special way, where i no longer copy the bytes?

weak plinth Oct 17, 2019, 9:55 PM

#

No

amber trench Oct 17, 2019, 9:55 PM

#

hmm

weak plinth Oct 17, 2019, 9:55 PM

#

You need to copy from the heap to the stack everytime you access

#

Or you convert pointers

#

But then you need to pin the array

#

And it loses all formatting

amber trench Oct 17, 2019, 9:56 PM

#

i understand from the point of view of assigning primtive variables, there will be a copy

weak plinth Oct 17, 2019, 9:56 PM

#

Any value there will be

#

Struct layout is not guaranteerd to be the same on every system

#

Nor even on a restart/rebuild of your application

amber trench Oct 17, 2019, 9:57 PM

#

what if there was a way to guarantee a struct layout

weak plinth Oct 17, 2019, 9:57 PM

#

So just copying an entire struct is very dangerous

amber trench Oct 17, 2019, 9:57 PM

#

do you think that would be useful?

weak plinth Oct 17, 2019, 9:57 PM

#

And should never be done unless it;s on the system itself

#

There isn't

amber trench Oct 17, 2019, 9:58 PM

#

hmm

#

but let's say that there were

#

would that be useful?

weak plinth Oct 17, 2019, 9:58 PM

#

Different types can be different sizes depending on architecture

#

You could copy directly to a struct

#

Rather than reading value per value into one

amber trench Oct 17, 2019, 9:58 PM

#

right, which sounds kind of like a bitstream right?

#

reading value by value

weak plinth Oct 17, 2019, 9:58 PM

#

That's what a bitstream is

#

yes

amber trench Oct 17, 2019, 9:58 PM

#

so copy directly to a struct, that sounds valuable right?

#

and the main problem is, actually, a canonical representation of structs, across process and runtime boundaries

#

i guess what i'm getting at is, cap'n'proto (and really protobufs), they sort of do these things. ECS is another idea for doing this thing

#

where actually, the thing that is a huge chore is gathering up all your little game structs and writing them out

weak plinth Oct 17, 2019, 10:00 PM

#

I'm sure they do exactly the same as a bitstream

amber trench Oct 17, 2019, 10:00 PM

#

it's a much bigger chore to write value by value, and then read value by value

weak plinth Oct 17, 2019, 10:00 PM

#

But rather have the data order embedded in the protocol

#

Instead of having strictly user-defined structs that you read top to bottom

amber trench Oct 17, 2019, 10:01 PM

#

i guess you should take a look at what it does, technologically, i think you would like it 🙂

weak plinth Oct 17, 2019, 10:01 PM

#

I think you are looking for some magic solution that doesn't exist

amber trench Oct 17, 2019, 10:02 PM

#

it just provides some hints as to why ECS looks the way it does, like what they are learning from

weak plinth Oct 17, 2019, 10:02 PM

#

That has no relation to network serialization

amber trench Oct 17, 2019, 10:03 PM

#

i think the idea is, they want to provide a general networking framework

#

something akin to the SC2/Overwatch engine's way of handling things

#

which is having a sort of runtime, and its associated memory, and working with the memory in an efficient way

#

a "game runtime"

#

that is itself embedded into a normal C# runtime

jade glacier Oct 17, 2019, 10:04 PM

#

most of that fun stuff in Overwatch isn't so much about the networking, that is more about the ability to resim quickly

weak plinth Oct 17, 2019, 10:04 PM

#

FYI, ECS has nothing to do with networking explicitely

amber trench Oct 17, 2019, 10:04 PM

#

well, surely you'd agree that if you can ship pieces of data to identical runtimes, but one happens to be client and the other server

#

and sort of "work on" that data

#

it would make resimming, as a general idea, easier

#

for normals like us

jade glacier Oct 17, 2019, 10:05 PM

#

networking poops on all of that for the most part, since you want to NOT have all of those nice structures for the network. You bitpack, compress and mangle the data any way possible to fit it through the pipes.

weak plinth Oct 17, 2019, 10:05 PM

#

Yes, that's why you serializa and deserialize from a bitstream the same way on both ends

amber trench Oct 17, 2019, 10:05 PM

#

the web development community used to call this "isomorphic" but i think they had a different name for it