Send 23MB byte array Client -> Server & Server -> ClientS

[thorny abyss]

Hello community,

I am developing a distributed Hololens based network application, where clients are locally connected to the same netwrok and share the same physical environment. I am using Mirror for handling the entire sync of virtual simulation. This far no issues at all, but I have one problem to solve and would like to know which is the best approach.

Just once per session, I need all participants to share a relatively large byte[] (approximately 20/30MB) containing information for aligning the various augmented objects. This step is quite crucial and the data contained in this byte array changes at each session, and must be distributed among all memebers.

My first approach has been to send a custom NetworkMessage containing a byte[] as property through NetworkServer.Send or NetworkClient.Send. However, I have been issued the warning:

NetworkConnection.ValidatePacketSize: cannot send packet larger than 298449 bytes, was 23054208 bytes

So my question is, which is the best approach to send such a large amount of data?

I had the system working with a custom c# socket implementation, but I would prefer to keep it straightforward and use the MirrorAPI and one of its transports. Are there any methods I can use which automatically fragment my byte[] into packets and have them rebuild on the other side? Should I implement such a beheviour myself? Any suggestion, or link to material is greatly appreciated. Many Thanks!!

[brittle kite]

If they're on the LAN, put it in a shared folder as a file and let them all load it directly. Game networking is designed for tiny fast msgs, not bulk file transfer.

[thorny abyss]

I see what you mean and makes sense, but in some configuration, there is not a dedicated server (a PC running as server) and only hololens devices connected (one running as host) and therefore I would have no place where to store such file.

[brittle kite]

Doesn't have to be a game server - you're on a LAN - any machine on the LAN can have a folder share

That said, I wouldn't count on the hololens being a great server

[thorny abyss]

Plus the file is generated runtime by one hololens at each application start and should be distributed to all connected devices. This operation again must be done each session.
Following your suggestion the configuration would be:

1. One Hololens creates the file and sends it to the PC server, which then stores it somewhere inside it
2. Once this operation is completed the server notifies all other clients of the completed operation and tells them the location of the files
3. When clients receive the message they download the file from the communicated location

[brittle kite]

How is it generating all this - if that's a seed-based procedural, then just pass the seed around and let them all gen the same result.

[thorny abyss]

No its a byte[] containing geometrical informations of the room and how the virtual objects are anchored within it. One device must create such and the others must receive it full to reconstruct the scene and have a shared experience

any suggestions on how to read from a shared folder (via c#) on a PC in the same lan?

[brittle kite]

ok then you're going to need to upload that via HTTP or store it via file share

System.File.IO is in Microsoft's docs and that's going to be the easiest and fastest delivery on LAN

[thorny abyss]

to write it in the file system that is not a problem, but still :

1. How do I send it to the PC server from the Hololens? Sockest?
2. How do the other Hololens (clients) retrieve the file from the Server PC file system?

[brittle kite]

I'm saying use filesystem write to save it to the shared folder directly

and same for them to read it

\\SomeLanPC\d$\Path\To\Shared\Folder\RoomData.txt
where the machine named SomeLanPC has done a public share of it's D drive

[thorny abyss]

I am checking the docs, it didn't thought it was that easy, sorry

And it looks I can create the shared folder also runtime on the machine. So the approach would be:

1. the server creates the shared folder and stores a reference to it.
2. when needed it communicates to clients the filename to download and where to get it from

[thorny abyss]

I do not like very much the idea of putting the entire D folder in share, it is possible to put as shared, simply a specific folder inside a given directory in D?

[brittle kite]

was just an example - there's various ways to share folders

[thorny abyss]

anyway, thank you @brittle kite for the opinion, I really didn't think about it but it is just easy and straightforward. I like it.

[brittle kite]

you can drill to a folder and right-click Share

google LAN folder sharing - several options available

mapped drives, etc.

[thorny abyss]

I actually checked and I can even do it runtime adding only specific clients to any given folder, thus restricting the access to the folder only for those connected at the session

But just wondering, if shared folder was not an option, which other approaches would you have suggested:

• direct transfer via sockets?
• remote server storing?
• others....

[brittle kite]

HTTP Post or FTP upload to a web service / CDN.

[thorny abyss]

@brittle kite can I actually ask you here a very short question regarding additive scene management, or shoud I write it in another post?

[brittle kite]

just ask in #🤝help-archive

[thorny abyss]

Just for curiosty, it seems shared folders are not available on UWP, thus it is impossible to communicate between the hololens and the PC server that way

[brittle kite]

ooof

[thorny abyss]

I was looking into and one approach would be via FTP, I need a solution without external servers...

pushing and getting the data via ftp from clients does not seem a big problem, and there are classes in the .NET api or I have found the FluentFTP. Now the problem is, how to run a simple FTP server which I can manage at runtime without the need to install external applications as FileZilla

[thorny abyss]

maybe I am not entirely sure about this, need to test via code. Will do it on monday. Let you know how it goes.

thanks again for the help!!

[brittle kite]

Does UWP also block local mapped drives?

[thorny abyss]

not sure

[brittle kite]

nvm - you probably can't set up mapped drives on hololens

or maybe you can - I don't know what that thing has for a file system.