I come from a python background with dynamic typing and garbage collection... does anyone have some suggestions on libraries to look at that implement de-serailization of byte streams into packed structs / slices?
#Resources for De-serialization Patterns?
jade grotto
for example, I am trying to serialize and deserialize frames of the following structure:
pub const DatagramHeader = packed struct {
command: Command,
idx: u8,
address: Address,
length: u11,
reserved: u3 = 0,
circulating: bool,
next: bool,
irq: u16,
};
pub const Datagram = struct {
header: DatagramHeader,
data: []u8,
wkc: u16,
};
pub const EtherCATHeader = packed struct {
length: u11,
reserved: u1 = 0,
type: u4 = 0x1,
};
pub const EtherCATFrame = struct {
header: EtherCATHeader,
datagrams: []Datagram,
};
pub const EthernetHeader = packed struct {
dest_mac: u48,
src_mac: u48,
ether_type: EtherType,
};
pub const EthernetFrame = struct {
header: EthernetHeader,
ethercat_frame: EtherCATFrame,
padding: []u8,
};
I have figured out that a std.io.FixedBufferStream is probably the easiest API for serialization since I can do something like:
try writer.writeStructEndian(
frame.header,
std.builtin.Endian.big,
);
etc.
but for deserialization I'm hitting a bit of a learning curve when it comes to allocations etc....
rough quest
a simple backing type would be std.ArrayList(u8).writer(). that way you don't have to calculate the buffer size in advance.
but if you did need to calc the size, you might first deserialize into a std.io.countingWriter(std.io.null_writer) to get the size. then allocate the buffer and actually deserialize into a io.fixedBufferStrrem(buffer)
this pattern is common in std.fmt. pretty sure that how its count() method works.
for packed structs, it may be easier to cast them to integers and use io.Writer.writeInt()
and io.Reader.readInt()
safe charm
have a look at https://github.com/getty-zig/json
and more general version https://github.com/getty-zig/getty
warm crest
@jade grotto I've done a lot of similar work in this library and got it to the point where I could craft, send, and received arbitrary Ethernet, IP, and TCP/UDP data across the wire (tested with Wireshark).
That said, I plan to rewrite large parts of that library using a Data Oriented Design style.
warm crest
for packed structs, it may be easier to cast them to integers and use io.Writer....
In my experience, this can be a little difficult with network headers since they don't all fall on the typical integer backed boundaries. Meaning you have to remove the padding manually when writing a string of headers (or their fields) together.
I could have just been doing it wrong as well though.