#Alternatives to tagged union?

1 messages · Page 1 of 1 (latest)

jagged bough
#

Hello.
I'm porting a pet-project to Zig, I originally prototyped it in ES6 a few years back.

But i'm a zig-noob so I struggle with the main interface design.

The problem:
I made the feed as an in-memory chain of blocks, so the entire construct was just a single readable/writable buffer.

This used to be my ES6-workflow:

const { sk: aliceKey } = Feed.signPair()
const { sk: bobKey } = Feed.signPair()
const a = new Feed()
a.append('Hello', aliceKey);
a.append('World', aliceKey);
a.length // => 2
const b = new Feed()
b.merge(a) // length is now 2
b.append('Hey Alice! /B', bobKey);
b.length // => 3
a.merge(b)
a.length // => 3

It's ok but wastes a lot of resources behind the scenes.

In Zig I implemented the Feed decoder as a simple Iterator, it was real nice.
But It also made me start thinking about "readonly" feeds since zig has real a pretty powerful const.

So now I'm stuck with a struct that tries to squeeze 2 concepts into a single field.. footgun🦶

pub const Variant = enum { readable, writable };
pub const Feed = struct {
    buffer: union(Variant) {
        /// Read-only feed
        readable: []u8,

        /// Backed by arraylist with growing capacity.
        writable: ArrayList(u8),
    },
}

Are there any better patterns i could use?
Help 🙏

jagged bough
#

Ugh. is it possible to normalize the slice and arraylist access as it's frustrating to self.buffer[n..m] and self.buffer.items[n..m] in common functions?

Or maybe there's some similar case in std-lib i can use as reference?

jagged bough
#

Alternatives to tagged union?

raven scarab
#

Firstly, a small note: your readable one should probably be a []const u8, to make sure you never accidentally mutate it and to allow it to be used with constant data.
Regarding cleaner structures: I don't know your exact use case, so there's a chance this is a bad idea, but with what you've said it sounds like it might be an idea to make your type templated at compile time on the const-ness, like this:

pub fn Feed(comptime mutability: enum { read, read_write }) type {
    const rw = mutability == .read_write;
    return struct {
        buffer: if (rw) std.ArrayList(u8) else []const u8,

        const Self = @This();

        // Regarding normalizing access, we can write a simple accessor method to get the underlying slice here.
        // If you never need to mutate existing elements you could fix this return type at []const u8
        pub inline fn items(feed: Self) if (rw) []u8 else []const u8 {
            return if (rw) feed.buffer.items else feed.buffer;
        }
    };
}

You can put whatever methods you want on this struct of course, and if you want an external function to accept both types, you can make the parameter anytype and use it accordingly.

#

This solution gives you better type safety (i.e. a function that only works on writable feeds will only accept writable feeds), although it will lead to functions being duplicated in the final binary, so if a very tiny binary is a major concern for some reason this might not be a good approach

#

The items accessor method also works fine with the union structure if you prefer to keep it as a runtime thing:

pub const Feed = struct {
    buffer: union(enum) { // union(enum) automatically creates the underlying enum for a tagged union, it's probably nicer here
        readable: []const u8,
        writable: std.ArrayList(u8),
    },

    pub inline fn items(feed: Feed) []const u8 {
        return switch (feed.buffer) {
            .readable => |buf| buf,
            .writable => |al| al.items,
        };
    }
};
elfin cove
#

I'd probably need to know more about the reason you need to merge there, and what you're really going for.
You could make it just encrypt as you append, all into a writer, for example, if that's what you're doing; it's not particularly obvious to me what that ES6 example code is really trying to accomplish, unfortunately.

jagged bough
#

@raven scarab wow thank you. I managed to try the tagged union approach yesterday before quits, using almost same inline getter/fn-technique you showed, binary duplication isn't a concern at this stage but it's itching. Also union(enum) 🔥

Anyway at the end I was just mindlessly updating function signatures; adding/removing const until i ended up in a deadlock where neither state compiled. I need to take another look..

I'm really struggling with the const keyword, in my world a function that takes a const variable basically promises that it will not attempt to alter the parameter, but that give me a lot of expected '[]const u8', found []u8 issues.. Stupid question, can i somehow cast []u8to []const u8?

Thank you for type templated struct pattern, that's a keeper, think i saw it in the ArrayList source, but it was a bit intimidating.
Honestly, I'm still trying to figure out which way to go, i'd love to expose a simple api and rely on meta-programming as little as possible. Hang on.

raven scarab
#

[]u8 should just coerce to []const u8, that error's pretty weird. Can I see some relevant code?

#

It's possible you're misremembering the issue and that it was the other way around, in which case: that conversion isn't really possible, Zig const is intentionally hard to get rid of (as opposed to C where it's basically a friendly suggestion lmao)

jagged bough
#

@elfin cove sorry for confusion, i had a link that got edited away in frustration.. I'm porting https://github.com/telamon/picofeed ==> https://github.com/decentlabs-north/pico-zig

Anyway continuing - my usecase dosen't really require a readble/writable separation
I think what i would like is something like this:

// Feed construct is like a "window" into a buffer
// to simplify read/write.
const remote: []u8 = ...; // from network/p2p/db/somewhere
var local: ArrayList(u8) = ...;

const ro_feed = Feed.from(remote);
var rw_feed = Feed.from(local) || Feed.create(allocator, initial_cap);

// Readable API (common ro+rw)
ro_feed.length // returns amount of blocks
ro_feed.blocks // list or iter of blocks
ro_feed.keys  // list or iter of public-keys
ro_feed.block(idx) // returns const Block
ro_feed.key(idx) // returns []const u8
ro_feed.diff(other: Feed) // returns (n_blocks: s16) or error.NotMergable
ro_feed.inspect() // Formatter? print ascii-table

// Writable API (should comptime error if used on ro_feed)

/// Attempt to copy/import blocks if mergable
try rw_feed.merge(ro_feed);
/// Create a new block
try rw_feed.append(data: []const u8, secret_key: [64]const u8);

Sorry for pesudo. that's mostly the full picture of my ambition... 🥲

jagged bough
raven scarab
#

Maybe it was something like [100]u8 to []const u8? An array is a fixed-size block of memory whereas a slice is a pointer, so you need to add an indirection for that with & - *const [100]u8 coerces to []const u8 (with ptr as that pointer and len as 100)

jagged bough
#

Found one case that i don't understand:

pub const BlockHeader = extern struct {
    signature: [U.size.sig]u8,
    parent_signature: [U.size.sig]u8,
    size: [U.size.body_counter]u8,

    pub inline fn readSize(self: *const BlockHeader) u32 {
        return std.mem.readIntBig(u32, self.size);
    }
};

src/block.zig:10:44: error: expected type '*const [4]u8', found '[4]u8'
        return std.mem.readIntBig(u32, self.size);
                                       ~~~~^~~~~

I think i declared self as a ptr to a constant instance of my struct, but the field size seems to have lost it's constantness along the way? O_o
OOPS Solved it.. &self.size

jagged bough
raven scarab
jagged bough
#

I see 🙂

jagged bough
#

o.m.g... the way to define a const field was lost to me up until now...
struct { ref_to_chunk: []const u8 } , i've been defining all fields as []u8 or [32]u8 all over the place confusing const inside struct was only static-variable declarations🤦‍♂️ 🥹

raven scarab
#

That's not haking the field const, it's making the pointer point to const data

#

This can be a bit tricky to get, but array types ([32]u8) and slice types ([]u8) are fundamentally different. The first is basically the same as a big struct filled with 32 u8s, whereas the second is a pointer and a length

#

Like any other pointer, you can specify that the data pointed to is const, i.e. can't be mutated, which you do with []const u8

#

But [32]const u8 wouldn't make any sense - like a struct, individual values (akin to fields) are const if the whole value is const, you can't get any more granularity

jagged bough
#

Thanks i think i understand, it's just my eyes weren't accustomed to the syntax. I think i know where to look now, cleaning duty

jagged bough
#

@raven scarab i chose to go with the union(enum) approach, do you know if i can somehow throw an comptime error to warn the user?

struct {
  buffer: union(enum) { ro: []const u8, rw: ArrayList(u8) },

  pub fn appendKey(self: *Self, key: [32]u8) {
    comptime if(!self.buffer.rw) // fail message ReadOnlyFeed
    // ...
  }
raven scarab
#

Zig doesn't really have the concept of a "partially comptime known" value (with a few exceptions), so that's unfortunately not possible - the ability to do that is the main advantage of the templating approach

jagged bough
#

oh i see.ah yeah cause then the type guards like a type-checker.. hmm

jagged bough
#

Uh is there any way i can save this thread so it i can refer to it later? My first support-thread seems to have been garbage-collected :/

elfin cove
solid knoll
#

make sure to react to your post with a ✅ to mark it as being solved