Improving my custom writer in 0.15.1

[wild current]

I'm working on a text editor. The base buffer implementation isn't very relevant here, but suffice it to say that it can supply ordered slices that together comprise any byte range in the logical document. It exposes this generic method for writing any sequence of bytes into some generic writer:

    pub fn materializeRange(self: *TextBuffer, w: anytype, start: usize, len: usize) @TypeOf(w).Error!void {
        // when a view should be written directly to a writer
        if (len == 0) return;
        var it = self.getSliceIter(start, len);
        while (it.next()) |slice| try w.writeAll(slice);
    }

Currently, my interface is implemented with sokol-zig and the actual text is rendered line-by-line using that library's debugtext binding. I need to implement a custom writer which

A) provides a writeAll (or something similar, I don't care what it's called since I control the callsite) method taking bytes
B) allows those writes to be accumulated into an internal buffer and flushed when the full line is written

So far after some iterating I've settled on this cludgy implementation, which works but has some hacks (like storing the pos by pointer, since the writer is passed as an immutable argument). I feel like this completely sidesteps the nice new rewrite, so I was wondering how I make this less horrible? Thanks!

(post too long so I'll put the implementation in a message beneath this)

const SdtxWriter = struct {
    // this will be passed as the writer into the document and text buffer.
    // Since function arguments are immutable, internal state must be mutated by pointer
    buffer: []u8,
    pos: *usize,

    pub const Error = error{};

    pub fn writeAll(self: *const SdtxWriter, bytes: []const u8) !void {
        // write into a private buffer, accumulate any number of writes as long as they fit in one line
        if (bytes.len == 0) return;
        debug.dassert(self.pos.* + bytes.len < self.buffer.len, "attempt to write past the end of line buffer");
        @memcpy(self.buffer[self.pos.*..self.pos.* + bytes.len], bytes);
        self.pos.* += bytes.len;
    }

    pub fn flush(self: *SdtxWriter) void {
        // null terminate the string and write it using sokol's standard debug
        if (self.buffer.len != 0) self.buffer[self.pos.*] = 0;
        const s: [:0]const u8 = self.buffer[0..self.pos.* :0];
        sdtx.putr(s, @as(i32, @intCast(self.pos.*)));
        self.pos.* = 0;
    }
};

[hybrid smelt]

the point of writergate is that you're not supposed to store buffers yourself

but rather it's part of the overall Io.Writer struct

[wild current]

Yeah I've watched the talk and read the release notes, my hangup is I'm just not sure how to leverage the new interface so I don't have to do this all myself

[eager stream]

I've been looking at the Allocating struct to help you, it seems like a good place to start

[hybrid smelt]

yeah well first you need to make your writer hold a Io.Writer object

[eager stream]

basically you need to create a Writer interface, only the drain function is necessary

[hybrid smelt]

which holds the buffer and tracks the seek head for you

[hybrid smelt]

yeah that

drain is a bit annoying to implement because it's designed with vectorized i/o in mind

[eager stream]

I'm still trying to understand what drain actually does, I'll come back when I understand its implementation in Allocating

[hybrid smelt]

but you can study Writer.Allocating to figure out how it works - reading the docs about its invariants is a good idea too

[hybrid smelt]

drain is called whenever the buffer is full or when the caller manually flushes

[eager stream]

that's not very helpful since it has 2 confusing parameters, what is data and what is splat?

[hybrid smelt]

and the input data is split into various chunks in one slice - first slice points to the buffer (always) and then more data potentially follows

the last slice should be processed splat times

i do realize this sounds like gibberish because vectorized i/o is really... alien to how we usually think about read/write operations

[random monolith]

buffer[0..end] is consumed first, followed by each slice of data in order.
...
The last element of data is repeated as necessary so that it is written splat number of times, which may be zero.
the documentation explains everything

[hybrid smelt]

reading more about posix iovecs might help

oh hey, wikipedia might help too https://en.wikipedia.org/wiki/Vectored_I/O

[tribal plume]

vector io >:( i hate

oh not the same as i thought

no wait it is what i thought, me not like

[wild current]

huh, thanks for the responses but I'm not sure I'm really understanding.. Ideally if I could find an example of what normal usage looks like I can figure it out. So I still need a custom writer struct but it's holding an Io.Writer? Not sure what to do with an Allocating object :/

[hybrid smelt]

it is slightly insane that writergate makes you think in terms of vectored io by default whereas e.g. rust uses the "simple" write interface by default and offers opt-in vectorized io

but its in the name of speed™

[hybrid smelt]

you can take a look at how Allocating is implemented

[wild current]

I love zig but the io has made me feel quite smooth-brained

[hybrid smelt]

to get a feeling of how a post-writergate writer impl looks like

[hybrid smelt]

i feel you, it took me a hot minute to realize wtf is going on as well

[wild current]

yeah I guess when in doubt look at the source. Probably doesn't help that I haven't worked with interfaces in zig yet

[eager stream]

const SdtxWriter = struct {
    pos: *usize,
    writer: std.Io.Writer,

    const vtable: std.Io.Writer.VTable = .{ .drain = SdtxWriter.drain };

    pub fn init(buffer: []u8, pos: *usize) SdtxWriter {
        return .{
            .pos = pos,
            .writer = .{
                .vtable = &vtable,
                .buffer = buffer,
            },
        };
    }

    pub fn drain(w: *std.Io.Writer, data: []const []const u8, splat: usize) std.Io.Writer.Error!usize {
        _ = splat;

        var self: *SdtxWriter = @fieldParentPtr("writer", w);

        @memcpy(w.buffer[self.pos.* .. self.pos.* + data[0].len], data[0]);
        self.pos.* += data[0].len;
        // null terminate the string and write it using sokol's standard debug
        if (w.buffer.len != 0) w.buffer[self.pos.*] = 0;
        const s: [:0]const u8 = w.buffer[0..self.pos.* :0];
        sdtx.putr(s, @as(i32, @intCast(self.pos.*)));
        self.pos.* = 0;
    }
};

i'm not sure how correct this is

but it may serve you as a starting point

[wild current]

thank you, I appreciate it. Still trying to wrap my head around why drain wants a 2d array in the first place

[eager stream]

just to get started, assume data is 1d and splat is 0