File reader returns EndOfStream and I don't quite understand why.

[whole lance]

I have this piece of testing code but it always returns EndOfStream error.
From what I understand and saw in the stdlib it can only return that if the read bytes are less than what the reader buffer can hold. But My file has over 440k characters so that shouldn't be the case here.

-# I know the while loop will overwrite the map_data variable if there are multiple files. This is just temporary for testing.

pub fn start() !void {
    const smp = std.heap.smp_allocator;

    const maps_path = "assets/maps";
    var maps_dir = try std.fs.cwd().openDir(maps_path, .{.iterate = true});
    defer maps_dir.close();

    var map_data: TiledMapData = undefined;

    var maps_iter = maps_dir.iterate();
    while (try maps_iter.next()) |map| {
        switch (map.kind) {
            .file => {
                var read_buf: [1024]u8 = undefined;
                const file = try maps_dir.openFile(map.name, .{});
                var file_reader = file.reader(&read_buf).interface;
                const json_str = try file_reader.readAlloc(
                    smp,
                    4_000_000
                );
                map_data = try parseData(smp, json_str);
                // TODO Change the function so it can return multiple maps!
            },
            else => unreachable,
        }
    }

    std.debug.print("{any}", .{map_data.tile_layers});
    std.debug.print("{any}", .{map_data.collision_layer});
    std.debug.print("{any}", .{map_data.marker_layer});
}

[humble pewter]

Seeing this error everywhere: don't copy the interface.

var file_reader = file.reader(&read_buf).interface;

needs to take a reference:

var file_reader = &file.reader(&read_buf).interface;

[whole lance]

but &file.reader(&read_buf).interface returns a *const Reader, so it can't change and therefore doesn't want to work when I call readAlloc.

I just assume that it is not intended to do a constCast

[tardy dagger]

it has to be two lines:

var file_reader = file.reader(&read_buf);
const reader = &file_reader.interface;

[whole lance]

that makes sense

[tardy dagger]

or you can just do file_reader.interface wherever you use it

[whole lance]

I still get the EndOfStream error, so it seems like I now only have a reference but that doesn't resolve the actual issue

[humble pewter]

One issue at a time. (thanks for fixing my bad fix).

[tardy dagger]

i think its because you’re requesting 4 million bytes and only getting x bytes. i also think theres a different function on the reader that treats EndOfStream as a success condition. i should really pull up my pc tho lol

docs say you get endofstream if it cant be filled completely, yea: https://ziglang.org/documentation/master/std/#std.Io.Reader.readSliceAll (readAlloc wraps readSliceAll)

you might want streamRemaining into a std.Io.Writer.Allocating

and then use writer.buffered() to get your json_str

[whole lance]

so the buffered writer replaces the read_buf array, is that correct?

[tardy dagger]

no you should still have a read_buf array. iiuc youll be copying byte by byte otherwise

[whole lance]

Ok, I haven't worked with readers and writers that even before the 0.15 changes.

So I need a buffered writer that uses the backing writer from the Allocating? This writer will then write the bytes from the file to the buffer for me.

And the old read_buf array is still used by the file reader?

[tardy dagger]

you should just need a buffered file reader and an allocating writer. and yea the array is still used by the file reader

ie just replace the line where you assign json_str with:

var json_str: std.Io.Writer.Allocator = .init(smp);
try file_reader.interface.streamRemaining(&json_str.writer);

[whole lance]

works like a charm

Thank you so much. I would never have figured that out on my own.