Questions about `std.net.stream.read`

[unborn cape]

Is it ok to pass in a huge buffer and read everything at once?

Is there a way to read the content in chunks - for example read the start line, then all the headers (line by line) and then the body?

References
https://developer.mozilla.org/en-US/docs/Web/HTTP/Messages
https://www.rfc-editor.org/rfc/rfc9112.html

[mental girder]

You should be able to use the standard "read until delimiter" functions from std.io.Reader

But that's pretty inefficient, so wrap it in a BufferedReader first

Passing in a large buffer should be fine, don't see why it wouldn't be

[unborn cape]

Back of the envelop guess how inefficient would the “read until delimiter” be? I ask because I want to set the max size start line I’m willing to accept as well as max amount of headers and max body size.
If I make a huge buffer I won’t know which section was too big.

[mental girder]

With a BufferedReader it's not inefficient

Without, it's extremely inefficient. One syscall per byte

[unborn cape]

If it’s not too much trouble could you share a simple example of how to use BufferedReader with std.net.stream.read?

[placid phoenix]

I forget how you get a reader from a connection, but the general idea is:

var buffered = std.io.bufferedReader(connection.reader());
const buffered_reader = buffered.reader();
var into: [1024]u8 = undefined;
buffered_reader.read(&into); // this is a buffered read

(buffered_reader has all the normal reader funcs too, of course.

[tribal root]

The idea of readers is that you generally shouldn't call std.net.stream.read directly: it's a primitive function which says "read as much as you immediately can into this buffer", but that's not very useful for most scenarios. Instead, you use std.io.Reader on top of it, an instance of which is constructed by my_stream.reader(). As squirl says, you can use the read-until-delimiter methods on Reader to help you here, but you'll want to use a BufferedReader for efficiency; here's an example:

// my_stream: std.net.Stream

var br = std.io.bufferedReader(my_stream.reader());
const r = br.reader(); // get the std.io.Reader from the buffered reader - this is backed by an in-memory buffer to prevent constant syscalls

// let's read the start line!
var line_buf: [1024]u8 = undefined; // reasonable max size
const start_line = try r.readUntilDelimiter(&line_buf, '\n');
// you'll have to check for the CR yourself
// then do stuff with start_line

[unborn cape]

Wow this is awesome! Thank you guys so much!

So let’s say I process the start line and then want to move onto the first header - can I just use r.readUntilDelimiter again - essentially keep using “r” until I hit the end of the request?

[tribal root]

Yup!

The buffered reader br contains a 4k buffer into which it reads blocks of data from the stream when need be, and also a current index into that buffer so it knows where we've reached in the input. r basically encapsulates a pointer to that buffered reader with a bunch of convenience methods for getting the data out