#Reader performance

1 messages · Page 1 of 1 (latest)

void siren
#

I have ported a decompression algorithm to Zig, and I want to maximise performance. It reads directly from an in-memory buffer and writes directly to another in-memory buffer, which can obviously potentially take a lot of memory and sucks compared to dealing with chunks at a time that can be stored into/read from files, but I also want the option to not have chunked reads at all. I also don't want to have a bunch of duplicated code with slight changes.

This sounds like a job for some reader/write interface with different possible underlying implementations - like std.io.GenericReader. Suppose I make my algorithm more generic so that it can take such a reader, and I make a reader implementation that reads from the in-memory buffer, wouldn't that run like ass due to added overhead? I suspect the compiler wouldn't be smart enough to reduce that down to just buffer reads like I have now. It kind of sounds like I would have to fight the compiler quite a bit to make this dream a reality. I'm very interested in best practices and I know none of them, so if you can point towards libraries you think have solved this problem (or similar), show me!

untold pawn
#

try see what godbolt generates when using something like a fixed buffer stream with ReleaseFast

#

with llvm I'd think it would optimise the generic reader away but I'm not sure

queen birch
#

I would've guessed that the compiler could optimise that fairly well but I don't really know anything about llvm or zig optimisation so yea basically useless input from me

#

I've never found the reader to be my bottleneck though