#open file binary

1 messages · Page 1 of 1 (latest)

sweet widget
#

I'm having issue loading a shader .spv file for vulkan. I get a []u8 using std.fs.cwd().openFile(vertSpvPath, .{ .mode = .read_only }); and fileVertSpv.readToEndAlloc(mem.MemMan.allocator, 8 * 1024 * 1024) that I read to get a [*]u32 using the right endianness (the file starts with a magic number that I'm reading correctly). In my previous version that was using c++ I used std::ifstream.open(file, ios::ate | ios::binary). Is there such a binary flag in zig or maybe a openBinary fn ?

I think my alignment is right since I fill a ArrayList(u32) and the endianess too. I don't know what to look for anymore.

thanks

warm crow
#

ios::binary avoids trashing line endings and such because the default is text mode, there is no such concept in zig. it's always binary mode

#

you could read into a buffer that's align(4) and then safely cast the pointer

#

since you're using readToEndAlloc, you could just switch to readToEndAllocOptions which takes an alignment argument

sweet widget
#

Thanks for clearing that up. I'll try readToEndAllocOptions

past dew
#

you shouldn't need to worry about alignment fyi since spv file size is always in multiples of 4
i would just @alignCast

warm crow
#

... you do need to worry, because increasing alignment is undefined behaviour

#

you can't pretend a *[4]u8 is a *u32 without consequences

#

now if it's *align(4) [4]u8 then yes that's okay

#

but you never increase alignment

#

unless the pointer happens to be aligned

past dew
#

is it not guaranteed to be aligned if the size is a multiple of 4?

warm crow
#

no

#

that would mean a [128]u8 is aligned to 128 or something no? but it's not, the alignment required for reading a u8 is 1

#

size and alignment are independent things

past dew
#

alright then
my own code doesn't account for that so i guess i should go fix that thanks lol

#

fwiw it hasn't ever failed for me though

warm crow
#

you're likely using an x86 processor yes?

#

with most instructions generated an unaligned read will just silently work, but subtly slower

#

it's a really ugly bug to run into later down the line

past dew
#

oh, didn't realise x86 allowed unaligned reads

#

yeah that's annoying

warm crow
#

it generally does yeah

#

not with the simd instructions though

#

so you'll one day pass some data to an optimised routine

#

and it'll just explode

warm crow
#

say you have an extern struct { x: u32, y: u8 }

#

yeah that's five bytes but in an array it'll take up eight because it'll need to pad out the memory so the next struct's first member aligns to a four byte boundary

#

so the alignment of this is 4, size is 5, but with padding it'll be 8

#

good to keep in mind ^_^

#

for accessing any primitive T it must be on a boundary denoted by the alignment

#

what's nice is that if you just *align(1) T then Zig will handle the unaligned read for you

lethal pagoda
#

e.g. a pointer with a value of 8 and 16 and so on have at least an alignment of 8, and thus would be eligible for any instructions requiring such alignments

warm crow
#

yeah align(N) is literally "the address value is a multiple of N"

lethal pagoda
#

(as opposed to a pointer with a value of 7 or 9 or whatever, which isn't aligned to anything but 1, so is only eligible for byte-level instructions)

past dew
#

wait, how do I read with a specific alignment when im not using readAllocate?

warm crow
#

read into an aligned array

past dew
#

ah okay

lethal pagoda
#

e.g.

var aligned_list = std.ArrayListAligned(u8, 4).init(allocator);
defer aligned_list.deinit();
try reader.streamUntilDelimiter(aligned_list.writer(), delim);
#

or using std.fifo

sweet widget
#

thanks

When using allocator.alloc( u32, N ) I expected the returned address to be aligned 4 but the slice is aligned to 8... Is it safe to alignCast to 4 ? I never had to deal with this yet

past dew
#

this works right?
var buffer align(@alignOf(*u32)) = try allocator.alloc(u8, size);

warm crow
#

nope

lethal pagoda
#

not quite

#

that would align the variable buffer itself

warm crow
#

the allocation is aligned to u8, still

sweet widget
#

I'm building it wrong then. that's my issue

lethal pagoda
#

the issue is only as it pertains to allocating raw bytes and assuming they're already aligned

past dew
#

should I use alignedAlloc?

lethal pagoda
#

you can fix that issue by using alignedAlloc

#

yeah

warm crow
#

yep

sweet widget
#

ho ok

lethal pagoda
#

if you're allocating bytes which should be at an aligned address

warm crow
lethal pagoda
# sweet widget ho ok

in the case of the file reading though, instead I'd recommend just using the ArrayListAligned

warm crow
#

or since you were already using readToEndAlloc you could use the Options variant with the alignment argument

lethal pagoda
#

is that a new thing?

#

that's pretty neat

warm crow
#

yes it can even add a sentinel

#

very useful function 🙂

lethal pagoda
#

very based