#Faster hash

1 messages · Page 1 of 1 (latest)

ornate pike
#

Hello,

I am using the following code to create a protocol version number.

fn derive_version() u32 {
    @setEvalBranchQuota(50000);
    const crc = std.hash.Crc32.hash;
    return crc(@embedFile("./msg.zig")) ^ crc(@embedFile("./client.zig"));
}

pub const VERSION = derive_version();

It works, but it is very slow to compile, and requires a huge "eval branch quota".

Do you have an idea for something faster?

The value does not really matter, the is just to detect version mismatch during development, in production the value will be hardcoded.

chilly pewter
#

try std.hash.Wyhash?

ornate pike
#

it's a bit faster but still quite slow

#

about 7 seconds

#

both files are about 30kb

obsidian pebble
#

Assuming there isn't a better solution that doesn't involve any hashing of files, you might want to move that to a build step so that you can get caching on it at least

honest nymph
#

I haven't looked into it, but maybe you can use some specific hash generated within the zig build cache itself in some step, extract that hash in a later step and put it in? Just a vague idea, but I would guess that something like that might work.

ornate pike
#

Well, I have a struct in the msg.zig file which includes type from the client.zig file, and I want to change the version if this struct changes

#

only idea I had was to hash the files, I could inspect the struct and iterate all the fields, but that's quite some work

#

I'll go the build step way

livid sequoia
boreal sparrow
#

@buoyant prairie I’ll let you do the magic

buoyant prairie
#

lol

boreal sparrow
#

Oh wait at Comptime oh goodness (maybe your work would still help here?)

buoyant prairie
#

huh?

#

xxh3 is 100% comptime

#

it works anywhere

boreal sparrow
#

No I mean

#

Comptime perf is really bad in general

buoyant prairie
#

oh lol

#

it is yeah

#

this is a job for Wyhash just saying, xxh3 is optimized in iterative hashing

#

if you were downloading something, and you wanted to hash it while downloading, xxh3 is what youd use

boreal sparrow
#

Just a Comptime issue then :(

buoyant prairie
#

cant solve comptime perf though :C

boreal sparrow
stiff jackal
#

how far do you think we got

ornate pike
#

I did a workaround with a shell script that I run when I build which write a .zigfile with const version = <someint>

boreal sparrow
#

Noooo

#

Just use an option step!

ornate pike
#

because build step wasn't faster

boreal sparrow
#

Wait really oh wow

ornate pike
#

it doesn't change anything

boreal sparrow
#

So std hashing is just slow?

#

That’s weird

#

How big are these files? And how are you hashing then with this new code

#

That’s super odd

ornate pike
#

30k each

#

aaah, but I was doing comptime hashing in my build script

boreal sparrow
#

That’s nothing how could it be this slow 🤔

ornate pike
#

it just hit me

boreal sparrow
#

Oh lol

ornate pike
#

I copied the code at the top of my build file

#

you just made me realize that