choosing and using a benchmark testing library

[serene laurel]

Coming from rust and the jvm and to a lesser degree golang, I’m used to there being a paved path for how one goes about writing benchmark tests. I couldn’t help but notice there are several benchmark testing libraries floating around the zig universe

Is there one that people tend to prefer and is there a standard way to separate benchmark tests from default tests that run?

with rust that looks like cargo bench with go that typically looks like go test -bench=. -run=^#

https://github.com/hendriknielaender/zBench, https://double-trouble.dev/post/zbench/ look fairly maintained

error: unable to determine name; fetched package has no build.zig.zon file```

though it doesn't look compatible with zig's package manager

[pallid tapir]

--save=name

[serene laurel]

derp. thanks!

test "bench example" {
    const zbench = @import("zbench");
    var bench = zbench.Benchmark.init(std.testing.allocator, .{
        .iterations = 1,
        .max_iterations = 10,
    });
    defer bench.deinit();
    try bench.add("Example", exampleBenchmark, .{});
    try bench.run(std.io.getStdOut().writer());
}

fn exampleBenchmark(_: std.mem.Allocator) void {
    // Code to benchmark here
    std.debug.print("test", .{});
}

adding this to zig build test just hangs (on zig 0.12.0). I've tried using the Benchmark default options and filling very low iteration numbers seen above

I've also tried the most basic example listed https://github.com/hendriknielaender/zBench/blob/main/examples/basic.zig

[dusky nova]

I just use poop

hyperfined in nature, but that means that each of your benchmarks are just individual programs

[serene laurel]

Performance Optimizer Observation Platform
Don't judge a library based on its name. But seriously this does look like it has a thoughtful UI.

I'm looking for something more on a micro benchmarking function level than executables

bookmarking that one though. thanks!

ah, I see this is more in the spirit of https://github.com/sharkdp/hyperfine.

[mystic mirage]

throwing out my tiny microbenchmark lib as a suggestion: https://github.com/silversquirl/benchmark.zig/
it needs updating to 0.12 which i've not gotten around to yet but i'd happily accept a PR :)

or if you like the look of it but don't wanna update it, ping me in a bit and i'll do it

shouldn't take too long, it's very small

[serene laurel]

here's a small pull for that https://github.com/silversquirl/benchmark.zig/pull/1

I noticed there weren't an git tags for versions, is that planned in the future?

build.zig.zon encourages, but doesn't enforce, some sense of semantic versioning typically with a git tag when it comes to github

I found another benchmark library but it's sandwiched between a number of other unrelated libraries as an uber util library https://github.com/karlseguin/zul?tab=readme-ov-file#zulbenchmarkrun I think I prefer the more focused libraries above

[serene laurel]

setting these up in a way that they can be running separately. zig's test options support's a notion of test filters but you can only filter in and not filter out tests. i.e. you can have a bench test filter that declares "bench" but you can't have a unit test filter that declares "!bench"