Diffference between "test runner" and "custom test runner"??

[lost trail]

Seeing these terms quite a bit. But, cannot discern the difference.

I assume a regular "test runner" is the part of the code that is in the brackets after the key-code "test". Correct??

Next, what differentiates a "custom test runner" from a "regular test runner"??

I am trying to figure out what the meaning of the latest pull on issue 14588, and the related 12201, 13970.

Specifically, the needed directory structures (and limitations) for where external functions called by the test blocks can reside.

[eternal surge]

Custom test runners let you plug in your own code to run tests with zig test

The default test runner can be found in lib/test_runner.zig

[lost trail]

but, don't all tests that I write do exactly that??

[eternal surge]

It's the root module when running zig test

[eternal surge]

Something has to run those tests

That something is the test runner

The test runner is what prints things like "All 10 tests passed." etc

[lost trail]

I am confused... if I write a function "foo" in the test "foo.zig", and in that file I have a test block. Is that a "custom runner"??

[eternal surge]

No, that is a test

[lost trail]

ok, so if I run "zig test foo.zig" .... that becomes a "custom test runner" then?

[eternal surge]

The test runner is that root module when you run zig test foo.zig. It's given a list of test functions by the compiler, and it loops through those, runs them, and collects the results

[lost trail]

(I believe you see my confusion from my stupid questions here.... My apologies, really, I feel quite dense !)

[eternal surge]

No, that's still using the default test runner. The compiler will look at all the test blocks in foo.zig and expose them to the test runner (in this case zig/lib/test_runner.zig) which can then run those tests

It's not something you'll typically have to worry about. It's an internal part of how zig test works

But it's exposed so you can plug in your own custom test runner that eg. outputs a report in a custom format, or runs tests in parallel, or takes performance benchmarks, etc etc

[lost trail]

hmmm.... interesting... this is some aspect I have not thought off... I appreciate the comment. So, when the test is using the zig/lib/test_runner.zig that is what is meant by a regular test runner. Okay, good, I have learned an important point here.

Is there a good example of a custom test runner and how one calls this instead??

[eternal surge]

I don't know any off the top of my head, it's a pretty niche feature

[lost trail]

I ask because I have been writing configurable tests (and benchmarking) using the "regular" runner (and having compilcated code blocks in the test block).

[eternal surge]

You can specify the custom runner by using the --test-runner flag

[lost trail]

It appears that "custom" test runners might be a better way...

[eternal surge]

The best example of how to write one is probably the regular test runner itself

[lost trail]

Okay, the --test-runner flag is something I will need to read about.

[eternal surge]

You can look at lib/test_runner.zig within your zig installation

[eternal surge]

Not much to read tbh

  --test-runner [path]           Specify a custom test runner

[lost trail]

YES!!! Good point, the lib/test_runner.zig is where I should be reading. Looking at the source code is the best learning source quite often.

(actually, it is kind of quite fun to read also.)

[eternal surge]

Yeah, reading std is an excellent resource :)

And it teaches you how to write Zig better too

[lost trail]

@eternal surge Thanks for the time with this silly question of mine... greatly appreciate your input.

[eternal surge]

No problem, happy I could help :)

[lost trail]

Now, I need to figure out how to mark the tag as "answered"

with the green check mark....

[eternal surge]

react to the top message with ✅