How to thread http stuff?

[ancient ginkgo]

youd need to show your threading code as well, can’t help you much without it
the api definitely works while threading though, ive used it like that before
the way I did it was having a http_done atomic bool and then setting it to true when the http request was done, and then the user knew that the arraylist they gave the http thread was filled and ready to be read

probably not the best way to do it but point is you can treat it like any other thread that has to return data to the user

[valid ether]

I dont have the thread code anymore, will try to do it again. thanks for the info

[valid ether]

This is my current attempt but printing the queue length gives me a wrong value. I guess thats not how to do it with threads. can you help me out?

pub const ThreadedComet = struct {
    comet: Comet,
    queue: MetricQueue,
    worker: ?std.Thread,

    pub fn init(allocator: std.mem.Allocator, api_key: []const u8, team_name: []const u8, project_name: []const u8, run_name: []const u8) !ThreadedComet {
        var comet = Comet.init(allocator, api_key, team_name, project_name, run_name);
        try comet.create_run();

        return .{
            .comet = comet,
            .queue = MetricQueue.init(allocator),
            .worker = null,
        };
    }

    pub fn deinit(self: *ThreadedComet) void {
        if (self.worker) |worker| {
            self.queue.shutdown();
            worker.join();
        }
        self.queue.deinit();
        self.comet.deinit();
    }

    // thread that runs and processes the requests one by one
    fn thread_worker(self: *ThreadedComet) void {
        std.debug.print("QUEUE: {d}\n", .{self.queue.request.items.len});

        while (true) {
            if (self.queue.pop()) |request| {
                self.comet.send_metric(request.metric) catch |err| {
                    std.debug.print("Error occured on send_metric [{}]", .{err});
                };
            } else {
                break; // queue shutdown and empty
            }
        }
    }

    pub fn start(self: *ThreadedComet) !void { // start thread
        if (self.worker != null) return error.AlreadyRunning;

        self.worker = try std.Thread.spawn(.{}, thread_worker, .{self});
    }

    pub fn send_metric(self: *ThreadedComet, metric: Metric) !void {
        try self.queue.push(.{ .metric = metric });
    }

    pub fn get_pending_count(self: *ThreadedComet) usize {
        return self.queue.get_pending_count();
    }
};

[ancient ginkgo]

Make sure you don't modify (or read if sync matters) ThreadedComet while the thread is running

[valid ether]

im calling send_metric to add new metric to the queue that should be processed by the thread. But im still stuck on the problem that the data thread_worker accesses is invalid

pub fn send_metric(self: *ThreadedComet, metric: Metric) !void {
    try self.queue.push(.{ .metric = metric });
}

[ancient ginkgo]

you should to put the queue behind some sort of mutex then, that way it won't be modified while your thread is reading it

although why not use a thread pool here
That's more or less what you're designing, but a thread pool allows multiple working threads depending on how much is queued

[valid ether]

I just want a single thread. not sure how well you can thread sending http requests? I want them to be send in order and 1 thread is good enough for me, it should just not block the main thread by any chance.

Tried now a bunch of different things to fix it but they didnt work. I guess I wont randomly stumble uppon a working solution without understanding whats wrong

[ancient ginkgo]

try putting the queue behind a mutex and see if that fixes it
So many things can happen otherwise since its not atomic

[valid ether]

does not fix it

[mint hill]

use .sanitize_thread option for b.addExecutable for debugging threading issues

[valid ether]

I get error:

zig build test
test
└─ run test
   └─ zig test Debug native 3 errors
error: sub-compilation of libcxx failed
    :1:1: note: unsupported option '-fsanitize=thread' for target 'x86_64-unknown-windows-gnu'
    :1:1: note: unsupported option '-fsanitize=thread' for target 'x86_64-unknown-windows-gnu'
    :1:1: note: unsupported option '-fsanitize=thread' for target 'x86_64-unknown-windows-gnu'
    :1:1: note: unsupported option '-fsanitize=thread' for target 'x86_64-unknown-windows-gnu'
    :1:1: note: unsupported option '-fsanitize=thread' for target 'x86_64-unknown-windows-gnu'
    :1:1: note: unsupported option '-fsanitize=thread' for target 'x86_64-unknown-windows-gnu'

pub fn build(b: *std.Build) void {
    const target = b.standardTargetOptions(.{});
    const optimize = b.standardOptimizeOption(.{});

    const mod_lib = b.addModule("zig_comet", .{ .root_source_file = b.path("src/root.zig"), .target = target, .optimize = optimize, .sanitize_thread = true });

    const mod_test = b.createModule(.{ .root_source_file = b.path("test/main.zig"), .target = target, .optimize = optimize, .sanitize_thread = true });
    mod_test.addImport("zig_comet", mod_lib);

    const test_lib = b.addTest(.{ .root_module = mod_test });
    b.installArtifact(test_lib); // builds the test.exe

    const test_step = b.step("test", "Run unit tests");
    test_step.dependOn(&(b.addRunArtifact(test_lib)).step);
}

[mint hill]

might not support windows