#How to use thread

1 messages · Page 1 of 1 (latest)

potent flicker
#

Hey !

I try to test the zig thread and I have this code. I don't understand why the sleep 1 second in the handler blocks the while(true) loop. I thought the detach() do what I want.

image.png
weak citrus
#

the sleep doesn't block the while loop

#

the accept will however block until it has a connection to accept

potent flicker
#

I send multiple request and the five messages "New request" wait 1 second between each request, I don't understand why

weak citrus
#

timestamp only has second granularity, are you sure it's actually waiting a full second, or if it's just slow because thread spawns are slow

potent flicker
#

Yes the message in the terminal is around 1 second

#

And when i remove the sleep it's faster but I use it to confirm that is blocking

weak citrus
# 
new request: 1683476658302
new request: 1683476658302
new request: 1683476658302
new request: 1683476658302
new request: 1683476658302
new request: 1683476658302
new request: 1683476658302
new request: 1683476658302
new request: 1683476658302

can't reproduce your issue, these are all different requests handled in the same millisecond

#

are you sure you aren't just sending requests in 1 second intervals?

potent flicker
#

Yes because you don't have the sleep I think no ? and the request is faster than 1 ms. what I understand it's the thread are synchrone

potent flicker
weak citrus
#

so evidently the thread sleeping isn't blocking the while loop

potent flicker
#

yes but when the handler is sleeping the next request is waiting no ?

weak citrus
#

no, std.Thread are os threads, they're completely separate threads of execution

#

once you spawn it, it goes and does it's own thing

potent flicker
#

So by default with this implementation I have a multithreading ?

weak citrus
#

yes

potent flicker
#

And the thread is balance on the different core by default ?

weak citrus
#

that's the os's problem

#

and just a warning, unless you're doing a lot of work or responding with a lot of data, spawning a thread will be slower than just calling the function and retaining single threadedness

potent flicker
#

Ok yes I understand, thank you :). A last question, how I can increase the performance with the number of connection ?

weak citrus
#

wdym?

potent flicker
#

For example in classic http server when I request with 1 connection I have for example 6000 req/s, if I switch to 10 connections I have 60000req/s. I don't know how the server manage the connection increase ?

weak citrus
#

the server "accepts" a new request when it calls accept, the system will buffer a few connections, but at some point it will just stop responding to new connections if that buffer fills

potent flicker
#

Ok so if I have 200 connections with 10 requests by connection, it will create 2000 threads ? It isn't optimal no ?

weak citrus
#

std.http will create a Response for each connection, keepalive isn't handled by accept, but yes, there's a reason no performant webservers spawn a new thread for every connection

potent flicker
#

Ok, so according to you what is the best practice ?

weak citrus
#

for simplicity? just use a single thread, accept a connection, handle it, rinse and repeat
for speed? use a pool of threads (usually best as the number of cpu cores you have available, any higher is bad) and distribute connections between them

potent flicker
#

Oh nice, ok so I just add for example an array of 4 threads. I distribute my thread in each index of array and the os automatically balance to the different core ?

weak citrus
#

something like that

potent flicker
#

But what is a difference between have an array of thread and juste one thread for each connection ? With just one thread like in my screen, the os doesn't balance to the cores ?

weak citrus
lime lily
#

See std.Thread.Pool

weak citrus
potent flicker
#

Okk! thanks you very much for your help 🙂

dusky canopy
#

Pretty sure in the OP's example, you will hit UB very quickly because its passing a stack var (res) into the handler thread. That response object will get clobbered by the next incoming connection.

Soon as you put that server under load / process a few of requests, you will get some nasty random crashes, and errors about "invalid alignment" with the response headers.

You will probably want to do something like this - https://gist.github.com/zigster64/7a2bc21c00629573f59765ff4f8d336b

The response header returned from accept needs to be cloned, then the clone passed to the thread handler. The handler can then clean it up when its all done.

Manual memory mgt is fun :). ... but give me MMM anyday over GC

Gist
simple threadpool webserver

simple threadpool webserver. GitHub Gist: instantly share code, notes, and snippets.

potent flicker
#

Yes I had this kind of problem lol

#

Thank you for the example 🙂

dusky canopy
#

yeah, that alignment error is a real headscratcher when you first see the error :). It gets worse once you have keepalives working, because each thread then lives a lot longer as it waits around for multiple requests on the same connection, so that gives plenty of time for the next accept() to come in on the main loop and completely trash the state of your handler thread.

That might be a common enough problem that it should be addressed in the way std.http allocates and returns a Response ? Dont know for sure.. Only seems to be the headers that get trashed though, so not 100% sure yet.

Cloning the response and handing the clone to the thread definitely does not have the same issue.

Anyway,. its all coming together nicely. If you want to do more experiments with threads in the context of std.http.Server / webserving in general, then you want to hang in there a bit longer and wait for 0.11 release - as there are some mass updates to the way that std.http works, and it looks very good.

potent flicker
#

oh nice, I see some PR that fixes some problems in http.server but I am enthusiast to use the stable version !

dusky canopy
#

yep, Im working off the code in that PR - if you want to try it out ... you can do this very unsafe and totally not recommended way of patching your Zig install without having to rebuild the world or wait for 0.11 :

  • cd to wherever Zig is installed
  • cd lib/std/http
  • mkdir backup && cp *.zig backup
  • copy all the files from that PR into lib/std/http. (or grab them off his fork, whatever is easier for you)
  • just rebuild your app (rm -rf zig-cache, then zig build)
  • to go back to the offiicial Zig stdlib, just cp the backup *.zig files back into that dir. Too easy

... because the stdlib is just code that gets lazy compiled in your app at comptime, you dont need to rebuild a custom version of Zig or anything, which is really nice and experimental-hacker-friendly.

Try with vs without cloning the response, and observe the alignment error when you put it under load.
Im serving up a react app with it (including POST endpoints for the api), and its pretty awesome - threads / keepalive / sub-millisecond round trip times for all the things -+ stable 12 MB total memory consumption on the server, and no discernable leaks. Excitement plus

potent flicker
#

Oh nice I will test this !

weak citrus
potent flicker
#

I thought that I need to re build zig

dusky canopy
#

^^ TIL !! thats even better. thx RIP

#

Pasty - yeah nah ... thought the same thing, because thats what just about every other dev tool would force me to do as well. (Like Go for example)
Zig is pretty hack friendly indeed

weak citrus
potent flicker
potent flicker
dusky canopy
#

^ that zolhttp is a much nicer pattern - pre-alloc 4 threads, and do the accept + handler loop per thread.
In that case, there is no chance of the res getting clobbered, because its all in the same thread

Suggest you also have a play with the pattern
Loop {
res = accept
spawn(handler, res)
}
Because a lot of ppl are going to try that first ... and fail with weird memory issues.

#

which I wouldnt mind fixing before 0.11 goes out, if its fixable

weak citrus
#

it's not fixable, accept doesn't allocate anymore

#

you either need to allocate it yourself, or put it somewhere it stays valid for the entire time you use it; just like any other variable

#

you should be able to theoretically pass it by value though afaik

dusky canopy
#

agree - because Zig will pass it by value., in theory it shouldnt have problems.
I will have a crack at debugging it, in case its either :

  • fixed anyway with the other recent updates to std.http
  • or something more insidious that is only surfacing under heavy load
#

"Pass by value" - Im assuming that means it literally copies the whole res struct onto the stack and calls thread.spawn()
... or would the optimizer pass a reference to the stack var, because it knows that res is already on the stack ?
No idea

weak citrus
#

I don't think it's allowed to do the ptr-to-const conversion