Zoi an ultra simple http/tcp server using only the Zig standard library

[drifting lotus]

I already posted about this in the net-web-dev channel but I am super proud of it so I thought I would share it here too.
Zoi is a server for http 1.1 that delivers static web pages from the directory the server is launched from. It is technically functional now but there are quite a few improvements I am hoping to make going forward.
It uses the tcp functionality instead of the built in http functionality so that I could work with the requests at the finest level possible but I might change that later it currently runs single threaded and was built in dev 0.11
https://github.com/AndrewGossage/Zoi

[sterile echo]

just a question, if i use this project am i considered a zoi boy?

[harsh aspen]

forgot to add zig-cache and zig-bin to gitignore

[drifting lotus]

I'm planning on continuing to work on this. Initially the speed actually looks pretty good I am able to beat out Flask which is a very popular micro framework written in Python and what I normally use for work. I'm sure with some work it could actually get much faster than it is now.

And yes I know python is slow so it isn't completely a fair comparison but flask is pretty small and pretty popular so I thought I would go ahead and compare.

[rustic berry]

If i'm reading correctly, before working on speed, you might want to work on correctness. The server allows directory traversal and doesn't seem to consider TCP fragmentation.

[drifting lotus]

Thanks for the feedback @rustic berry. To be more precise I haven't worked on speed yet. This is just as it is right out the gate. I do intend to work on correctness.

[drifting lotus]

I added the ability to select the port the server will run in based off a toml file rather than requiring it to be hard coded into the server at compile time.

I am also implementing the toml parsing from scratch to maintain the 'zero external dependencies' goal (which is really to help me get better by having to do everything from scratch). Although I'm sure somebody else has already implemented a great toml parser.

[drifting lotus]

I launched a site that runs off my custom server: http://zoi-server.com/

[drifting lotus]

I have added basic support for dynamic content.

[drifting lotus]

I have been running a simple site in production basically nonstop for about a week it has remained stable

[drifting lotus]

Zoi now runs multithreaded! This should greatly improve performance and allow use in higher traffic applications.

[drifting lotus]

Coming back to work on this some more and doing some testing to ensure that it is stable to run long term. I have confirmed that the project will build and launch on FreeBSD which is not too surprising given how solid Zig works on just about everything but it is when I am coding something nothing is a guarantee haha.

[drifting lotus]

The multi-threading functionality needs an overhaul as the way I did it was not ideal. I have however, confirmed that this Zoi runs properly on the current development version of Zig.

[spice veldt]

I looked at the code and you seem to spawn a GPA for every function call and you don't deinit it either.

        _ = self;
        var gpa = std.heap.GeneralPurposeAllocator(.{}){};
        const allocator = gpa.allocator();
        //create the ArrayList for message
        var response = std.ArrayList(u8).init(allocator);
        defer response.deinit();

Maybe in later updates if you continue working on it you could have this be more efficient by reusing it. Cool stuff otherwise.

Also I think that it does mean it leaks memory.

[drifting lotus]

Thanks, i am definitely planning on continuing to work with it. I appreciate you calling that out.

[drifting lotus]

I have made some improvements on memory leaks and the server is now more stable. I also am beginning to implement some security features such as setting what file types the server is allowed to send which should be fully operational soon.

[drifting lotus]

A couple of security upgrades went through. You now can configure which file types you will let through. Hidden files and folders are now also filtered out,

[native yoke]

replying to this old message because the thread was bumped, but would TCP fragmentation really be an issue you have to handle at the userspace level? i don't see why it would matter here since they're just reading some amount of bytes.

[rustic berry]

You can't assume that 1 read of the socket contains the full request

While it's unlikely to happen, a "message" of N bytes could be spread out across N reads.

[native yoke]

ah, i didn't notice that it was assuming that, at a further glance, i see it now. yeah, i get what you mean now.

assuming that the entire request fits in 1024 bytes isn't the best idea either, for a generic file server (e.g. long paths would trip it up)

[rustic berry]

There are a number of other concerns. read_file leaks $file_size if the file fails to be read. Creating a GPA + allocator per connection. read_url will leak list if any of the the append's fail. Mutex is never released if accept fails.

[drifting lotus]

Thank you for pointing out some of the issues they will eventually all be addressed as my schedule allows

[drifting lotus]

Many of the issues that had been pointed out previously have now been fixed as well as some other bugs and shortcomings I found on my own. The server still expects a message to only take one read and there are definitely other improvements that will be important to make but overall I think Zoi has been a successful experiment so far.

[eager juniper]

Nice. Do you have a roadmap and/or testing plan and/or intended targets for tcp and html behavior? html in particular is a beast with 1.1 being annoyingly underspecified and more annoyingly perf-wise html 1.1 is missing pipelining and a sane header default.
tcp itself is also quite underspecified regarding time behavior of control channel behavior or any sort of configurability, so we are currently in the state of non-severe portability of performance behavior and the alternatives are overly complex beasts like html 2.0.

[drifting lotus]

Not currently, I am still working on fleshing out a baseline and then I will be able to build out a more detailed roadmap.

[drifting lotus]

I've been continuing to flesh this out and there are several positive changes mostly around support for dynamic content.

[drifting lotus]

As a brief recap of the development that has been done on this. The server no longer assumes the request headers will fit in 1024 bytes, several bugs/potential bugs have been fixed. Memory management has been significantly improved, Custom headers can be added to responses and much more.