A naive benchmark https://github.com/FarhanAliRaza/fastapi-vs-litestar-vs-django-bolt-vs-django-ninja-benchmarks comparing these frameworks.
#Django Ninja Vs Django bolt vs Fast API vs Litestar
12 messages · Page 1 of 1 (latest)
I'm surprised bolt is so much faster for DB queries - I thought the ORM was left mostly as-is
I only try to optimze if user return a queryset than i try to get values that is much faster then a normal for loop over objects. But in this case i do not anything like that. Because it already return a list. It can be improved using values it will be faster than this i think.
I do not have much experience with sqlalchemy. Maybe i have made some mistake in there.
Someone added drf and reran it on its machine.
https://github.com/tanrax/python-api-frameworks-benchmark
I'm curious if these benchmarks are parallel to the what you should expect from a live application on production?
I guess the quick answer it depends, but I'm also confused if anyone is actaully limited by the RPS being 3k but not 20k when using Django
Sorry for the long answer. Seeing so many comments like this.
That is the most common kind of argument. Seeing a lot of it on Reddit. When building a framework, I can only test the internals of the framework. I am only responsible for the work that is done on the hotpath to be fast. That also includes user business logic. I can only measure what framework offers. And these benchmarks are designed to do that. I want to write a detailed blog post about it. But replying here shortly. I started from about 6k rps, which i optimized to now 20k rps for a single worker. In this benchmark, I am measuring framework code performance. I am not measuring user business logic that changes with every application. DB latency and slow user code, database performance, I cannot measure. Nor does this framework promise to fix bad database design. It is an understood part that production code will not get the same performance. Because most of us are not using SQLite hosted locally with only 10 items in it. But this benchmark proves that if your database latency of db that is constant, consider that x milliseconds. The framework overhead is minimal compared to other frameworks.
None of my personal projects has scaled to the limits of Django. I just like the syntax. Latency being lower for the same view written in drf vs django bolt is much lower what satisfies my inner engineer.
Sorry, I was just asking to learn, it wasn't a counter argument
I mean, I didn't know how you measured those results (maybe I should've checked before, sorry) and if you had a model serialization in them, but it's also really fast and gives closer results to real world
Or maybe you'd have some usecases that I am not aware of, where that kind of speed is important
Suppose you found a way to make your DB code very fast, like 20k rps fast. You can just use 1 django-bolt worker for it on one server (Theoretically). You can not do that with gunicorn because gunicorn scales using threads. That is the issue with most sync-first frameworks. It is not using ModelSerializer; it is using a msgspec struct as a serializer that validates the dict returned, so they have correct fields and no extra fields. Serializer implementation that I implemented on top of msgspec that has a similar feature to Pydantic is a little slow as compared to raw struct (2200 RPS vs 2400 RPS (same data)). Serializer is just a msgspec struct class extended to support ORM and functional validation.
How did I measure it?
The code link is there in the post. I just created api endpoints and use bombardier to measure RPS.