Hello,
We are trying to do some benchmarking in our self hosted Novu instance.
Is there any prior limitations on RPS on different Novu APIs. I do see in official documentation on some rate limiting applied, but for cloud versions.
On increasing the RPS for trigger events, we do see system get hung up a bit on Novu API service (tested with 3 pods), causing timeout errors as well.
#Load tests - Self hosted Novu instance
1 messages · Page 1 of 1 (latest)
@tribal river, you just advanced to level 4!
Hi @frank lintel
Can you give some info on this?
Hi Ragu did you get any help on this item? We have a similar issue with load testing here: https://discord.com/channels/895029566685462578/1436388040980304044
@torn pollen
I did a bit of changes on my end. I'm only using chat webhooks, to eliminate provider dependencies.
- Changed workflow type from BRIDGE to REGULAR. >> This doubled performance as no api calls were made from worker service to api layer.
- Document DB CPU, high load caused CPU to be exhausted. Increased cpu limit here.
- No of connections open per service. This also had major impact. This requires testing against all services you're running. And concluding a proper value. I'm using 10-50. Could be high. But needs more testing.
Also set MONGO_AUTO_CREATE_INDEX to True on first run. This will improve read queries in services as certain collection can increase over time.
Hey @tribal river , can you elaborate on how you changed the workflow type from BRIDGE -> REGULAR? I am not seeing any workflow type options in the dashbaord or API. What version of Novu are you using?
@burnt lance
I'm using Api to create Workflows through v2 Api. You will need to add a field called type with REGULAR as value. Steps also has type I think.
When using dashboard to create Workflows. This type is automatically set as BRIDGE. You can check directly in db.
Hey @tribal river , can you please share a sample API request? I have tried hitting the v2/workflows POST API with type: "REGULAR" but in the db i still see type is BRIDGE
@burnt lance
Yeah. I went through the code. For now I'm using v1 api. Which accepts this type and also allows to attach content to the templates unfortunately.
I'm still trying to get some get some clarity of v1 and v2 Workflows what's the differences.