sick phoenix
Hello,
On our NAS mgwd logs, I can see that we have a lot of this kind of events :
ZAPILIB:823f03000:8503e80001f1ae19 :: ontapi :: [IP of NABOX] :: netapp-harvest :: GET /api/private/cli/aggr/show-space :: This request exceeds the configured session limit (20) for your application ("Harvest/24.11.1"). :: Throttled
Based on the following KB, there is a workaround by using a timeout modification in the request. Is there a way to customize it on NABox / harvest ?
spice stump
hi @sick phoenix yes, you can add a return_timeout property to any individual template or to all the templates by adding it to your default.yaml. If you want to add it to all your templates, create a custom.yaml file beside your existing conf/rest/default.yaml or conf/restperf/default.yaml)`
Add the following line to your custom.yaml
The other thing you can configure in your templates is jitter, which will smear collector startup across a longer window so there are fewer concurrent requests. Documented here https://netapp.github.io/harvest/24.11.1/configure-rest/#collector-configuration-file and instructions for applying here https://github.com/NetApp/harvest/discussions/2856
I also want to ask internally because I think this KB is incorrect. Yes, the request may be throttled, but that means queued by ONTAP and serviced once another request finishes. The KB says the request will fail, that is not the case unless the queue is full or the request has been pending too long and in both cases there is an ERROR returned to the client. Actually that's a good question, do you see errors from corresponding to the same timestamp you saw in the mgwd log? We're happy to check your logs to confirm if you share them. https://netapp.github.io/harvest/24.11.1/help/log-collection/
Can you share which version of ONTAP you're running on this cluster?
sick phoenix
Yes i'm using 9.12.1P13
spice stump
thanks, and I confirmed that what I shared above is correct. In other words, the KB is incorrect. Your request will be queued but that mgwd log message is informational and should not cause an error on the client
that said, jitter would still be worth doing to spread the requests out some, but you don't need to increase the return_timeout