smoky mist
Hello,
We’re starting to leverage S3 object duality. I was wondering what the recommended monitoring we have in Harvest to illustrate the health of the object-store-server. Are there any NAS specific health metrics that I should make sure are being collected?
For example, It was suggested that I look at nas_list_objects_total, nas_list_object_latency, and nas_list_objects_rate from ASUP. Are we collecting those?
violet mural
Hi @smoky mist do you know which REST API includes those? I took a quick look at https://docs.netapp.com/us-en/ontap-restapi/ontap/swagger-ui/index.html and don't see them. Alternatively, do you know which ONTAP CLI returns these metrics and if so we can help you create a template to collect them. Most of the S3 metrics we collect today are included in the ONTAP: S3 Object Storage dashboard
smoky mist
Talking to some folks, I believe there are counters under the same endpoint for object_store_server
I don’t have a CLI to retrieve as of yet.
Perhaps it’s in statistics show?
violet mural
We collect api/cluster/counter/tables/object_store_server via this template https://github.com/NetApp/harvest/blob/main/conf/restperf/9.14.1/ontap_s3_svm.yaml
I don't see anything about nas_list_objects_total but perhaps that is equivalent to the counter list_objects_total?
smoky mist
Quoting some other engineers here: “list_bucket_latency counters tracks latency for the S3 Op where S3 client does an op to list all the buckets in the system. It'll get back both "S3" and "NAS" type buckets back in the result. However, this counter isn't the one that is tracking the latency of the LIST Object Op from clients
For tracking latency of listing of objects for NAS buckets: we must look at following counters within the same "object_store_server"
• nas_list_objects_total
• nas_list_objects_latency
• nas_list_objects_rate
SImilarly for tracking latency of listing of objects for S3 buckets, we must look at
• list_objects_total
• list_objects_latency
• list_objects_rate”
It seems there is a difference
I actually sent him the collector for one of the list_bucket metrics Harvest has already. And that’s what I got from him
violet mural
Unfortunately the nas_* counters are not included via the counter manager REST endpoint (they are via ZAPI) You can see that by running this against a cluster. (replace user, pass, and ip with your values) curl -sk -u$user:$pass 'https://$ip/api/cluster/counter/tables/object_store_server' | grep nas
smoky mist
Oh great lol
violet mural
you'll need to ask ONTAP to add them if those are important or use ZAPI for this collector
smoky mist
Well I’m talking to them now so I’ll figure something out.
violet mural
sounds good, let us know if we can help!
smoky mist
Thanks for the help Chris
smoky mist
Update: they’re filing a Burt as it should be in REST.
Have you ever retrieved statistics show via private cli? The counters are available via CLI so that would be my backup plan
We tried but got an error saying “statistics” wasn’t supported
violet mural
no, it should be possible, but requires a new collector to be built. A better stop-gap would be to use ZAPI
smoky mist
Zapi wouldn’t be an option for us. We use oauth for authentication (compliance restrictions)
violet mural
unfortunately, private CLI statistics would require a fair bit of work to add. And last conversation I had with the CM team, not recommended
smoky mist
Oh so even if I put together a template for it, it still wouldn’t work?