New NABOX install no iops no latency information

[trim gust]

Hello,
I would like to know if anyone has experienced this: after installing a NAbox (4.0.10) and registering a NetApp storage running ONTAP 9.16.1P3, metrics and IOPS data are not being reported.
The rest of the information is visible in Grafana, but everything related to metrics and IOPS does not seem to be reported correctly.
On another NAbox (same version) connected to a different storage system, all data, including metrics and IOPS, is displayed correctly — and that system is also running ONTAP 9.16.1P3

Some screens attached to show you guys what I´m talking about.

Thanks in advance 🙂

[hybrid gale]

hi @trim gust can you create a support bundle from nabox and upload it here https://upload.nabox.org/siru-fiko-payy

[trim gust]

Hi Chris uploaded!, so i check out the harvest and seems to be working on one of the storage the one called netappD, the difference is that i force the API to ZAPI instead of AUTO.
Thanks 🙂

[hybrid gale]

Thanks for the logs @trim gust After forcing ZAPI do you see metrics for netappA now?

[muted bear]

That would point to acl issues ?

[hybrid gale]

we thought so too, but the logs show none

@trim gust can you run the following and upload the cluster.txt file to https://upload.nabox.org/siru-fiko-payy
curl -sk -u$user:$pass 'https://$ip/api/cluster?fields=*' > cluster.txt

Anything special about this cluster? Looks like it might be ASA?

What's interesting about this cluster is that it is running 9.16.1 but RestPerf fails with StatusCode: 404, Message: API not found, Code: 3 but there are no permission issues

[trim gust]

Yes after forcing ZAPI i have metrics indeed but I test the specific user created for that purpose and the default admin user from the netapps.

[trim gust]

yes, this are 2 ASA C190 configure with SMBC between them

[trim gust]

When I regain access to the network clusters I will push the command, NetApp OS 9.16.1P3.

[trim gust]

Hey Chris no matter how I execute the given command I got not result, the file cluster.txt has been created but there is no content inside.

[hybrid gale]

Thanks for trying the curl. If the curl command is not working, the Harvest REST collector also will not work. Sounds like REST is disabled on this cluster.

Can you ssh into your cluster and verify that the role your created for the harvest user has web access?

something like this - replacing role with your role name

vserver services web access show -name rest -role harvest2-rest-role

Also run this and paste the output
system services web show

Also can you run curl with verbose and paste the results here?
curl -v -sk -u$user:$pass 'https://$ip/api/cluster?fields=*'

[trim gust]

On Netapp cluster

netappD::> system services web show
External Web Services: true
Status: online
HTTP Protocol Port: 80
HTTPS Protocol Port: 443
HTTP Enabled: false

netappD::> vserver services web access show -name rest -role harvest2-rest-role
There are no entries matching your query.

I just upload the files from user admin and nabox, there is some cluster IP info that I remove the rest of the output is intact.

Regards

[hybrid gale]

thanks for the files Ruso. Earlier you said that curl was not working. These files show that it is working now. What changed?

What does this curl return when you execute it?
curl -sk -u$user:$pass 'https://$ip/api/cluster/counter/tables/system:node'

[trim gust]

It was failing because there is a space between the -u and the variable $user, this commands works `curl -sk -u "$user:$pass" https://$ip/api/cluster?fields=* > cluster.txt´

[hybrid gale]

got it and what about the counter/tables curl in the message above?

[trim gust]

Sorry I was talking about the command with the output to TXT, the command that you send does not bring any result even if I add a space after -u
I double check variables nad there are fine.

[hybrid gale]

what if you add -v, so curl -sk -v -u$user:$pass 'https://$ip/api/cluster/counter/tables/system:node'

BTW this works fine for me, no space required between -u and username
curl -sk -uuser:pass 'https://10.193.48.154/api/cluster?fields=*'

curl -V
curl 8.7.1 (x86_64-apple-darwin24.0) libcurl/8.7.1 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.64.0
Release-Date: 2024-03-27

looks like you are using the version of curl installed on nabox. What version of curl is installed there?

Thanks. I deleted the image since it includes credentials

[trim gust]

I´m using the curl from netbox:

admin@nabox ~ $ curl -V
curl 8.10.1 (x86_64-cros-linux-gnu) libcurl/8.10.1 OpenSSL/3.2.3 zlib/1.3.1 c-ares/1.33.1 libpsl/0.21.5 nghttp2/1.62.1
Release-Date: 2024-09-18
Protocols: dict file ftp ftps http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz NTLM PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets

[hybrid gale]

thanks for debug output. It's odd. At the cluster-level, Rest is enabled, ZapiPerf is enabled, RestPerf is disabled. This isn't a Harvest issue, but an ONTAP one. I'll check with some ONTAP folks and see what I can find out

[trim gust]

Thanks Chris, i. will try to replicate this on lab with a C190 that I have, I´ll let you know, thanks for your support 😀

[hybrid gale]

@trim gust I got confirmation from ONTAP that this is a bug in 9.16.X. Can you email us with you company details so I can update the issue with those? Alternatively, you can open a case and associate it that way. Let us know which you would prefer. The issue is CONTAP-454572

A workaround is to continue using the Rest and ZapiPerf collectors instead of RestPerf

[trim gust]

Hi Chris,
Thank you for the confirmation. I’ll open a case, but in this situation, as a workaround, I’ll continue using the ZAPI that’s currently working.
Thanks again for all your support.

[hybrid gale]

Hey @trim gust can you share the case number?

[trim gust]

Hey Chris sorry for the delay response I haven´t been in the office.
Case number 2010398092.