sick spire
Hi team,
We’ve noticed that Harvest doesn’t always report the full set of volumes from a cluster.
For example, an SVM with dozens of online RW volumes only show a fraction of them in Grafana/Prometheus via Harvest.
Those volumes are not falling under any exclude options mentioned in the conf file. Any idea what might be the reason or issue?
We are on 9.14+ ONTAP and use REST collectors.
Harvest version 25.08.1
sour token
@sick spire Are the missing metrics volume capacity metrics or performance metrics? Could you share the Harvest logs for the affected poller at https://upload.nabox.org/gife-quce-bety ?
sick spire
volume metrics under REST, sure let me share the harvest logs
performance metrics are visible for all the volumes in the identified cluster
sour token
Please also share output of below endpoint on the relevant cluster in the same location.
api/private/cli/volume?return_records=true&fields=aggr_list,autosize_grow_threshold_percent,autosize_mode,autosize_shrink_threshold_percent,available,clone_parent_name,clone_parent_vserver,clone_volume,compression_space_saved,compression_space_saved_percent,dedupe_space_saved,dedupe_space_saved_percent,expected_available,files,files_used,filesystem_size,is_encrypted,is_sis_volume,is_space_enforcement_logical,is_space_reporting_logical,junction_path,logical_available,logical_used,logical_used_by_afs,logical_used_by_snapshots,logical_used_percent,max_autosize,min_autosize,nodes,overwrite_reserve,overwrite_reserve_used,percent_snapshot_space,percent_used,performance_tier_inactive_user_data,performance_tier_inactive_user_data_percent,sis_space_saved,sis_space_saved_percent,size,size_available_for_snapshots,size_used_by_snapshots,snaplock_type,snapshot_count,snapshot_policy,snapshot_reserve_available,snapshot_reserve_size,snapshot_space_used,space_mgmt_try_first,state,tags,tiering_policy,total,type,used,uuid,virtual_used,virtual_used_percent,volume,volume_style_extended,vserver,vsroot&is_constituent=*&ignore_unknown_fields=true
Let's focus on a capacity metric which is missing. Ideally volume_labels should have all volumes.
sick spire
these are NFS volumes which are having issue.. harvest logs shared
sour token
Thanks. From the logs, I see that we collected 192 volumes but exported only 81. Possible reasons include FlexGroup constituents being excluded by default or the volumes being transient. Once you share the API output, I can check.
sick spire
from my checks, volumes which are missing are not falling under these categories
sour token
Could you check if these are object store volumes? We exclude them as well
https://github.com/NetApp/harvest/blob/main/cmd/collectors/rest/plugins/volume/volume.go#L215
You can check it with api api/storage/volumes?fields=is_object_store
Could you share output of below api call via email
api/storage/volumes?return_records=true&fields=anti_ransomware.dry_run_start_time,anti_ransomware.state,clone.parent_snapshot.name,clone.split_estimate,is_object_store,name,snapmirror.destinations.is_cloud,snapmirror.destinations.is_ontap,snapmirror.is_protected,svm.name&is_constituent=false
none of the volumes are storing objects..
sour token
Thanks for the API output. I see the issue, the second API call returns only 80 volumes. The public API call is missing records that the private API/CLI returns. We are excluding them on that line.
https://github.com/NetApp/harvest/blob/main/cmd/collectors/rest/plugins/volume/volume.go#L229-L230
sick spire
but these are not node root or temp volumes.. not getting the logic why harvest is excluding them
sour token
Could you check if there is anything different about volumes which are not returned in public api api/storage/volumes
sick spire
yeah will dig into it
sour token
I’ve sent you the names via DM. Some are root volumes, and others look like backup-type volumes. These are not returned by the public ONTAP REST API.
sick spire
I officially gave up 🙂 and opened NetApp support case to understand the discrepancy with private cli and public REST on data volumes on our cluster
will keep you posted..
@sour token as always thank you for jumping in quickly and checking the issue 🙏