Snapshot Dashboard | NetApp | Page 1

foggy palm Sep 13, 2023, 1:59 PM

#

i m trying to setup a dashboard per vol see size of each snapshot ,

Query for api would be /api/storage/volumes/*/snapshots ?

unborn cloud Sep 13, 2023, 2:47 PM

#

hi @foggy palm yes, that is the correct REST API to use to gather snapshot information, including size

foggy palm Sep 13, 2023, 2:59 PM

#

@unborn cloud Thanks , let try to have conf file ,

foggy palm Sep 14, 2023, 8:41 AM

#

@unborn cloud

Query has more details that what we need

/storage/volumes/{volume.uuid}/snapshots

How do i configure pass volume uuid in conf fille

`name: Snapshots
query: /api/storage/volumes/*/snapshots
object: snapshots_detail

counters:
-^^volume.name => volume
-^^volume.uuid => uuid

endpoints:
- query /api/storage/volumes/{volume.uuid}/snapshots
counters:
-^^uuid => snapshot_uuid
-^^name => snapshot_name

endpoints:
- query /api/storage/volumes/{volume.uuid}/snapshots/{uuid}
counters:
-^^volume.name => volume_name
-^^name => snapshot_name
-^^svm.name => svm
-^^create_time
-^^size => snapshot_size

export_options:
instance_keys:
- volume
- uuid
- svm
- snapshot_size
- snapshot_name`

patent sundial Sep 14, 2023, 11:18 AM

#

@foggy palm
You can follow this template

name:                     Snapshot
query:                    api/storage/volumes/*/snapshots
object:                   snapshot

counters:
  - ^^uuid                   => uuid
  - ^^volume.uuid            => volume_uuid
  - ^name                    => snapshot
  - ^svm.name                => svm
  - ^volume.name             => volume
  - create_time(timestamp)   => create_time
  - size                     => size

export_options:
  instance_keys:
    - uuid
    - volume_uuid
  instance_labels:
    - svm
    - volume
    - snapshot

#

patent sundial Sep 14, 2023, 11:46 AM

#

Please take this updated template as you would need all the labels in metrics.

name:                     Snapshot
query:                    api/storage/volumes/*/snapshots
object:                   snapshot

counters:
  - ^^uuid                   => uuid
  - ^^volume.uuid            => volume_uuid
  - ^name                    => snapshot
  - ^svm.name                => svm
  - ^volume.name             => volume
  - create_time(timestamp)   => create_time
  - size                     => size

export_options:
  include_all_labels: true

foggy palm Sep 14, 2023, 12:00 PM

#

@patent sundial Thank you , will run and let you know

foggy palm Sep 14, 2023, 12:31 PM

#

@patent sundial i see this is working but not did not get data from all svm that i have ,

foggy palm Sep 14, 2023, 1:21 PM

#

Also noticed size is not same in rest call and in exporter

patent sundial Sep 14, 2023, 3:25 PM

#

I have checked prometheus and rest call values for couple of snapshots and they are matching.

% curl -s http://localhost:13008/metrics  |grep snapshot_size | grep snapshot-75661f95-038c-4371-954e-8fb9c6518a52
snapshot_size{datacenter="rest",cluster="umeng-aff300-01-02",snapshot="snapshot-75661f95-038c-4371-954e-8fb9c6518a52",svm="astra_300",volume="trident_pvc_be8b6f8d_4aeb_4222_ba47_84966ecc59b2",uuid="72fd2a46-f4e2-4440-a7e8-49d419190a4a",volume_uuid="2d9401a2-ce28-11ed-b61b-00a098d39e12"} 45977600

% ./bin/harvest rest -p sar show data --api storage/volumes/2d9401a2-ce28-11ed-b61b-00a098d39e12/snapshots --field="name=snapshot-75661f95-038c-4371-954e-8fb9c6518a52" 
[
 {
  "poller": "sar",
  "addr": "x.x.x.x",
  "api": "storage/volumes/2d9401a2-ce28-11ed-b61b-00a098d39e12/snapshots",
  "version": "9.13.1",
  "cluster_name": "umeng-aff300-01-02",
  "records": [
   {
    "create_time": "2023-03-30T01:04:50+12:00",
    "logical_size": 203251712,
    "name": "snapshot-75661f95-038c-4371-954e-8fb9c6518a52",
    "provenance_volume": {
     "uuid": "2d9401a2-ce28-11ed-b61b-00a098d39e12"
    },
    "size": 45977600,
    "svm": {
     "name": "astra_300",
     "uuid": "835c0d25-ceb0-11eb-80e3-00a098d39e12"
    },
    "uuid": "72fd2a46-f4e2-4440-a7e8-49d419190a4a",
    "version_uuid": "72fd2a46-f4e2-4440-a7e8-49d419190a4a",
    "volume": {
     "name": "trident_pvc_be8b6f8d_4aeb_4222_ba47_84966ecc59b2",
     "uuid": "2d9401a2-ce28-11ed-b61b-00a098d39e12"
    }
   }
  ],
  "num_records": 1,
  "poll_ms": 1160
 }
]

foggy palm Sep 15, 2023, 5:45 AM

#

@patent sundial still it does not report from other SVM in same cluster

foggy palm Sep 15, 2023, 6:45 AM

#

is there something spcial we need to do for unit conversion ? i m tried bytes but value show as MB/GB/TB but not KB ,

patent sundial Sep 15, 2023, 1:01 PM

#

@foggy palm when you say other SVMs, do you mean to say data are not coming from other node SVM or admin SVM ? Or couple of data SVMs' data are also not coming in the REST response/Harvest metrics ?

#

The unit of this size metric is in bytes from ONTAP and Harvest export them as-is and generally unit conversion would be in Grafana panels and Grafana would show them TB/GB/MB/KB based on the value.

patent sundial Sep 15, 2023, 1:30 PM

#

Also, When you say size is not same for exported data and rest call, Could you share specific Rest o/p and exported data o/p where it's not same ?

foggy palm Sep 18, 2023, 6:01 AM

#

@patent sundial

Other SVM are data type only ( when manully run rest api it does how but it does not show in harvest , checking logs from harvest to see )
Yup i used bytes in unit and it should convert it to TB/GB/MB/KB but it does not do well
3.Size is now matching probably i was looking at different values.

patent sundial Sep 19, 2023, 10:55 AM

#

Thanks for update.
For 1, Could you share harvest logs for that poller with us @ ng-harvest-files@netapp.com?
For 2, Could share more light on units conversion part, what kind of issue/problem you may have faced ?
For 3, Seems we are good here.

unborn cloud Sep 19, 2023, 12:51 PM

#

@foggy palm in addition to the log files that @muted shuttle asked for, there was a REST pagination bug we fixed in an earlier release of Harvest. Can you let us know which version of Harvest you are using?

foggy palm Sep 20, 2023, 3:26 PM

#

@unborn cloud @muted shuttle Sure

foggy palm Sep 20, 2023, 3:53 PM

#

@unborn cloud Version is harvest version 23.08.0-1 (commit 874a1f87) (build date 2023-08-21T08:36:53-0400) linux/amd64

unborn cloud Sep 20, 2023, 3:55 PM

#

that's good, you'll have the pagination fix

foggy palm Sep 21, 2023, 6:17 AM

#

i sent email with subject snapshot details

patent sundial Sep 21, 2023, 6:28 AM

#

Thanks @foggy palm , we have received the logs, checking for any abnormality in that.

foggy palm Sep 22, 2023, 6:27 AM

#

@patent sundial i suspect rest call consider default SVM only ?

patent sundial Sep 22, 2023, 1:46 PM

#

This wildcard * Rest call would not give all the snapshots data from the cluster. I have shared the template in mail to get the all the snapshot data via Rest private cli

foggy palm Sep 22, 2023, 1:49 PM

#

@patent sundial Thank you let me run new template that you shared

#

@patent sundial This is working , Thank you

kindred cairn Jun 5, 2024, 5:38 PM

#

@patent sundial @unborn cloud could you please share the snapshot template using pri cli? there used to be a snapshot template in zapi but not in recent releases

unborn cloud Jun 5, 2024, 5:48 PM

#

@kindred cairn can you let me know how you installed Harvest? native, compose, nabox, etc.?

kindred cairn Jun 5, 2024, 5:49 PM

#

using docker image in K8s version 24.02.0

#

there is snapshotpolicy.yaml, query: api/private/cli/snapshot/policy

#

so for snapshots, is api/private/cli/snapshot equvelent to "snap show"? is there document on pri api? if so i may be able to write a template myself 😀

unborn cloud Jun 5, 2024, 5:58 PM

#

i'm nearly done validating the template and will share in a moment. Yes, how to use private CLI is documented https://netapp.github.io/harvest/nightly/configure-rest/#ontap-private-cli and yes api/private/cli/snapshot is equivalent to snap show

kindred cairn Jun 5, 2024, 6:05 PM

#

i did a curl call it returned a list of snapshots but not much details, such as creation time and size
{
"records": [
{
"vserver": "flc1-01-lab-phx-storage",
"volume": "vol0",
"snapshot": "nightly.1"
},
{
"vserver": "flc1-01-lab-phx-storage",
"volume": "vol0",
"snapshot": "hourly.5"
},

unborn cloud Jun 5, 2024, 6:08 PM

#

See if this helps https://github.com/NetApp/harvest/discussions/2970

#Snapshot Dashboard