Metrics which require resource owner UUID | NetApp | Page 1

molten zealot May 24, 2024, 1:43 PM

#

Hi team, is there a way to collect REST metrics that require providing resource UUID to be exposed?
Example: i need delta.size_consumed metric provided by /storage/volumes/{UUID}/snaphots

CLI command snapshot show-delta is not implemented in private cli API and requires providing a snapshot name.

restive lava May 24, 2024, 2:05 PM

#

@molten zealot You can try api/storage/volumes/*/snapshots?fields=delta.size_consumed

molten zealot May 24, 2024, 3:42 PM

#

doesn't seem that query parameter value is handled correctly or expected to be such

restive lava May 24, 2024, 3:58 PM

#

Could you share your template.

molten zealot May 24, 2024, 4:17 PM

#

name: TX_SnapMirror
query: api/storage/volumes/*/snapshots?fields=delta.size_consumed
object: snapshot

counters:

delta.size_consumed

#

there is no object "snapshot" as far as i see

restive lava May 24, 2024, 4:23 PM

#

Can you try below template

name:                     TX_SnapMirror
query:                    api/storage/volumes/*/snapshots
object:                   snapshot

counters:
  - ^^volume.uuid                                 => uuid
  - ^^name                                        => snapshot
  - ^volume.name                                  => volume
  - delta.size_consumed                           => size_consumed


export_options:
  instance_keys:
    - snapshot
    - volume
    - uuid

molten zealot May 24, 2024, 4:40 PM

#

i see this in poller log:

#

but metric isn't exported

restive lava May 24, 2024, 4:41 PM

#

Could you share your harvest version

molten zealot May 24, 2024, 4:43 PM

#

i have 2 days ago nightly build instance. Harvest isn't even starting with the same configuration

#

config files are identical

#

restive lava May 24, 2024, 4:47 PM

#

Can you upgrade to 24.05 version. It has better logging

molten zealot May 24, 2024, 4:56 PM

#

while im trying to update, can i have a bonus question - are metrics sharable between objects somehow? let's say i create custom metric with metric agent and then use it's value in another object to make the final metric i need

restive lava May 24, 2024, 4:59 PM

#

Each template is independent. You may join their results in a prometheus query.
Rest templates do support endpoints feature as documented here
https://netapp.github.io/harvest/24.05/configure-rest/#endpoints if it helps.
Can you share your usecase?

molten zealot May 24, 2024, 5:03 PM

#

use case is to calculate the time predicted for the upcoming snapmirror update (delta.size_consumed) having last_transfer_size/last_transfer_duration pre-calculated from snapmirror object

#

i have yet come up to ansible playbook pushing metrics to pushgateway which is very flexible but maybe there is a native way with harvest

restive lava May 24, 2024, 5:08 PM

#

Okay sounds like a custom plugin kind of requirement if it cannt be done with prometheus queries.

molten zealot May 24, 2024, 5:08 PM

#

restive lava Each template is independent. You may join their results in a prometheus query. ...

endpoints look like the feature i could play with!!

restive lava May 24, 2024, 5:08 PM

#

custom plugin is more of a go code which we'll need to write.

molten zealot May 24, 2024, 5:09 PM

#

i looked at snapmirror plugin and if it does additional calls to see peering details then it can look at many other places

restive lava May 24, 2024, 5:09 PM

#

https://netapp.github.io/harvest/24.05/plugins/

molten zealot May 24, 2024, 5:10 PM

#

i saw this page but didn't find "developers documentation"

restive lava May 24, 2024, 5:10 PM

#

Yes in a plugin we can handle this. You can also open a Github issue and we can take a look.

restive lava May 24, 2024, 5:12 PM

#

molten zealot i saw this page but didn't find "developers documentation"

Yes we don't have details around this but architecture can give you a basic idea.
https://github.com/NetApp/harvest/blob/main/ARCHITECTURE.md

molten zealot May 24, 2024, 5:15 PM

#

i will ping you in netapp teams with more business related details for this

#

got it updated to nightly 24.05

#

#

restive lava May 24, 2024, 5:29 PM

#

Okay.

#

Strange. As per error, query URL is empty in template.

#

seems to work fine for me

restive lava May 27, 2024, 11:56 AM

#

molten zealot

@molten zealot Could you check that your default.yaml for Rest includes this line: https://github.com/NetApp/harvest/blob/main/conf/rest/default.yaml#L5 under the schedule section? Also verify that this file matches.

molten zealot May 27, 2024, 3:16 PM

#

hi Rahul, yes, this line was missing since i was upgrading from ver 23.08 -> 24.05 simply copying default.yaml file

#

Upgrade page in Hravest documentation is pretty simple (doc update request needed?)

#

i actually was wrong - it hasn't fixed. It is now starting the harvest without an error, but empty url error appears in poller log

#

#

when upgrading i only copied harvest.yaml, default.yaml and my custom tx_snapmirror.yaml files. As you may see the error is also seen for qtree metrics - which file i never edited, only enabled it in default.yaml

restive lava May 28, 2024, 4:29 AM

#

Upgrade instructions for the relevant platform are available here: https://netapp.github.io/harvest/24.05/install/overview/. Could you confirm your installation mode? Is it rpm, deb, or native? The default.yaml file doesn't need to be updated. It should be used as it was shipped with the 24.05 version. If you need to use tx_snapmirror.yaml, then a custom.yaml file needs to be created. For more details, see here: https://netapp.github.io/harvest/24.05/configure-templates/#how-to-extend-a-restrestperfstoragegridems-collectors-existing-object-template.

molten zealot May 28, 2024, 6:51 AM

#

i downloaded last available release from git and expanded to a new folder, then copied harvest.yaml, default.yaml (as i have many collectors disabled) and tx_snapmirror files. And also snapmirror.yaml that worked in 23.08 release as i have custom metrics configured there

restive lava May 28, 2024, 6:52 AM

#

Ok could you paste your default.yaml here

#

For Rest collector

molten zealot May 28, 2024, 10:18 AM

#

here it is

#

hm. didn't change anything but after host reboot it started to work without empty url message

📎 default.yaml

restive lava May 28, 2024, 10:42 AM

#

File looks alright. It's possible that during a previous start, it used an older default.yaml which had an issue.

molten zealot May 29, 2024, 8:08 AM

#

hi Rahul, returning to the original question. Do i understand correctly that the UUID required for the data extraction from the underlying resource cannot be used this way as below?

#

#

its not clear how to follow the REST API sematics if UUID is required in the API call to address "next" resource data

#

also as per my understanding an endpoint query should be used to extend the data collection for the main query but not to incorporate a logic when you can collect some data from multiple independent resources to calculate a new metric based on collected data.

#

example: i need a new metric to understand snapmirror bandwidth based on last_trasfer_size and last_transfer_duration (snapmirror resources). Then take delta.size_consumed from a snapshot data (snapshot resource) and make a new metric calucated on these 2 to get the approx time of the upcoming snapmirror update

#

that'd be even more accurate:

#

restive lava May 29, 2024, 10:57 AM

#

@molten zealot Endpoints are used to enrich the data collected from the main query. Your key can be composite as well, and it doesn't necessarily have to be named uuid. You can name it anything as long as the key names match between the main and endpoint queries.

In the example you provided, it appears there could be a many-to-one relationship between snapshots and volumes. Endpoints are typically useful where there is a one-to-one mapping. For this particular scenario, you might consider collecting this data via two separate templates and then, if possible, using Prometheus queries to join the results later.

molten zealot May 29, 2024, 3:06 PM

#

yes, in my exmaple it can be rather many to many. I am trying to understand the limits of endpoints data collection and representation. I am looking up snapmirror protected volumes by uuid (as i don't need any other information from the volume itself) and having them identified listing the snapshots'd delta. Which may result in many-to-many. Would it be hard to represent for the export?

#Metrics which require resource owner UUID