two workload for iscsi volume qos_ops metric

[flint basin]

Hi All, when using qos_ops and qos_latency to monitor iscsi volume , there are two output.
One have the "User-Best-Effort" Policy group which is default for volume level.
Is it possible to exclude this default policy group from harvest poller?
Also, Based on this KB:
https://kb.netapp.com/onprem/ontap/Performance/Why_does_the_statistics_volume_show_command_show_the_total_ops_which_donot_match_the_read_write_and_other_ops
For the SAN protocol, using "volume_total_ops" metric will have the same value as "qos_ops"? Is this correct?

[glass helm]

@flint basin Do you want to exclude workloads with policy User-Best-Effort from Harvest?
For volume_total_ops vs qos_ops, #1062049169520476220 channel may help.

[flint basin]

Yes, to exclude workloads with policy User-Best-Effort from harvest, not from prometheus metric

[glass helm]

Okay, that is confusing. If we exclude this policy from Harvest, it will also not be shown in Prometheus. Essentially, a query like qos_ops{policy_group="User-Best-Effort"} should not return anything, right?

[flint basin]

Yes, I think we should exclude all the system-defined policy group from harvest
For example, these policy group are not shown in cmd "qos policy-group show"

[glass helm]

Ok, We can do that with template change. Harvest supports workload filtering as mentioned here
https://netapp.github.io/harvest/24.02/configure-zapi/#workload-templates

[flint basin]

To edit workload.yaml like this

[glass helm]

So, there are two workload templates:

• workload.yaml collects workloads with the classes system-defined and user-defined.
• workload_volume.yaml collects the autovolume class.

None of the autovolume or system-defined classes are visible in the qos policy-group show command. In my setup, all autovolume class workloads have the policy group set to User-Best-Effort.

You may make changes as mentioned here for workload.yaml to only collect user-defined ones

https://github.com/NetApp/harvest/discussions/2827

[glass helm]

If you want to exclude policy-group User-Best_Effort then filter looks like below which can be added to both workload.yaml and workload_volume.yaml with similiar steps as mnetioned here https://github.com/NetApp/harvest/discussions/2827

  - filter:
      - policy-group: "!User-Best-Effort"

[flint basin]

I dont't see "workload-class" as a supported fields in the document https://netapp.github.io/harvest/24.02/configure-zapi/#workload-templates

[glass helm]

Ok. It's a documentation gap. We'll fix. It is supported.

It will be added in docs via https://github.com/NetApp/harvest/pull/2828 for next release.

[flint basin]

After follwing the 2827 step,it excludes policy group like "_System-Work" and "_System-Best-Effort", but "User-Best-Effort" still exist

[glass helm]

That must be coming from workload_volume.yaml
you can add a filter in that template as below

  - filter:
      - policy-group: "!User-Best-Effort"

similiar steps as in 2827 but for workload_volume.yaml this time.

Added here
https://github.com/NetApp/harvest/discussions/2827#discussioncomment-9093072

[flint basin]

@glass helm Thanks for your great help!!
To sum up,
We select only user-defined policy-group by filtering in workload.yaml
And the autovolume which shows "User-Best-Effort" is filtered in workload_volume.yaml

[flint basin]

@glass helm We followed the steps in 2827, and still see User-Best-Effort policy group in metrics,
So do we need to add

• filter:
  • policy-group: "!User-Best-Effort"
    into workload_volume.yaml?