Volume QoS Resource Latency Graphs | NetApp | Page 1

rotund compass Jan 21, 2026, 6:58 AM

#

Hi,

I'm using a OVA deployment for our nabox. I would like to see some volume qos details. (ONTAP: Volume Deep Dive Dashboard -> QoS Resource Latencies)

I know I need to activate this, what I did (hopefully):

un-commented:
Workload: workload.yaml
WorkloadVolume: workload_volume.yaml

In
/data/packages/harvest/conf/zapiperf/default.yaml
/data/packages/harvest/conf/restperf/default.yaml

and run docker compose up -d --force-recreate

Anything else I need to activate to get the QoS Resource Latencies dashboard working?

amber ether Jan 21, 2026, 7:00 AM

#

Hi. This will be overwritten with harvest upgrade. You should enable it in NAbox preferences instead.

rotund compass Jan 21, 2026, 7:02 AM

#

Hi, thanks! Can you point me to the right place? What do you mean with NAbox preferences?

#

Not sure if you mean this: (was already enabled)

rotund compass Jan 21, 2026, 7:30 AM

#

Ok, now I'm a bit confused. I have duplIcated dashboards since the last update. (I know there is a bug)
In my duplicated dashboard I see "QoS Resource Latencies" but on the 'correct' dashboard it's not even there.

pliant moon Jan 21, 2026, 8:00 AM

#

@rotund compass Qos resource latency metrics are removed since Harvest 25.02. More details are available https://github.com/NetApp/harvest/issues/3423

Templates
Workload: workload.yaml
WorkloadVolume: workload_volume.yaml

are enabled by default in harvest if you are using Harvest 25.02 or higher.

rotund compass Jan 21, 2026, 1:45 PM

#

what is the correct way to get the "ONTAP: Workload" Dashboard working? Currently I have no data there. What file I need to modify to activate the counter & also have it not overwritter by an update? ("This dashboard requires Workload/QOS counter templates.")

Enable workload collection in preferences is active (nabox)

pliant moon Jan 21, 2026, 1:46 PM

#

What is your current Harvest version?

Workload: workload.yaml
WorkloadVolume: workload_volume.yaml

These templates must be enabled. Could you share NABox logs @ https://upload.nabox.org/wowi-hetu-yyzy ?

rotund compass Jan 21, 2026, 1:54 PM

#

NAbox v4.1.0 (963b3e1)
Harvest 25.11.0
Grafana 12.2.1
Victoria Metrics 2.24.0

pliant moon Jan 21, 2026, 1:55 PM

#

That should be good. We'll know more from logs.

rotund compass Jan 21, 2026, 1:56 PM

#

I mean, what is the correct file to modify? 🙂

#

this is the content of: /etc/nabox/harvest/nabox/collect-workloads/zapiperf/custom.yaml

objects:
Workload: workload.yaml
WorkloadDetail: workload_detail.yaml
WorkloadVolume: workload_volume.yaml
WorkloadDetailVolume: workload_detail_volume.yaml

Is this generelly the correct place for this settings?

pliant moon Jan 21, 2026, 1:59 PM

#

I will need to check. @amber ether Do you recall this folder?

pliant moon Jan 21, 2026, 1:59 PM

#

rotund compass I mean, what is the correct file to modify? 🙂

You shouldn't need modification anymore. If you can share nabox support bundle. We can check.

rotund compass Jan 21, 2026, 2:06 PM

#

@pliant moon logs are uploaded

pliant moon Jan 21, 2026, 2:20 PM

#

Data collection seem fine w.r.t to workloads. If you don't have any customisations in dashboard then can you delete all dashboards. NABox should create new ones and then see if it is a dashboard related issue?

rotund compass Jan 21, 2026, 2:26 PM

#

I deleted the grafana db already today, as I had duplicated dashboards after update.

pliant moon Jan 21, 2026, 2:29 PM

#

Okay. If you check ONTAP:Workload dashboard, Do you see data in all dropdowns?

#

#

Similiary for other dropdowns.

rotund compass Jan 21, 2026, 2:32 PM

#

No, every panel is "no data"

I can see data in "workload"

#

also in datacenter, cluster, ...

pliant moon Jan 21, 2026, 2:32 PM

#

What is this 422 error? Could you share full detail?

rotund compass Jan 21, 2026, 2:32 PM

#

Thats the error in every panel

#

pliant moon Jan 21, 2026, 2:33 PM

#

Data not present in datacenter and cluster points to something different error.

#

Could you paste full error?

#

Let's try selecting top 5 records instead of 500.

rotund compass Jan 21, 2026, 2:34 PM

#

It's only visible while hover... so here is a screenshot with more info: (will sent you in PM)

pliant moon Jan 21, 2026, 2:36 PM

#

Ok. Have you tried selecting top 5 from download and see if is still same error?

rotund compass Jan 21, 2026, 2:37 PM

#

No change with 5

#

But, I can see some date when I select a specific volume

amber ether Jan 21, 2026, 2:38 PM

#

pliant moon I will need to check. <@323825973718155265> Do you recall this folder?

Still need me ?

pliant moon Jan 21, 2026, 2:38 PM

#

amber ether Still need me ?

We are past that now. 🙂

amber ether Jan 21, 2026, 2:39 PM

#

Sorry !

rotund compass Jan 21, 2026, 2:39 PM

#

I can see some data when I select a volume:

pliant moon Jan 21, 2026, 2:40 PM

#

If you edit dashboard and edit workload variable and add custom all value as .* ?

rotund compass Jan 21, 2026, 2:40 PM

#

Still a generell question: What path / file I can modify to keep the changes after update? For example if I need qtree data @amber ether

pliant moon Jan 21, 2026, 2:41 PM

#

You would need to customize NABox like this
https://nabox.org/faq/#customize-harvest

#

https://github.com/NetApp/harvest/discussions/3446

rotund compass Jan 21, 2026, 2:42 PM

#

alright, thanks. I'll have a look at this tomorrow 🙂

rotund compass Jan 21, 2026, 2:45 PM

#

pliant moon If you edit dashboard and edit workload variable and add custom all value as `.*...

I'll check this later. Looks like I can't save it from WebUI

This dashboard cannot be saved from the Grafana UI because it has been provisioned from another source. Copy the JSON or save it to a file below, then you can update your dashboard in the provisioning source.

pliant moon Jan 21, 2026, 2:46 PM

#

NABox probably have it disabled. Yes, You can import the dashboard as a different name and check.

#

It seems related to UI only.

pliant moon Jan 22, 2026, 6:30 AM

#

@rotund compass Unrelated to this issue: when you were running Harvest version 25.08, there were errors like the ones below in the logs. They shouldn’t impact any Harvest features, but I’d like to check whether we need to fix something in the code to avoid these errors.

Jan 21 06:33:43 de9899sa6 dc[3907]: havrest           | time=2026-01-21T06:33:43.385Z level=ERROR source=volume.go:92 msg="Failed to create new instance" Poller=xxxx plugin=RestPerf:Volume object=Volume error="duplicate instance key

Could you please share output of below curl command. You can upload to same location https://upload.nabox.org/wowi-hetu-yyzy . `Please replace USERNAME,PASSWORD,CLUSTER_IP as applicable.

curl -sk -u USER:PASSWORD 'https://CLUSTER_IP/api/cluster/counter/tables/volume/rows?fields=*'

rotund compass Jan 22, 2026, 7:00 AM

#

Sure, I uploaded the output 🙂

#

by the way, since the last update to latest release I have a failed systemd service:
Failed Units: 1
issuegen.service

Is this something known?

pliant moon Jan 22, 2026, 7:01 AM

#

@amber ether

rotund compass Jan 22, 2026, 7:02 AM

#

here is the journalctl output:

Jan 21 07:03:23 server1 systemd[1]: Starting issuegen.service - Generate /run/issue...
Jan 21 07:03:23 server1 cp[706888]: cp: cannot stat '/usr/share/nabox/issue/webui': No such file or directory
Jan 21 07:03:23 server1 systemd[1]: issuegen.service: Control process exited, code=exited, status=1/FAILURE
Jan 21 07:03:23 server1 systemd[1]: issuegen.service: Failed with result 'exit-code'.
Jan 21 07:03:23 server1 systemd[1]: Failed to start issuegen.service - Generate /run/issue.

pliant moon Jan 22, 2026, 7:08 AM

#

rotund compass Sure, I uploaded the output 🙂

Did you make any template customizations in your environment during Harvest 25.08.1 related to volume performance?

rotund compass Jan 22, 2026, 7:15 AM

#

As far as I know I only edited the following files:

/etc/nabox/.env.custom
/data/packages/harvest/container/prometheus/alert_rules.yml (AFAIK: will be overwritten after update)
/data/packages/harvest/container/prometheus/ems_alert_rules.yml (AFAIK: will be overwritten after update)

And I added a new dashboard. (for StorageGrid)
Everything else should be default.

pliant moon Jan 22, 2026, 7:18 AM

#

Could you also share output of https://CLUSTER_IP/api/private/cli/volume?fields=volume,vserver,volume_style_extended&is_constituent=*

rotund compass Jan 22, 2026, 7:20 AM

#

pliant moon Could you also share output of https://CLUSTER_IP/api/private/cli/volume?fields=...

done

pliant moon Jan 22, 2026, 7:20 AM

#

Thanks

amber ether Jan 22, 2026, 10:31 AM

#

rotund compass by the way, since the last update to latest release I have a failed systemd serv...

No that's fine, That's not the beta version right ?

rotund compass Jan 22, 2026, 12:33 PM

#

No, it's not.

NAbox v4.1.0 (963b3e1)
Harvest 25.11.0
Grafana 12.2.1
Victoria Metrics 2.24.0

amber ether Jan 22, 2026, 3:39 PM

#

Ok so yes that's been adressed for 4.1.1 that will be out soon

#Volume QoS Resource Latency Graphs