Hello

[plush mortar]

I need to find/create similar graphs in NAbox as are in Acive IQ Unified manager. In particular
Node - latency (read/write)
Node - MB/s (read/write)
Node - IOPS (read/write/other)

Aggregate - latency (read/write)
Aggregate - MB/s (read/write)
Aggregate - IOPS (read/write/other)
Aggregate - utilization

I browsed the counters a lot but I can't find anything like this.
Thank you very much in advance

[spare bear]

Hi @plush mortar I'm not familiar with which ONTAP counters AIQ.UM is showing in those views. Can you share the counters and we can help you find the same in Harvest.

Also take a look at Harvest's ONTAP: Node and ONTAP: Aggregate dashboards.

[plush mortar]

I can get may be the counters that exists but I didn't find what counters are behind the graphs. But basicaly if we take for example the first what I need - Node - latency (read/write). What Harvest counters would you use for this? I can't find any suitable... Thank you

[spare bear]

Checkout the Node dashboard. You can see which counters are used for any panel by hovering over the panel and pressing e . For example, hovering over the Backend > Latency panel and pressing e shows the following query

[plush mortar]

Yes, thank you but this is what brings even more confusion to the topic. Why counters volume_* are for backend? I would expect counters aggr_* to be for backend.

If volume counter is for backend what counter is then for node frontend?

[spare bear]

Do you have Harvest installed or are you reading through the docs and trying to match with AIQ.UM? The frontend counters and panels are on that same dashboard.

Which aggregate counter are you interested in? There are four: headroom_aggregate, wafl_comp_aggr_bin, wafl_comp_aggr_vol_bin, and wafl_hya_per_aggregate. Harvest collects all of those except for wafl_comp_aggr_bin

[plush mortar]

I have NAbox in production running with real data. I know this dasboards quite well. I'm just confused with the use of counters. Volume counters are used in Backend. But also in Frontend. The description of counters in Harvest doc is very bad I don't know what the description wants to tell me. I don't see clear distinction what we measure on backend and what on frontend. Also, honestly, if we talk about backend latency, for example, where do we start to measure and where we end? What is the trip of the datablock that we measure on the backend and what we measure on frontend. I'm trying to undestatnd this for years (intermittently) but I always end up confused, disappointed and annoyed and I give up.

[spare bear]

I understand your frustration. Maybe @distant dew can help with some of your ONTAP counter questions or knows someone who can. The volume counters used in the backend row of the Node dashboard are volume counters aggregated across node(s).

Do you have specific documentation feedback that we can improve? Are you talking about the hover text for dashboard panels or the documentation at https://netapp.github.io/harvest/nightly/ontap-metrics or both? We're happy to improve all of it, just need to know what you find confusing. Some parts of the documentation are taken from ONTAP's counter documentation which is authoritative, but perhaps confusing. Can you share specifics we can discuss?

[distant dew]

There are two objects in ONTAP for measuring volume latency. The workload object is the same data used in qos statistics.

The QoS subsystem measures data from the nansecond it enters the nblade to the nanosecond it exits the nblade. For SAN workloads, it includes the RTT of the SCSI start to finish. For S3, it can include the time between requests for large reads/writes (not FabricPool, but ONTAP S3 server).

Volume objects only measure the time from when the IOP is started to be processed and is finished. It doesn't include any nblade delays such as vscan/fpolicy/rewinds.

Frontend = nblade/network/Network processing layer, backend=dblade/WAFL/Data Processing/data layer

Confusing terms for sure.

[spare bear]

thanks Paul

[distant dew]

Yw

[solemn dirge]

@plush mortar You can find these metrics in Harvest, as documented in the following link: Harvest ONTAP Metrics. These metrics are available in the Volume Dashboard, as shown in the attached screenshot. They are also accessible in the Workload dashboards. Please note that these metrics are disabled by default.

[plush mortar]

Thank you very much for the help. I will look into it and come back with more questions, I guess. (This is a keepalive reply to let you know I'm working on it and that I really appreciate you fast and helpful reaction.)

[primal glade]

@distant dew thanks for the insight. Is there like a light weight documentation about d and nblade to get a better understanding?

[plush mortar]

this is DB table with counters for Active IQ Unified Manager. I think that they must pull the same data as Harvest to fill this table. But it is not clear from the table what netapp counters are used to fill this table. It would be perfect to have a guy who knows Active IQ UM in detail and could help us map the correct counters to this table and then we could pair it with Harvest....

[solemn dirge]

Several of these look similiar to Harvest template https://github.com/NetApp/harvest/blob/main/conf/zapiperf/cdot/9.8.0/system_node.yaml

[plush mortar]

Now I use graphs from AciveID UM and there are graphs like Aggregate latency ,MB/s, IOPS ... Don't know hat counters are behind this.

[distant dew]

Workload.

[distant dew]

I think they don't let the design of AIQUM out, so it may not be available to release publicly. @gilded palm do you know?

[distant dew]

I don't think we have much, but it's just the difference in layers of handling.

[heady bronze]

In general, we don't know the exact metrics we use. anything UM grabs that is perf related comes from the ccma files, which are parsed through code.

[gilded palm]

The exact metrics used are not exposed in either the UI or the documentation. Certain values may be reported as is from Ontap while others such as Utilization, Performance Capacity Used, Takeover planning, etc are calculated based on multiple metrics.