Enabling FlexGroup collection | NetApp | Page 1

median prawn Aug 5, 2024, 6:49 PM

#

I'm trying to find the harvest directory in NABOX, but the Github instructions appear to be for version 3, not 4. Can anyone point me to the location in NABOX 4?

hollow light Aug 5, 2024, 6:50 PM

#

hi @median prawn see if https://nabox.org/faq/#customize-harvest helps

median prawn Aug 5, 2024, 6:51 PM

#

🙂 Different location. Thank you

#

@hollow light I found one volume.yaml. The note in the dashboard implies there may be more? And I presume it will take some time before I see any data

#

It looks like it is starting to populate. I see some members...

#

Thank you

hollow light Aug 5, 2024, 7:13 PM

#

great! There are a total of four templates: zapi, zapiperf, rest, restperf

median prawn Aug 5, 2024, 7:14 PM

#

Ah...

grave eagle Feb 6, 2025, 4:23 PM

#

I know this topic is old, but I am also struggling getting the FlexGroup working.. On my nabox I only have /etc/nabox/harvest/user/zapi, and not rest etc.. do I have to create these manually? also.. inside the zapi folder there is a cdot folder where I have placed the custom.yaml that has the custom_volume_flexgroup.yaml reference, which is then again located in /etc/nabox/..../zapi/cdot/9.8/ even though we are running 9.16.1 😉 I'm sorry but why is this so complex? 😉 and what am I missing?

#

...it would be very cool if there was a "config" option inside the nabox GUI where you could enable these features... FlexGroup and CIFS Sessions etc.. 🙂

hollow light Feb 6, 2025, 4:32 PM

#

hi @grave eagle sorry that you're having trouble. Are you wanting to enable flexgroup constituents for Zapi or Rest?

#

And what version of NAbox and Harvest are you running?

grave eagle Feb 6, 2025, 4:39 PM

#

9.0.8 and 24.11.1

#

The configuration is set to "auto" 😉 so I'm not sure which to use... I have now created all 4 like described here: https://github.com/NetApp/harvest/wiki/Enabling-FlexGroup-Constituents#extend-zapi-volume-template-1

GitHub

Enabling FlexGroup Constituents

Open-metrics endpoint for ONTAP and StorageGRID. Contribute to NetApp/harvest development by creating an account on GitHub.

#

but not sure how long to wait for data in grafana?

hollow light Feb 6, 2025, 4:41 PM

#

5 to 10 minutes should be sufficient

grave eagle Feb 6, 2025, 4:42 PM

#

I have subsistuted: /opt/harvest2-conf/conf/ with /etc/nabox/harvest/user/

hollow light Feb 6, 2025, 4:42 PM

#

yes, that page was written for nabox3, not nabox4. Nabox 4 changed the paths

grave eagle Feb 6, 2025, 4:43 PM

#

ok lets give it 30 mins and lets see 🙂

hollow light Feb 6, 2025, 4:44 PM

#

sounds good. i'll look for an nabox4 system in the meantime

grave eagle Feb 6, 2025, 5:07 PM

#

still nothing..

hollow light Feb 6, 2025, 5:10 PM

#

which is to say the flexgroup dashboard constituent drop down is empty?

#

Can you grab a support bundle and upload to https://upload.nabox.org/yypy-wymu-sefu

grave eagle Feb 6, 2025, 5:12 PM

#

its showin "no data"

#

bundle uploaded

hollow light Feb 6, 2025, 5:13 PM

#

thanks!

hollow light Feb 6, 2025, 5:57 PM

#

ok, what you have is close. Your harvest.yml has the following list of collectors (see screenshot below), which means that Rest and RestPerf will be used before Zapi and ZapiPerf so let's focus on Rest and RestPerf. I see that you created the custom_volume_flexgroup.yaml in the correct place, but you haven't told Harvest to use those files.

For REST, create a file like so: sudo vim /etc/nabox/harvest/user/rest/custom.yaml
and it's contents should be

objects:
  Volume: custom_volume_flexgroup.yaml

Same for RestPerf, create a file like so: sudo vim /etc/nabox/harvest/user/restperf/custom.yaml with the same yaml contents as above.

After making those changes: dc restart havrest and then if you want to check that the changes were picked up faster, run this:
docker logs havrest --since 20m 2>&1 | grep -E 'custom_volume_flexgroup'

You should see something like

time=2025-02-06T17:42:36.823Z level=INFO source=helpers.go:98 msg="best-fit template" Poller=vsim collector=Rest:Volume path=/harvest-conf/active/rest/9.12.0/custom_volume_flexgroup.yaml v=9.14.1 jitter=none
time=2025-02-06T17:42:36.825Z level=INFO source=helpers.go:98 msg="best-fit template" Poller=vsim collector=RestPerf:Volume path=/harvest-conf/active/restperf/9.12.0/custom_volume_flexgroup.yaml v=9.14.1 jitter=none

where you can see that the best-fit algorithm picked the custom_volume_flexgroup.yaml file instead of the out-of-the-box template

grave eagle Feb 6, 2025, 6:04 PM

#

nabox4 /etc/nabox/harvest/user/restperf # docker logs havrest --since 20m 2>&1 | grep -E 'custom_volume_flexgroup' time=2025-02-06T18:02:37.581Z level=WARN source=poller.go:828 msg="init collector-object" Poller=s01 error="no best-fit template for custom_volume_flexgroup.yaml on confPath ../../../data/packages/harvest/conf:../harvest/conf:active %!w(<nil>)" collector=Rest object=Volume time=2025-02-06T18:02:37.581Z level=WARN source=poller.go:828 msg="init collector-object" Poller=s01 error="no best-fit template for custom_volume_flexgroup.yaml on confPath ../../../data/packages/harvest/conf:../harvest/conf:active %!w(<nil>)" collector=RestPerf object=Volume

#

So just to be clear I need to have both the custom.yaml and the custom_volume_flexgroup.yaml in the same folder?

hollow light Feb 6, 2025, 6:07 PM

#

custom.yaml should be in /etc/nabox/harvest/user/restperf/ and /etc/nabox/harvest/user/rest/ while custom_volume_flexgroup.yaml will be in a versioned directory, so like this:

grave eagle Feb 6, 2025, 6:09 PM

#

hmm so the "cdot" folder isn't needed?

hollow light Feb 6, 2025, 6:10 PM

#

cdot is only needed for zapi/zapiperf to differentiate 7-mode and cdot. rest and restperf only work with cdot so it was dropped

#

everything you have is correct, you're just missing the two custom.yaml files

grave eagle Feb 6, 2025, 6:14 PM

#

ok much better without the cdot..

#

got some graphs as well now..

#

thanks for the help 🙂 a toggle on the GUI woud have been nice 😉

hollow light Feb 6, 2025, 6:15 PM

#

your welcome. The Harvest team doesn't work on NAbox, but I'll pass that along to @young fossil

grave eagle Feb 6, 2025, 6:16 PM

#

ah OK I thought you were all in it together 😉 maybe NetApp should consider "buying" the Nabox..

hollow light Feb 6, 2025, 6:17 PM

#

🙂 we talk and we're all NetAppers but Conway's Law will always win

Conway's law

Conway's law describes the link between communication structure of organizations and the systems they design. It is named after the computer programmer Melvin Conway, who introduced the idea in 1967. His original wording was:

[O]rganizations which design systems (in the broad sense used here) are constrained to produce designs which are copies ...

grave eagle Feb 6, 2025, 6:21 PM

#

Just see what happened to Snap Creator... seems like the development kinda died... I think this was also a development from "outsiders" which was merged into NetApp?

hollow light Feb 6, 2025, 6:24 PM

#

I've not used Snap Creator before, but looks like that's a NetApp technology https://kb.netapp.com/Legacy/Snap_Creator/What_is_Snap_Creator

NetApp Knowledge Base

What is Snap Creator?

grave eagle Feb 6, 2025, 6:26 PM

#

Pretty sure some Germans started it... I might be wrong 😉

#

PS: Iave almost all graphs in the FlexGroup dashboard, but "Top 5 Constituents by Average Throughput" still reports "No Data"... and I put a load on the volumes which I can see on the other graphs... am I missing something?

#

...also is it possible to remove the top banner "Important Information about FlexGroup dashboard" from the dash? !Update, I managed to edit the dash.. guess it will be updated with a harvest update? 😉

hollow light Feb 6, 2025, 6:44 PM

#

try importing the latest version of the flexgroup dashboard - we made a change Nov 22 that you won't have yet (it will be included in the 25.02 release) https://github.com/NetApp/harvest/blob/main/grafana/dashboards/cmode/flexgroup.json

Paste the text from GitHub into the "Import via dashboard JSON model" textbox and press Load

GitHub

harvest/grafana/dashboards/cmode/flexgroup.json at main · NetApp/ha...

Open-metrics endpoint for ONTAP and StorageGRID. Contribute to NetApp/harvest development by creating an account on GitHub.

grave eagle Feb 6, 2025, 6:46 PM

#

nice, much better 🙂 thanks again

fierce bear Jun 23, 2025, 4:10 PM

#

@hollow light I know this is an old topic but I'm wondering if it's an all or nothing for Flex Group monitoring or can it be enabled for only specific aggregates or volumes?

hollow light Jun 23, 2025, 4:14 PM

#

Hi @fierce bear by default, flex group monitoring is on or off for all volumes. You should be able to add other filters to the filter section if you want to limit it

fierce bear Jun 23, 2025, 4:15 PM

#

This is still the latest link to use to enable monitoring for REST - https://github.com/NetApp/harvest/wiki/Enabling-FlexGroup-Constituents#extend-zapi-volume-template-1

GitHub

Enabling FlexGroup Constituents

Open-metrics endpoint for ONTAP and StorageGRID. Contribute to NetApp/harvest development by creating an account on GitHub.

hollow light Jun 23, 2025, 4:18 PM

#

that's correct

fierce bear Jun 23, 2025, 4:26 PM

#

Thank you sir, a big help as always!

fierce bear Aug 12, 2025, 8:21 PM

#

@hollow light The steps you gave to buller7929 are still correct for enabling FlexGroup monitoring? If so, when will this link be updated because none of those steps work. I thnk you mentioned those were menat for NAbox version 3? https://github.com/NetApp/harvest/discussions/3778

GitHub

Enabling FlexGroup Constituents · NetApp harvest · Discussion #3778

Harvest Guide for Enabling FlexGroup Constituents You must extend all the volume.yaml templates with the include_constituents flag to true to enable flexgroup constituents counters. Note: Enabling ...

#

@hollow light Also are we still pasting the contents of the link below into the custom_volume_flexgroup.yaml https://github.com/NetApp/harvest/blob/main/conf/restperf/9.12.0/volume.yaml#L3

GitHub

harvest/conf/restperf/9.12.0/volume.yaml at main · NetApp/harvest

Open-metrics endpoint for ONTAP and StorageGRID. Contribute to NetApp/harvest development by creating an account on GitHub.

grave flame Aug 13, 2025, 5:56 AM

#

@fierce bear If you scroll down in that page. You'll find NABox4 steps as well

fierce bear Aug 13, 2025, 5:56 PM

#

It seems the steps could be a little clearer. A lot to be interpreted and it seems a little out of order in steps.

#

Just an observation.

hollow light Aug 13, 2025, 8:37 PM

#

thanks for the feedback, @fierce bear we'll give it another look

vague socket Aug 14, 2025, 12:54 PM

#

@fierce bear , We have updated the doc: https://github.com/NetApp/harvest/discussions/3778, You can have a look when you get a chance.

GitHub

Enabling FlexGroup Constituents · NetApp harvest · Discussion #3778

Harvest Guide for Enabling FlexGroup Constituents You must extend all the volume.yaml templates with the include_constituents flag to true to enable flexgroup constituents counters. Note: Enabling ...

scarlet ferry Oct 2, 2025, 8:51 PM

#

Any plans on making this settings default in any future version so no one has to go edit those files for flexgroup constituent reporting

vague socket Oct 3, 2025, 7:45 AM

#

@scarlet ferry Enabling FlexGroup Constituents counters may slow down data collection due to a high number of constituents metrics. Currently, we do not have any plans to enable it by default. However, based on customer usage of these counters and requests for enablement, we might plan to enable it in the future.

potent widget Oct 3, 2025, 3:51 PM

#

hollow light `custom.yaml` should be in `/etc/nabox/harvest/user/restperf/` and `/etc/nabox/h...

Chris, I couldn't fine custom_volume_flexgroup.yaml in the version directories. Where is it located or what is the content of it? Thanks!

#Enabling FlexGroup collection