changing retention days | NetApp | Page 1

foggy sluice Feb 6, 2024, 7:56 AM

#

Hi team

I have changed retention day to 90d , can you please let me know if it should look like this
I have attached screens shot

Thank you

raven veldt Feb 6, 2024, 8:26 AM

#

@foggy sluice Yes looks correct. You can verify the same from prometheus UI

foggy sluice Feb 6, 2024, 9:04 AM

#

I am getting EMS as inactive , how can I activate it I’m using ZAPI

raven veldt Feb 6, 2024, 9:09 AM

#

From screenshot, EMS alert rules are being checked. Do you have EMS collector enabled in your Harvest configuration?

foggy sluice Feb 6, 2024, 9:16 AM

#

It says to find harvest.yml file , I’m really getting difficulty in finding its location can you please help me

raven veldt Feb 6, 2024, 9:16 AM

#

Can you share your installation type? rpm, deb, native, container or nabox?

foggy sluice Feb 6, 2024, 9:19 AM

#

It’s NAbox and I have found it now

raven veldt Feb 6, 2024, 9:20 AM

#

okay. You can run below command to share it here (NABox as root user)

dc exec -w /conf nabox-harvest2 /netapp-harvest/bin/harvest doctor --print

#

Better to run above command as it will remove any PII information from your configuration.

#

Also if EMS collector is enabled then check if Rest permissions are configured as mentioned here https://netapp.github.io/harvest/nightly/prepare-cdot-clusters/#create-rest-role

foggy sluice Feb 6, 2024, 9:26 AM

#

It’s says - - print is unknown flag

#

My harvest yaml file looks like this

raven veldt Feb 6, 2024, 9:27 AM

#

Okay it looks good

#

EMS collector looks good

foggy sluice Feb 6, 2024, 9:27 AM

#

So why it says it’s not collecting

#

I will delete this picture

raven veldt Feb 6, 2024, 9:27 AM

#

okay

foggy sluice Feb 6, 2024, 9:28 AM

#

It says harvest ems alert is inactive in Prometheus

raven veldt Feb 6, 2024, 9:28 AM

#

From prometheus screenshot it seems active to me

#

do you mean this line?

foggy sluice Feb 6, 2024, 9:29 AM

#

Yes this one is having inactive in green

#

Like this

raven veldt Feb 6, 2024, 9:31 AM

#

Okay these are just tags. It means that currently all EMS events are inactive (none are firing) => means no condition is matched as per alert rules

#

#

Can you share your harvest logs @ ng-harvest-files@netapp.com
More details here https://netapp.github.io/harvest/nightly/help/log-collection/#nabox

foggy sluice Feb 6, 2024, 9:33 AM

#

Ok I will do so

raven veldt Feb 6, 2024, 9:33 AM

#

Can you check Rest permissions once if user has permissions to call REST API
https://netapp.github.io/harvest/nightly/prepare-cdot-clusters/#create-rest-role

foggy sluice Feb 6, 2024, 9:35 AM

#

Ok it has only access for Ontapi

#

I have created role for rest and docs-api

raven veldt Feb 6, 2024, 9:44 AM

#

okay let’s restart harvest

foggy sluice Feb 6, 2024, 9:45 AM

#

Ok I got this now

#

It shows in my panel

raven veldt Feb 6, 2024, 9:46 AM

#

nice!

foggy sluice Feb 6, 2024, 9:47 AM

#

I have one more question

raven veldt Feb 6, 2024, 9:48 AM

#

Yes tell me

#

Okay. I ll share documentation shortly

foggy sluice Feb 6, 2024, 9:50 AM

#

Thank you

#

Ok I have uncommented in default.yaml file

#

Cifs session and nfs session

#

And restarted my docker

#

Do I need to do more

#

But after I save it it’s still gets commented out

raven veldt Feb 6, 2024, 9:55 AM

#

You want to modify file /opt/harvest2-conf/conf/zapi/default.yaml But it will get overwritten post upgrade

#

I'll share steps on how you can extend these so that you won't have to modify default.yaml

foggy sluice Feb 6, 2024, 9:56 AM

#

Ok

raven veldt Feb 6, 2024, 10:04 AM

#

Added here
https://github.com/NetApp/harvest/discussions/2633

foggy sluice Feb 6, 2024, 10:19 AM

#

Thanks a lot Rahul I also want to know about NFS

#

Do I need to create same file

raven veldt Feb 6, 2024, 10:19 AM

#

Can you share what's the template name?

foggy sluice Feb 6, 2024, 10:23 AM

#

NFS Clients

raven veldt Feb 6, 2024, 10:29 AM

#

Added here https://github.com/NetApp/harvest/discussions/2635

#

Please note, Both of these templates may slow down data collection due to high number of metrics involved.

foggy sluice Feb 6, 2024, 10:30 AM

#

Thanks a lot Rahul , I will try to enable it and if any issue will bother you again , thanks a lot again

raven veldt Feb 6, 2024, 10:30 AM

#

sure np.

foggy sluice Feb 6, 2024, 1:03 PM

#

I am trying to setup emails , am I doing it in right way , can you please let me know

#

Im editing /usr/local/nabox/docker-compose/docker-compose.override.yml

raven veldt Feb 6, 2024, 1:20 PM

#

Looks alright to me as per NABox docs https://nabox.org/faq/#customize-grafana
@frozen cliff can add more on this.

frozen cliff Feb 6, 2024, 3:10 PM

#

Looks ok to be as well

foggy sluice Feb 6, 2024, 3:21 PM

#

Thank you guys but I’m getting error as -failed to send test alert failed to send notification to email address storage@project2.com:tls :failed to verify certificate:x509: certificate is valid for prdstorage:sap.ir not for mail.project2.com

frozen cliff Feb 6, 2024, 3:24 PM

#

try and add GF_SMTP_SKIP_VERIFY=true

weak cobalt Feb 6, 2024, 3:24 PM

#

sounds like a cert issue, try adding GF_SMTP_SKIP_VERIFY=true

#

jinks!

foggy sluice Feb 6, 2024, 3:33 PM

#

Thanks a lot it solved issues

foggy sluice Feb 6, 2024, 5:17 PM

#

My last question- some volumes are having used percentages above 90 percent, and severity = critical and state is in firing in promethus but I’m not getting alert mails for these volumes, (I have got test messages from grafana) , what more I need to check

weak cobalt Feb 6, 2024, 5:31 PM

#

the alerts you're seeing are from the Prometheus alert rules that Harvest ships with. https://github.com/NetApp/harvest/blob/main/container/prometheus/alert_rules.yml

Typically you would use Prometheus Alertmanager to manage, notify, silence, etc those events. https://prometheus.io/docs/alerting/latest/alertmanager/

Grafana doesn't know anything about those Prometheus alerts, but I believe you can configure Grafana to do something similar with https://grafana.com/docs/grafana/latest/alerting/ But to be clear, the Harvest team has not tried Grafana alerts so not sure what all is possible. This article from last 2022 compares the two https://alexandre-vazquez.com/grafana-alerting-vs-alert-manager/

foggy sluice Feb 6, 2024, 8:49 PM

#

Thanks @weak cobalt it worked

#

I have one more question- can we add other storage like isilon , object storage , VMax , thank you

frozen cliff Feb 6, 2024, 9:25 PM

#

Harvest does only deal with ONTAP and StorageGRID.
If you had other software that pulls metrics from other arrays and publishes prometheus compatible metrics we can talk about it, but the short answer is no 🙂

#changing retention days