silence alerts?

[molten robin]

What's the best way to silence alerts generated by the alerting rules that are defined in the alert_rules.yml and ems_alert_rules.yml? I can silence notifications on custom rules that I've added myself to grafana either from the silence link in the alert or from the More button and silence notifications but neither of these seem to be available for the default defined alerts.

[fallow terrace]

I think grafana is integrated into prom/vm advertized alerts and has the ability to silence them. I think the alternative would be to silence in alert manager. Now, there might be a bug in 4.1 with regards to access to alertmanager

[wraith nexus]

hello guys, i've the same issue and i'm going crazy, how can I silence default defined alerts?
for example Lun Destroyed?
I've also tried to add mute timings under notification policies with the same result :\

NAbox v4.0.10 (f9baaca)

Harvest 25.02.0

Grafana 11.5.2

Victoria Metrics 2.24.0

thanks!

[gloomy osprey]

@fallow terrace

[fallow terrace]

That part is really WIP and there is not a lot of flexibility. The next version will add access to alertmanager, I'll check if that can help to silence individual alerts, for now I think the /am/ url isn't effective

Do you need some alerts ? You could always disable smtp

[wraith nexus]

Ciao Yann! Yes I need alerts :\ for the moment I've moved it in the clutter folder otherwyse it's impossible to work 😄 and I check sometimes if there is some new alert, because, as you know, the alerts are incorporated in the same email and then I cannot exclude just 1 alert :
Then the only way it's to wait for the upgrade right? There is no way to change something in victoria or similar?

Thanks 🙂

[fallow terrace]

You should be able to edit /etc/nabox/vmalert/*_rules.yml

[wraith nexus]

I don't have it, I've
/usr/share/nabox/vmalert
but here I've just 1 alert

root@empl-nabox01 /usr/share/nabox/vmalert # cat nabox_alert_rules.yml
groups:

• name: NAbox Rules
  rules:
  • alert: CapacityLow
    expr: (node_filesystem_avail_bytes{mountpoint=~"(/|/run|/data|/tmp)"}/node_filesystem_size_bytes{mountpoint=~"(/|/run|/data|/tmp)"}) < 0.15
    for: 5m
    labels:
    severity: "critical"
    annotations:
    summary: "Mountpoint [{{ $labels.mountpoint }}] low capacity"
    description: "[{{ $labels.mountpoint }}] has less than 15% available."

but I see that I've /etc/nabox/alertmanager/alertmanager.yml
maybe I can try to add it the mute notifications here?

Something like this:
route:
group_by:
- cluster
receiver: email
routes:
- matchers:
- alertname = "Snapmirror lag time is high"
mute_time_intervals:
- Forever

    - matchers:
        - alertname = "LUN Destroyed"
      mute_time_intervals:
        - Forever

time_intervals:

• name: Forever
  time_intervals:
  • times:
    • start_time: "00:00"
      end_time: "23:59"
      weekdays:
    • monday
    • tuesday
    • wednesday
    • thursday
    • friday
    • saturday
    • sunday
      location: Europe/Rome

Thanks!

[fallow terrace]

you would be willing to give 4.1.1 a shot before I publish it ? It should give you the possibility to manages silences

[fallow terrace]

4.1.1b1 is available with the improvement : https://nabox.org/downloads/

[wraith nexus]

if you need an help i can take a look but i'll place in a new vm

i've tested the WA that i've sent you and it works 🙂

[molten robin]

This is great to hear, I went the remove the snmp server route since I was getting inundated. I'll have to get upgraded and test it again.