#Best way to alert on HA pair going down?

1 messages · Page 1 of 1 (latest)

maiden jewel
#

Hi all,

Before I started a deep dive on this, I thought I'd see if anyone else had been down this road before. Pretty simple ask on the surface, customer just wants to know the quickest way to get pinged in Prometheus when an HA pair is down.

Thanks in advance

jovial estuary
fathom linden
#

AIQUM supports EMS alerts.

ancient geode
#

Harvest too. We just don't know which event corresponds to HA pair down

jovial estuary
cosmic drum
#

As a customer who has been doing this for years, my recommendation would be to run AIQUM alongside nabox/harvest

#

AIQUM has nearly all the alerting capabilities you should ever require. There have been times where we've created out own rule set for our own prom/alert manager stack such as RX/TX light levels for SFP's where we've requested enhancements to AIQUM

#

We typically alert from AIQUM for hardware/software specifics, and then harvest becomes our performance alerts

ancient geode
#

thanks for sharing your experience @cosmic drum I'm curious if you've tried using Harvest's EMS collector? Maybe you haven't since AIQUM meets your needs? (which is totally fine. great even). The Harvest team is interested in any feedback you have if you used the EMS collector. Especially if you hit any gaps

maiden jewel
maiden jewel
maiden jewel
jovial estuary
#

@maiden jewel I have taken down 1 node on a 2-node cluster. We may be able to detect HA down if Takeover Possible is false or - for all?

image.png
maiden jewel
jovial estuary
#

Yes same here!

maiden jewel
#

@jovial estuary what have you found so far? My week has gotten away from me so I haven't done much yet

jovial estuary
ancient geode
maiden jewel
#

I forgot I posted there lol

cosmic drum
jovial estuary