#Hi guys I ve just upgraded harvest 22 11

1 messages · Page 1 of 1 (latest)

strange spire
#

On the source it looks like this

image.png
#

and on the destination is this

image.png
hazy spoke
#

@strange spire as per your screenshot, it seems source cluster would have 5 relationships which are initiated from pvprxxxx10 cluster. Could you please confirm these : 1. are these flexgroup relationships or flexvols? 2. Mentioned 5 relationships are local or remote? and if remote, then all the destination clusters are being monitored by Harvest ?

#

we would not be exposing the flexgroup consituent relationships in snapmirror dashboard

strange spire
#

Hi @hazy spoke 1. currently we only use FelxVols
2.- the first screenshot is from the source (snapmirror list-destination) and the second screenshot is from the destination (backup system) and all cluster are being collected by the NAbox

#

what it also is worth to mention wo dont have any scheduler on the snapmirror since this will be done from the cleondris backup instance

hazy spoke
#

could you share the o/p of the command snapmirror show in destination cluster?

strange spire
#

can I send you the output via mail? so you see the whole paths

hazy spoke
#

sure, you can mail us at ng-harvest-files@netapp.com for these zapi response from both the clusters: snapmirror-get-iter and vserver-peer-get-iter

strange spire
#

also we are not allowed to send these data over discorder and a official netapp mail would be better 🙂

hazy spoke
#

yup, the above mail address is accessible by Harvest team only.

#

you can collect the zapi response via Harvest cli : bin/zapi -p POLLERNAME show data --api snapmirror-get-iter --max 10000

strange spire
#

Great thank you, about the ZAPI repsone, how will this be done in the nabox?

hazy spoke
#

for NABox, just a min

rancid peak
hazy spoke
#

You can run above CLI in NABox from root as shown example in screenshot

image.png
strange spire
#

Perfect thank you both

rancid peak
#

Shorter version : dc exec nabox-harvest2 bash

strange spire
#

Mail is out

hazy spoke
#

@strange spire I have replayed the zapi responses in Harvest, and validated the values in promQL and grafana. so far no issues identified from snapmirror perspective. Still 2 open item, 1 is sourceCluster var is empty which is join of snapmirror and volume. Could you share the volume-get-iter zapi response at the same email address ? 2 In your screenshot above, panels snapmirrors and volume protection shows value 2, which I don't see when I reply your responses. Can you click edit on that panel and copy paste the metric expression?

strange spire
#

Hi @hazy spoke you should recieve the mail in a few minutes

hazy spoke
#

Thanks, we got it.

I have replayed all zapi responses in local Harvest system and zapi responses are as expected.
If source_cluster var is not populated then whole dashboard would not be working because it's consumed in all sub sequent vars. It's strange that you could get destination node var properly.
Also, the snapmirrors and volume protection both the panels should show either 0 value(due to source_cluster missing) or 5(actual), but should not be 2.

We need to validate from prometheus side as well and need further detail to confirm.

Could you please run the below commands and share the harvest.yml.out and prom.out file to us:
bin/harvest doctor --print > harvest.yml.out
curl 'http://{{prometheus_url}:9090/api/v1/query?query=volume_labels' >> prom.out
curl 'http://{{prometheus_url}}:9090/api/v1/query?query=snapmirror_labels' >> prom.out

strange spire
#

Hi @hazy spoke short way how the curl can be done inside the nabox? 🙂

magic canyon
#

@strange spire login as root to NABox and try below commands for curl

curl -k 'https://NABOX_IP/prometheus/api/v1/query?query=volume_labels' >> volume.out
curl -k 'https://NABOX_IP/prometheus/api/v1/query?query=snapmirror_labels' >> snapmirror.out

strange spire
#

perfekt thanks, I will send the mail in the next few minutes

frail delta
#

Hi all, I hit the same issue. If we select all values in $datacenter or change it to “All”. Then we can see the snapmirror pairs and their status. Hope that it would be a hint to resolve it.

magic canyon
#

@frail delta Yes @mellow tartan has the solution for this issue and it should be available soon. And you are right it is related with datacenter filter.

frail delta
#

@magic canyon Thanks for prompt response. We are looking forward to the available solution.

hazy spoke
#

@strange spire @frail delta , We have detected the issue yesterday after evaluating Falcon667's prometheus response. SnapMirror is different kind of object and data only would be available from destination side and Harvest plugin would able to manage to get source side detail via vserver-peer and add those source labels in it. But it still populated from destination side, so datacenter would be showing that. Our earlier assumption was source and destination both the cluster would be in same datacenter but it's not the case always. I have recreated the issue locally and then tested the fix available in PR: https://github.com/NetApp/harvest/pull/1546.

#

If you would like to test the dashboard changes, You can copy from above PR and import it in your grafana.

magic canyon