Reinstalling Harvest and Grafana on a larger site (164 nodes and growing, 23k+ vols), global environment. Outgrew NAbox. Currently, want to ensure they won't outgrow this but with Docker I don't believe scaling out should be an issue. With Docker backend communication, coupled with their firewall, do I need to worry about poller issues being in different locations? (I've read the documentation and installed via lab but different environment 🙂 ). Thanks in advance!
#Harvest with Docker Global Install
shell flume
austere wave
How much resource did you give NAbox ?
shell flume
It was a very old installation, many years ago when they planned only 8-16 nodes.
Just checked my notes
obsidian vine
he's asking what resources the netapp install/appliance has, how many vcpu, how much memory, etc.
shell flume
I understand. I don't have that information outside of what they originally projected a few years ago when they installed. I can obtain those details.
austere wave
https://nabox.org/documentation/installation/#compute-resources this might be useful, if you do your own install it should be the same, those a loose guidances of course, just a starting point
shell flume
https://nabox.org/documentation/installation/#compute-resources this might be us...
I believe that was the sizing that the customer used when they implemented NAbox. Now we're installing with Docker due to environment size. Waiting for response from customer, they're located PDT.
austere wave
Yes I meant it should be similar, then you can spread on multiple hosts.
shell flume
With regards to pollers, I tried to do it in the lab but with Dockers back-end, do I need to worry about network issues with pollers across the globe?
austere wave
I’d be curious to know what you end up doing and if can help with nabox scalability.
shell flume
Can I email you the spreadsheet I have with what I'm trying to accomplish?
austere wave
With regards to pollers, I tried to do it in the lab but with Dockers back-end, ...
I would be careful yes, harvest does a lot of api calls which builds up latency, it’s better to have it close to your systems.
shell flume
Current issues with NAbox is severe latency between pollers and Grafana
shell flume
I would be careful yes, harvest does a lot of api calls which builds up latency,...
Definitely one of the issues we're trying to tackle!
austere wave
Can you be more specific ? Pollers talk to ONTAP systems and Prometheus pulls the metrics. Grafana only communicates with Prometheus.
austere wave
Can I email you the spreadsheet I have with what I'm trying to accomplish?
Sure yann.bizeul@netapp.com I’ll forward to the harvest folks
shell flume
Sure, yes. From what I've seen it appears to be a few issues: sizing with NAbox and outgrown environment. Not enough pollers located around the globe to properly pull the data and feed it into Grafana. Prometheus is what they're currently using and would like to stay with.
austere wave
Just to clarify : data is not fed to Grafana, they're fed in Prometheus. You need proximity between harvest and the systems. You can sustain more latency from prometheus to harvest as the data is streamed by batches, and you probably want grafana close to prometheus because it's doing lots of queries and round trips