QoS scheduling | NetApp | Page 1

waxen herald Sep 1, 2022, 11:01 AM

#

Could you share logs? It could be due to workload Zapi call taking long time and impacting next poll.

sterile plinth Sep 1, 2022, 12:36 PM

#

Hey Rahul 🙂 Here's the log from nabox-harvest2 container and workload, workload-detail, zapiperf/default yaml files for you to take a look at if you can. I can see from the log that the zapiperf is being collected 4 times in a minute but I see the matrics(workload/qos_ops) are updating on the Grafana every minute though. Thanks for the help as always!

📎 nabox-harvest2_log.txt 📎 workload_detail.yaml.txt 📎 Workload.yaml.txt 📎 ZapiPerf_default.yaml.txt

#

https://github.com/NetApp/harvest/discussions/679
I have also tried this, but it didn't seem to affect the result.

GitHub

Need Information Regarding the scrape interval & collector schedule...

I need some information about the below use-case. If Prometheus servers are scraping Harvest targets (end-points) every minute, and the default zapi collectors have below schedule, how can I ensure...

quartz crest Sep 5, 2022, 10:21 AM

#

@sterile plinth As per the shared log file, It doesn't seem to have any issue relate to workload and workloadDetail templates. And yes, collection happens 4 times in minute. Just to make sure Prometheus scraps at 15s interval, run this query in Prometheus and count should be exactly 4. count_over_time(node_uptime[1m])

#

From Grafana perspective, Could you check what interval it would be showing in your panel? Based on the Data point and selected time range, Grafana will decide interval. Attaching screenshot for one panel just for reference.

sterile plinth Sep 6, 2022, 2:00 AM

#

Hi @quartz crest! I appreciate your comment. The interval on Grafana is set for 15 seconds but I see the counts are 1 instead of 4 in Prometheus. I feel like I am doing something wrong with Prometheus scraping. Would it be possible to share with me which config I should change to shorten the interval? I have tried adding scrape_interval on prometheus.yaml but it didn't work.

#

These are from
/opt/prometheus-scrapers/scrapers.yml
/opt/packages/harvest2/docker/prometheus/prometheus.yml

#

📎 scrapers.txt 📎 prometheus.txt

#

BTW I am using NAbox. Thanks for your help 🙂

waxen herald Sep 6, 2022, 5:18 AM

#

@sterile plinth As per your prometheus configuration screenshot, scarape_interval is 1m. Adding @night garnet for NABox related things

night garnet Sep 6, 2022, 2:41 PM

#

There is no way to customize scrape intervals in NAbox currently, give me some time to figure out how default interval is applied, I don't have scrape_interval anywhere in NAbox code, so I think it's just a default to 60s

#

Prom default is 1min indeed

#

so you can change it in /usr/local/nabox/files/prometheus/prometheus.yml and it should be persistent

#

Well, except during upgrades...

sterile plinth Sep 7, 2022, 1:01 AM

#

@night garnet Thank you so much! It's been applied to the prometheus.

#QoS scheduling