Need Help integrating NetApp Harvest with kube-prometheus

[hidden cypress]

Hi

I have setup Monitoring for my Kubernetes Cluster in a test environment using kube-prometheus repository.
https://computingforgeeks.com/setup-prometheus-and-grafana-on-kubernetes/
https://github.com/prometheus-operator/kube-prometheus

Grafana, Prometheus, AlertManager services are exposed via LoadBalancer.

I have a NetApp storage cluster which i want to monitor using Harvest and have installed Cloud Deployment of Harvest. However, i can't figure out how to get Harvest data in Prometheus and get the NetApp storage data in Dashboards.
https://github.com/NetApp/harvest/blob/3a9ebddcc2c20b2eaf2383978c4fc890660281ed/container/k8/README.md#cloud-deployment
The documentation in Github states "Networking between Harvest and Prometheus must be configured, and this can be accomplished by adding the network configuration in harvest-compose.yaml." However, can't find a sample harvest-compose.yaml that shows how is this done.

If i update "ntapc1" service to use LoadBalancer. How do i connect Harvest with Prometheus?

Any help is much appreciated.

Output of following is attached:
kubectl get svc -n monitoring
kubectl get pods -n monitoring
cat harvest.yaml
cat harvest-compose.yml

[dreamy aurora]

@hidden cypress You can find examples here
https://netapp.github.io/harvest/23.05/install/k8/

You would need to configure NetworkPolicy in k8 for pod connectivity.

[hidden cypress]

Creating a Service Monitor worked for me

[dreamy aurora]

Nice!

[hidden cypress]

Thank .. however, i am seeing an issue where the logs from the Pod "netapp-cluster" states .. "connection refused" Poller ...
the polling works for some time and i see Grafana charts being populated .. and then the connection fails to establish

the storage system is up and i can connect to it fine. When i reboot the kubernetes cluster and storage simulator .. the polling starts and then fails after some time.
kubectl logs ntapc1-65578fd799-fc66r --all-containers -n monitoring

[dreamy aurora]

@hidden cypress Could you please share full logs with us via [e-mail]?

1. How many pollers are experiencing this issue?
2. Have you tried to connect to the cluster when you encounter these logs in Harvest to rule out if cluster is having any connectivity issue?

I noticed that you are currently using a storage simulator. To better diagnose the issue, I would recommend testing against a real cluster to see if the problem persists.