Querying Prometheus metrics collected via Netdata with PromQL (in Grafana)

[raw fog]

We have netdata running on each of our servers, and we saw that there was a great Prometheus plugin available. we loaded it up and saw that everything just works off the bat, all the way from the application emitting the metrics up to the netdata cloud and into the Grafana instance we use for building dashboards with the metrics.

The only issue is that we would like to be able to query the metrics collected as if we had collected them with Prometheus. Is that supported in the netdata cloud grafana plugin?

[granite trout]

Netdata data source interacts with the same APIs the Cloud UI uses and allows Grafana to query the data that way. We haven't yet made any of these Cloud APIs "public", though. I.e. even though the UIs are publicly accessible with the required token, we do not make any guarantees that the APIs are stable and don't provide documentation for them.

I should say "yet", because we do intend to do this eventually, but we have to pick what we work on and this hasn't been a priority so far. If there is sufficient demand for it, it may come higher on the list.

Can you expand a bit on your use case?

[raw fog]

I'm just wondering if it would be possible to somehow query the metrics scraped from an OpenMetrics endpoint by the netdata agent with PromQL. I'll give you an example:

application x exposes a counter metric called x_requests. a netdata agent scrapes that metric and streams it to netdata cloud. In grafana, I'd like to make a panel called "X requests rate" with the query rate(x_requests[5m]) or something.

My thinking is to have netdata cloud export metrics to a prometheus remote write compatible datasource, but I imagine that netdata will marshal that initial scrape data from prometheus format to netdata format back to prometheus, instead of just adding some labels to the original metric and calling it a day