topaz venture
As per title: what are some data monitoring tools we can use pecifically for Alibaba cloud?
Currently we are using springboot scheduler to pull data on a monthly basis (monthly rainfall) and writing the values to our mySQL DB in a single table. I asked chatGPT and it gave me the following options:
Great Expectations: An open-source library for data validation, documentation, and profiling. It allows you to set expectations about your data and monitor changes over time.
Deequ: An open-source library by Amazon for data quality assessment and monitoring. It helps you define and verify data quality constraints.
Apache Griffin: A data quality solution that supports data quality measurement, monitoring, and reporting.
Databand: This tool helps monitor, troubleshoot, and optimize data pipelines. It provides visibility into data lineage, dependencies, and transformations.
DataDog: While commonly used for infrastructure monitoring, DataDog also offers features for monitoring data pipelines and workflows.
Apache NiFi: Primarily used for data integration, Apache NiFi can also be employed to monitor and manage data flows across various sources and sinks.
Presto: A distributed SQL query engine that can be utilized for monitoring and analyzing data across different data sources.
Grafana: As mentioned earlier, Grafana can be configured to visualize and monitor data metrics, making it suitable for data monitoring as well.
Are all of these tools applicable to a sql table setup that we have? Alternatively, what might fit better?
I might have the framing of this question entirely wrong, let me know how I can provide more information.
dire warren