All geek questions in one place

Monitoring log files using some exporter metrics + Prometheus + Grafana

I need to track very different log files for errors, success status, etc. And I need to capture the relevant indicators using Prometheus and show in Grafana + set some warnings on it. Prometheus + Grafana is fine. I already use them a lot with different exporters like node_exporter or mysql_exporter etc. Also a warning in the new Grafana 4.x works very well.

But I have a rather difficult task to find a suitable exporting program / program that could analyze the log files on the fly and extract metrics from them.

So far I have tried:

  • mtail ( https://github.com/google/mtail ) - works, but the existing version cannot easily control more files - in general, it cannot link a specific mtail program (receive for analysis) to some specific log file + I don’t I can easily add the log file name to the tag
  • grok_exporter ( https://github.com/fstab/grok_exporter ) - it works, but I can extract only limited information + one instance can control only one log file, which means that I will have to run more copies, exporting to more ports and tuning them in prometheus, making too many new points of failure
  • fluentd prometheus exporter ( https://github.com/kazegusuri/fluent-plugin-prometheus ) - works, but it looks like I can extract only very simple indicators, and I can not do any advanced analysis of regular expressions of the string (s ) from the log file

Does anyone really have a running solution for monitoring advanced indicators from log files using "some exporter" + "Prometheus + Grafan"? Or instead of an exporter, some program from which I could get results using the Prometheus gateway. Thanks.

+10
source share
2 answers

Look at the Telegraph . It supports tail logs using the logparser and tail input plugins. To export metrics as a Prometheus endpoint, use the prometheus_client output plugin. You can also apply for aggregations on the fly. I found it easier to configure multiple log files than grok_exporter or mtail

+6
source

These are 3 answers at the moment for getting log data in Prometheus.

You can also see how magazines are created to directly show Prometheus metrics.

+1
source

Source: https://habr.com/ru/post/1261315/

More articles:


All Articles