How to calculate the use of the container processor in cubers with prometheus as monitoring?

Question

I want to calculate the processor usage by all modules in a cluster of quaternets. I found two metrics in prometheus that may be useful:

container_cpu_usage_seconds_total: Cumulative cpu time consumed per cpu in seconds.
process_cpu_seconds_total: Total user and system CPU time spent in seconds.

Cpu Usage of all pods = increment per second of sum(container_cpu_usage_seconds_total{id="/"})/increment per second of sum(process_cpu_seconds_total)

However, I found that every second increment is container_cpu_usage{id="/"}larger than the increment sum(process_cpu_seconds_total). Thus, the use can be more than 1 ...

Answer 1

This I use to get the CPU utilization at the cluster level:

sum (rate (container_cpu_usage_seconds_total{id="/"}[1m])) / sum (machine_cpu_cores) * 100

I also track CPU usage for each module.

sum (rate (container_cpu_usage_seconds_total{image!=""}[1m])) by (pod_name)

kubernetes-prometheus GitHub, , : https://github.com/camilb/prometheus-kubernetes

Answer 2

, :

avg (rate (container_cpu_usage_seconds_total{id="/"}[1m]))

Answer 3

prometheus (https://github.com/google-cloud-tools/kube-eagle), , . . :

sum(eagle_pod_container_resource_usage_cpu_cores)

, .