Name: Monitoring and Debugging Prometheus
Author: Julius Volz

Monitoring and Debugging Prometheus

Training Overview
Introduction
Prerequisites
Monitoring and Debugging Overview
Motivation
Overview
Web Status Pages
Overview
Target Scrape Status
Service Discovery Status
TSDB Status Information
Knowledge Check
Logs
Overview
Standard Error Logs
Query Log
Active Queries Log
Knowledge Check
Metrics-Based Meta-Monitoring
Overview
Prometheus' Own Metrics
Meta-Monitoring Architecture
Alerting Rule Examples
End-to-End Watchdog Alerts
Knowledge Check
Profiling
Overview
Profiling Heap Memory Usage
Profiling CPU Usage
Inspecting Goroutine States
Knowledge Check
Review and Outlook
Summary
Further Steps

Summary

In this training, you learned:

How you can debug many aspects of Prometheus' behavior directly using its web-based status pages,
How to interpret and configure Prometheus' various logging options to understand run-time behavior as well as executed query statistics,
How to build a robust meta-monitoring setup to monitor Prometheus and Alertmanager servers based on their own metrics,
How to profile Prometheus and Alertmanager servers to diagnose memory and CPU usage issues, as well as goroutine states.

You should now be able to monitor and debug your Prometheus alerting setups in an effective way.