Monitoring and Debugging Prometheus

Training Overview
Introduction
Prerequisites
Monitoring and Debugging Overview
Motivation
Overview
Web Status Pages
Overview
Target Scrape Status
Service Discovery Status
TSDB Status Information
Knowledge Check
Logs
Overview
Standard Error Logs
Query Log
Active Queries Log
Knowledge Check
Metrics-Based Meta-Monitoring
Overview
Prometheus' Own Metrics
Meta-Monitoring Architecture
Alerting Rule Examples
End-to-End Watchdog Alerts
Knowledge Check
Profiling
Overview
Profiling Heap Memory Usage
Profiling CPU Usage
Inspecting Goroutine States
Knowledge Check
Review and Outlook
Summary
Further Steps

Knowledge Check

How can you make Prometheus' standard error (stderr) logging more verbose?

You can set the --log.level=debug command-line flag.

You can set the global.log-verbosity: verbose option in the main Prometheus configuration file.

By default, Prometheus already logs at the most verbose level, so there is no way to make its logging more verbose.

None of the above.

How does Prometheus log PromQL queries?

It writes all completed queries, along with some execution statistics, to a log file when setting the global.query_log_file option in the main configuration file.

It shows all currently running and recently run PromQL queries on the /logs web status page.

It keeps track of all currently running queries in a special file in the TSDB's data directory so that Prometheus can log them after a crash.

All of the above.

Which kinds of information does the query log show you for each executed query?

How long different phases of the execution took.

Which client IP triggered the query, if the query was triggered via the HTTP API.

What the exact PromQL expression and input time parameters were.

All of the above.