Recording rules allow you to precompute a PromQL query on a periodic basis and store its results in Prometheus' time series database (TSDB) as a new set of time series with a new metric name.

This can be helpful in a number of situations, such as:

Precomputing expensive queries

Some PromQL expressions are expensive to evaluate, especially when they involve reading and processing a large number of time series. In these cases, it can be helpful to precompute a query on an ongoing basis and store its resulting output time series as a new metric name. You can then query the recorded metric name to both reduce the load on the Prometheus server and get faster query results.

This is especially effective when the query is used in multiple dashboards or alerts and when the query aggregates many time series into fewer series (think of 1000s of individual request rates being summed up into a single aggregate rate – querying the stored result will be much cheaper, since Prometheus will only need to retrieve a single time series).

Precomputing frequently used queries

You can also use recording rules to precompute frequently used queries or expression fragments that are used in multiple queries. Besides improving the performance of a query, this can make it easier to reuse and maintain shared queries. For example, if you have a query expression that you use in multiple dashboards or alerts, you can use a recording rule to precompute the result once and then simply use the recording rule's output metric name in all of the relevant dashboards and alerts. This makes it easier to change the shared query, as you only need to modify it in one place.

Precomputing metrics for use in federation

Prometheus' federation feature allows you to scrape metrics from one Prometheus server's TSDB into another. This is useful for aggregating metrics from multiple Prometheus servers into a single server (for example, to get a global view). However, you can only federate metrics that are already persisted in the TSDB of the source Prometheus server. If you want to federate the result of a PromQL expression, you need to precompute and store it using a recording rule first.

Recording synthetic series with custom labels

Recording rules also allow you to record arbitrary scalar values (for example, the PromQL expression 42) into time series with custom labels that you can then correlate to other time series data. For example, you could use a set of recording rules with the same output metric name, but different path label values, to synthetically record different error rate thresholds for each path of an HTTP-serving application. You can then write a single alerting rule referencing the generated threshold metric name to produce alerts with a custom threshold for each path's current error rate.

See the "Using time series as alert thresholds" blog post by Robust Perception to learn more about this pattern.