Hidden Labels and Metadata

Labels starting with a double underscore (__) are automatically removed after the last relabeling rule and thus do not make it into the final labels of an object. These specially prefixed labels can be used for storing and passing around extra metadata during the relabeling phase to make relabeling decisions or changes to the object's labels. For example, a service discovery mechanism may initially supply additional metadata for each target in these "hidden" labels.

Scrape control labels

For targets, some of these hidden labels have a special meaning and control how Prometheus should scrape a target:

  • __address__: This contains the TCP address that should be scraped for the target. It is initially set to the target's <host>:<port> pair provided by the service discovery mechanism. After relabeling, Prometheus sets the instance label to the value of __address__ if you do not set the instance label explicitly to another value before that.
  • __scheme__: This contains the HTTP scheme (http or https) with which the target should be scraped. Defaults to http.
  • __metrics_path__: This contains the HTTP path to scrape metrics from. Defaults to /metrics.
  • __param_<name>: This contains HTTP query parameter names and their values that should be sent along with a scrape request.

You can set or override each of these labels using relabeling rules to produce custom scrape behaviors for individual targets.

Service discovery metadata labels

Additionally, service discovery mechanisms can provide a set of labels starting with __meta_ that contain discovery-specific metadata about a target. For example, when discovering pods in a Kubernetes cluster, the Kubernetes service discovery engine will provide a __meta_kubernetes_pod_name label for each pod target, containing the name of the discovered pod, and a __meta_kubernetes_pod_ready label indicating whether the pod is in the ready state or not. You can find all the labels that the Kubernetes service discovery provides in the configuration documentation. Other service discovery mechanisms provide similar metadata.

Temporary labels

If you need to store a temporary label value in a relabeling step (for use in a subsequent relabeling step), you can use the __tmp label name prefix. Labels starting with __tmp are guaranteed to never be used by Prometheus itself.