Hidden Labels and Metadata
Labels starting with a double underscore (
__) are treated specially in that they are removed after relabeling. A source of labeled objects (such as a service discovery mechanism producing labeled targets) may initially attach these "hidden" labels to provide extra metadata about a labeled object. These special labels can be used during the relabeling phase to make decisions or changes to the object's labels.
Scrape Control Labels
For targets, some of these hidden labels have a special meaning and control how a target should be scraped:
__address__: Contains the TCP address that should be scraped for the target. It initially defaults to the
<host>:<port>pair provided by the service discovery mechanism. Prometheus sets the
instancelabel to the value of
__address__after relabeling if you do not set the
instancelabel explicitly to another value before that.
__scheme__: Contains the HTTP scheme (
https) with which the target should be scraped. Defaults to
__metrics_path__: Contains the HTTP path to scrape metrics from. Defaults to
__param_<name>: Contains HTTP query parameter names and their values.
Each of these labels can be set or overwritten using relabeling rules to produce custom scrape behaviors for individual targets.
Service Discovery Metadata Labels
Additionally, service discovery mechanisms can provide a set of labels starting with
__meta_ that contain discovery-specific metadata about a target. For example, when discovering pods in a Kubernetes cluster, the Kubernetes service discovery engine will provide a
__meta_kubernetes_pod_name label for each pod target, containing the name of the discovered pod, and a
__meta_kubernetes_pod_ready label indicating whether the pod is in the ready state or not. You can find all the labels that the Kubernetes service discovery provides in the configuration documentation.
If a relabeling step needs to save a value to a temporary label (to be processed in a subsequent relabeling step), you can use the
__tmp label name prefix. Labels starting with
__tmp are guaranteed to never be used by Prometheus itself.