Target, Metric, Write, and Alert Relabeling: What's the Difference?

Prometheus applies relabeling at several points in its data pipeline, each with a distinct purpose. While relabel_configs is the one most users encounter, there are also metric_relabel_configs, write_relabel_configs, and alert_relabel_configs. Together, they let you reshape targets, metrics, and alerts at different stages of collection and delivery.

1. Target Relabeling (`relabel_configs`)

After service discovery and before scraping, Prometheus applies target relabeling. At this point, targets have already been discovered and enriched with metadata. The relabel_configs block gives you control over that discovery data-deciding which targets to keep, rename, or ignore. It’s the tool you use to clean up or transform target labels before Prometheus begins collecting metrics.

For instance, you might rename a discovered label to something more meaningful:

relabel_configs:
  - source_labels: [__meta_kubernetes_pod_name]
    target_label: pod
  - source_labels: [__meta_kubernetes_namespace]
    target_label: namespace

Or drop unnecessary targets:

relabel_configs:
  - source_labels: [__meta_kubernetes_pod_label_app]
    regex: test-.*
    action: drop

2. Metric Relabeling (`metric_relabel_configs`)

Once metrics have been scraped and before they’re ingested into storage, Prometheus can still adjust them before storing anything in its database. Metric relabeling operates here-between scraping and ingestion. It’s where you refine the data: removing heavy time series, renaming labels, or scrubbing sensitive information that shouldn’t persist.

For example, to rename a label that’s too verbose:

metric_relabel_configs:
  - source_labels: [kubernetes_namespace]
    target_label: namespace

Or to drop metrics that aren’t needed:

metric_relabel_configs:
  - source_labels: [job]
    regex: unwanted-job
    action: drop

Think of it as a post-scrape filter that keeps your database organized and efficient.

ℹ️ To avoid any confusion, remember this: Prometheus first discovers targets and this is where the Discovered Labels come from. After discovery, Prometheus starts seeing the labels of the target and this is where the Target Labels come from. The relabel_configs operates on the target labels but the metric_relabel_configs operates on the metric labels. Therefore, changes made in relabel_configs will be seen in the Service Discovery page, while changes made in metric_relabel_configs will not be seen there but will be reflected in the actual metrics stored in Prometheus.

3. Remote Write Relabeling (`write_relabel_configs`)

Prometheus can forward metrics to remote systems like Cortex, Thanos, or VictoriaMetrics. Before those metrics leave your server to go to these external systems, write_relabel_configs gives you a final chance to curate what’s sent. You can drop nonessential data, tweak labels for downstream routing, or enrich metrics with context needed by the receiver.