Grafana Tempo 2.9 Supercharges Distributed Tracing with LLM Integration

Image source: https://grafana.com/blog/2025…

TL;DR

Grafana Tempo 2.9 debuts with MCP server support and TraceQL metrics sampling, enhancing data analysis and query efficiency.

Key Points

Highlight key points with color coding based on sentiment (positive, neutral, negative).

Grafana Tempo 2.9 introduces MCP server support, enabling integration with LLMs for enhanced data analysis and faster issue diagnosis by providing direct access to distributed tracing data.

The release includes TraceQL metrics sampling, which allows for quicker query results through probabilistic sampling, offering approximate results that are beneficial in high-volume environments.

New query hints in TraceQL, such as with(sample=true) and with(sample=0.xx), enable dynamic and fixed probabilistic sampling, respectively, to improve query performance by reducing execution time.

Operational improvements for multi-tenant environments include new metrics for monitoring query I/O and span timestamp distances, enhancing visibility into performance and potential issues.

The update also features enhancements in TraceQL metrics performance, with specific improvements for simple queries and overall search and metrics performance, contributing to more efficient data processing.

Stakeholder Relationships

An interactive diagram mapping entities directly or indirectly involved in this news. Drag nodes to rearrange them and see relationship details.

Organizations

Key entities and stakeholders, categorized for clarity: people, organizations, tools, events, regulatory bodies, and industries.

Grafana Labs Software Company

Developed and released Grafana Tempo 2.9 with new features like MCP server support and TraceQL metrics sampling.

Tools

Key entities and stakeholders, categorized for clarity: people, organizations, tools, events, regulatory bodies, and industries.

Grafana Tempo 2.9 Tracing Tool

The latest version released by Grafana Labs, featuring enhancements for data analysis and query performance.

TraceQL Query Language

Introduced in Grafana Tempo 2.9 to improve metrics sampling and query performance.

MCP Server Server Support

A new feature in Grafana Tempo 2.9 aimed at enhancing data analysis capabilities.

OTLP Exporter Data Exporter

The standard exporter used in Grafana Tempo 2.9, replacing the deprecated Jaeger agent/exporter.

Timeline of Events

Timeline of key events and milestones.

2025-11 Grafana Tempo 2.9 released

Grafana Labs released the latest version of Grafana Tempo, version 2.9, which includes new features for enhanced data analysis.

Future Project Rhythm announced

Grafana Labs is developing a new Tempo architecture, codenamed Project Rhythm. This experimental initiative aims to remove the replication factor 3 (RF3) requirement, decouple read and write paths for better scalability and reliability, and establish a foundation for reduced total cost of ownership and long-term scalability.

Grafana Tempo 2.9 has arrived, and it's packed with features that developers and data scientists might find pretty exciting. One of the big highlights? The support for the Model Context Protocol (MCP) server. This isn't just a simple add-on; it's about integrating with large language models (LLMs) to supercharge data analysis. By letting LLMs tap into distributed tracing data through TraceQL queries and other endpoints, this experimental feature aims to speed up issue diagnosis and give users a clearer picture of service interactions.

But wait, there's more. The update also brings a fresh take on TraceQL metrics sampling. With new query hints, the goal is to boost query performance. Probabilistic sampling is in play here, meaning users can get results faster, albeit with a touch of approximation. This is a lifesaver in data-heavy environments. Users can choose to control the sampling dynamically or stick with a fixed percentage, allowing them to balance speed and accuracy based on what they need at the moment.

And the improvements keep coming. The release is also geared towards making life easier in multi-tenant environments, simplifying the monitoring and refining of tracing pipelines. New metrics are on board to keep an eye on query I/O and span timestamp distances, offering better insights into query performance and potential issues like clock skew. Plus, there are enhancements in TraceQL performance, especially in search and metrics queries. The release notes and changelog are chock-full of details on these updates, including bug fixes and breaking changes. Looking ahead, Grafana Tempo is gearing up for a new architecture, Project Rhythm, which promises to boost scalability and reliability while cutting down costs.