Join us

Grafana Pushes the Limits of Metrics Performance with Mimir 3.0

Grafana Pushes the Limits of Metrics Performance with Mimir 3.0

TL;DR

Grafana Mimir 3.0 debuts with a new query engine and architecture, boosting performance, reliability, and cost efficiency.

Key Points

Highlight key points with color coding based on sentiment (positive, neutral, negative).

Grafana Mimir 3.0 introduces a new query engine that streams results, significantly enhancing performance by reducing memory usage and speeding up query execution.

The update features a decoupled architecture that separates read and write operations, improving reliability and allowing independent scaling of these paths.

Apache Kafka is integrated as an asynchronous buffer between ingestion and query processes, facilitating smoother ingestion and faster queries.

Ingest storage in the new architecture ensures that spikes in query volume do not affect ingestion performance, simplifying scaling and operational maintenance.

The architectural changes in Mimir 3.0 lead to up to 15% lower resource usage in large clusters, optimizing costs while maintaining high performance and reliability.

Key Numbers

Present key numerics and statistics in a minimalist format.
4.7k

The number of GitHub stars for Grafana Mimir.

1 billion

The target scale for active series in Mimir.

15 %

The reduction in resource usage for large clusters.

92 %

The reduction in peak memory usage with the new query engine.

Stakeholder Relationships

An interactive diagram mapping entities directly or indirectly involved in this news. Drag nodes to rearrange them and see relationship details.

Organizations

Key entities and stakeholders, categorized for clarity: people, organizations, tools, events, regulatory bodies, and industries.
Grafana Labs Software Company

Responsible for the development and release of Grafana Mimir 3.0.

Tools

Key entities and stakeholders, categorized for clarity: people, organizations, tools, events, regulatory bodies, and industries.
Apache Kafka Distributed Event Streaming Platform

Used in the new decoupled architecture of Mimir 3.0 to allow independent scaling of read and write paths.

Mimir Query Engine (MQE) Query Engine

Introduced in Grafana Mimir 2.17 and foundational in Mimir 3.0, offering faster and more memory-efficient performance.

Prometheus Monitoring System

Configured as a write client to send data to multiple endpoints in the new architecture of Mimir 3.0.

OpenTelemetry Collector Telemetry Data Collector

Configured as a write client to send data to multiple endpoints in the new architecture of Mimir 3.0.

Grafana Visualization Tool

Reconfigured as a read client to point to new endpoints in the updated architecture of Mimir 3.0.

Events

Key entities and stakeholders, categorized for clarity: people, organizations, tools, events, regulatory bodies, and industries.
Release of Grafana Mimir 3.0 Software Release

Introduced significant architectural changes, including a new query engine and a decoupled architecture using Apache Kafka.

Timeline of Events

Timeline of key events and milestones.
2022 Introduction of Grafana Mimir

Grafana Mimir was introduced as an open-source, horizontally scalable, multi-tenant time series database for long-term storage of Prometheus and OpenTelemetry metrics.

Release of Grafana Mimir 2.17 Introduction of the Mimir Query Engine (MQE)

The release of Grafana Mimir 2.17 included the Mimir Query Engine, which offers faster and more memory-efficient performance.

Release of Grafana Mimir 3.0 Announcement of significant improvements

Grafana Mimir 3.0 was announced with significant improvements including a new query engine, decoupled architecture, and enhanced reliability and performance. This version separates read and write operations, introduces Apache Kafka for independent scaling, and optimizes resource usage.

Grafana Mimir has been stirring up interest in the time series database arena ever since it hit the scene. Originally rolled out as an open-source, horizontally scalable, multi-tenant database, it was crafted for the long-term storage of Prometheus and OpenTelemetry metrics. This was a pretty big deal for developers and DevOps folks on the hunt for solid solutions to manage their metric data. The platform's horizontal scalability means it can grow alongside your needs, which is always a win in our ever-expanding tech world.

Jumping ahead to Grafana Mimir 2.17, we saw the debut of the Mimir Query Engine (MQE). This new engine promised faster and more memory-efficient performance, which is music to the ears of anyone who's ever wrestled with sluggish query responses. The MQE marked a significant leap forward, offering a more efficient experience for users who need to crunch large volumes of data quickly and efficiently. But it's not just about speed - the memory efficiency means you can do more with less, an important factor when resources are tight.

Then, Grafana Mimir 3.0 arrived, bringing a slew of improvements that caught the tech community's eye. This version introduced a decoupled architecture, separating read and write operations, which allows for more flexibility and reliability. By incorporating Apache Kafka, Mimir 3.0 allows independent scaling, meaning you can tweak your system's capacity without a hitch. These changes not only boost performance but also optimize resource usage, making it a more appealing option for those managing large-scale deployments.

Enjoyed it?

Get weekly updates delivered straight to your inbox, it only takes 3 seconds!

Subscribe to our weekly newsletter Kaptain to receive similar updates for free!

What is FAUN.news()?

Let's keep in touch!

Stay updated with my latest posts and news. I share insights, updates, and exclusive content.

Unsubscribe anytime. By subscribing, you share your email with @kaptain and accept our Terms & Privacy.

Give a Pawfive to this post!


Only registered users can post comments. Please, login or signup.

Start writing about what excites you in tech — connect with developers, grow your voice, and get rewarded.

Join other developers and claim your FAUN.dev() account now!

FAUN.dev()
FAUN.dev()

FAUN.dev() is a developer-first platform built with a simple goal: help engineers stay sharp without wasting their time.

Avatar

Kaptain #Kubernetes

FAUN.dev()

@kaptain
Kubernetes Weekly Newsletter, Kaptain. Curated Kubernetes news, tutorials, tools and more!
Developer Influence
1

Influence

1

Total Hits

32

Posts