Skip to content

[prod] Update grafana/mimir Docker tag to v2.17.1

Soosrequested to merge
renovate/prod-grafana-mimir-2.x into master

This MR contains the following updates:

Package Type Update Change
grafana/mimir (source) prod minor 2.15.0 -> 2.17.1

⚠️ Warning

Some dependencies could not be looked up. Check the warning logs for more information.

Release Notes

grafana/mimir (grafana/mimir)

v2.17.1

Grafana Mimir

v2.17.0

Grafana Mimir
  • [CHANGE] Query-frontend: Ensure that cache keys generated from cardinality estimate middleware are less than 250 bytes in length by hashing the tenant IDs that are included in them. This change invalidates all cardinality estimates in the cache. #​11568
  • [CHANGE] Ruler: Remove experimental CLI flag -ruler-storage.cache.rule-group-enabled to enable or disable caching the contents of rule groups. Caching rule group contents is now always enabled when a cache is configured for the ruler. #​10949
  • [CHANGE] Ingester: Out-of-order native histograms are now enabled whenever both native histogram and out-of-order ingestion is enabled. The -ingester.ooo-native-histograms-ingestion-enabled CLI flag and corresponding ooo_native_histograms_ingestion_enabled runtime configuration option have been removed. #​10956
  • [CHANGE] Distributor: removed the cortex_distributor_label_values_with_newlines_total metric. #​10977
  • [CHANGE] Ingester/Distributor: renamed the experimental max_cost_attribution_cardinality_per_user config to max_cost_attribution_cardinality. #​11092
  • [CHANGE] Frontend: The subquery spin-off feature is now enabled with -query-frontend.subquery-spin-off-enabled=true instead of -query-frontend.instant-queries-with-subquery-spin-off=.* #​11153
  • [CHANGE] Overrides-exporter: Don't export per-tenant overrides that are set to their default values. #​11173
  • [CHANGE] gRPC/HTTP clients: Rename metric cortex_client_request_invalid_cluster_validation_labels_total to cortex_client_invalid_cluster_validation_label_requests_total. #​11237
  • [CHANGE] Querier: Use Mimir Query Engine (MQE) by default. Set -querier.query-engine=prometheus to continue using Prometheus' engine. #​11501
  • [CHANGE] Memcached: Ignore initial DNS resolution failure, meaning don't depend on Memcached on startup. #​11602
  • [CHANGE] Ingester: The -ingester.stream-chunks-when-using-blocks CLI flag and ingester_stream_chunks_when_using_blocks runtime configuration option have been deprecated and will be removed in a future release. #​11711
  • [CHANGE] Distributor: track cortex_ingest_storage_writer_latency_seconds metric for failed writes too. Added outcome label to distinguish between success and failure. #​11770
  • [CHANGE] Distributor: renamed few metrics used by experimental ingest storage. #​11766
    • Renamed cortex_ingest_storage_writer_produce_requests_total to cortex_ingest_storage_writer_produce_records_enqueued_total
    • Renamed cortex_ingest_storage_writer_produce_failures_total to cortex_ingest_storage_writer_produce_records_failed_total
  • [CHANGE] Distributor: moved HA tracker timeout config to limits. #​11774
    • Moved distributor.ha_tracker.ha_tracker_update_timeout to limits.ha_tracker_update_timeout.
    • Moved distributor.ha_tracker.ha_tracker_update_timeout_jitter_max to limits.ha_tracker_update_timeout_jitter_max.
    • Moved distributor.ha_tracker.ha_tracker_failover_timeout to limits.ha_tracker_failover_timeout.
  • [CHANGE] Distributor: Memberlist marked as stable as an option for backend storage for the HA tracker. #​11861
  • [CHANGE] Distributor: etcd deprecated as an option for backend storage for the HA tracker. #​12047
  • [CHANGE] Memberlist: Apply new default configuration values for MemberlistKV. This unlocks using it as backend storage for the HA Tracker. We have observed better performance with these defaults across different production loads. #​11874
    • memberlist.packet-dial-timeout: 500ms
    • memberlist.packet-write-timeout: 500ms
    • memberlist.max-concurrent-writes: 5
    • memberlist.acquire-writer-timeout: 1s These defaults perform better but may cause long-running packets to be dropped in high-latency networks.
  • [CHANGE] Query-frontend: Apply query pruning and check for disabled experimental functions earlier in query processing. #​11939
  • [FEATURE] Distributor: Experimental support for Prometheus Remote-Write 2.0 protocol. Limitations: Created timestamp is ignored, per series metadata is merged on metric family level automatically, ingestion might fail if client sends ProtoBuf fields out of order. The label version is added to the metric cortex_distributor_requests_in_total with a value of either 1.0 or 2.0 depending on the detected Remote-Write protocol. #​11100 #​11101 #​11192 #​11143
  • [FEATURE] Query-frontend: expand query-frontend.cache-errors and query-frontend.results-cache-ttl-for-errors configuration options to cache non-transient response failures for instant queries. #​11120
  • [FEATURE] Query-frontend: Allow use of Mimir Query Engine (MQE) via the experimental CLI flags -query-frontend.query-engine or -query-frontend.enable-query-engine-fallback or corresponding YAML. #​11417 #​11775
  • [FEATURE] Querier, query-frontend, ruler: Enable experimental support for duration expressions in PromQL, which are simple arithmetics on numbers in offset and range specification. #​11344
  • [FEATURE] You can configure Mimir to export traces in OTLP exposition format through the standard OTEL_ environment variables. #​11618
  • [FEATURE] distributor: Allow configuring tenant-specific HA tracker failover timeouts. #​11774
  • [FEATURE] OTLP: Add experimental support for promoting OTel scope metadata (name, version, schema URL, attributes) to metric labels, prefixed with otel_scope_. Enable via the -distributor.otel-promote-scope-metadata flag. #​11795
  • [FEATURE] Distributor: Add experimental -distributor.otel-native-delta-ingestion option to allow primitive delta metrics ingestion via the OTLP endpoint. #​11631
  • [FEATURE] MQE: Add support for experimental sort_by_label and sort_by_label_desc PromQL functions. #​11930
  • [FEATURE] Ingester/Block-builder: Handle the created timestamp field for remote-write requests. #​11977
  • [FEATURE] Cost attribution: Labels specified in the limit configuration may specify an output label in order to override emitted label names. #​12035
  • [ENHANCEMENT] Dashboards: Add "Influx write requests" row to Writes Dashboard. #​11731
  • [ENHANCEMENT] Mixin: Add MimirHighVolumeLevel1BlocksQueried alert that fires when level 1 blocks are queried for more than 6 hours, indicating potential compactor performance issues. #​11803
  • [ENHANCEMENT] Querier: Make the maximum series limit for cardinality API requests configurable on a per-tenant basis with the cardinality_analysis_max_results option. #​11456
  • [ENHANCEMENT] Querier: Add configurable concurrency limit for remote read queries with the --querier.max-concurrent-remote-read-queries flag. Defaults to 2. Set to 0 for unlimited concurrency. #​11892
  • [ENHANCEMENT] Dashboards: Add "Queries / sec by read path" to Queries Dashboard. #​11640
  • [ENHANCEMENT] Dashboards: Add "Added Latency" row to Writes Dashboard. #​11579
  • [ENHANCEMENT] Ingester: Add support for exporting native histogram cost attribution metrics (cortex_ingester_attributed_active_native_histogram_series and cortex_ingester_attributed_active_native_histogram_buckets) with labels specified by customers to a custom Prometheus registry. #​10892
  • [ENHANCEMENT] Distributor: Add new metrics cortex_distributor_received_native_histogram_samples_total and cortex_distributor_received_native_histogram_buckets_total to track native histogram samples and bucket counts separately for billing calculations. Updated cortex_distributor_received_samples_total description to clarify it includes native histogram samples. #​11728
  • [ENHANCEMENT] Store-gateway: Download sparse headers uploaded by compactors. Compactors have to be configured with -compactor.upload-sparse-index-headers=true option. #​10879 #​11072.
  • [ENHANCEMENT] Compactor: Upload block index file and multiple segment files concurrently. Concurrency scales linearly with block size up to -compactor.max-per-block-upload-concurrency. #​10947
  • [ENHANCEMENT] Ingester: Add per-user cortex_ingester_tsdb_wal_replay_unknown_refs_total and cortex_ingester_tsdb_wbl_replay_unknown_refs_total metrics to track unknown series references during WAL/WBL replay. #​10981
  • [ENHANCEMENT] Added -ingest-storage.kafka.fetch-max-wait configuration option to configure the maximum amount of time a Kafka broker waits for some records before a Fetch response is returned. #​11012
  • [ENHANCEMENT] Ingester: Add cortex_ingester_tsdb_forced_compactions_in_progress metric reporting a value of 1 when there's a forced TSDB head compaction in progress. #​11006
  • [ENHANCEMENT] Ingester: Add cortex_ingest_storage_reader_records_batch_fetch_max_bytes metric reporting the distribution of MaxBytes specified in the Fetch requests sent to Kafka. #​11014
  • [ENHANCEMENT] All: Add experimental support for cluster validation in HTTP calls. When it is enabled, HTTP server verifies if a request coming from an HTTP client comes from an expected cluster. This validation can be configured by the following experimental configuration options: #​11010 #​11549
    • -server.cluster-validation.label
    • -server.cluster-validation.http.enabled
    • -server.cluster-validation.http.soft-validation
    • -server.cluster-validation.http.exclude-paths
  • [ENHANCEMENT] Query-frontend: Add experimental support to include the cluster validation label in HTTP request headers. When cluster validation is enabled on the HTTP server side, cluster validation labels from HTTP request headers are compared with the HTTP server's cluster validation label. #​11010 #​11145
    • By setting -query-frontend.client-cluster-validation.label, you configure the query-frontend's client cluster validation label.
    • The flag -common.client-cluster-validation.label, if set, provides the default for -query-frontend.client-cluster-validation.label.
  • [ENHANCEMENT] Distributor: Add ignore_ingest_storage_errors and ingest_storage_max_wait_time flags to control error handling and timeout behavior during ingest storage migration. #​11291
    • -ingest-storage.migration.ignore-ingest-storage-errors
    • -ingest-storage.migration.ingest-storage-max-wait-time
  • [ENHANCEMENT] Memberlist: Add -memberlist.abort-if-fast-join-fails support and retries on DNS resolution. #​11067
  • [ENHANCEMENT] Querier: Allow configuring all gRPC options for store-gateway client, similar to other gRPC clients. #​11074
  • [ENHANCEMENT] Ruler: Log the number of series returned for each query as result_series_count as part of query stats log lines. #​11081
  • [ENHANCEMENT] Ruler: Don't log statistics that are not available when using a remote query-frontend as part of query stats log lines. #​11083
  • [ENHANCEMENT] Ingester: Remove cost-attribution experimental max_cost_attribution_labels_per_user limit. #​11090
  • [ENHANCEMENT] Update Go to 1.24.2. #​11114
  • [ENHANCEMENT] Query-frontend: Add cortex_query_samples_processed_total metric. #​11110
  • [ENHANCEMENT] Query-frontend: Add cortex_query_samples_processed_cache_adjusted_total metric. #​11164
  • [ENHANCEMENT] Ingester/Distributor: Add cortex_cost_attribution_* metrics to observe the state of the cost-attribution trackers. #​11112
  • [ENHANCEMENT] Querier: Process multiple remote read queries concurrently instead of sequentially for improved performance. #​11732
  • [ENHANCEMENT] gRPC/HTTP servers: Add cortex_server_invalid_cluster_validation_label_requests_total metric, that is increased for every request with an invalid cluster validation label. #​11241 #​11277
  • [ENHANCEMENT] OTLP: Add support for converting OTel explicit bucket histograms to Prometheus native histograms with custom buckets using the distributor.otel-convert-histograms-to-nhcb flag. #​11077
  • [ENHANCEMENT] Add configurable per-tenant limited_queries, which you can only run at or less than an allowed frequency. #​11097
  • [ENHANCEMENT] Ingest-Storage: Add ingest-storage.kafka.producer-record-version to allow control Kafka record versioning. #​11244
  • [ENHANCEMENT] Ruler: Update <prometheus-http-prefix>/api/v1/rules and <prometheus-http-prefix>/api/v1/alerts to reply with HTTP error 422 if rule evaluation is completely disabled for the tenant. If only recording rule or alerting rule evaluation is disabled for the tenant, the response now includes a corresponding warning. #​11321 #​11495 #​11511
  • [ENHANCEMENT] Add tenant configuration block ruler_alertmanager_client_config which allows the Ruler's Alertmanager client options to be specified on a per-tenant basis. #​10816
  • [ENHANCEMENT] Distributor: Trace when deduplicating a metric's samples or histograms. #​11159 #​11715
  • [ENHANCEMENT] Store-gateway: Retry querying blocks from store-gateways with dynamic replication until trying all possible store-gateways. #​11354 #​11398
  • [ENHANCEMENT] Mimirtool: Support multiple --selector flags in remote read commands to send multiple queries in a single protobuf request, leveraging the remote read protocol's native batching capabilities. #​11733
  • [ENHANCEMENT] Mimirtool: Added --use-chunks flag to remote read commands to control response type preference (chunked streaming vs sampled). #​11733
  • [ENHANCEMENT] Query-frontend: Add optional reason to blocked_queries config. #​11407 #​11434
  • [ENHANCEMENT] Distributor: Gracefully handle type assertion of WatchPrefix in HA Tracker to continue checking for updates. #​11411 #​11461
  • [ENHANCEMENT] Querier: Include chunks streamed from store-gateway in Mimir Query Engine memory estimate of query memory usage. #​11453 #​11465
  • [ENHANCEMENT] Querier: Include chunks streamed from ingester in Mimir Query Engine memory estimate of query memory usage. #​11457
  • [ENHANCEMENT] Query-frontend: Add retry mechanism for remote reads, series, and cardinality prometheus endpoints #​11533
  • [ENHANCEMENT] Ruler: Ignore rulers in non-operation states when getting and syncing rules #​11569
  • [ENHANCEMENT] Query-frontend: add optional reason to blocked_queries config. #​11407 #​11434
  • [ENHANCEMENT] Tracing: Add HTTP headers as span attributes when -server.trace-request-headers is enabled. You can configure which headers to exclude using the -server.trace-request-headers-exclude-list flag. #​11655
  • [ENHANCEMENT] Ruler: Add new per-tenant limit on minimum rule evaluation interval. #​11665
  • [ENHANCEMENT] store-gateway: download sparse headers on startup when lazy loading is enabled. #​11686
  • [ENHANCEMENT] Distributor: added more metrics to troubleshoot Kafka records production latency when experimental ingest storage is enabled: #​11766 #​11771
    • cortex_ingest_storage_writer_produce_remaining_deadline_seconds: measures the remaining deadline (in seconds) when records are requested to be produced.
    • cortex_ingest_storage_writer_produce_records_enqueue_duration_seconds: measures how long it takes to enqueue produced Kafka records in the client.
    • cortex_ingest_storage_writer_kafka_write_wait_seconds: measures the time spent waiting to write to Kafka backend.
    • cortex_ingest_storage_writer_kafka_write_time_seconds: measures the time spent writing to Kafka backend.
    • cortex_ingest_storage_writer_kafka_read_wait_seconds: measures the time spent waiting to read from Kafka backend.
    • cortex_ingest_storage_writer_kafka_read_time_seconds: measures the time spent reading from Kafka backend.
    • cortex_ingest_storage_writer_kafka_request_duration_e2e_seconds: measures the time from the start of when a Kafka request is written to the end of when the response for that request was fully read from the Kafka backend.
    • cortex_ingest_storage_writer_kafka_request_throttled_seconds: measures how long Kafka requests have been throttled by the Kafka client.
  • [ENHANCEMENT] Distributor: Add per-user cortex_distributor_sample_delay_seconds to track delay of ingested samples with regard to wall clock. #​11573
  • [ENHANCEMENT] Distributor: added circuit breaker to not produce Kafka records at all if the context is already canceled / expired. This applied only when experimental ingest storage is enabled. #​11768
  • [ENHANCEMENT] Compactor: Optimize the planning phase for tenants with a very large number of blocks, such as tens or hundreds of thousands, at the cost of making it slightly slower for tenants with a very a small number of blocks. #​11819
  • [ENHANCEMENT] Query-frontend: Accurate tracking of samples processed from cache. #​11719
  • [ENHANCEMENT] Store-gateway: Change level 0 blocks to be reported as 'unknown/old_block' in metrics instead of '0' to improve clarity. Level 0 indicates blocks with metadata from before compaction level tracking was added to the bucket index. #​11891
  • [ENHANCEMENT] Compactor, distributor, ruler, scheduler and store-gateway: Makes -<component-ring-config>.auto-forget-unhealthy-periods configurable for each component. Deprecates the -store-gateway.sharding-ring.auto-forget-enabled flag. #​11923
  • [ENHANCEMENT] otlp: Stick to OTLP vocabulary on invalid label value length error. #​11889
  • [ENHANCEMENT] Ingester: Display user grace interval in the tenant list obtained through the /ingester/tenants endpoint. #​11961
  • [ENHANCEMENT] kafkatool: add consumer-group delete-offset command as a way to delete the committed offset for a consumer group. #​11988
  • [ENHANCEMENT] Block-builder-scheduler: Detect gaps in scheduled and completed jobs. #​11867
  • [ENHANCEMENT] Distributor: Experimental support for Prometheus Remote-Write 2.0 protocol has been updated. Created timestamps are now supported. This feature includes some limitations. If samples in a write request aren't ordered by time, the created timestamp might be dropped. Additionally, per-series metadata is automatically merged on the metric family level. Ingestion might fail if the client sends ProtoBuf fields out-of-order. The label version is added to the metric cortex_distributor_requests_in_total with a value of either 1.0 or 2.0, depending on the detected remote-write protocol. #​11977
  • [ENHANCEMENT] Query-frontend: Added labels query optimizer that automatically removes redundant __name__!="" matchers from label names and label values queries, improving query performance. You can enable the optimizer per-tenant with the labels_query_optimizer_enabled runtime configuration flag. #​12054 #​12066 #​12076 #​12080
  • [ENHANCEMENT] Query-frontend: Standardise non-regex patterns in query blocking upon loading of config. #​12102
  • [ENHANCEMENT] Ruler: Propagate GCS object mutation rate limit for rule group uploads. #​12086
  • [ENHANCEMENT] Stagger head compaction intervals across zones to prevent compactions from aligning simultaneously, which could otherwise cause strong consistency queries to fail when experimental ingest storage is enabled. #​12090
  • [ENHANCEMENT] Compactor: Add -compactor.update-blocks-concurrency flag to control concurrency for updating block metadata during bucket index updates, separate from deletion marker concurrency. #​12117
  • [ENHANCEMENT] Query-frontend: Allow users to set the query-frontend.extra-propagated-headers flag to specify the extra headers allowed to pass through to the rest of the query path. #​12174
  • [BUGFIX] OTLP: Fix response body and Content-Type header to align with spec. #​10852
  • [BUGFIX] Compactor: fix issue where block becomes permanently stuck when the Compactor's block cleanup job partially deletes a block. #​10888
  • [BUGFIX] Storage: fix intermittent failures in S3 upload retries. #​10952
  • [BUGFIX] Querier: return NaN from irate() if the second-last sample in the range is NaN and Prometheus' query engine is in use. #​10956
  • [BUGFIX] Ruler: don't count alerts towards cortex_prometheus_notifications_dropped_total if they are dropped due to alert relabelling. #​10956
  • [BUGFIX] Querier: Fix issue where an entire store-gateway zone leaving caused high CPU usage trying to find active members of the leaving zone. #​11028
  • [BUGFIX] Query-frontend: Fix blocks retention period enforcement when a request has multiple tenants (tenant federation). #​11069
  • [BUGFIX] Query-frontend: Fix -query-frontend.query-sharding-max-sharded-queries enforcement for instant queries with binary operators. #​11086
  • [BUGFIX] Memberlist: Fix hash ring updates before the full-join has been completed, when -memberlist.notify-interval is configured. #​11098
  • [BUGFIX] Query-frontend: Fix an issue where transient errors could be inadvertently cached. #​11198
  • [BUGFIX] Ingester: read reactive limiters should activate and deactivate when the ingester changes state. #​11234
  • [BUGFIX] Query-frontend: Fix an issue where errors from date/time parsing methods did not include the name of the invalid parameter. #​11304
  • [BUGFIX] Query-frontend: Fix a panic in monolithic mode caused by a clash in labels of the cortex_client_invalid_cluster_validation_label_requests_total metric definition. #​11455
  • [BUGFIX] Compactor: Fix issue where MimirBucketIndexNotUpdated can fire even though the index has been updated within the alert threshold. #​11303
  • [BUGFIX] Distributor: fix old entries in the HA Tracker with zero valued "elected at" timestamp. #​11462
  • [BUGFIX] Query-scheduler: Fix issue where deregistered querier goroutines can cause a panic if their backlogged dequeue requests are serviced. #​11510
  • [BUGFIX] Ruler: Failures during initial sync must be fatal for the service's startup. #​11545
  • [BUGFIX] Querier and query-frontend: Fix issue where aggregation functions like topk and quantile could return incorrect results if the scalar parameter is not a constant and Prometheus' query engine is in use. #​11548
  • [BUGFIX] Querier and query-frontend: Fix issue where range vector selectors could incorrectly ignore samples at the beginning of the range. #​11548
  • [BUGFIX] Querier: Fix rare panic if a query is canceled while a request to ingesters or store-gateways has just begun. #​11613
  • [BUGFIX] Ruler: Fix QueryOffset and AlignEvaluationTimeOnInterval being ignored when either recording or alerting rule evaluation is disabled. #​11647
  • [BUGFIX] Ingester: Fix issue where ingesters could leave read-only mode during forced compactions, resulting in write errors. #​11664
  • [BUGFIX] Ruler: Fix rare panic when the ruler is shutting down. #​11781
  • [BUGFIX] Block-builder-scheduler: Fix data loss bug in job assignment. #​11785
  • [BUGFIX] Compactor: start tracking -compactor.max-compaction-time after the initial compaction planning phase, to avoid rare cases where planning takes longer than -compactor.max-compaction-time and so actual compaction never runs for a tenant. #​11834
  • [BUGFIX] Distributor: Validate the RW2 symbols field and reject invalid requests that don't have an empty string as the first symbol. #​11953
  • [BUGFIX] Distributor: Check max_inflight_push_requests_bytes before decompressing incoming requests. #​11967
  • [BUGFIX] Query-frontend: Allow limit parameter to be 0 in label queries to explicitly request unlimited results. #​12054
  • [BUGFIX] Distributor: Fix a possible panic in the OTLP push path while handling a gRPC status error. #​12072
  • [BUGFIX] Query-frontend: Evaluate experimental duration expressions before sharding, splitting, and caching. Otherwise, the result is not correct. #​12038
  • [BUGFIX] Block-builder-scheduler: Fix bugs in handling of partitions with no commit. #​12130
  • [BUGFIX] Ingester: Fix issue where ingesters can exit read-only mode during idle compactions, resulting in write errors. #​12128
  • [BUGFIX] otlp: Reverts #​11889 which has a pooled memory re-use bug. #​12266
Mixin
  • [CHANGE] Alerts: Update the query for MimirBucketIndexNotUpdated to use max_over_time to prevent alert firing when pods rotate. #​11311, #​11426
  • [CHANGE] Alerts: Make alerting threshold for DistributorGcUsesTooMuchCpu configurable. #​11508
  • [CHANGE] Remove support for the experimental read-write deployment mode. #​11975
  • [CHANGE] Alerts: Replace namespace with job label in golang_alerts. #​11957
  • [FEATURE] Add an alert if the block-builder-scheduler detects that it has skipped data. #​12118
  • [ENHANCEMENT] Dashboards: Include absolute number of notifications attempted to alertmanager in 'Mimir / Ruler'. #​10918
  • [ENHANCEMENT] Alerts: Make MimirRolloutStuck a critical alert if it has been firing for 6h. #​10890
  • [ENHANCEMENT] Dashboards: Add panels to the Mimir / Tenants and Mimir / Top Tenants dashboards showing the rate of gateway requests. #​10978
  • [ENHANCEMENT] Alerts: Improve MimirIngesterFailsToProcessRecordsFromKafka to not fire during forced TSDB head compaction. #​11006
  • [ENHANCEMENT] Alerts: Add alerts for invalid cluster validation labels. #​11255 #​11282 #​11413
  • [ENHANCEMENT] Dashboards: Improve "Kafka 100th percentile end-to-end latency when ingesters are running (outliers)" panel, computing the baseline latency on max(10, 10%) of ingesters instead of a fixed 10 replicas. #​11581
  • [ENHANCEMENT] Dashboards: Add "per-query memory consumption" and "fallback to Prometheus' query engine" panels to the Queries dashboard. #​11626
  • [ENHANCEMENT] Alerts: Add MimirGoThreadsTooHigh alert. #​11836 #​11845
  • [ENHANCEMENT] Dashboards: Add autoscaling row for ruler query-frontends to Mimir / Remote ruler reads dashboard. #​11838
  • [BUGFIX] Dashboards: fix "Mimir / Tenants" legends for non-Kubernetes deployments. #​10891
  • [BUGFIX] Dashboards: fix Query-scheduler RPS panel legend in "Mimir / Reads". #​11515
  • [BUGFIX] Recording rules: fix cluster_namespace_deployment:actual_replicas:count recording rule when there's a mix on single-zone and multi-zone deployments. #​11287
  • [BUGFIX] Alerts: Enhance the MimirRolloutStuck alert, so it checks whether rollout groups as a whole (and not spread across instances) are changing or stuck. #​11288
Jsonnet
  • [CHANGE] Increase the allowed number of rule groups for small, medium_small, and extra_small user tiers by 20%. #​11152
  • [CHANGE] Update rollout-operator to latest release. #​11232 #​11748
  • [CHANGE] Memcached: Set a timeout of 500ms for the ruler-storage cache instead of the default 200ms. #​11231
  • [CHANGE] Ruler: If ingest storage is enabled, set the maximum buffered bytes in the Kafka client used by the ruler based on the expected maximum rule evaluation response size, clamping it between 1 GB (default) and 4 GB. #​11602
  • [CHANGE] All: Environment variable JAEGER_REPORTER_MAX_QUEUE_SIZE is no longer set. Components will use OTel's default value of 2048 unless explicitly configured. You can still configure JAEGER_REPORTER_MAX_QUEUE_SIZE if you configure tracing using Jaeger env vars, and you can always set OTEL_BSP_MAX_QUEUE_SIZE OTel configuration. #​11700
  • [CHANGE] Removed jaeger-agent-mixin and _config.jaeger_agent_host configuration. You can configure tracing using an OTLP endpoint through _config.otlp_traces_endpoint, see tracing.libsonnet for more configuration options. #​11773
  • [CHANGE] Removed ingester_stream_chunks_when_using_blocks option. #​11711
  • [CHANGE] Enable memberlist.abort-if-fast-join-fails for ingesters using memberlist #​11931 #​11950
  • [CHANGE] Remove average per-pod series scaling trigger for ingest storage ingester HPA and use one based on max owned series instead. #​11952
  • [CHANGE] Add store_gateway_grpc_max_query_response_size_bytes config option to set the max store-gateway gRCP query response send size (and corresponsing querier receive size), and set to 200MB by default. #​11968
  • [CHANGE] Removed support for the experimental read-write deployment mode. #​11974
  • [FEATURE] Make ingest storage ingester HPA behavior configurable through _config.ingest_storage_ingester_hpa_behavior. #​11168
  • [FEATURE] Add an alternate ingest storage HPA trigger that targets maximum owned series per pod. #​11356
  • [FEATURE] Make tracing of HTTP headers as span attributes configurable through _config.trace_request_headers. You can exclude certain headers from being traced using _config.trace_request_exclude_headers_list. #​11655 #​11714
  • [FEATURE] Allow configuring tracing with OTel environment variables through $._config.otlp_traces_endpoint. When configured, the $.jaeger_mixin is no longer available for use. #​11773 #​11981 #​12074
  • [FEATURE] Updated rollout-operator to support OTEL_ environment variables for tracing. #​11787
  • [ENHANCEMENT] Add query_frontend_only_args option to specify CLI flags that apply only to query-frontends but not ruler-query-frontends. #​11799
  • [ENHANCEMENT] Make querier scale up ($_config.autoscaling_querier_scaleup_percent_cap) and scale down rates ($_config.autoscaling_querier_scaledown_percent_cap) configurable. #​11862
  • [ENHANCEMENT] Set resource requests and limits for the Memcached Prometheus exporter. #​11933 #​11946
  • [ENHANCEMENT] Add assertion to ensure ingester ScaledObject has minimum and maximum replicas set to a value greater than 0. #​11979
  • [ENHANCEMENT] Add ingest_storage_migration_ignore_ingest_storage_errors and ingest_storage_migration_ingest_storage_max_wait_time configs to control error handling of the partition ingesters during ingest storage migrations. #​12105
  • [ENHANCEMENT] Add block-builder job processing duration timings and offset-skipped errors to the Block-builder dashboard. #​12118
  • [BUGFIX] Honor weight argument when building memory HPA query for resource scaled objects. #​11935
Mimirtool
  • [FEATURE] Add --enable-experimental-functions flag to commands that parse PromQL to allow parsing experimental functions such as sort_by_label().
  • [ENHANCEMENT] Add --block-size CLI flag to remote-read export that allows setting the output block size. #​12025
  • [BUGFIX] Fix issue where remote-read doesn't behave like other mimirtool commands for authentication. #​11402
  • [BUGFIX] Fix issue where remote-read export could omit some samples if the query time range spans multiple blocks. #​12025
  • [BUGFIX] Fix issue where remote-read export could omit some output blocks in the list printed to the console or fail with read/write on closed pipe. #​12025
Mimir Continuous Test
  • [FEATURE] Add -tests.client.cluster-validation.label flag to send the X-Cluster header with queries. #​11418
Query-tee
Documentation
  • [ENHANCEMENT] Update Thanos to Mimir migration guide with a tip to add the __tenant_id__ label. #​11584
  • [ENHANCEMENT] Update the MimirIngestedDataTooFarInTheFuture runbook with a note about false positives and the endpoint to flush TSDB blocks by user. #​11961
Tools
  • [ENHANCEMENT] kafkatool: Add offsets command for querying various partition offsets. #​11115
  • [ENHANCEMENT] listblocks: Output can now also be JSON or YAML for easier parsing. #​11184
  • [ENHANCEMENT] mark-blocks: Allow specifying blocks from multiple tenants. #​11343
  • [ENHANCEMENT] undelete-blocks: Support removing S3 delete markers to avoid copying data when recovering blocks. #​11256
  • [BUGFIX] screenshots: Update to tar-fs v3.1.0 to address CVE-2025-48387. #​12030

v2.16.1

Grafana Mimir
  • [BUGFIX] Update to Go v1.23.9 to address CVE-2025-22871. #​11543
  • [BUGFIX] Update golang.org/x/net to v0.38.0 to address CVE-2025-22872. #​11281
  • [BUGFIX] Query-frontend: Fix a panic in monolithic mode caused by a clash in labels of the cortex_client_invalid_cluster_validation_label_requests_total metric definition. #​11455

v2.16.0

Grafana Mimir
  • [CHANGE] Querier: pass context to queryable IsApplicable hook. #​10451
  • [CHANGE] Distributor: OTLP and push handler replace all non-UTF8 characters with the unicode replacement character \uFFFD in error messages before propagating them. #​10236
  • [CHANGE] Querier: pass query matchers to queryable IsApplicable hook. #​10256
  • [CHANGE] Build: removed Mimir Alpine Docker image and related CI tests. #​10469
  • [CHANGE] Query-frontend: Add topic label to cortex_ingest_storage_strong_consistency_requests_total, cortex_ingest_storage_strong_consistency_failures_total, and cortex_ingest_storage_strong_consistency_wait_duration_seconds metrics. #​10220
  • [CHANGE] Ruler: cap the rate of retries for remote query evaluation to 170/sec. This is configurable via -ruler.query-frontend.max-retries-rate. #​10375 #​10403
  • [CHANGE] Query-frontend: Add topic label to cortex_ingest_storage_reader_last_produced_offset_requests_total, cortex_ingest_storage_reader_last_produced_offset_failures_total, cortex_ingest_storage_reader_last_produced_offset_request_duration_seconds, cortex_ingest_storage_reader_partition_start_offset_requests_total, cortex_ingest_storage_reader_partition_start_offset_failures_total, cortex_ingest_storage_reader_partition_start_offset_request_duration_seconds metrics. #​10462
  • [CHANGE] Ingester: Set -ingester.ooo-native-histograms-ingestion-enabled to true by default. #​10483
  • [CHANGE] Ruler: Add user and reason labels to cortex_ruler_write_requests_failed_total and cortex_ruler_queries_failed_total; add user to cortex_ruler_write_requests_total and cortex_ruler_queries_total metrics. #​10536
  • [CHANGE] Querier / Query-frontend: Remove experimental -querier.promql-experimental-functions-enabled and -query-frontend.block-promql-experimental-functions CLI flags and respective YAML configuration options to enable experimental PromQL functions. Instead access to experimental PromQL functions is always blocked. You can enable them using the per-tenant setting enabled_promql_experimental_functions. #​10660 #​10712
  • [CHANGE] Store-gateway: Include posting sampling rate in sparse index headers. When the sampling rate isn't set in a sparse index header, store gateway rebuilds the sparse header with the configured blocks-storage.bucket-store.posting-offsets-in-mem-sampling value. If the sparse header's sampling rate is set but doesn't match the configured rate, store gateway either rebuilds the sparse header or downsamples to the configured sampling rate. #​10684 #​10878
  • [CHANGE] Distributor: Return specific error message when burst size limit is exceeded. #​10835
  • [CHANGE] Ingester: enable native histograms ingestion by default, meaningingester.native-histograms-ingestion-enabled defaults to true. #​10867
  • [FEATURE] Query Frontend: Expose query stats in the Server-Timing header when the X-Mimir-Response-Query-Stats: true header is present in the request. #​10192
  • [FEATURE] Distributor: Add experimental -distributor.otel-keep-identifying-resource-attributes option to allow keeping service.instance.id, service.name and service.namespace in target_info on top of converting them to the instance and job labels. #​10216
  • [FEATURE] Ingester/Distributor: Add support for exporting cost attribution metrics (cortex_ingester_attributed_active_series, cortex_distributor_received_attributed_samples_total, and cortex_discarded_attributed_samples_total) with labels specified by customers to a custom Prometheus registry. This feature enables more flexible billing data tracking. #​10269 #​10702
  • [FEATURE] Ruler: Added /ruler/tenants endpoints to list the discovered tenants with rule groups. #​10738
  • [FEATURE] Distributor: Add experimental Influx handler. #​10153
  • [FEATURE] Query-frontend: Configuration options query-frontend.cache-errors and query-frontend.results-cache-ttl-for-errors for caching non-transient error responses are no longer experimental. #​10927
  • [FEATURE] Distributor: Add experimental memberlist KV store for ha_tracker. You can enable it using the -distributor.ha-tracker.kvstore.store flag. You can configure Memberlist parameters via the -memberlist-* flags. #​10054
  • [ENHANCEMENT] Compactor: Expose cortex_bucket_index_last_successful_update_timestamp_seconds for all tenants assigned to the compactor before starting the block cleanup job. #​10569
  • [ENHANCEMENT] Query Frontend: Return server-side samples_processed statistics. #​10103
  • [ENHANCEMENT] Distributor: OTLP receiver now converts also metric metadata. See also prometheus/prometheus#15416. #​10168
  • [ENHANCEMENT] Distributor: discard float and histogram samples with duplicated timestamps from each timeseries in a request before the request is forwarded to ingesters. Discarded samples are tracked by cortex_discarded_samples_total metrics with the reason sample_duplicate_timestamp. #​10145 #​10430
  • [ENHANCEMENT] Ruler: Add cortex_prometheus_rule_group_last_rule_duration_sum_seconds metric to track the total evaluation duration of a rule group regardless of concurrency #​10189
  • [ENHANCEMENT] Distributor: Add native histogram support for electedReplicaPropagationTime metric in ha_tracker. #​10264
  • [ENHANCEMENT] Ingester: More efficient CPU/memory utilization-based read request limiting. #​10325
  • [ENHANCEMENT] OTLP: In addition to the flag -distributor.otel-created-timestamp-zero-ingestion-enabled there is now -distributor.otel-start-time-quiet-zero to convert OTel start timestamps to Prometheus QuietZeroNaNs. This flag is to make the change rollout safe between Ingesters and Distributors. #​10238
  • [ENHANCEMENT] Ruler: When rule concurrency is enabled for a rule group, its rules will now be reordered and run in batches based on their dependencies. This increases the number of rules that can potentially run concurrently. Note that the global and tenant-specific limits still apply #​10400
  • [ENHANCEMENT] Query-frontend: include more information about read consistency in trace spans produced when using experimental ingest storage. #​10412
  • [ENHANCEMENT] Ingester: Hide tokens in ingester ring status page when ingest storage is enabled #​10399
  • [ENHANCEMENT] Ingester: add active_series_additional_custom_trackers configuration, in addition to the already existing active_series_custom_trackers. The active_series_additional_custom_trackers configuration allows you to configure additional custom trackers that get merged with active_series_custom_trackers at runtime. #​10428
  • [ENHANCEMENT] Query-frontend: Allow blocking raw http requests with the blocked_requests configuration. Requests can be blocked based on their path, method or query parameters #​10484
  • [ENHANCEMENT] Ingester: Added the following metrics exported by PostingsForMatchers cache: #​10500 #​10525
    • cortex_ingester_tsdb_head_postings_for_matchers_cache_hits_total
    • cortex_ingester_tsdb_head_postings_for_matchers_cache_misses_total
    • cortex_ingester_tsdb_head_postings_for_matchers_cache_requests_total
    • cortex_ingester_tsdb_head_postings_for_matchers_cache_skips_total
    • cortex_ingester_tsdb_head_postings_for_matchers_cache_evictions_total
    • cortex_ingester_tsdb_block_postings_for_matchers_cache_hits_total
    • cortex_ingester_tsdb_block_postings_for_matchers_cache_misses_total
    • cortex_ingester_tsdb_block_postings_for_matchers_cache_requests_total
    • cortex_ingester_tsdb_block_postings_for_matchers_cache_skips_total
    • cortex_ingester_tsdb_block_postings_for_matchers_cache_evictions_total
  • [ENHANCEMENT] Add support for the HTTP header X-Filter-Queryables which allows callers to decide which queryables should be used by the querier, useful for debugging and testing queryables in isolation. #​10552 #​10594
  • [ENHANCEMENT] Compactor: Shuffle users' order in BlocksCleaner. Prevents bucket indexes from going an extended period without cleanup during compactor restarts. #​10513
  • [ENHANCEMENT] Distributor, querier, ingester and store-gateway: Add support for limit parameter for label names and values requests. #​10410
  • [ENHANCEMENT] Ruler: Adds support for filtering results from rule status endpoint by file[], rule_group[] and rule_name[]. #​10589
  • [ENHANCEMENT] Query-frontend: Add option to "spin off" subqueries as actual range queries, so that they benefit from query acceleration techniques such as sharding, splitting, and caching. To enable this feature, set the -query-frontend.instant-queries-with-subquery-spin-off=<comma separated list> option on the frontend or the instant_queries_with_subquery_spin_off per-tenant override with regular expressions matching the queries to enable. #​10460 #​10603 #​10621 #​10742 #​10796
  • [ENHANCEMENT] Querier, ingester: The series API respects passed limit parameter. #​10620 #​10652
  • [ENHANCEMENT] Store-gateway: Add experimental settings under -store-gateway.dynamic-replication to allow more than the default of 3 store-gateways to own recent blocks. #​10382 #​10637
  • [ENHANCEMENT] Ingester: Add reactive concurrency limiters to protect push and read operations from overload. #​10574
  • [ENHANCEMENT] Compactor: Add experimental -compactor.max-lookback option to limit blocks considered in each compaction cycle. Blocks uploaded prior to the lookback period aren't processed. This option helps reduce CPU utilization in tenants with large block metadata files that are processed before each compaction. #​10585 #​10794
  • [ENHANCEMENT] Distributor: Optionally expose the current HA replica for each tenant in the cortex_ha_tracker_elected_replica_status metric. This is enabled with the -distributor.ha-tracker.enable-elected-replica-metric=true flag. #​10644
  • [ENHANCEMENT] Enable three Go runtime metrics: #​10641
    • go_cpu_classes_gc_total_cpu_seconds_total
    • go_cpu_classes_total_cpu_seconds_total
    • go_cpu_classes_idle_cpu_seconds_total
  • [ENHANCEMENT] All: Add experimental support for cluster validation in gRPC calls. When it is enabled, gRPC server verifies if a request coming from a gRPC client comes from an expected cluster. This validation can be configured by the following experimental configuration options: #​10767
    • -server.cluster-validation.label
    • -server.cluster-validation.grpc.enabled
    • -server.cluster-validation.grpc.soft-validation
  • [ENHANCEMENT] gRPC clients: Add experimental support to include the cluster validation label in gRPC metadata. When cluster validation is enabled on gRPC server side, the cluster validation label from gRPC metadata is compared with the gRPC server's cluster validation label. #​10869 #​10883
    • By setting -<grpc-client-config-path>.cluster-validation.label, you configure the cluster validation label of a single gRPC client, whose grpcclient.Config object is configurable through -<grpc-client-config-path>.
    • By setting -common.client-cluster-validation.label, you configure the cluster validation label of all gRPC clients.
  • [ENHANCEMENT] gRPC clients: Add cortex_client_request_invalid_cluster_validation_labels_total metrics, that are used by Mimir's gRPC clients to track invalid cluster validations. #​10767
  • [ENHANCEMENT] Add experimental metric cortex_distributor_dropped_native_histograms_total to measure native histograms silently dropped when native histograms are disabled for a tenant. #​10760
  • [ENHANCEMENT] Compactor: Add experimental -compactor.upload-sparse-index-headers option. When enabled, the compactor will attempt to upload sparse index headers to object storage. This prevents latency spikes after adding store-gateway replicas. #​10684
  • [ENHANCEMENT] Ruler: add support for YAML aliases in alert, record and expr fields in rule groups. prometheus/prometheus#14957 #​10884
  • [ENHANCEMENT] Memcached: Add experimental -<prefix>.memcached.addresses-provider flag to use alternate DNS service discovery backends when discovering Memcached hosts. #​10895
  • [BUGFIX] Distributor: Use a boolean to track changes while merging the ReplicaDesc components, rather than comparing the objects directly. #​10185
  • [BUGFIX] Querier: fix timeout responding to query-frontend when response size is very close to -querier.frontend-client.grpc-max-send-msg-size. #​10154
  • [BUGFIX] Query-frontend and querier: show warning/info annotations in some cases where they were missing (if a lazy querier was used). #​10277
  • [BUGFIX] Query-frontend: Fix an issue where transient errors are inadvertently cached. #​10537 #​10631
  • [BUGFIX] Ruler: fix indeterminate rules being always run concurrently (instead of never) when -ruler.max-independent-rule-evaluation-concurrency is set. prometheus/prometheus#15560 #​10258
  • [BUGFIX] PromQL: Fix various UTF-8 bugs related to quoting. prometheus/prometheus#15531 #​10258
  • [BUGFIX] Ruler: Fixed an issue when using the experimental -ruler.max-independent-rule-evaluation-concurrency feature, where if a rule group was eligible for concurrency, it would flap between running concurrently or not based on the time it took after running concurrently. #​9726 #​10189
  • [BUGFIX] Mimirtool: remote-read commands will now return data. #​10286
  • [BUGFIX] PromQL: Fix deriv, predict_linear and double_exponential_smoothing with histograms prometheus/prometheus#15686 #​10383
  • [BUGFIX] MQE: Fix deriv with histograms #​10383
  • [BUGFIX] PromQL: Fix <aggr_over_time> functions with histograms prometheus/prometheus#15711 #​10400
  • [BUGFIX] MQE: Fix <aggr_over_time> functions with histograms #​10400
  • [BUGFIX] Distributor: return HTTP status 415 Unsupported Media Type instead of 200 Success for Remote Write 2.0 until we support it. #​10423 #​10916
  • [BUGFIX] Query-frontend: Add flag -query-frontend.prom2-range-compat and corresponding YAML to rewrite queries with ranges that worked in Prometheus 2 but are invalid in Prometheus 3. #​10445 #​10461 #​10502
  • [BUGFIX] Distributor: Fix edge case at the HA-tracker with memberlist as KVStore, where when a replica in the KVStore is marked as deleted but not yet removed, it fails to update the KVStore. #​10443
  • [BUGFIX] Distributor: Fix panics in DurationWithJitter util functions when computed variance is zero. #​10507
  • [BUGFIX] Ingester: Fixed a race condition in the PostingsForMatchers cache that may have infrequently returned expired cached postings. #​10500
  • [BUGFIX] Distributor: Report partially converted OTLP requests with status 400 Bad Request. #​10588
  • [BUGFIX] Ruler: fix issue where rule evaluations could be missed while shutting down a ruler instance if that instance owns many rule groups. prometheus/prometheus#15804 #​10762
  • [BUGFIX] Ingester: Add additional check on reactive limiter queue sizes. #​10722
  • [BUGFIX] TSDB: fix unknown series errors and possible lost data during WAL replay when series are removed from the head due to inactivity and reappear before the next WAL checkpoint. prometheus/prometheus#16060 prometheus/prometheus#16231 #​10824 #​10955
  • [BUGFIX] Querier: fix issue where label_join could incorrectly return multiple series with the same labels rather than failing with vector cannot contain metrics with the same labelset. prometheus/prometheus#15975 #​10826
  • [BUGFIX] Querier: fix issue where counter resets on native histograms could be incorrectly under- or over-counted when using subqueries. prometheus/prometheus#15987 #​10871
  • [BUGFIX] Querier: fix incorrect annotation emitted when quantile_over_time is evaluated over a range with both histograms and floats. prometheus/prometheus#16018 #​10884
  • [BUGFIX] Querier: fix duplicated double quotes in invalid label name error from count_values. prometheus/prometheus#16054 #​10884
  • [BUGFIX] Ingester: fix goroutines and memory leak when experimental ingest storage enabled and a server-side error occurs during metrics ingestion. #​10915
  • [BUGFIX] Alertmanager: Avoid fetching Grafana state if Grafana AM compatibility is not enabled. #​10857
  • [BUGFIX] Alertmanager: Fix decoding of queryFromGeneratorURL in templates. #​8914
  • [BUGFIX] Alertmanager: DedupStage to stop notification pipeline when the timestamp of notification log entry is after the pipeline was flushed #​10989
Mixin
  • [CHANGE] Alerts: Only alert on errors performing cache operations if there are over 10 request/sec to avoid flapping. #​10832
  • [FEATURE] Add compiled mixin for GEM installations in operations/mimir-mixin-compiled-gem. #​10690 #​10877
  • [ENHANCEMENT] Dashboards: clarify that the ingester and store-gateway panels on the 'Reads' dashboard show data from all query requests to that component, not just requests from the main query path (ie. requests from the ruler query path are included as well). #​10598
  • [ENHANCEMENT] Dashboards: add ingester and store-gateway panels from the 'Reads' dashboard to the 'Remote ruler reads' dashboard as well. #​10598
  • [ENHANCEMENT] Dashboards: add ingester and store-gateway panels showing only requests from the respective dashboard's query path to the 'Reads' and 'Remote ruler reads' dashboards. For example, the 'Remote ruler reads' dashboard now has panels showing the ingester query request rate from ruler-queriers. #​10598
  • [ENHANCEMENT] Dashboards: 'Writes' dashboard: show write requests broken down by request type. #​10599
  • [ENHANCEMENT] Dashboards: clarify when query-frontend and query-scheduler dashboard panels are expected to show no data. #​10624
  • [ENHANCEMENT] Alerts: Add warning alert DistributorGcUsesTooMuchCpu. #​10641
  • [ENHANCEMENT] Dashboards: Add "Federation-frontend" dashboard for GEM. #​10697 #​10736
  • [ENHANCEMENT] Dashboards: Add Query-Scheduler <-> Querier Inflight Requests row to Query Reads and Remote Ruler reads dashboards. #​10290
  • [ENHANCEMENT] Alerts: Add "Federation-frontend" alert for remote clusters returning errors. #​10698
  • [BUGFIX] Dashboards: fix how we switch between classic and native histograms. #​10018
  • [BUGFIX] Alerts: Ignore cache errors performing delete operations since these are expected to fail when keys don't exist. #​10287
  • [BUGFIX] Dashboards: fix "Mimir / Rollout Progress" latency comparison when gateway is enabled. #​10495
  • [BUGFIX] Dashboards: fix autoscaling panels when Mimir is deployed using Helm. #​10473
  • [BUGFIX] Alerts: fix MimirAutoscalerNotActive alert. #​10564
Jsonnet
  • [CHANGE] Update rollout-operator version to 0.23.0. #​10229 #​10750
  • [CHANGE] Memcached: Update to Memcached 1.6.34. #​10318
  • [CHANGE] Change multi-AZ deployments default toleration value from 'multi-az' to 'secondary-az', and make it configurable via the following settings: #​10596
    • _config.multi_zone_schedule_toleration (default)
    • _config.multi_zone_distributor_schedule_toleration (distributor's override)
    • _config.multi_zone_etcd_schedule_toleration (etcd's override)
  • [CHANGE] Ring: relaxed the hash ring heartbeat timeout for store-gateways: #​10634
    • -store-gateway.sharding-ring.heartbeat-timeout set to 10m
  • [CHANGE] Memcached: Use 3 replicas for all cache types by default. #​10739
  • [ENHANCEMENT] Enforce persistentVolumeClaimRetentionPolicy Retain policy on partition ingesters during migration to experimental ingest storage. #​10395
  • [ENHANCEMENT] Allow to not configure topologySpreadConstraints by setting the following configuration options to a negative value: #​10540
    • distributor_topology_spread_max_skew
    • query_frontend_topology_spread_max_skew
    • querier_topology_spread_max_skew
    • ruler_topology_spread_max_skew
    • ruler_querier_topology_spread_max_skew
  • [ENHANCEMENT] Validate the $._config.shuffle_sharding.ingester_partitions_shard_size value when partition shuffle sharding is enabled in the ingest-storage mode. #​10746
  • [BUGFIX] Ports in container rollout-operator. #​10273
  • [BUGFIX] When downscaling is enabled, the components must annotate prepare-downscale-http-port with the value set in $._config.server_http_port. #​10367
Mimirtool
  • [BUGFIX] Fix issue where MIMIR_HTTP_PREFIX environment variable was ignored and the value from MIMIR_MIMIR_HTTP_PREFIX was used instead. #​10207
  • [ENHANCEMENT] Unify mimirtool authentication options and add extra-headers support for commands that depend on MimirClient. #​10178
  • [ENHANCEMENT] mimirtool grafana analyze now supports custom panels. #​10669
  • [ENHANCEMENT] mimirtool grafana analyze now supports bar chart, pie chart, state timeline, status history, histogram, candlestick, canvas, flame graph, geomap, node graph, trend, and XY chart panels. #​10669
Mimir Continuous Test
Query-tee
  • [ENHANCEMENT] Allow skipping comparisons when preferred backend fails. Disabled by default, enable with -proxy.compare-skip-preferred-backend-failures=true. #​10612
Documentation
  • [CHANGE] Add production tips related to cache size, heavy multi-tenancy and latency spikes. #​9978
  • [ENHANCEMENT] Update MimirAutoscalerNotActive and MimirAutoscalerKedaFailing runbooks, with an instruction to check whether Prometheus has enough CPU allocated. #​10257
Tools
  • [CHANGE] copyblocks: Remove /pprof endpoint. #​10329
  • [CHANGE] mark-blocks: Replace markblocks with added features including removing markers and reading block identifiers from a file. #​10597

v2.15.3

Grafana Mimir
Mimirtool
  • [BUGFIX] Upgrade Alpine Linux to 3.20.6, fixes CVE-2025-26519. #​11530
Mimir Continuous Test
  • [BUGFIX] Upgrade Alpine Linux to 3.20.6, fixes CVE-2025-26519. #​11530

v2.15.2

Grafana Mimir

v2.15.1

Grafana Mimir

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻️ Rebasing: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this MR and you won't be reminded about this update again.

  • If you want to rebase/retry this MR, check this box

This MR has been generated by Renovate Bot.

Edited by Soos

Merge request reports