Increase in 500 errors due to Prometheus integration not found

Summary

A GitLab Dedicated customer on version 17.9.3 reported an increase in HTTP 500 errors since 2025/04/04 8AM UTC. For the majority of the errors, it seems to be generally affecting 5 projects, although there might be others.

We identified in the logs undefined method can_query?' for nil:NilClass error on lib/gitlab/prometheus/adapter.rb:14:in prometheus_adapter (full stacktrace in below), and it seems to be related to the recent removal of the Prometheus integration (!175395 (merged)).

Users are seeing this when loading the MR: Something went wrong while fetching the environments for this merge request. Please try again.
image

Steps to reproduce

  • Create a prometheus integration (project level works).
    • Do this, either in the rails console, or disable the remove_monitor_metrics feature flag to enable the UI again.
  • Go to Settings -> General -> Integration Settings and make it allow-list only, ensuring that Prometheus is not checked.
  • Create a deployment and environment in the Rails console.
  • Gitlab::Prometheus::Adapter.new(proj, deployment).prometheus_adapter should fail with the reported error message.

Example Project

What is the current bug behavior?

What is the expected correct behavior?

Relevant logs and/or screenshots

Projects::MergeRequestsController#ci_environments_status

NoMethodError
undefined method `can_query?' for nil:NilClass

lib/gitlab/prometheus/adapter.rb:14:in `prometheus_adapter', app/models/deployment_metrics.rb:37:in `block in prometheus_adapter', app/models/deployment_metrics.rb:36:in `prometheus_adapter', app/models/deployment_metrics.rb:16:in `has_metrics?', app/models/environment_status.rb:61:in `block in has_metrics?', app/models/environment_status.rb:60:in `has_metrics?', app/serializers/environment_status_entity.rb:57:in `has_metrics?', app/serializers/environment_status_entity.rb:14:in `block in <class:EnvironmentStatusEntity>', app/serializers/base_serializer.rb:16:in `represent', app/controllers/projects/merge_requests_controller.rb:339:in `ci_environments_status', lib/gitlab/ip_address_state.rb:11:in `with', ee/app/controllers/ee/application_controller.rb:45:in `set_current_ip_address', app/controllers/application_controller.rb:491:in `set_current_admin', lib/gitlab/session.rb:11:in `with_session', app/controllers/application_controller.rb:482:in `set_session_storage', lib/gitlab/i18n.rb:114:in `with_locale', lib/gitlab/i18n.rb:120:in `with_user_locale', app/controllers/application_controller.rb:473:in `set_locale', app/controllers/application_controller.rb:466:in `set_current_context', lib/gitlab/middleware/action_controller_static_context.rb:23:in `call', lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call', lib/gitlab/middleware/sidekiq_shard_awareness_validation.rb:20:in `block in call', lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled', lib/gitlab/middleware/sidekiq_shard_awareness_validation.rb:20:in `call', lib/gitlab/middleware/memory_report.rb:13:in `call', lib/gitlab/middleware/speedscope.rb:13:in `call', lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call', lib/gitlab/middleware/rails_queue_duration.rb:33:in `call', lib/gitlab/etag_caching/middleware.rb:21:in `call', lib/gitlab/metrics/rack_middleware.rb:16:in `block in call', lib/gitlab/metrics/web_transaction.rb:46:in `run', lib/gitlab/metrics/rack_middleware.rb:16:in `call', lib/gitlab/middleware/go.rb:21:in `call', lib/gitlab/middleware/query_analyzer.rb:11:in `block in call', lib/gitlab/database/query_analyzer.rb:83:in `within', lib/gitlab/middleware/query_analyzer.rb:11:in `call', lib/ci/job_token/middleware.rb:11:in `call', lib/gitlab/middleware/multipart.rb:173:in `call', lib/gitlab/middleware/read_only/controller.rb:50:in `call', lib/gitlab/middleware/read_only.rb:18:in `call', lib/gitlab/middleware/unauthenticated_session_expiry.rb:18:in `call', lib/gitlab/middleware/same_site_cookies.rb:27:in `call', lib/gitlab/middleware/path_traversal_check.rb:35:in `call', lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call', lib/gitlab/middleware/basic_health_check.rb:25:in `call', lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call', lib/gitlab/middleware/request_context.rb:15:in `call', lib/gitlab/middleware/webhook_recursion_detection.rb:15:in `call', config/initializers/fix_local_cache_middleware.rb:11:in `call', lib/gitlab/middleware/compressed_json.rb:44:in `call', lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call', lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call', lib/gitlab/metrics/requests_rack_middleware.rb:83:in `call', lib/gitlab/middleware/release_env.rb:12:in `call'

Output of checks

This bug happens on GitLab Dedicated

Possible fixes

  • Ensure any call to #find_or_initialize_integration handles nil correctly.
Edited by Max Woolf