Prometheus triggering kernel panic


Summary

As reported by the customer:

One of our GitLab rails hosts crashed and rebooted. It appears that prometheus caused a kernel panic. We don't currently have prometheus configured in our setup. Do you have any idea why this would happen?

Steps to reproduce

This has only occurred once and we are unable to reproduce the issue.

What is the current bug behavior?

  1. Kernel panic triggered
  2. Server restarts
  3. Comes back online working again

What is the expected correct behavior?

Kernel calls should not cause a kernel panic

Relevant logs and/or screenshots

For privacy reasons please see internal Zendesk support issue for log output: https://gitlab.zendesk.com/agent/tickets/86291

Output of checks

Results of GitLab environment info

Customer is on 10.0.4. Have requested this information.

Results of GitLab application Check

Possible fixes

This is the only kernel call that I can find that is made by Prometheus: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/models/project_services/prometheus_service.rb#L82

I think we should review what is happening here.

Edited Dec 15, 2017 by Adam Mulvany
Assignee Loading
Time tracking Loading