2020-04-23 Recent registry logs are missing from Elasticsearch
Summary
Less than 24 hours ago, the Elasticsearch index for registry logs has no events.
This might be related to https://gitlab.com/gitlab-com/gl-infra/k8s-workloads/logging/-/merge_requests/20
From Slack conversation with @mwasilewski-gitlab:
timing of registry logs certainly would suggest the two are related, here's the job that deployed the change: https://ops.gitlab.net/gitlab-com/gl-infra/k8s-workloads/logging/-/jobs/1114367
and the log seemed to have stop being sent around the same time: https://log.gprd.gitlab.net/goto/3287c0bf2f7333c5c31d8ed58a8475d6
More information will be added as we investigate the issue.
Timeline
All times UTC.
2020-04-22
- 17:30 - Registry logs stop appearing in Elasticsearch index
pubsub-registry-inf-gprd*