Importer log messages are absent on Kibana
Problem
Importer log entries are absent in Kibana
Some of our importers produce log messages during failure handling that are sent to the importer.log file. One example is the GitHub Importer
Prior to GitLab.com switching to Kubernetes, the Omnibus log file at /var/log/gitlab/gitlab-rails/importer.log used to be available for search in the indexed data on Kibana. This appears to have regressed after we moved to using Kubernetes.
The logs are required for the GitLab Support team (and other teams) to diagnose unexpected issues with the use of our various importers.
Related Slack thread: https://gitlab.slack.com/archives/CB3LSMEJV/p1647261249242809
Desired outcome
Importer log entries are searchable in Kibana
Acceptance criteria
-
Entries such as the code line here should be searchable on Kibana's GPRD indexes (Rails/Sidekiq)