GithubImporter: Count fetched/import objects globally and by project

What does this MR do?

Github Importer counts the number of fetched and imported objects, scoped by project (redis) for report and globally (prometheus), without scoping by projects, for healthy dashboards.

Github Labels, Releases and Milestones uses a different approach to be imported from the other objects and these objects were not being counted, or logged, in the same way of the rest of Github objects.

Related to: #335199 (closed)

Screenshots or Screencasts (strongly suggested)

{"message":"starting importer","importer":"Gitlab::GithubImport::Importer::LabelsImporter","import_source":"github","project_id":32,"import_stage":"Gitlab::GithubImport::Stage::ImportBaseDataWorker"}
{"import_source":"github","project_id":32,"importer":"Gitlab::GithubImport::Importer::LabelsImporter","message":"15 labels fetched"}
{"import_source":"github","project_id":32,"importer":"Gitlab::GithubImport::Importer::LabelsImporter","message":"15 labels imported"}
{"message":"starting importer","importer":"Gitlab::GithubImport::Importer::MilestonesImporter","import_source":"github","project_id":32,"import_stage":"Gitlab::GithubImport::Stage::ImportBaseDataWorker"}
{"import_source":"github","project_id":32,"importer":"Gitlab::GithubImport::Importer::MilestonesImporter","message":"2 milestones fetched"}
{"import_source":"github","project_id":32,"importer":"Gitlab::GithubImport::Importer::MilestonesImporter","message":"2 milestones imported"}
{"message":"starting importer","importer":"Gitlab::GithubImport::Importer::ReleasesImporter","import_source":"github","project_id":32,"import_stage":"Gitlab::GithubImport::Stage::ImportBaseDataWorker"}
{"import_source":"github","project_id":32,"importer":"Gitlab::GithubImport::Importer::ReleasesImporter","message":"31 releases fetched"}
{"import_source":"github","project_id":32,"importer":"Gitlab::GithubImport::Importer::ReleasesImporter","message":"31 releases imported"}

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Kassio Borges

Merge request reports

Loading