Add Reliability Metrics to Github Importer Grafana Dashboard
Current Github Grafana Dashboard
We currently have a simple grafana dashboard that counts some object importing and limit rates
Proposed changed
The current dashboard leverages from a global counter of imported objects (https://gitlab.com/gitlab-org/gitlab/blob/master/app/workers/concerns/gitlab/github_import/object_importer.rb#L50). Using the introduction of the number of fetched and imported objects for each import log (!64256 (merged)) as example, a similar global counter for the fetched objects could be introduced.
Having the expected number of objects (fetched number of object) and the imported number of objects enables us to calculate a success rate which would be imported/fetched.
Ideally we could have a global success rate and by object type success rates, like:
- Global success rate: sum(all imported objects)/sum(all fetched objects)
- Merge Requests imported success rate: number of imported merge requests/number of fetched merge requests
- Issues imported success rate: number of imported issues/number of fetched issues
- ...
TODO
-
Count fetched and imported objects globally !65700 (merged) -
Count objects imported in batches ( Gitlab::GithubImport::BulkImporting) !65773 (merged)-
Enable the feature flag created import_redis_increment_byon staging -
Enable the feature flag created import_redis_increment_byon production
-
-
Update dashboard to use the new counters (gitlab-com/runbooks!3793 (merged), gitlab-com/runbooks!3817 (merged)) -
Share the dashboard with the group in slack -
Add the dashboard link to the dev documentation
Edited by Kassio Borges