Store GitHub Import stats in DB
Problem
Currently, when the GitHub import is completed(finished) we are showing the statistics of imported/fetched items.
On the UI for partial import only, so actually based on these stats we can detect whether it was a partially completed:
On the apiv4 for all finished imports via {host}/api/v4/projects/{id}/import
endpoint:
Proposal
Store stats that are currently handled by Gitlab::GithubImport::ObjectCounter
in the DB in the same table where the status is stored so it will not disappear after 2 days (Redis cache expiration time). These data should be put to DB after the import is finished. Based on this statistic we can detect whether the import was Partially Completed
or fully Completed
and have a general end state of import: Completed, Partially completed, Failed, Cancelled.
Technilac details
In addition change all places where current stats are fetched to be fetched from DB and not from the Redis Cache so it will not disappear when Redis key is expired. Most likely that it should be column for ProjectImportState#checksums
(project_mirror_data.checksums
column)
Import::GithubController#realtime_changes
API::Entities::ProjectImportStatus