Skip to content

[SPIKE] Evaluate deprecation of project statistics increment in Git operations

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Context

The Project statistics API retrieves the number of times the project is either cloned or pulled with the HTTP methods (SSH fetches are not included). Internally, every Git clone/pull operation increments the project statistics fetch counter.

Problem

Due to performance issues, the Projects::FetchStatisticsIncrementService was executed in every git-upload-pack, leading to significant drops in the workhorse_auth_api Apdex. To address this, the disable_git_http_fetch_writes FF was created and enabled in production. However, this solution does not apply to Dedicated instances, as feature flags are not supported on GitLab Dedicated.

Incidents

Proposal

We must reevaluate the current implementation and find a solution that doesn't involve writing to the database during Git operations.

Validate with the product team how relevant this API is for our users and customers.

If this is still relevant:

  1. As described in this comment. Evaluate if we can use historical log data to return this counter instead of using our primary OLTP database as a time-series database.
  2. Evaluate if we should include missing SSH fetches in the statistics.
  3. What other approaches should we consider?

If this is not relevant:

  1. Identify how many references we have for the ProjectDailyStatistic model.
  2. Check if this model's removal/deprecation is feasible and define the following steps needed.
  3. Define the following steps to deprecate the Project statistics API. Reference: https://docs.gitlab.com/ee/development/documentation/restful_api_styleguide.html#deprecations
Edited by 🤖 GitLab Bot 🤖