Skip to content

GitLab Next

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
GitLab FOSS
GitLab FOSS
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
    • Iterations
  • Merge Requests 1
    • Merge Requests 1
  • Requirements
    • Requirements
    • List
  • Security & Compliance
    • Security & Compliance
    • Dependency List
    • License Compliance
  • Operations
    • Operations
    • Incidents
    • Environments
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • Code Review
    • Insights
    • Issue
    • Repository
    • Value Stream
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
  • GitLab.org
  • GitLab FOSSGitLab FOSS
  • Merge Requests
  • !28996

Merged
Opened May 31, 2019 by Mayra Cabrera@mayra-cabrera⚡Maintainer13 of 14 tasks completed13/14 tasks

Step 2 of #62214: Includes logic to update namespaces storage statistics

  • Overview 134
  • Commits 7
  • Pipelines 44
  • Changes 20

What does this MR do?

This is the second part of #62214 (closed). This merge request includes logic to update namespaces storage statistics in an async way

Built upon https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/29570

Does this MR meet the acceptance criteria?

Conformity

  • [-] Changelog entry
  • [-] Documentation created/updated or follow-up review issue created
  • Code review guidelines
  • Merge request performance guidelines
  • Style guides
  • Database guides
  • Separation of EE specific content

Performance and testing

  • Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
  • [-] Tested in all supported browsers

Closes #62214 (closed)

To do (backend)

  • Whenever the statistics of a project changes, we insert a row into namespace_aggregation_schedules
    • We don't insert a new row if there's already one related to the namespace.
    • Insertion is done through a callback and with a Sidekiq job. We can't do it in the same transaction as ProjectStatistics is already involved in a large one (https://gitlab.com/gitlab-org/gitlab-ce/issues/62488)
  • After inserting the row, we schedule a Namespaces::RootStatisticsWorker job, 3 hours into the future.
  • This job will:
    • Update the namespace storage statistics through Namespaces::StatisticsRefresherService
    • Delete the related namespace_aggregation_schedules after the update
  • Create the service that will update the namespace statistics (Namespaces::StatisticsRefresherService)
  • We also need to create another Sidekiq job that will traverse any remaining rows on namespace_aggregation_schedules and schedule jobs for every pending row (Namespaces::PruneAggregationSchedulesWorker)
  • Hide changes behind a group feature flag
  • Read the interval of caching time form redis defaulting to once every 3 hours
  • Refactor and add specs.
Edited Jun 26, 2019 by Mayra Cabrera
Assignee
Assign to
Reviewer
Request review from
12.1
Milestone
12.1 (Past due)
Assign milestone
Time tracking
Reference: gitlab-org/gitlab-foss!28996
Source branch: 62214-namespaces-storage-statistics-on-database

Revert this merge request

This will create a new commit in order to revert the existing changes.

Switch branch
Cancel
A new branch will be created in your fork and a new merge request will be started.

Cherry-pick this merge request

Switch branch
Cancel
A new branch will be created in your fork and a new merge request will be started.