Skip to content

GitLab Next

    • GitLab: the DevOps platform
    • Explore GitLab
    • Install GitLab
    • How GitLab compares
    • Get started
    • GitLab docs
    • GitLab Learn
  • Pricing
  • Talk to an expert
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
    • Menu
    Projects Groups Snippets
  • Sign up now
  • Login
  • Sign in / Register
  • GitLab FOSS GitLab FOSS
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 0
    • Merge requests 0
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages & Registries
    • Packages & Registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Metrics
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • Code review
    • Insights
    • Issue
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
Collapse sidebar
  • GitLab.org
  • GitLab FOSSGitLab FOSS
  • Issues
  • #52247
Closed (moved) (moved)
Open
Created Oct 04, 2018 by James Ramsay (ex-GitLab)@jramsay-gitlab🔴Contributor

Partition the events table by year

We have a prune events worker which prunes user activity every 2 years.

This was extended from 12 months in https://gitlab.com/gitlab-org/gitlab-ce/issues/52246, where we implemented this short-term fix to give us some additional time to consider a scaling strategy for the related table (the events table). This fix was merged in 11.4, which means a more permanent solution must take place by October 2019.

This data is very useful, and we should not ever prune data unless explicitly done by an instance administrator.

Further details

See context from @yorickpeterse in https://gitlab.com/gitlab-org/gitlab-ce/issues/24244#note_60995986 on the DB challenges.

Proposal

  • Partition the events table by range on the created_at column.
  • Tables should be created for each year with a schema like events-yyyy specifying the year of the relevant records.
  • Once events is no longer being pruned, we should remove prune_old_events_worker.rb.

Links / references

  • prune events worker: https://gitlab.com/gitlab-org/gitlab-ce/blob/1fbda39c8dd8774d13832d5492e800b66ba90d6d/app/workers/prune_old_events_worker.rb
Edited Jul 30, 2019 by Jeremy Watson (ex-GitLab)
Assignee
Assign to
Time tracking