Sidekiq memory leaks on GitLab.com

Ever since deploying GitLab 7.4 Enterprise Edition on GitLab.com we have been experiencing memory leaks with Sidekiq. The graph below shows a plateau at 2GB RSS before 7.4, with wild peaks exceeding 10GB starting around the deploy of 7.4.

omnibus_gitlab_sidekiq_rss-month

To our knowledge no other GitLab installations are experiencing these Sidekiq memory issues. We are creating this issue in case anybody else does run into it, and to share what we learn in our investigation.

cc @dzaporozhets @sytses @vsizov @marin @JobV @patricio