Skip to content

GitLab Next

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
Operations
Operations
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 1
    • Issues 1
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
    • Iterations
  • Merge Requests 0
    • Merge Requests 0
  • Requirements
    • Requirements
    • List
  • Security & Compliance
    • Security & Compliance
    • Dependency List
    • License Compliance
  • Operations
    • Operations
    • Incidents
  • Analytics
    • Analytics
    • Code Review
    • Insights
    • Issue
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
  • GitLab.com
  • OperationsOperations
  • Issues
  • #276

Closed
Open
Opened May 19, 2016 by Kamil Trzciński@ayufanDeveloper

Unicorn workers are restarted too often

On GitLab.com after looking into unicorn logs to check why the git push timings are not deterministic I saw that workers are restarted roughly every 3-5s. Giving the time required to kill worker and spawn another one (according to log) it is 400ms it appears that we build long backlog of request on unicorn masters.

W, [2016-05-19T23:20:35.944824 #12127]  WARN -- : #<Unicorn::HttpServer:0x000000036fd2b0>: worker (pid: 12127) exceeds memory limit (383347712.0 bytes > 333834411 bytes)
W, [2016-05-19T23:20:35.945003 #12127]  WARN -- : Unicorn::WorkerKiller send SIGQUIT (pid: 12127) alive: 6 sec (trial 1)
W, [2016-05-19T23:20:35.968973 #11488]  WARN -- : #<Unicorn::HttpServer:0x000000036fd2b0>: worker (pid: 11488) exceeds memory limit (387458048.0 bytes > 361337562 bytes)
I, [2016-05-19T23:20:36.333073 #1931]  INFO -- : reaped #<Process::Status: pid 12127 exit 0> worker=6
I, [2016-05-19T23:20:36.343287 #13165]  INFO -- : worker=6 spawned pid=13165
I, [2016-05-19T23:20:36.344133 #13165]  INFO -- : worker=6 ready

It looks like that we should increase memory limit for workers or separate workers that are processing git requests from other types.

cc @pcarranza @yorickpeterse

Assignee
Assign to
Milestone-2
Milestone
Milestone-2 (Past due)
Assign milestone
Time tracking
None
Due date
None
Reference: gitlab-com/operations#276