Skip to content

GitLab Next

Merged
Created by Sean McGivern@smcgivernMaintainer3 of 3 tasks completed3/3 tasks

API endpoint to remove jobs from Sidekiq based on metadata

What does this MR do?

This API endpoint uses job metadata to remove jobs from a queue. It can only be used by admins, and deletes as many jobs as it can in 30 seconds. If it exceeds 30 seconds, it returns a flag indicating that the user should try again to finish processing the queue.

(Because of the way Sidekiq queues work, it can't resume where it left off, so you just have to start from scratch each time.)

To test this manually, I created a group with a bunch of users, turned off Sidekiq, then created three projects: two as root, and one as another user.

Sidekiq::Queue.new('authorized_projects').to_enum.map { |j| j['meta.user'] }
#=> ["olivia.towne", "olivia.towne", "olivia.towne", "olivia.towne", "olivia.towne", "olivia.towne", "olivia.towne", "root", "root", "root", "root", "root", "root", "root", "root", "root", "root", "root", "root", "root", "root"]

I then hit the API endpoint to delete root's jobs:

$ curl --request DELETE --header "Private-Token: $GITLAB_API_TOKEN_LOCAL" http://localhost:3000/api/v4/admin/sidekiq/queues/authorized_projects?user=root
{"completed":true,"deleted_jobs":14,"queue_size":7}

And I can see the relevant jobs are gone:

Sidekiq::Queue.new('authorized_projects').to_enum.map { |j| j['meta.user'] }
# => ["olivia.towne", "olivia.towne", "olivia.towne", "olivia.towne", "olivia.towne", "olivia.towne", "olivia.towne"]

I also tested this in the GraphQL explorer with:

mutation($input:AdminSidekiqQueuesDeleteJobsInput!) {
  adminSidekiqQueuesDeleteJobs(input: $input) {
    result {
      completed
      deletedJobs
      queueSize
    }
    errors
  }
}

And variables:

{
  "input":{
    "queueName": "authorized_projects",
    "user": "olivia.towne"
  }
}

With the result:

{
  "data": {
    "adminSidekiqQueuesDeleteJobs": {
      "result": {
        "completed": true,
        "deletedJobs": 7,
        "queueSize": 0
      },
      "errors": []
    }
  }
}

And therefore the queue is empty:

Sidekiq::Queue.new('authorized_projects').to_enum.map { |j| j['meta.user'] }
# => []

Running as non-admin gives:

{
  "data": {
    "adminSidekiqQueuesDeleteJobs": null
  },
  "errors": [
    {
      "message": "You must be an admin to use this mutation",
      "locations": [
        {
          "line": 31,
          "column": 3
        }
      ],
      "path": [
        "adminSidekiqQueuesDeleteJobs"
      ]
    }
  ]
}

For gitlab-com/gl-infra/scalability#116 (closed).

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Edited by Marcel Amirault