Configure per-runner timeout for shared-runners-manager-X on GitLab.com

As it's described in https://gitlab.com/gitlab-com/infrastructure/issues/4010#note_69372298, we should use our new per-runner job timeout feature and set the value for shared-runners-manager-X on GitLab.com.

Copying important part from the linked issue:

After a discussion we've decided that 3h timeout for Shared Runners would be a good compromise. It's 3 times more than most of currently active projects are using and still feels to be reasonable.

To apply this we should:

• choose the deadline,
• find which projects will be affected when we'll set the 3h timeout,
• notify owners of such projects, specifying when we'll update the timeout,
• find affected projects and notify owners before 2018-04-30: #4092
• apply the 3h timeout after 2018-05-08:
  • shared-runners-manager-1.gitlab.com
  • shared-runners-manager-2.gitlab.com
  • shared-runners-manager-3.gitlab.com
  • shared-runners-manager-4.gitlab.com

cc @ayufan

mentioned in issue #4010 (closed)

*** PRODUCTION *** production> Project.where(shared_runners_enabled: true).where('build_timeout > ?', 3.hour.seconds).count
=> 585

For:

• find which projects will be affected when we'll set the 3h timeout,
• notify owners of such projects, specifying when we'll update the timeout,

I'll create a separate, confidential issue, so we can work on preparing the notification and collect contact data.

changed the description

changed the description

As for the date, when the change should be applied: I propose to make it at May 7th. The notification should be then sent before April 30th.

WDYT @ayufan?

It works for me.

changed the description

marked the checklist item choose the deadline, as completed

changed the description

We've moved the date of configuration change to 2018-05-08 since the notification will be sent today.

marked the checklist item find affected projects and notify owners before 2018-04-30: #4092 as completed

The notification was scheduled for 2018-05-01 09am PT, so it should be sent 8 hours ago. The only thing left is to update the configuration at May 8th.

marked the checklist item shared-runners-manager-1.gitlab.com as completed

marked the checklist item shared-runners-manager-2.gitlab.com as completed

marked the checklist item shared-runners-manager-3.gitlab.com as completed

marked the checklist item shared-runners-manager-4.gitlab.com as completed

marked the checklist item apply the 3h timeout after 2018-05-08: as completed