Skip to content

Move the blackout window code to its own project so it can be included from different projects

This is a corrective action for https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/8528 so that blackout windows can be consumed from multiple projects

  • Settle on a new project name and location for blackout windows
  • Create a ruby script and image that can be used by other projects that implement the same functionality in the existing blackout window checker (written in python).

Design

  • This will be in its own project, with a published docker image. The image will be built on both gitlab.com and ops
  • Blackout windows are tagged, so that different projects can limit checks to specific periods
  • Start and finish dates are provided for each blackout period, this can either be a cron or date syntax
  • Two tags will be defined initially, infra and delivery
  • A single job will be defined in all consuming pipelines to call the script ex: ./bin/blackout --tags {delivery,infra}
  • Blackout failure notification will be sent to channels that have the corresponding tags
  • When the blackout check fails, a non-zero return status is given which will cause the job to fail
  • An env variable can be set to override the blackout check, which will prevent a non-zero return status

Config:

---

##############################################################
#
# Blackout periods are defined here, two formats are supported:
#   - Cron format
#   - Timestamp (ex: 2020-11-26 00:00:00 UTC)
#
##############################################################


blackout_periods:
    #####################################################################
    ##### Weekend blackout period
  -
    tags:
      - delivery
      - infra
    start: '00 23 * * 5'  # At 23:00 on Friday https://crontab.guru/#0_23_*_*_5
    finish: '00 06 * * 1'  # At 06:00 on Monday https://crontab.guru/#0_6_*_*_1

    #####################################################################
    ##### Thanksgiving + Blackfriday needs be specified using a timestamp
  -
    tags:
      - delivery
      - infra
    start: '2020-11-26 00:00:00 UTC'
    finish: '2020-11-27 23:59:59 UTC'

    # American Thanksgiving to Black Friday 2021
  -
    tags:
      - delivery
      - infra
    start: '2020-11-25 00:00:00 UTC'
    finish: '2020-11-26 23:59:59 UTC'

    # American Thanksgiving to Black Friday 2022
  -
    tags:
      - delivery
      - infra
    start: '2020-11-24 00:00:00 UTC'
    finish: '2020-11-25 23:59:59 UTC'

    #####################################################################
    # Christmas Day + Day after Christmas
  -
    tags:
      - delivery
      - infra
    start: '00 00 25 12 *' # At 00:00 on day-of-month 25 in December https://crontab.guru/#00_00_25_12_* (Gregorian calendar Christmas https://gitlab.com/gitlab-com/gl-infra/delivery/issues/541)
    finish: '59 23 26 12 *' # At 23:59 on day-of-month 25 in December https://crontab.guru/#59_23_26_12_*.

    # New Years
  -
    tags:
      - delivery
      - infra
    start: '00 00 01 01 *' # At 00:00 on the day-of-month 1 in January https://crontab.guru/#00_00_1_1_*
    finish: '59 23 01 01 *' # At 23:59 on day-of-month 1 in January https://crontab.guru/#59_23_1_1_*.


##############################################################
#
# Notification channels for when we are in the blackout period
#
##############################################################

slack:
  - tags:
      - delivery
      - infra
    channels:
      - '#announcements'
  - tags:
      - delivery
    channels:
      - '#g_delivery'
  - tags:
      - infra
    channels:
      - '#infrastructure-lounge'
Edited by John Jarvis