DAST Full Scans fail to execute

Incident review: https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/8179

Summary

A recent release of DAST caused full scans to fail to execute. Baseline scans were not affected. Note that this issue in no way would have affected GitLab.com in general.

Service(s) affected : DAST (full scan)

Team attribution : Secure, DAST

Minutes downtime or degradation : 2027 minutes (1 day, 9 hours, 47 minutes)

Timeline

2019-10-08

• 16:44 UTC - commit Upgrade to Python 3 gitlab-org/security-products/dast@5100d514 is merged into DAST master branch

2019-10-13

• 04:30 UTC - scheduler runs and automatically deploys DAST https://gitlab.com/gitlab-org/security-products/dast/pipelines/88455397
• 23:40 UTC - DAST engineer(s) learn that DAST end to end tests have been silently failing, submit fixes

2019-10-14

• 03:21 UTC - DAST engineer(s) learn that recent release is affected
• 14:17 UTC - DAST engineer(s) test and release, resolving issue

Analysis

Considering this a S2 because less than 50% of DAST users will be affected (most DAST users will be using a Baseline scan). There was no workaround while the feature was broken.

/cc @sethgitlab