Add --no-tags to GIT_FETCH_EXTRA_FLAGS in gitlab-com/www-gitlab-com CI settings

Production Change

Change Summary

Through #3278 (closed) we learned that for repositories with lots of refs, it can help reduce Gitaly server load if CI is configured to fetch with the --no-tags option.

We already applied this to gitlab-org/gitlab. This issue is for doing the same on gitlab-com/www-gitlab-com.

Change Details

1. Services Impacted - ServiceGitaly
2. Change Technician - @cmiskell
3. Change Criticality - C3
4. Change Type - changescheduled
5. Change Reviewer - @cmiskell
6. Due Date - 2021-01-19 22:15 UTC
7. Time tracking - 15 minutes
8. Downtime Component - none

Detailed steps for the change

Pre-Change Steps - steps to be completed before execution of the change

Estimated Time to Complete (mins) - 1 minute

• Verify that file-praefect-03 is the active Gitaly server for www-gitlab-com. If not, update log links below.

Change Steps - steps to take to execute the change

Estimated Time to Complete (mins) - 1 minute

• As an admin, browse to https://gitlab.com/gitlab-com/www-gitlab-com/-/settings/ci_cd
• Expand the Variables section, and add a new variable called GIT_FETCH_EXTRA_FLAGS
• Set the value to --prune --progress --no-tags
  • If the variable already exists, add --no-tags to whatever is there.
• Click "Add variable"

Post-Change Steps - steps to take to verify the change

Estimated Time to Complete (mins) - 10 minutes

• Monitor per monitoring section; the change is expected to provide performance improvements, but otherwise no change to behavior.

Rollback

Rollback steps - steps to be taken in the event of a need to rollback this change

Estimated Time to Complete (mins) - 1 minute

• Repeat the change, reverting to the original value of GIT_FETCH_EXTRA_FLAGS

Monitoring

Key metrics to observe

Rollback is preferred (it is quick) if there is the slightest suggestion the change has caused troubles; there should be sufficient logs from failing builds to help us debug after the fact.

• Metric: gitlab_runner_failed_jobs_total
  • Location: Thanos
  • What changes to this metric should prompt a rollback: A noticeable rise; there is some low grade/low-rate of failed jobs, but a noticeable increase may suggest this change was problematic.
• Metric: CPU usage on file-praefect-03
  • Location: Host stats dashboard
  • What changes to this metric should prompt a rollback: A noticeable rise. Unfortunately the base metric is bursty, so it would require a substantial and obvious change. We're hoping this will go down, but it may be hard to identify this in the short term; it will be more visible over several hours, so we're just initially monitoring for degradation.
• Metric: gitaly apdex/errors
  • Location: gitaly host dashboard for file-praefect-03
  • What changes to this metric should prompt a rollback: Decrease in apdex or increase in error ratio.

Summary of infrastructure changes

• Does this change introduce new compute instances?
• Does this change re-size any existing compute instances?
• Does this change introduce any additional usage of tooling like Elastic Search, CDNs, Cloudflare, etc?

None

Changes checklist

• This issue has a criticality label (e.g. C1, C2, C3, C4) and a change-type label (e.g. changeunscheduled, changescheduled) based on the Change Management Criticalities.
• This issue has the change technician as the assignee.
• Pre-Change, Change, Post-Change, and Rollback steps and have been filled out and reviewed.
• Necessary approvals have been completed based on the Change Management Workflow.
• Change has been tested in staging and results noted in a comment on this issue.
• A dry-run has been conducted and results noted in a comment on this issue.
• SRE on-call has been informed prior to change being rolled out. (In #production channel, mention @sre-oncall and this issue and await their acknowledgement.)
• There are currently no active incidents.