Bump Nginx resources for LicenseDot
Production Change
Change Summary
LicenseDot (AutoDevops) is under-provisioned and Nginx has timed out a few times with a significant increase in CPU (not application related).
The Blackbox probes failed a few times in the last 24 hours with:
ts=2021-10-11T07:10:26.836891252Z caller=main.go:119 module=http_2xx target=https://license.gitlab.com level=error msg="Error for HTTP request" err="Get \"https://license.gitlab.com/users/sign_in\": net/http: TLS handshake timeout"
ts=2021-10-11T07:07:05.181131982Z caller=main.go:119 module=http_tls_upgrade target=http://license.gitlab.com level=error msg="Error for HTTP request" err="Get \"http://35.237.177.242\": context deadline exceeded"
Change Details
- Services Impacted - ServiceLicense
- Change Technician - DRI for the execution of this change
- Change Reviewer - DRI for the review of this change
- Time tracking - Time, in minutes, needed to execute all change steps, including rollback
- Downtime Component - If there is a need for downtime, include downtime estimate here
Detailed steps for the change
Pre-Change Steps - steps to be completed before execution of the change
Estimated Time to Complete (mins) - Estimated Time to Complete in Minutes
-
Set label changein-progress on this issue -
Pre-Change Step 1 -
Pre-Change Step 2 -
Pre-Change Step 3
Change Steps - steps to take to execute the change
Estimated Time to Complete (mins) - Estimated Time to Complete in Minutes
-
Change Step 1 -
Change Step 2 -
Change Step 3
Post-Change Steps - steps to take to verify the change
Estimated Time to Complete (mins) - Estimated Time to Complete in Minutes
-
Post-Change Step 1 -
Post-Change Step 2 -
Post-Change Step 3
Rollback
Rollback steps - steps to be taken in the event of a need to rollback this change
Estimated Time to Complete (mins) - Estimated Time to Complete in Minutes
-
Rollback Step 1 -
Rollback Step 2 -
Rollback Step 3
Monitoring
Key metrics to observe
- Metric: Metric Name
- Location: Dashboard URL
- What changes to this metric should prompt a rollback: Describe Changes
Summary of infrastructure changes
-
Does this change introduce new compute instances? -
Does this change re-size any existing compute instances? -
Does this change introduce any additional usage of tooling like Elastic Search, CDNs, Cloudflare, etc?
Summary of the above
Changes checklist
-
This issue has a criticality label (e.g. C1, C2, C3, C4) and a change-type label (e.g. changeunscheduled, changescheduled) based on the Change Management Criticalities. -
This issue has the change technician as the assignee. -
Pre-Change, Change, Post-Change, and Rollback steps and have been filled out and reviewed. -
This Change Issue is linked to the appropriate Issue and/or Epic -
Necessary approvals have been completed based on the Change Management Workflow. -
Change has been tested in staging and results noted in a comment on this issue. -
A dry-run has been conducted and results noted in a comment on this issue. -
SRE on-call has been informed prior to change being rolled out. (In #production channel, mention @sre-oncall
and this issue and await their acknowledgement.) -
Release managers have been informed (If needed! Cases include DB change) prior to change being rolled out. (In #production channel, mention @release-managers
and this issue and await their acknowledgment.) -
There are currently no active incidents.