Loading
Add IP fallback errors to Gitlab::HTTP HTTP_ERRORS
What does this MR do and why?
- Adds
Errno::EADDRNOTAVAIL,Errno::EAFNOSUPPORTandErrno::ETIMEDOUTtoGitlab::HTTP_V2::HTTP_ERRORS(whichGitlab::HTTP::HTTP_ERRORSpoints to). These errors trigger the next-IP fallback added in !239733 (merged), but they were not in the list. When the last IP also failed, callers that rescue*Gitlab::HTTP::HTTP_ERRORSdid not catch the error. - Webhooks:
WebHookServicedid not rescue the error, so the failure was not logged in the hook's logs, the hook did not get the normal backoff or auto-disable, and Sidekiq retried the job. - CI
include:remote: pipelines failed with a misleading "Unable to run pipeline - Undefined error". - Adds a spec for each fallback error, checking that
Gitlab::HTTP_V2.try_getreturns nil when every resolved IP fails with that error.
References
- #344788 (closed)
- #630166
- !239733 (merged)
- https://new-sentry.gitlab.net/organizations/gitlab/issues/GITLABCOM-ERSC
Expected side effects
- Webhooks that fail on every attempt with these errors now count as failures (rescue), so they get backed off and eventually auto-disabled. Previously they crashed before the failure was recorded.
- Audit event streaming treats these errors as destination failures: they are logged instead of sent to Sentry, and they count toward the destination's circuit breaker.
How to set up and validate locally
- From
gems/gitlab-http, runbundle exec rspec spec/gitlab/http_v2/ip_fallback_spec.rb. - Optionally, revert the change in
gems/gitlab-http/lib/gitlab/http_v2/exceptions.rband rerun. The new examples for ETIMEDOUT, EADDRNOTAVAIL and EAFNOSUPPORT fail.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.