Skip to content

Handle 500 errors in get requests

Mark Fletcher requested to merge handle-500-errors into master

It's possible for requests to the API to fail with a 500 error. The error that we have encountered appears to be related to a sporadic Gitaly timeout.

Here we will handle Net::HTTPInternalServerError from the server and retry when encountered. If retries fail the entire query operation will fail.

Notes:

  • The raised Net::HTTPFatalError is only retried for get requests

    • We haven't encountered problems posting so that hasn't been changed

  • Upon error, the logs will output the details when in debug mode like:

.[DEBUG] query_api: https://gitlab.com/api/v4/groups/gitlab-org/merge_requests?per_page=100&state=opened&page=2&page=3&page=4&page=5&page=6&page=7&page=8&page=9&page=10&page=11&page=12&page=13
[DEBUG] #<HTTParty::Response:0x7fbd1b6ad990 parsed_response={"message"=>"500 Internal Server Error"}, @response=#<Net::HTTPInternalServerError 500 Internal Server Error readbody=true>, @headers={"server"=>["nginx"], "date"=>["Mon, 03 Feb 2020 14:15:41 GMT"], "content-type"=>["application/json"], "content-length"=>["39"], "connection"=>["close"], "cache-control"=>["no-cache"], "vary"=>["Origin"], "x-request-id"=>["ZQMv8VlQ7q3"], "x-runtime"=>["57.377019"], "ratelimit-limit"=>["600"], "ratelimit-observed"=>["1"], "ratelimit-remaining"=>["599"], "ratelimit-reset"=>["1580739401"], "ratelimit-resettime"=>["Mon, 03 Feb 2020 14:16:41 GMT"], "gitlab-lb"=>["fe-13-lb-gprd"], "gitlab-sv"=>["localhost"]}>
[DEBUG] query_api: https://gitlab.com/api/v4/groups/gitlab-org/merge_requests?per_page=100&state=opened&page=2&page=3&page=4&page=5&page=6&page=7&page=8&page=9&page=10&page=11&page=12&page=13

Closes #174 (closed)

Merge request reports