Production error - JSON decode error when http status is not success

Problem

When Duo Workflow Executor fails to execute an action, it returns to Duo Workflow Service a string like 'Error running tool: ....

In case the action is http, and GitLab API returns a non-success status, this payload is something like 'Error running tool: unexpected status code: 404. Result: {"message":"unexpected status code: 404","status":404}', which is not a valid json. Duo Workflow Service does not able to parse this and raises an error JSON decode error for GET ...

sentry error

log error

Note: This is one of the most frequently occurring bugs, causing 2 different alerts, triggered 44 and 27 times within a week.

Outcome

Non-success http responses does not cause an exception

Implementation plan

• 1. @halilcoban Add a new http_response field to the proto definitions. MR is merged.
• 2. @halilcoban In Golang executor, write to http response in addition to existing legacy response. MR is under review.
• 3. @afontaine In Typescript executor, write to http response in addition to existing legacy response. gitlab-org/editor-extensions/gitlab-lsp!1775 (merged)
• 4. @halilcoban In duo workflow service, read from the new http response field, and use status code and body in one of the calls (update workflow status). MR is under review.
• 5. @halilcoban Create tickets to migrate other tools. Epic
• 6. @halilcoban Create tickets to cleanup legacy response. Issue