Fix or remove 'zap-addons-up-to-date` job
Context
This came up while attempting to run a pipeline in a personal fork of dast
; see Make DAST pipelines fork-friendly (#409726).
The zap-addons-up-to-date
job was originally added to the pipeline to signal when ZAP addons needed to be updated in the image. It was added with allow_failure: true
so that it wouldn't block unrelated work.
Later it was changed to create an issue if updates were available. I can't find any documentation on why this was changed, but it could be because allow_failure: true
meant that the failing job was too easy to ignore when the focus was on getting other work merged. After this update, the job no longer contained allow_failure: true
.
Problem
Since at least 2021-06-02, the job has been attempting to create an issue about ZAP addons being out of date, but has been failing to do so. However, the job has always completed successfully, leaving no indication that it was actually failing to do what it was intended to do.
In the oldest available pipeline (2021-06-02), the job attempted to create a new issue, and seems to think it was successful; however, the following line is missing the expected issue url, potentially indicating that the call failed (the preceding curl
command is missing the --fail
argument, meaning that if the server returned an HTTP error code, the script would have continued as if the call was successful).
This behavior continued until 2023-03-03, when the jq
command that was parsing the result of the issue creation began to fail with the error parse error: Invalid numeric literal at line 1, column 3
. The script then recognized that a failure had occurred: Failed to create issue: response was An error has occurred and reported in the system's low-level error handler.
This seems to indicate that the payload for creating the issue is invalid in a way that was previously producing a json error response, but at this point changed to producing an unhandled error. However, although the script recognized the error, it did not exit with a non-zero status code, resulting in the job still appearing to succeed.
Although we can't say for certain whether the calls ever succeeded, we can say with relative certainty that no issue was ever created, as searching for the intended issue details produces no relevant results. Correction: there are 4 closed issues that were created by this process, the last one created in April 2021.
Proposal
Since we have been apparently been working all of this time without this feature working as intended, it suggests that we don't really need it. Particularly given our focus on browser-based scanning, my recommendation would be:
-
Remove this job and its associated scripts
However, if we believe this functionality would still be valuable and is worth fixing, then we should:
-
Fix the issue creation call -
Ensure the job fails appropriately -
Change when the job runs. - Currently it runs on all MRs, which could potentially contain old code if the MR has not been rebased recently.
- It would probably make the most sense to run it on a schedule (nightly?).