ci_pipelines.yaml_errors and pipeline_messages are recording duplicate data
The Problem
It was very helpfully noticed by @mbobin and @fabiopitino that ci_pipelines.yaml_errors and pipeline_messages are recording the same data. They appear to have been doing this for several years
module Pipeline
module Chain
module Helpers
def error(message, failure_reason: nil)
sanitized_message = ActionController::Base.helpers.sanitize(message, tags: [])
pipeline.yaml_errors = sanitized_message if failure_reason == :config_error
pipeline.add_error_message(sanitized_message)
while our database creaks under the weight of its tables.
Proposal (updated)
- Update yaml_errors in the Ci::Pipeline model to use error_messages <- workflowin review !178041 (diffs)
-
Remove duplicate collection of errors in the being done via
pipeline.yaml_errors<- this is the for stopping the recording of duplicate data - THIS ISSUE AIMS TO SOLVE THIS - it's blocked by the MR up above. Once that is merged, this can be reviewed again (MR is already created and "pre-reviewed") -
Drop
pipeline.yaml_errorscolumn - this will take 3 milestones as per guidelines -
Update locations for
yaml_errorsto useerror_messagesinstead in the FE- These can be updated as technical debt although some have already been updated:
-
FRONTEND:~~ app/assets/javascripts/ci/pipelines_page/components/pipeline_labels.vue no need to fix, ref is changed here~~ #507283 (closed) -> !179041 (merged) seems like the FE code isn't actually in use and is under a feature flag, so no need to fix here. More info in the MR.FRONTEND:~~ app/assets/javascripts/ci/pipeline_details/header/components/header_badges.vue -
FRONTEND:~~ app/assets/javascripts/ci/merge_requests/utils.js -
FRONTEND:app/assets/javascripts/ide/stores/modules/pipelines/mutations.js~~ <- this is going away in 18.0 so we can ignore until and is unrelated to the dupe errors. -
**
ANY:**~~app/views/projects/pipelines/show.html.haml~~~ -
BACKEND:~~ app/models/integrations/chat_message/pipeline_message.rb~~
-
- These can be updated as technical debt although some have already been updated:
Historical context: Drop one or the other. It appears that we created pipeline_messages to try to take some data off the ci_pipelines table, so the move is probably to drop the yaml_errors column and use the related pipeline messages everywhere in the app.