Regression in project_auto_devops model
There is a bug in app/models/project_auto_devops.rb
that prevents Runner from parsing the payload and generally makes all jobs set to manual
to be stuck and finally dropped by stuck_ci_jobs_worker
.
The bug is at https://gitlab.com/gitlab-org/gitlab-ce/blob/v11.0.0/app/models/project_auto_devops.rb#L32 and https://gitlab.com/gitlab-org/gitlab-ce/blob/v11.0.0/app/models/project_auto_devops.rb#L33, where variable values are defined as integers:
if manual?
variables.append(key: 'STAGING_ENABLED', value: 1)
variables.append(key: 'INCREMENTAL_ROLLOUT_ENABLED', value: 1)
end
Variables are next passed to Runner without any change in how they were defined (https://gitlab.com/gitlab-org/gitlab-ce/blob/11-0-stable/lib/api/entities.rb#L1241):
class Response < Grape::Entity
(...)
expose :variables
(...)
Finally, when Runner receives the payload of job with such variable, it ends with JSON parsing error, since Runner expects the value to be a string:
time="2018-06-21T06:52:06Z" level=warning msg="Checking for jobs... failed" runner=ed2dce3a status="Error decoding json payload json: cannot unmarshal number into Go struct field JobVariable.value of type string"
The best, and only valid fix would be to improve the Response
Grape entity, so it will transform each value to be JSONed string, no matter what value was set at models. However, the regression itself was created by adding new variables in project_auto_devops.rb
.
The easiest fix would be to change:
if manual?
variables.append(key: 'STAGING_ENABLED', value: 1)
variables.append(key: 'INCREMENTAL_ROLLOUT_ENABLED', value: 1)
end
to:
if manual?
variables.append(key: 'STAGING_ENABLED', value: "1")
variables.append(key: 'INCREMENTAL_ROLLOUT_ENABLED', value: "1")
end
and ~"Pick into 11.0", and then create a follow-up issue to improve the Response
Grape entity