Dotenv report does not handle variables with already existing names

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Summary

When a variable already exists in an environment of current job, an attempt to write a variable with this name to a dotenv report artifact fails with a non-human-readable error.

Steps to reproduce

1. Open example project
2. Navigate to a "repro" job.
3. Input any value for DEPLOY_HOST variable, like so:
4. Start the job.

Example Project

https://gitlab.com/Ansile/dotenv-duplicate-bug-repro

What is the current bug behavior?

Artifact upload fails with error 500.

WARNING: Uploading artifacts to coordinator... failed  id=*** responseStatus=500 Internal Server Error status=500 Internal Server Error token=***
WARNING: Retrying...                                context=artifacts-uploader error=invalid argument

What is the expected correct behavior?

Artifacts should upload, or, at least, CI log should show a human-readable error instead of a 500.

Relevant logs and/or screenshots

Error stack:

"ua":"gitlab-runner 12.9.0 (12-9-stable; go1.13.8; linux/amd64)","route":"/api/:version/jobs/:id/artifacts","exception.class":"ActiveRecord::RecordNotUnique","exception.message":"PG::UniqueViolation: ERROR:  duplicate key value violates unique constraint \"index_ci_job_variables_on_key_and_job_id\"\nDETAIL:  Key (key, job_id)=(DEPLOY_HOST, 6910977) already exists.\n","exception.backtrace":["app/models/concerns/bulk_insert_safe.rb:151:in `block (2 levels) in _bulk_insert_all!'","app/models/concerns/bulk_insert_safe.rb:145:in `each'","app/models/concerns/bulk_insert_safe.rb:145:in `each_slice'","app/models/concerns/bulk_insert_safe.rb:145:in `each'","app/models/concerns/bulk_insert_safe.rb:145:in `flat_map'","app/models/concerns/bulk_insert_safe.rb:145:in `block in _bulk_insert_all!'","app/models/concerns/bulk_insert_safe.rb:144:in `_bulk_insert_all!'","app/models/concerns/bulk_insert_safe.rb:85:in `bulk_insert!'","app/services/ci/parse_dotenv_artifact_service.rb:15:in `execute'","app/services/ci/create_job_artifacts_service.rb:163:in `parse_dotenv_artifact'","app/services/ci/create_job_artifacts_service.rb:120:in `parse_artifact'","app/services/ci/create_job_artifacts_service.rb:41:in `execute'","lib/api/ci/runner.rb:291:in `block (2 levels) in <class:Runner>'","ee/lib/gitlab/middleware/ip_restrictor.rb:14:in `block in call'","ee/lib/gitlab/ip_address_state.rb:10:in `with'","ee/lib/gitlab/middleware/ip_restrictor.rb:13:in `call'","lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call'","lib/gitlab/middleware/rails_queue_duration.rb:33:in `call'","lib/gitlab/metrics/rack_middleware.rb:16:in `block in call'","lib/gitlab/metrics/transaction.rb:61:in `run'","lib/gitlab/metrics/rack_middleware.rb:16:in `call'","lib/gitlab/request_profiler/middleware.rb:17:in `call'","ee/lib/gitlab/jira/middleware.rb:19:in `call'","lib/gitlab/middleware/go.rb:20:in `call'","lib/gitlab/etag_caching/middleware.rb:13:in `call'","lib/gitlab/middleware/multipart.rb:145:in `block in call'","lib/gitlab/middleware/multipart.rb:58:in `with_open_files'","lib/gitlab/middleware/multipart.rb:144:in `call'","lib/gitlab/middleware/read_only/controller.rb:51:in `call'","lib/gitlab/middleware/read_only.rb:18:in `call'","lib/gitlab/middleware/same_site_cookies.rb:27:in `call'","lib/gitlab/middleware/basic_health_check.rb:25:in `call'","lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'","lib/gitlab/middleware/request_context.rb:23:in `call'","config/initializers/fix_local_cache_middleware.rb:9:in `call'","lib/gitlab/metrics/requests_rack_middleware.rb:60:in `call'","lib/gitlab/middleware/release_env.rb:12:in `call'"]

Output of checks

This bug happens on gitlab.com (and self-managed)