Pipelines silently fail with 500 error when improperly escaped ANSI escape code variables exist

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Summary

If a pipeline contains a variable with an ANSI escape code that is not properly escaped with a double backslash (\\), pipelines generated from a push will fail silently in the background with no indication to the end user. If you try and manually run a pipeline when the .gitlab-ci.yml contains a variable like this, an unhelpful "Pipeline cannot be run" error is presented to the user, due to a 500 error being returned for that POST request.

This is caused by a PG::UntranslatableCharacter error being encountered in the background.

Steps to reproduce

Use the following .gitlab-ci.yml:

variables:
  BLUE: "\033[0;34m"

job:
  script:
    - echo ""

This can be fixed by properly escaping the variable like so:

variables:
  BLUE: "\\033[0;34m"

What is the current bug behavior?

Pipelines generated from a push fail silently in the background when improperly escaped ANSI escape codes are present. An unhelpful "pipeline cannot be run" error is presented when trying to trigger the pipeline manually.

What is the expected correct behavior?

We either:

  • Properly handle these variables and do not treat them as unicode escape escape sequences

  • We present a helpful error for push-generated pipelines and failed manual executions

Relevant logs and/or screenshots

Exception cause: PG::UntranslatableCharacter

Exception class: ActiveRecord::StatementInvalid

Exception message:

PG::UntranslatableCharacter: ERROR:  unsupported Unicode escape sequence
LINE 1: ...53835343166306563373261343831643333373765303662', '{"options...
                                                             ^
DETAIL:  \u0000 cannot be converted to text.
CONTEXT:  JSON data, line 1: ...},"yaml_variables":[{"key":"BLUE","value":"\u0000...

Exception sql:

/*application:web,correlation_id:9aef655a930d4e4b-ATL,endpoint_id:GraphqlController#execute,db_config_database:gitlabhq_production,db_config_name:ci*/ INSERT INTO "p_ci_job_definitions" ("partition_id","project_id","created_at","interruptible","checksum","config") VALUES ($1, $2, $3, $4, $5, $6) ON CONFLICT ("project_id","checksum","partition_id") DO NOTHING

Output of checks

This bug happens on GitLab.com

Patch release information for backports

If the bug fix needs to be backported in a patch release to a version under the maintenance policy, please follow the steps on the patch release runbook for GitLab engineers.

Refer to the internal "Release Information" dashboard for information about the next patch release, including the targeted versions, expected release date, and current status.

High-severity bug remediation

To remediate high-severity issues requiring an internal release for single-tenant SaaS instances, refer to the internal release process for engineers.

Edited by 🤖 GitLab Bot 🤖