Strip characters Atlassian rejects from Jira Connect payloads

Jorge Tomásrequested to merge
565633-sanitize-jira-connect-payload into master

What does this MR do and why?

Fixes #565633 (closed). The GitLab-for-Jira Cloud app fails to sync data (dev-info, deployments, builds, feature flags) when a free-text field contains an emoji. Atlassian's Data Depot rejects the whole request with:

Invalid binary character '#xD83D' was found in the request body, the set of
allowed characters is #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF]

Root cause

#xD83D is a UTF-16 high surrogate — the first half of an emoji such as 🚝 (U+1F69D → surrogate pair D83D DE9D). It is not a length problem: the reported failure is a repository named 🚝 d2c (5 characters). GitLab sends the emoji as raw UTF-8, and Atlassian re-encodes it into UTF-16 surrogates for its XML store, whose allowed set excludes the surrogate range (D800–DFFF). So any astral (non-BMP) character fails the sync, on every sync path — the customer log is the dev-info/repository path, not only deployments.

Fix

Every Jira Connect payload funnels through Atlassian::JiraConnect::Client#post. It now strips characters that are invalid in XML 1.0 (astral code points, control characters, surrogates) from every string in the payload, before serialization. Keeps the documented allowed set: #x9 #xA #xD [#x20-#xD7FF] [#xE000-#xFFFD].

  • 🚝 d2c → d2c (emoji removed, sync succeeds)
  • café-ñ → café-ñ (valid BMP characters preserved)

A follow-up could convert emoji to their :shortcode: (e.g. :monorail: d2c) instead of stripping, using TanukiEmoji — but that reintroduces the field-length question and is a display/UX choice, so it is intentionally out of scope here.

How to set up and validate locally

  1. Configure a project with the GitLab for Jira Cloud app and sync enabled.
  2. Put an emoji in a synced field (repository/project name, branch, MR title, deployment metadata).
  3. Trigger a sync and confirm it succeeds (previously a 400 from Atlassian's Data Depot).

Testing

  • New regression coverage in client_spec.rb: #post strips a rejected character from the request body (the reported repository-name scenario) and preserves valid multi-byte characters; #sanitize_xml_chars unit tests cover stripping, recursion into arrays/hashes, and non-string passthrough.
  • bin/rspec spec/lib/atlassian/jira_connect/client_spec.rb — 75 examples, 0 failures, in both EE and FOSS_ONLY=1.

Related to

#565633 (closed)

Merge request reports