fix(codegen): keep emoji outside the BMP intact in generated GraphQL

What does this MR do?

Fixes GLQL label filters (and any other quoted value) containing emoji outside the Basic Multilingual Plane, such as 🌐 or 🚦.

Root cause

validate_and_format_graphql parses the generated query with graphql_parser and pretty-prints it back. graphql_parser 0.4.1 prints every character above U+FFFF as \u followed by its decimal code point (src/format.rs#L148), so 🌐 (U+1F310 = 127760) was emitted as \u127760. GraphQL reads that as \u1277 (ቷ) plus the literal digits 60:

curl -s -X POST https://gitlab.com/api/graphql -H "Content-Type: application/json" \
  -d '{"query":"{ a: echo(text: \"\\u127760Network\") b: echo(text: \"🌐Network\") }"}'
# {"data":{"a":"nil says: ቷ60Network","b":"nil says: 🌐Network"}}

So label = "🌐Network::❗Operational Failure" asked GitLab for the label ቷ60Network::❗Operational Failure, which does not exist. Emoji inside the BMP (⏳ U+23F3, ❗ U+2757) are printed verbatim and kept working, which explains the "hit or miss" reports in the issue.

Upstream has an open fix, graphql-rust/graphql-parser#97, but no release ships it (0.4.1 is the latest). Patching the crate is not an option either: the gem's native extension pulls glql as a git dependency, so a [patch] in this crate's manifest would not reach it, and vendoring ~4.4k lines for a one-line fix is not worth it.

Fix

AstralPlaceholders swaps each character above U+FFFF for an unused BMP Private Use Area code point before the parse/format round trip and restores it on the printed output. Stand-ins are chosen from code points absent from the query both as characters and as \uXXXX escapes, so restoring cannot touch user text or chain. Parse errors are reported against the original query so they name the user's characters. A canary unit test asserts that graphql_parser still mangles astral characters, so a future dependency bump that fixes it flags the workaround for removal.

Follow-ups

  • Release glql and bump gitlab_query_language / @gitlab/query-language-rust in gitlab-org/gitlab; the monolith only gets the fix then.

Example Usage

Build prerequisite:

cd glql_rb
bundle install
bundle exec rake compile

Filter by a scoped label whose emoji sit outside the BMP

mode: standard
query: group = "gitlab-org" and label = "Anne 🔍"
fields: title, labels
Test via ruby extension
cd glql_rb # after the build prerequisite above

bundle exec ruby -Ilib - <<'RUBY'
require "gitlab_query_language"
require "json"
require "net/http"

query   = 'group = "gitlab-org" and label = "Anne 🔍"'
context = { fields: "title, labels" }

compiled = Glql.compile(query, context)
puts "== Generated GraphQL =="
puts compiled["output"]

uri  = URI("https://gitlab.com/api/graphql")
http = Net::HTTP.new(uri.host, uri.port); http.use_ssl = true
req  = Net::HTTP::Post.new(uri, "Content-Type" => "application/json")
req.body = { query: compiled["output"], variables: { limit: 3 } }.to_json
response = JSON.parse(http.request(req).body)

transformed = Glql.transform(response["data"], { mode: context[:mode], fields: compiled["fields"] })
puts "\n== Transformed rows =="
puts JSON.pretty_generate(transformed["data"])
RUBY

Generated GraphQL (truncated):

query GLQL($before: String, $after: String, $limit: Int) {
  group(fullPath: "gitlab-org") {
    workItems(labelName: "Anne 🔍", before: $before, after: $after, first: $limit, includeDescendants: true) {
      nodes {
        id
        iid
        title
        ...

Before this MR the same line read workItems(labelName: "Anne \u128269", ...). Against gitlab.com the fixed query returns count: 9; the escaped form returns count: 0.

Related to gitlab#572851 (closed)

Edited by Daniele Rossetti

Merge request reports

Loading
Loading