Commit list message search with non-ASCII (e.g. Korean) query raises Encoding::UndefinedConversionError → 500 / "not valid JSON"

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Summary

On the new commit list (epic &17482, GitLab 19.1+), searching the message filter with a non-ASCII term — e.g. Korean — raises Encoding::UndefinedConversionError in the backend. The GraphQL request (getCommitList) returns 500, and the frontend surfaces it as Unexpected token '<', "<!DOCTYPE "... is not valid JSON. ASCII-only terms work fine.

Reported on GitLab Self-Managed 19.1.2. Reproduced by a customer with plain single-word Korean terms (no special characters, no whitespace).

Root cause

In lib/gitlab/gitaly_client/commit_service.rb#list_commits, the message-pattern assignment does not encode the string to binary before appending it to the protobuf bytes field:

# lib/gitlab/gitaly_client/commit_service.rb:341
if params[:commit_message_patterns]
  request.commit_message_patterns += Array.wrap(params[:commit_message_patterns])
end

request.author = encode_binary(params[:author]) if params[:author]          # ← encoded
request.paths.push(encode_binary(params[:path])) if params[:path].present?   # ← encoded

commit_message_patterns is a bytes (ASCII-8BIT) field, but the query string arrives as UTF-8 and is appended without encode_binary. For a Korean term this throws U+D5C8 from UTF-8 to ASCII-8BIT — unlike author and path on the adjacent lines, which are wrapped in encode_binary. ASCII terms don't hit this because they convert to ASCII-8BIT losslessly, which is why Latin-script search works and Korean does not.

This is the same class of bug as gitlab-org/gitlab-foss#42161 (fixed in 10.4 for commit_count's path); the new commit-list path reintroduced it in a different field.

Server log (customer, 19.1.2)

Encoding::UndefinedConversionError
message : U+D5C8(허) from UTF-8 to ASCII-8BIT
lib/gitlab/gitaly_client/commit_service.rb:341:in `+'
lib/gitlab/gitaly_client/commit_service.rb:341:in `list_commits'
lib/gitlab/git/repository.rb:1185:in `block in list_commits'
lib/gitlab/git/wraps_gitaly_errors.rb:7:in `wrapped_gitaly_errors'
app/models/repository.rb:213:in `list_commits'
app/graphql/resolvers/repositories/commits_resolver.rb:45:in `resolve_with_lookahead'
...
operation_name: getCommitList

The GraphQL variables show query: "[FILTERED]" with all other args nil — i.e. a plain message-only search.

Steps to reproduce

  1. On a project that has Korean-language commit messages, open Code > Commits.
  2. In the message filter, enter a Korean term (e.g. 허브).
  3. The request to POST /api/graphql (getCommitList) returns 500 Internal Server Error; the UI shows Unexpected token '<', "<!DOCTYPE "... is not valid JSON.

Example URL: …/-/commits/master?message=%ED%97%88%EB%B8%8C (허브)

Expected behavior

A non-ASCII message search returns matching commits (or an empty result), the same way an ASCII search does.

Suggested fix

Wrap the pattern in encode_binary before appending, consistent with the author / path handling on the adjacent lines:

if params[:commit_message_patterns]
  request.commit_message_patterns += Array.wrap(params[:commit_message_patterns]).map { |p| encode_binary(p) }
end

Impact

Blocks commit message search entirely for non-ASCII (CJK and others) repositories on 19.1+. Self-managed customers with Korean / Japanese / Chinese commit histories cannot use the feature at all.

Related

  • gitlab-org/gitlab-foss#42161 — same Encoding::UndefinedConversionError class, fixed in 10.4 for commit_count
  • #598401 — feedback issue for the new commit list experience
Edited by 🤖 GitLab Bot 🤖