Add structured error responses for Conan package registry

David Fernandezrequested to merge
595495-conan-formatted-error-responses into master

🔔 Context

GitLab's Conan package registry returns error responses in a custom JSON format ({"message":"403 Forbidden - Package protected."}) that Conan clients cannot parse cleanly. When Conan's response_to_str() encounters this format, it falls back to dumping the raw JSON string, producing unhelpful messages like:

ERROR: Permission denied for user: 'dev': {"message":"403 Forbidden - Package protected."}. [Remote: gitlab]

The Conan client natively supports an errors array format ({"errors":[{"status":N,"message":"..."}]}). When response_to_str() finds this key, it extracts and formats the message cleanly.

🤔 What does this MR do and why?

  • Add an errors key to all 4xx/5xx error responses from Conan API endpoints (v1 and v2)
  • The existing message key is preserved for backward compatibility
  • Gated behind the conan_structured_error_responses feature flag (disabled by default)

The implementation introduces an ApiErrorFormatter helper module that overrides render_structured_api_error!. When the feature flag is enabled and the status code is >= 400, it adds the errors key to the response hash and calls super to proceed with the normal error flow.

Before: (raw response body dumped)

ERROR: Permission denied for user: 'dev': {"message":"403 Forbidden - Package protected."}. [Remote: gitlab]

After:

ERROR: Permission denied for user: 'dev': 403: Forbidden - Package protected.. [Remote: gitlab]

📚 References

🧪 How to set up and validate locally

1. Create a test project and users

# Rails console
user = User.find_by(username: 'root')
project = Project.find_by_full_path('root/conan-test') ||
  Projects::CreateService.new(user, { name: 'conan-test', namespace_id: user.namespace.id, visibility_level: 0 }).execute[:project]

token = user.personal_access_tokens.create!(name: 'conan-test', scopes: [:api], expires_at: 30.days.from_now)
puts "PAT: #{token.token}"
puts "Project ID: #{project.id}"

2. Enable the feature flag and restart

# Rails console
Feature.enable(:conan_structured_error_responses)

Then restart rails: gdk restart rails-web

3. Test with curl

The Conan PUT authorize endpoints are routed through Workhorse. To test them directly with curl, connect to the Puma unix socket:

PROJECT_ID=<your_project_id>
PAT=<your_pat>
SOCKET=~/gdk/gitlab.socket
BASE="http://localhost/api/v4/projects/$PROJECT_ID/packages/conan"

# Get a JWT token
JWT=$(curl -s --max-time 30 -u "root:$PAT" \
  "http://localhost:3000/api/v4/projects/$PROJECT_ID/packages/conan/v1/users/authenticate")

# 404 — Nonexistent package (v2)
curl -s --unix-socket $SOCKET -H "Authorization: Bearer $JWT" \
  "$BASE/v2/conans/nonexistent/1.0/root%2Bconan-test/stable/latest"
# Expected: {"message":"404 Package Not Found","errors":[{"status":404,"message":"404 Package Not Found"}]}

# 400 — Search term too long
curl -s -H "Authorization: Bearer $JWT" \
  "http://localhost:3000/api/v4/projects/$PROJECT_ID/packages/conan/v1/conans/search?q=$(python3 -c "print('A'*201)")"
# Expected: {"message":"400 Bad request - ...","errors":[{"status":400,"message":"400 Bad request - ..."}]}

# 200 — Ping (no errors key added)
curl -s "http://localhost:3000/api/v4/projects/$PROJECT_ID/packages/conan/v1/ping"
# Expected: "revisions" (no errors key)

4. Test protected package with Conan client

# Rails console — create protection rule
project = Project.find(<project_id>)
Packages::Protection::Rule.create!(
  project: project,
  package_type: :conan,
  package_name_pattern: '*',
  pattern: '*',
  minimum_access_level_for_push: :owner
)

# Create a developer user (blocked by protection rule)
result = Users::CreateService.new(User.find(1), {
  name: 'Conan Dev', username: 'conandev', email: 'conandev@example.com',
  password: 'Str0ngP4ss!xYz', skip_confirmation: true
}).execute
dev = result.payload[:user]
project.add_developer(dev)
dev_token = dev.personal_access_tokens.create!(name: 'dev', scopes: [:api], expires_at: 30.days.from_now)
puts "Dev PAT: #{dev_token.token}"

Then test with the Conan client:

pip install conan
conan profile detect --force
conan remote add gdk-test http://localhost:3000/api/v4/projects/$PROJECT_ID/packages/conan --force
conan remote login gdk-test conandev -p $DEV_PAT

# Create a test package
mkdir /tmp/conan-test && cd /tmp/conan-test
cat > conanfile.py << 'EOF'
from conan import ConanFile
class TestPkg(ConanFile):
    name = "test-pkg"
    version = "1.0"
    user = "root+conan-test"
    channel = "stable"
EOF
conan export .

# Upload (should show clean protection error)
conan upload "test-pkg/1.0@root+conan-test/stable" -r gdk-test --confirm
# Expected: ERROR: Permission denied for user: 'conandev': 403: Forbidden - Package protected.. [Remote: gdk-test]

5. Verify flag off (backward compatibility)

# Rails console
Feature.disable(:conan_structured_error_responses)

Restart rails (gdk restart rails-web), then repeat the tests. The errors key should be absent from all responses and the Conan client should show the raw JSON dump instead.

🏎️ MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by David Fernandez

Merge request reports