Package Registry: forward backend error messages to users
<!--IssueSummary start-->
<details>
<summary>
Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards.
</summary>
- [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=595485)
</details>
<!--IssueSummary end-->
## Package Registry: Backend Error Passthrough to End Users
## Actionable Changes
| Format | Change required | Mechanism | Status |
|---|---|---|---|
| **Maven** | Return RFC 9457 `application/problem+json` body on error responses | Response body format + Content-Type | [Task](https://gitlab.com/gitlab-org/gitlab/-/work_items/595487) |
| **npm** | Extend `{"error":"..."}` field to auth errors (401/403). Set `npm-notice` header for important messages. | Response body field + response header | _Skipped as we already have a passthrough. npm-notice would be a nice to have_ |
| **NuGet** | Set `X-NuGet-Warning` response header on error responses | Response header | [Task](https://gitlab.com/gitlab-org/gitlab/-/work_items/595489) |
| **RubyGems** | Set `X-Error-Message` response header on error responses | Response header | [Task](https://gitlab.com/gitlab-org/gitlab/-/work_items/595492) |
| **Go Proxy** | Return `Content-Type: text/plain` with plain text body on errors | Response body format + Content-Type | [Task](https://gitlab.com/gitlab-org/gitlab/-/work_items/595493) |
| **Composer** | Return `{"warning":"..."}` field in JSON error body | Response body field | [Task](https://gitlab.com/gitlab-org/gitlab/-/work_items/595494) |
| **Conan** | Return Artifactory JSON `{"errors":[{"status":N,"message":"..."}]}` on errors | Response body format | [Task](https://gitlab.com/gitlab-org/gitlab/-/work_items/595495) |
| **PyPI** | Nothing actionable — `pip` discards response body entirely | — | No client support ([pip#12037](https://github.com/pypa/pip/issues/12037)) |
| **Debian** | `apt` discards body. `dput-ng` shows body on 403. | Response body (publish only) | Limited client support |
| **Terraform** | Nothing actionable for most operations (only module location reads body) | — | Limited client support |
## Current State
GitLab returns most package registry errors as `Content-Type: application/json` with body `{"message": "NNN StatusPhrase - reason"}` (e.g., `{"message": "403 Forbidden - Package protected."}`).
**Exception**: npm endpoints return both fields: `{"message": "...", "error": "..."}` for npm-specific errors (400, 403, 404). Standard auth errors (401/403 from base Grape helpers) still return only `{"message": "..."}`.
## Per-Format Analysis
### Maven
| Client | Passthrough today? | Mechanism | Details |
|---|---|---|---|
| `mvn` 3.9.11+ | No | **RFC 9457**: `Content-Type: application/problem+json` with `{type, status, title, detail}` body | Exact Content-Type required (no `; charset=utf-8`). `detail` field shown to user. |
| `mvn` < 3.9.11 | No | None — body discarded | Status code + reason phrase only. HTTP/2 loses reason phrase. |
| `gradle` 8.x | No | None — body discarded | RFC 9457 planned for Gradle 9.x ([gradle/gradle#36203](https://github.com/gradle/gradle/issues/36203)) |
| `sbt`/Coursier (GET) | No | None — body, reason phrase, headers all discarded | Status code only |
| `sbt`/Ivy (PUT) | Partial | **Plain text body** shown up to 1024 chars ([sbt#8537](https://github.com/sbt/sbt/pull/8537)) | Current JSON body appears as raw JSON |
### npm
| Client | Passthrough today? | Mechanism | Details |
|---|---|---|---|
| `npm` | **Yes** (npm-specific errors) | **`body.error` JSON field** + **`npm-notice` response header** | GitLab npm endpoints return `{"error": "..."}` for 400/403/404. Auth errors (401) from base helpers lack the `error` field. `npm-notice` header is displayed on **all responses** (success and error) at default log level. |
| `yarn` v1 | **Yes** (npm-specific errors) | **`body.error`** then **`body.message`** fallback | Both fields present in GitLab npm responses. `body.message` matches GitLab's standard format. Frozen/maintenance mode. |
| `yarn` v2+ (Berry) | **Yes** (npm-specific errors) | **`body.error`** only | GitLab npm endpoints include `error` field. Auth errors from base helpers don't. |
**Key mechanisms**:
- **`npm-notice` header**: Displayed on every response (success + error). Not redacted. Deduplicated per command. Suppressed on cache hits. This is the official npm server-to-user messaging channel.
- **`body.error` field**: Appended to error messages on 4xx/5xx. All npm ecosystem clients read this field.
- **`www-authenticate` header**: Parsed on 401 to classify auth errors (OTP, IP-based, Bearer vs Basic).
### PyPI
| Client | Passthrough today? | Mechanism | Details |
|---|---|---|---|
| `pip` | **No** | None available today | Body discarded entirely. Only status code + reason phrase + URL. No custom headers read. Active [pre-PEP RFC 9457 discussion](https://discuss.python.org/t/pre-pep-discussion-rfc-9457-error-responses-for-package-registries/105453). [pip#12037](https://github.com/pypa/pip/issues/12037) |
| `twine` | **No** (without `--verbose`) | **Response body** shown raw with `--verbose` flag | No JSON parsing. No custom headers read. Non-verbose shows generic "Retry with --verbose". |
### NuGet
| Client | Passthrough today? | Mechanism | Details |
|---|---|---|---|
| `nuget` / `dotnet` | **Partial** (header only) | **`X-NuGet-Warning` response header** + HTTP ReasonPhrase | `X-NuGet-Warning` is logged as a warning on **every** HTTP response via `ServerWarningLogHandler`. Multiple values supported. ReasonPhrase included in error exceptions but **breaks on HTTP/2**. Response body is **never** read on errors. Open [proposal for structured error body](https://github.com/NuGet/NuGetGallery/issues/5818) — unimplemented. |
| `choco` | **Partial** | Inherits NuGet's `X-NuGet-Warning` + ReasonPhrase | Also string-matches exception messages for status codes like "406", "409". |
**Key mechanisms**:
- **`X-NuGet-Warning` header**: Defined in `ProtocolConstants.ServerWarningHeader`. Processed by `ServerWarningLogHandler` in the HTTP pipeline. Logged at warning level. Works on all responses (success + error). GitLab does **not** currently set this header.
- **ReasonPhrase**: Included in `EnsureSuccessStatusCode()` exception. Works on HTTP/1.1 only. Truncated to 512 chars by NuGet server.
### Composer
| Client | Passthrough today? | Mechanism | Details |
|---|---|---|---|
| `composer` | **Partial** | **JSON body** (first 200 chars) if `Content-Type: application/json` (exact match). Also supports structured **`warning`/`warnings`** JSON fields. | Does NOT recognize `application/problem+json`. GitLab's current `{"message":"..."}` appears as raw JSON. No custom headers read. |
**Structured format** for clean display:
```json
{"warning": "message"}
{"warnings": [{"message": "...", "versions": ">=2.0"}]}
```
### Conan
| Client | Passthrough today? | Mechanism | Details |
|---|---|---|---|
| `conan` v1/v2 (upload, auth: 400/401/403/500) | **Partial** | **Response body** via `response_to_str()` | Parses Artifactory JSON: `{"errors":[{"status":N,"message":"..."}]}` if `Content-Type: application/json` (exact). Plain text (`text/plain`) passed through as-is. HTML discarded. GitLab's current JSON shown as raw text. |
| `conan` v1/v2 (download 404) | **No** | None — hardcoded to show URL only | |
| `conan` v1/v2 (download 5xx) | **No** | None — hardcoded to show status code + URL only | |
No custom error headers read. Only `X-Conan-Server-Capabilities` (for capabilities, not errors).
### Helm
| Client | Passthrough today? | Mechanism | Details |
|---|---|---|---|
| `helm` (HTTP classic repos) | **No** | None — body discarded | Only shows `"failed to fetch <URL> : <status line>"`. No headers read. |
| `helm cm-push` | **Partial** | **`{"error":"..."}`** JSON field or raw body dump | If JSON parse fails, entire raw body is dumped to user. No headers read. |
### Go Proxy
| Client | Passthrough today? | Mechanism | Details |
|---|---|---|---|
| `go` | **No** | **`text/plain` response body** (US-ASCII or UTF-8) | Up to 8 lines / ~648 bytes. Validates UTF-8, rejects non-printable chars. Body ONLY read when `Content-Type: text/plain`. GitLab returns JSON, so body is discarded. No custom headers read. [go#30748](https://github.com/golang/go/issues/30748) |
### RubyGems
| Client | Passthrough today? | Mechanism | Details |
|---|---|---|---|
| `gem push` | **Partial** | **Response body** displayed as plain text via `clean_text()` sanitization | GitLab's JSON appears as raw JSON. No custom headers read for push. |
| `gem install` / `gem fetch` | **No** | **`X-Error-Message` response header** | If present, replaces HTTP status text in error message. Falls back to HTTP reason phrase. Body is never read. GitLab does **not** currently set this header. |
### Debian
| Client | Passthrough today? | Mechanism | Details |
|---|---|---|---|
| `apt` | **No** | None — body discarded via `RunDataToDevNull()` | Only `"Failed to fetch <URL> <status> <reason>"`. `Retry-After` header read for retry timing only. |
| `dput-ng` | **Partial** | **Response body** on 403 errors | Charset-aware decoding. No custom headers read. |
### Terraform
| Client | Passthrough today? | Mechanism | Details |
|---|---|---|---|
| `terraform` (module location) | **Partial** | **Response body** appended raw to error | No parsing. No custom headers read. |
| `terraform` (module versions) | **No** | None — `resp.Status` only | |
| `terraform` (provider ops) | **No** | None — `resp.Status` only | |
### Generic
| Client | Passthrough today? | Mechanism | Details |
|---|---|---|---|
| `curl` / API | **Yes** | Full HTTP response visible | No dedicated client |
## Summary: All Error Passthrough Mechanisms
### Response Headers
| Header | Client(s) | Behavior |
|---|---|---|
| **`npm-notice`** | `npm` | Displayed on **all** responses (success + error). Default log level. Not redacted. |
| **`X-NuGet-Warning`** | `nuget`, `dotnet`, `choco` | Logged as warning on **all** responses via `ServerWarningLogHandler`. Multiple values supported. |
| **`X-Error-Message`** | `gem` (install/fetch only) | Replaces HTTP status text in error message. |
| **`www-authenticate`** | `npm` | Parsed on 401 to classify auth error type (OTP, IP, Bearer/Basic). |
| **`Warning`** (RFC 7234) | `helm` (OCI/ORAS only) | Code 299 warnings passed to optional callback. |
### Response Body Formats
| Format | Client(s) | Details |
|---|---|---|
| **RFC 9457** (`application/problem+json`) | `mvn` 3.9.11+ | `{type, status, title, detail}`. Exact Content-Type match. |
| **`{"error": "..."}`** (CouchDB convention) | `npm`, `yarn` v1/v2+, `helm cm-push` | The `error` string field is extracted and shown. |
| **`{"errors":[{"status":N,"message":"..."}]}`** (Artifactory) | `conan` v1/v2 | Only when `Content-Type: application/json` (exact). |
| **`{"warning":"..."}` / `{"warnings":[...]}`** | `composer` | Structured warning display. Version filtering. |
| **JSON body** (first 200 chars, any structure) | `composer` | Raw JSON appended to error if `Content-Type: application/json`. |
| **Plain text body** | `go`, `gem push`, `twine` (verbose), `conan` (upload/auth), `dput-ng`, `sbt` (PUT), `terraform` (module location) | Various limits and conditions apply. |
### What GitLab Can Do Per Format
| Format | Actionable change | Impact |
|---|---|---|
| **Maven** | Return RFC 9457 body on Maven endpoints (feature-flagged) | `mvn` 3.9.11+ users see `detail` field |
| **npm** | Already returns `{"error":"..."}` on npm-specific errors. Extend to auth errors. Set `npm-notice` header for important messages. | All npm/yarn users see error details + notices |
| **NuGet** | Set `X-NuGet-Warning` header on error responses | All nuget/dotnet/choco users see warnings |
| **RubyGems** | Set `X-Error-Message` header on error responses | `gem install` users see error details |
| **Go Proxy** | Return `Content-Type: text/plain` with plain text body on errors | `go` users see error details (up to 8 lines) |
| **Composer** | Return `{"warning":"..."}` in JSON error body | Composer users see formatted warnings |
| **Conan** | Return Artifactory JSON format on errors | Conan users see parsed error messages on upload/auth |
| **PyPI** | Nothing actionable — `pip` discards body | Wait for pip RFC 9457 adoption |
| **Debian** | Nothing actionable — `apt` discards body | |
| **Terraform** | Nothing actionable for most operations | Only module location benefits |
issue