Return metav1.Status on Kubernetes proxy errors
Closes gitlab-org/gitlab#338305 (closed).
Current behavior:
➜ ~ k get pods
error: You must be logged in to the server (the server has asked for the client to provide credentials)
➜ ~ k -v 9 get pods
I0524 20:40:24.836334 29828 loader.go:372] Config loaded from file: ~/.kube/config
I0524 20:40:24.836798 29828 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" -H "Authorization: Bearer <masked>" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:40:24.837225 29828 round_trippers.go:510] HTTP Trace: Dial to tcp:127.0.0.1:8154 succeed
I0524 20:40:24.863423 29828 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 26 milliseconds
I0524 20:40:24.863439 29828 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 25 ms ServerProcessing 0 ms Duration 26 ms
I0524 20:40:24.863445 29828 round_trippers.go:577] Response Headers:
I0524 20:40:24.863450 29828 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:40:24.863454 29828 round_trippers.go:580] X-Content-Type-Options: nosniff
I0524 20:40:24.863459 29828 round_trippers.go:580] X-Request-Id: 01G3TTEM8ZZWQ8EKTWM2CSHPJ3
I0524 20:40:24.863463 29828 round_trippers.go:580] Date: Tue, 24 May 2022 10:40:24 GMT
I0524 20:40:24.863467 29828 round_trippers.go:580] Content-Length: 36
I0524 20:40:24.863472 29828 round_trippers.go:580] Content-Type: text/plain; charset=utf-8
I0524 20:40:24.883270 29828 request.go:1073] Response Body: Authorization header: invalid value
I0524 20:40:24.902200 29828 request.go:1264] body was not decodable (unable to check for Status): Object 'Kind' is missing in 'Authorization header: invalid value
'
I0524 20:40:24.902220 29828 cached_discovery.go:119] skipped caching discovery info due to the server has asked for the client to provide credentials
I0524 20:40:24.902301 29828 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" -H "Authorization: Bearer <masked>" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:40:24.903091 29828 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 0 milliseconds
I0524 20:40:24.903103 29828 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 0 ms Duration 0 ms
I0524 20:40:24.903109 29828 round_trippers.go:577] Response Headers:
I0524 20:40:24.903115 29828 round_trippers.go:580] X-Content-Type-Options: nosniff
I0524 20:40:24.903120 29828 round_trippers.go:580] X-Request-Id: 01G3TTEMA60E4ZXVA2C0QNYE1B
I0524 20:40:24.903125 29828 round_trippers.go:580] Date: Tue, 24 May 2022 10:40:24 GMT
I0524 20:40:24.903130 29828 round_trippers.go:580] Content-Length: 36
I0524 20:40:24.903135 29828 round_trippers.go:580] Content-Type: text/plain; charset=utf-8
I0524 20:40:24.903140 29828 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:40:24.922101 29828 request.go:1073] Response Body: Authorization header: invalid value
I0524 20:40:24.941260 29828 request.go:1264] body was not decodable (unable to check for Status): Object 'Kind' is missing in 'Authorization header: invalid value
'
I0524 20:40:24.941274 29828 cached_discovery.go:119] skipped caching discovery info due to the server has asked for the client to provide credentials
I0524 20:40:24.941287 29828 shortcut.go:89] Error loading discovery information: the server has asked for the client to provide credentials
I0524 20:40:24.941366 29828 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" -H "Authorization: Bearer <masked>" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:40:24.941989 29828 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 0 milliseconds
I0524 20:40:24.941999 29828 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 0 ms Duration 0 ms
I0524 20:40:24.942008 29828 round_trippers.go:577] Response Headers:
I0524 20:40:24.942014 29828 round_trippers.go:580] X-Request-Id: 01G3TTEMBDSC2H86E15NJKMD6C
I0524 20:40:24.942019 29828 round_trippers.go:580] Date: Tue, 24 May 2022 10:40:24 GMT
I0524 20:40:24.942025 29828 round_trippers.go:580] Content-Length: 36
I0524 20:40:24.942031 29828 round_trippers.go:580] Content-Type: text/plain; charset=utf-8
I0524 20:40:24.942036 29828 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:40:24.942041 29828 round_trippers.go:580] X-Content-Type-Options: nosniff
I0524 20:40:24.960181 29828 request.go:1073] Response Body: Authorization header: invalid value
I0524 20:40:24.980410 29828 request.go:1264] body was not decodable (unable to check for Status): Object 'Kind' is missing in 'Authorization header: invalid value
'
I0524 20:40:24.980436 29828 cached_discovery.go:119] skipped caching discovery info due to the server has asked for the client to provide credentials
I0524 20:40:24.980550 29828 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" -H "Authorization: Bearer <masked>" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:40:24.981403 29828 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 0 milliseconds
I0524 20:40:24.981420 29828 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 0 ms Duration 0 ms
I0524 20:40:24.981426 29828 round_trippers.go:577] Response Headers:
I0524 20:40:24.981432 29828 round_trippers.go:580] Content-Length: 36
I0524 20:40:24.981438 29828 round_trippers.go:580] Content-Type: text/plain; charset=utf-8
I0524 20:40:24.981445 29828 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:40:24.981450 29828 round_trippers.go:580] X-Content-Type-Options: nosniff
I0524 20:40:24.981456 29828 round_trippers.go:580] X-Request-Id: 01G3TTEMCNTD9H79EFH22RXY63
I0524 20:40:24.981461 29828 round_trippers.go:580] Date: Tue, 24 May 2022 10:40:24 GMT
I0524 20:40:25.001150 29828 request.go:1073] Response Body: Authorization header: invalid value
I0524 20:40:25.021818 29828 request.go:1264] body was not decodable (unable to check for Status): Object 'Kind' is missing in 'Authorization header: invalid value
'
I0524 20:40:25.021857 29828 cached_discovery.go:119] skipped caching discovery info due to the server has asked for the client to provide credentials
I0524 20:40:25.022029 29828 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" -H "Authorization: Bearer <masked>" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:40:25.023248 29828 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 1 milliseconds
I0524 20:40:25.023262 29828 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 0 ms Duration 1 ms
I0524 20:40:25.023270 29828 round_trippers.go:577] Response Headers:
I0524 20:40:25.023279 29828 round_trippers.go:580] Date: Tue, 24 May 2022 10:40:25 GMT
I0524 20:40:25.023286 29828 round_trippers.go:580] Content-Length: 36
I0524 20:40:25.023292 29828 round_trippers.go:580] Content-Type: text/plain; charset=utf-8
I0524 20:40:25.023299 29828 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:40:25.023305 29828 round_trippers.go:580] X-Content-Type-Options: nosniff
I0524 20:40:25.023311 29828 round_trippers.go:580] X-Request-Id: 01G3TTEMDYZBPGX37MD6YNW28Z
I0524 20:40:25.043278 29828 request.go:1073] Response Body: Authorization header: invalid value
I0524 20:40:25.063848 29828 request.go:1264] body was not decodable (unable to check for Status): Object 'Kind' is missing in 'Authorization header: invalid value
'
I0524 20:40:25.063913 29828 cached_discovery.go:119] skipped caching discovery info due to the server has asked for the client to provide credentials
I0524 20:40:25.064161 29828 helpers.go:222] server response object: [{
"metadata": {},
"status": "Failure",
"message": "the server has asked for the client to provide credentials",
"reason": "Unauthorized",
"details": {
"causes": [
{
"reason": "UnexpectedServerResponse",
"message": "Authorization header: invalid value"
}
]
},
"code": 401
}]
error: You must be logged in to the server (the server has asked for the client to provide credentials)As you can see, kubectl is trying and failing to parse kas' response - I0524 20:40:24.902200 29828 request.go:1264] body was not decodable (unable to check for Status): Object 'Kind' is missing in 'Authorization header: invalid value'. User also gets a generic error: You must be logged in to the server (the server has asked for the client to provide credentials) rather than what kas actually sends in the response body Authorization header: invalid value.
New behavior:
➜ ~ k get pods
error: You must be logged in to the server (GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMA86JFW3SBS31WAVDHG0)
➜ ~ k -v 9 get pods
I0524 20:43:43.985391 29895 loader.go:372] Config loaded from file: ~/.kube/config
I0524 20:43:43.986389 29895 round_trippers.go:466] curl -v -XGET -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" -H "Authorization: Bearer <masked>" -H "Accept: application/json, */*" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:43:43.987981 29895 round_trippers.go:510] HTTP Trace: Dial to tcp:127.0.0.1:8154 succeed
I0524 20:43:44.017253 29895 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 30 milliseconds
I0524 20:43:44.017264 29895 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 28 ms ServerProcessing 0 ms Duration 30 ms
I0524 20:43:44.017271 29895 round_trippers.go:577] Response Headers:
I0524 20:43:44.017275 29895 round_trippers.go:580] Content-Type: application/json
I0524 20:43:44.017279 29895 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:43:44.017282 29895 round_trippers.go:580] X-Request-Id: 01G3TTMPRGC4Z7GJ33W6ZB5JXZ
I0524 20:43:44.017285 29895 round_trippers.go:580] Date: Tue, 24 May 2022 10:43:44 GMT
I0524 20:43:44.017288 29895 round_trippers.go:580] Content-Length: 231
I0524 20:43:44.038388 29895 request.go:1073] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPRGC4Z7GJ33W6ZB5JXZ","reason":"Unauthorized","code":401}
I0524 20:43:44.056407 29895 cached_discovery.go:119] skipped caching discovery info due to GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPRGC4Z7GJ33W6ZB5JXZ
I0524 20:43:44.056498 29895 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "Authorization: Bearer <masked>" -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:43:44.059073 29895 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 2 milliseconds
I0524 20:43:44.059084 29895 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 0 ms Duration 2 ms
I0524 20:43:44.059088 29895 round_trippers.go:577] Response Headers:
I0524 20:43:44.059092 29895 round_trippers.go:580] Content-Type: application/json
I0524 20:43:44.059097 29895 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:43:44.059100 29895 round_trippers.go:580] X-Request-Id: 01G3TTMPSRDKN4DKB28MVTMCV7
I0524 20:43:44.059104 29895 round_trippers.go:580] Date: Tue, 24 May 2022 10:43:44 GMT
I0524 20:43:44.059109 29895 round_trippers.go:580] Content-Length: 231
I0524 20:43:44.082077 29895 request.go:1073] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPSRDKN4DKB28MVTMCV7","reason":"Unauthorized","code":401}
I0524 20:43:44.101247 29895 cached_discovery.go:119] skipped caching discovery info due to GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPSRDKN4DKB28MVTMCV7
I0524 20:43:44.101500 29895 shortcut.go:89] Error loading discovery information: GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPSRDKN4DKB28MVTMCV7
I0524 20:43:44.101556 29895 round_trippers.go:466] curl -v -XGET -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" -H "Authorization: Bearer <masked>" -H "Accept: application/json, */*" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:43:44.102167 29895 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 0 milliseconds
I0524 20:43:44.102176 29895 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 0 ms Duration 0 ms
I0524 20:43:44.102180 29895 round_trippers.go:577] Response Headers:
I0524 20:43:44.102185 29895 round_trippers.go:580] Content-Length: 231
I0524 20:43:44.102189 29895 round_trippers.go:580] Content-Type: application/json
I0524 20:43:44.102194 29895 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:43:44.102198 29895 round_trippers.go:580] X-Request-Id: 01G3TTMPV5C52VB87VPS081815
I0524 20:43:44.102202 29895 round_trippers.go:580] Date: Tue, 24 May 2022 10:43:44 GMT
I0524 20:43:44.104077 29895 request.go:1073] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPV5C52VB87VPS081815","reason":"Unauthorized","code":401}
I0524 20:43:44.123329 29895 cached_discovery.go:119] skipped caching discovery info due to GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPV5C52VB87VPS081815
I0524 20:43:44.123443 29895 round_trippers.go:466] curl -v -XGET -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" -H "Accept: application/json, */*" -H "Authorization: Bearer <masked>" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:43:44.124380 29895 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 0 milliseconds
I0524 20:43:44.124394 29895 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 0 ms Duration 0 ms
I0524 20:43:44.124399 29895 round_trippers.go:577] Response Headers:
I0524 20:43:44.124405 29895 round_trippers.go:580] Content-Length: 231
I0524 20:43:44.124409 29895 round_trippers.go:580] Content-Type: application/json
I0524 20:43:44.124414 29895 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:43:44.124419 29895 round_trippers.go:580] X-Request-Id: 01G3TTMPVV7627JZMX0D9VQT3N
I0524 20:43:44.124424 29895 round_trippers.go:580] Date: Tue, 24 May 2022 10:43:44 GMT
I0524 20:43:44.143208 29895 request.go:1073] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPVV7627JZMX0D9VQT3N","reason":"Unauthorized","code":401}
I0524 20:43:44.162461 29895 cached_discovery.go:119] skipped caching discovery info due to GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPVV7627JZMX0D9VQT3N
I0524 20:43:44.162572 29895 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" -H "Authorization: Bearer <masked>" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:43:44.163716 29895 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 1 milliseconds
I0524 20:43:44.163726 29895 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 0 ms Duration 1 ms
I0524 20:43:44.163731 29895 round_trippers.go:577] Response Headers:
I0524 20:43:44.163737 29895 round_trippers.go:580] Content-Length: 231
I0524 20:43:44.163741 29895 round_trippers.go:580] Content-Type: application/json
I0524 20:43:44.163746 29895 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:43:44.163750 29895 round_trippers.go:580] X-Request-Id: 01G3TTMPX3MB77B6FY24SF9V6E
I0524 20:43:44.163755 29895 round_trippers.go:580] Date: Tue, 24 May 2022 10:43:44 GMT
I0524 20:43:44.183170 29895 request.go:1073] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPX3MB77B6FY24SF9V6E","reason":"Unauthorized","code":401}
I0524 20:43:44.203349 29895 cached_discovery.go:119] skipped caching discovery info due to GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPX3MB77B6FY24SF9V6E
I0524 20:43:44.203556 29895 helpers.go:222] server response object: [{
"metadata": {},
"status": "Failure",
"message": "GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPX3MB77B6FY24SF9V6E",
"reason": "Unauthorized",
"code": 401
}]
error: You must be logged in to the server (GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPX3MB77B6FY24SF9V6E)Improvements:
-
GitLab Agent Serverin the message allows to distinguish kas-generated errors vs upstream-generated errors. - Actual error message is displayed to the user.
- No parsing errors.
- Correlation ID is displayed.