Skip to content

Add self-signing for testing ai-gateway using gitlab-qa

Jay McCure requested to merge jmc-aigw-self-sign into master

What does this MR do and why?

Part of: gitlab-org/modelops/applied-ml/code-suggestions/ai-assist#508

In order to call /api/v4/code_suggestions/direct_access and get a token to perform code completions directly to ai-gateway, the ai-gateway instance must have a signing and validation key passed in as env variables. We can use test keys from the ai-gateway repo: https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/blob/main/example.env?ref_type=heads#L79

I've hardcoded the key into gitlab-qa as it isn't secret and it saves us from having to pass in another environment variable to the job itself.

This MR adds the test keys to ai-gateway env variables in gitlab-qa so we can test code completions via direct connections.

Without these keys direct connectivity does not work and we get the following error. Discussion: https://gitlab.slack.com/archives/C04KWTK3GFJ/p1717419586128949?thread_ts=1717419517.383519&cid=C04KWTK3GFJ

error 
{
    "status_code": null,
    "exception_class": "JWKError",
    "backtrace": "Traceback (most recent call last):\n  File \"/opt/venv/ai-gateway-9TtSrW0h-py3.10/lib/python3.10/site-packages/jose/backends/cryptography_backend.py\", line 265, in __init__\n    self.prepared_key = load_pem_public_key(key, self.cryptography_backend())\nValueError: Unable to load PEM file. See https://cryptography.io/en/latest/faq/#why-can-t-i-import-my-pem-file for more details. MalformedFraming\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/opt/venv/ai-gateway-9TtSrW0h-py3.10/lib/python3.10/site-packages/jose/backends/cryptography_backend.py\", line 267, in __init__\n    self.prepared_key = load_pem_private_key(key, password=None, backend=self.cryptography_backend())\n  File \"/opt/venv/ai-gateway-9TtSrW0h-py3.10/lib/python3.10/site-packages/cryptography/hazmat/backends/openssl/backend.py\", line 494, in _handle_key_loading_error\n    raise ValueError(\nValueError: ('Could not deserialize key data. The data may be in an incorrect format, it may be encrypted with an unsupported algorithm, or it may be an unsupported key type (e.g. EC curves with explicit parameters).', [<OpenSSLError(code=503841036, lib=60, reason=524556, reason_text=unsupported)>])\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/app/ai_gateway/auth/providers.py\", line 122, in jwks\n    jwk.RSAKey(\n  File \"/opt/venv/ai-gateway-9TtSrW0h-py3.10/lib/python3.10/site-packages/jose/backends/cryptography_backend.py\", line 269, in __init__\n    raise JWKError(e)\njose.exceptions.JWKError: ('Could not deserialize key data. The data may be in an incorrect format, it may be encrypted with an unsupported algorithm, or it may be an unsupported key type (e.g. EC curves with explicit parameters).', [<OpenSSLError(code=503841036, lib=60, reason=524556, reason_text=unsupported)>])\n",
    "correlation_id": "01J1XZ9G4KY3HNTGR0TKFNRGB9",
    "extra":
    {},
    "logger": "exceptions",
    "level": "error",
    "type": "mlops",
    "stage": "main",
    "timestamp": "2024-07-04T04:15:56.167849Z",
    "message": "('Could not deserialize key data. The data may be in an incorrect format, it may be encrypted with an unsupported algorithm, or it may be an unsupported key type (e.g. EC curves with explicit parameters).', [<OpenSSLError(code=503841036, lib=60, reason=524556, reason_text=unsupported)>])"
}

How to set up and validate locally

I have tested this via a pipeline in the gitlab project using a draft of the direct access code completion test.

Using this MR's version of gitlab-qa, existing tests and code completion direct connection works: https://gitlab.com/gitlab-org/gitlab/-/jobs/7263419748#L569

With the master version of gitlab-qa we get a 503 https://gitlab.com/gitlab-org/gitlab/-/jobs/7262113520#L585 (the above error can be found in the modelgateway.log logs)

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Jay McCure

Merge request reports