Support parsing manifests with yet unknown Custom Resources
Hey there,
I am in the process of evaluating the GitLab Agent for Kubernetes to move our infrastructure to gitlab.com
Sadly, there seems to be no setting to ignore validation of manifests. This results to the following errors of the agent:
Failed to decode GitOps objects
and
unknown resource types:
I would propose to have a setting in the agents gitops configuration (or per manifest file) to ignore model validation.
Also, after the error above my agent log is full of:
{"level":"error","time":"2022-02-01T11:18:28.577Z","msg":"GetObjectsToSynchronize failed","mod_name":"gitops","project_id":"x/x/x/x","agent_id":x,"error":"rpc error: code = Unavailable desc = connection error: desc = \"transport: failed to write client preface: failed to write msg: failed to close writer: failed to write fin frame: WebSocket closed: received close frame: status = StatusGoingAway and reason = \\\"Shutting down\\\"\""}
{"level":"error","time":"2022-02-01T11:18:28.577Z","msg":"Error handling a connection","mod_name":"reverse_tunnel","error":"Connect(): rpc error: code = Unavailable desc = connection error: desc = \"transport: failed to write client preface: failed to write msg: failed to close writer: failed to write fin frame: WebSocket closed: received close frame: status = StatusGoingAway and reason = \\\"Shutting down\\\"\""}
(Replaced sensitive data with 'x')
Related Urls:
- https://about.gitlab.com/blog/2021/02/22/gitlab-kubernetes-agent-on-gitlab-com/
- https://docs.gitlab.com/ee/user/clusters/agent/
Proposal
Ok, I've looked at the code and I see what the problem is. We retry applying the resources, but parsing them happens outside of that loop. This needs to be fixed to support this use case.
Implementation/fix thoughts (for engineers): As part of that, when manifestreader.SetNamespaces()
returns a UnknownTypesError
error, we should reset the rest mapper (meta.MaybeResetRESTMapper()
) to refresh the discovery information (registered Kubernetes API types that could have been updated with CRDs) and try again. This can likely be done in the syncDecoder
. Additionally, we should avoid re-parsing and re-setting namespaces once we have done that successfully (memoize the result).