Support deploying directories and sub-directories containing manifest files

Relese notes

Users of the GitLab Kubernetes Agent until now had to collect all their Kubernetes resources into a single manifest file. Starting with GitLab 13.6, the GitLab Kubernetes Agent can grab Kubernetes manifests recursively from specified directories in a project. This way platform engineers can use a single repository to manage different clusters from one place, and can describe large deployments easily to be deployed with a single Agent.

TODO: add docs link

Details

Currently GitLab Kubernetes Agent does not allow to specify which files or directories should be fetched from a manifest project. manifest.yaml in the root of the repository is the file that is used (was done so as part of POC to cut scope).

This issue is to allow to specify directories with files to be fetched, recursively or not. All *.yaml, *.yml and *.json files are fetched. Hidden directories (the one with a name starting with .) are ignored.

By default path scanning recurses into nested directories.

The agent configuration changes to:

gitops:
  manifest_projects:
  - id: "path-to/your-awesome-project"
    paths:
    - glob: "/environments/production/*.yaml"