Delete unused resources

What does this MR do?

This MR deletes unused resources.

It catalogs the existing and desired managed resources and at the end of the reconciliation loop deletes the unused managed resources, i.e. those that exist but are not desired/required anymore.

This is done via an interface and helper functions to minimize the impact on the existing controller code. This is not ideal but they will be removed once we could replace it with the framework.

Testing

1. Deploy a GitLab instance using the Operator with the default list of components:

   apiVersion: apps.gitlab.com/v1beta1
   kind: GitLab
   metadata:
     name: issue-889
     namespace: <the namespace that Operator watches>
   spec:
     chart:
       version: 7.1.2
       values:
         global:
           hosts:
             domain: issue-889.<a real domain>
           ingress:
             class: nginx
             configureCertmanager: true
         certmanager-issuer:
           email: <a real email address>

2. Once GitLab resource is Available go ahead and edit the .spec and disable one of the components, e.g. KAS:

   ...
   spec:
     chart:
       version: 7.1.2
       values:
         global:
           kas:
             enabled: false
         ...

3. Wait for the changes to take effect and GitLab resource to become Available again.

4. Check KAS resources, e.g. its Deployment or ConfigMap. They still exist.

5. Change the controller manager image to the image that is built for this branch in build_branch_image job. Use image SHA for accuracy. You can do this by editing the gitlab-controller-manager Deployment.

6. Wait for the changes to take effect and GitLab resource become Available.

7. Check the logs of manager container for gitlab-controller-manager Pod. You should see Object deleted messages at the end of the reconciliation loop.

8. Check KAS resources, e.g. its Deployment or ConfigMap. They are gone.

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

• Merge Request Title and Description are up to date, accurate, and descriptive
• MR targeting the appropriate branch
• MR has a green pipeline on GitLab.com
• When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow

Expected (please provide an explanation if not completing)

• Test plan indicating conditions for success has been posted and passes
• Documentation created/updated
• Tests added (depends on &28)
• Integration tests added to GitLab QA
• Equivalent MR/issue for omnibus-gitlab opened
• Validate potential values for new configuration settings. Formats such as integer 10, duration 10s, URI scheme://user:passwd@host:port may require quotation or other special handling when rendered in a template and written to a configuration file.

Related issues

Closes #889 (closed)

Closes #315 (closed)