Follow-up from "Add Prometheus metrics endpoint and basic infrastructure to meter code"

The following discussion from !11553 (merged) should be addressed:

• @grzesiek started a discussion:

  Because of https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/11553#note_31671685 we do not clean-up this directory in after hook. We should devise a way to improve this in the future.

/cc @pchojnacki