Fix changelog entries deletion

When we compile the changelog, files are not deleted from the working directory.

We never found this because we usually commit specific files during a release tagging. With Gitaly, we merge the stable branch into master and these untracked files prevent master branch checkout (because those same files are still present on master branch)

Closes https://gitlab.com/gitlab-org/release-tools/-/issues/424