Add an `annotate` command
This adds a new command, Annotate, which drops a user defined annotation on Grafana.
Sometimes it is useful to manually mark down specific periods in a Grafana graph during which an event happened, so as to see how the metrics responded to that event. More often than not this is done for us e.g. by the feature flag command. However, I've run into several cases already where I wish was able to drop annotations so as to demarcate experiments and fence in graphs during that time, for instance:
- Running expensive or long-running tasks on staging or prod (such as imports & exports, often triggered manually)
- Experimenting with switches that are no feature flags, such as env vars being turned on or off (https://ops.gitlab.net/gitlab-cookbooks/chef-repo/-/merge_requests/3390)
Grafana supports this via CTRL+click even, but that requires admin access, and I believe it also does not record who created the annotation, at leats not which GitLab user.
I think we should give developers broader access to mark down important events in our dashboards outside of automated tasks, and ChatOps seems like the right tool for this.
The new command also records the GitLab user who created the annotation.