CI: Metrics in MR are not displayed although Metrics in Environments show up correctly - likely an issue with documentation and unclear error messages
PROJECT_URL/merge_requests/1
Deployment statistics are not available currently
PROJECT_URL/environments/ENVIRONMENT_ID/metrics
Docs improvements
The docs are vague on how the performance metrics in MR work. https://docs.gitlab.com/ce/user/project/integrations/prometheus.html#determining-the-performance-impact-of-a-merge
When a source branch has been deployed to an environment, a sparkline and numeric comparison of the average memory consumption will appear.
This sounds like it would compare it to the environment on target branch... Target branch environment's memory usage is 100 MB, source branch environment's memory usage is 150 MB. We have a 50 MB difference. (Hypothesis no 1)
The comparison shows the difference between the 30 minute average before and after the deployment.
How can it compare the metrics before the deployment, when source environment didn't exist? This leads me to think the comparison is between the current revision and a previous revision in the same environment (Hypothesis no 2) But this doesn't make sense for review apps where each branch gets its own environment.
https://docs.gitlab.com/ee/ci/review_apps/ should either mention "performance impact of MR" feature and explain how to make it work. If the metrics-in-MR feature doesn't fit the review apps pattern and the metrics comparison in MR is of no practical use (comparing between feature branch's revisions), this should be explicitly mentioned in both documents.
Error message improvements
No metrics show up on our end although the metrics show up correctly within environments. This error message isn't helpful in any way:
Deployment statistics are not available currently
Assuming Hypothesis no 1 is true, here's how we can improve the wording:
If it cannot display the comparison because it's missing metrics from a different environment (Hypothesis no 1), then the error message should say, e.g. "Cannot compare deployment metrics to target branch. We couldn't locate metrics for target branch environment." The link to source branch environment metrics is key. We can at least see how it performs by itself.
If it cannot display the comparison because it cannot detect what the source branch environment is, it should say: "Cannot compare deployment metrics to target branch. We couldn't find the target branch environment."
There's probably more use cases, and maybe it's Hypothesis no 2 that's true. Either way, the messaging needs to be clear and helpful.