Feature monitoring

Description

There are at least three kinds of monitoring that a developer cares about:

1. Application performance monitoring (APM)
2. System monitoring
3. Business metrics/analytics

What I'm excited about is a 4th kind, which I'm calling "feature monitoring". Being able to know after shipping a merge request that we didn't blow up production. Or that error rates grew by 5% after enabling a feature flag on a subset of users.

Proposal

Here are three potential evolutions:

1. Measure before/after deploying merge request and attribute any changes to the relevant merge request(s)
2. Canary deploys where new features (via merge requests) can be shipped to a portion of a production fleet so they get real traffic, then monitor, report, and alert on differences in performance.
3. Feature flags to ship one or more merge requests behind a flag that can be enabled for a portion of production users, then monitor, report, and alert on differences in performance.

Links / references

• Feature deploys: https://gitlab.com/gitlab-org/gitlab-ee/issues/779
• Related: #23841 (closed)
• Original vision: https://about.gitlab.com/direction/cicd/#monitor
• Feature deploys: https://gitlab.com/gitlab-org/gitlab-ee/issues/779