Feature instrumentation reminder - release 16.10

Problem

Analysis conducted by the CSM team reveals that only 50% of our premium features are currently instrumented. While we released on an average of 30 features per milestone between versions 16.6 and 16.9, we observed that, on average, only 6 features are instrumented per milestone.

The absence or delay of instrumentation presents several challenges:

1. Missed Insights: Without instrumentation, we're missing out on valuable insights that could be derived from the data.
2. Deferred Instrumentation: When instrumentation is delayed to future milestones, self-managed customers must upgrade to the instrumented version for us to gather insights. This delay means we have less data from self-managed sources.
3. Hindered Customer Support: Lack of instrumentation impedes the ability of our CSM team to assist customers seeking to understand feature usage and adoption within their organisations.

Goal

The primary objective of this issue is to track features released in each milestone and gain insights from Product Managers and Engineering Managers regarding the current status of instrumentation for these features. By doing so, we aim to identify any obstacles hindering instrumentation and collaborate with teams to address these gaps effectively. Our overarching aim is to increase the number of instrumented features at the time of release, ensuring the availability of timely data essential for guiding data-driven decisions.

Furthermore, by highlighting the number of metrics being instrumented across different stages in each milestone, we aim to inspire more teams to kickstart instrumentation efforts.

Product Manager tasks

• Please review the feature list for your stage and category and fill in the Instrumentation and Additional Information columns. Here are further details:

For the Instrumentation column, please select one of the following options:

1. Instrumentation added
2. Planned for X milestone
3. Not Planned
4. Other

For the Additional information column,

• If you have added instrumentation, please provide the link to the instrumentation issue or MR. This will help us link features to their respective events/metrics and maintaining a single source of truth for feature-to-event/metrics mapping. Currently, this process is manual, but we are actively working on automating it. You won't need to add this information in the future.
• If you choose any of the other options, please provide additional information that will help us understand any blockers or pain points regarding feature instrumentation.

Feature list

Table contains list of features that made a release post in 16.10 and is sorted by stage and category. Additional details for the features available here

Stage	Category	Feature	Reporter	Instrumentation	Additional Information
ai-powered	Duo Chat	GitLab Duo access governance control	@pwietchner
create	Code Review Workflow	Expanded checks in merge widget	@phikai	Not Planned	These are additional views that are on by default, not things people interact with
create	Code Review Workflow	Automatically collapse generated files in merge requests	@phikai	Not Planned	These are additional views that are on by default, not things people interact with
data_stores	User Profile	Blocked users are excluded from the followers list	@lohrc	Not Planned	Team is at capacity for the foreseeable future.
data_stores	Groups Projects	Filter groups by visibility in the REST API	@lohrc	Not Planned	This was a community contribution. Team is at capacity for the foreseeable future.
data_stores	Groups Projects	Filter members by Enterprise users with GraphQL API	@lohrc	Not Planned	This was a community contribution. Team is at capacity for the foreseeable future.
data_stores	Groups Projects	Updated project deletion functionality	@lohrc	Not Planned	These were updates to the deletion flow by the UX Paper Cuts team. Team is at capacity for the foreseeable future.
deploy	Environment Management	Manually refresh the dashboard for Kubernetes	@nagyv-gitlab	Not planned	This feature complements a bigger feature (Flux usage) that is instrumented separately and improved instrumentation is planned
deploy	Environment Management	Improved environment details page	@nagyv-gitlab	Not planned	This was a redesign. Instrumentation is already in place.
govern	Security Policy Management	Scan result policies are now "Merge request approval policies"	@g.hickman
govern	Security Policy Management	Use merge base pipeline for merge request approval policy comparison	@g.hickman
govern	Permissions	Audit event for assigning a custom role	@jrandazzo
govern	System Access	Improved error message for authentication rate limit	@hsutor	Not Planned	Simply a UX improvement
govern	User Management	Smart card support for Active Directory LDAP	@hsutor	Planning (no timeline)	Would be a good thing to understand usage of, but is not a high priority
govern	System Access	Webhooks support mutual TLS	@hsutor	Not planned	Not something easily instrument-able, resulting data would not inform any decisions, so not worth the effort
govern	Permissions	New permissions for custom roles	@jrandazzo
govern	Compliance Management	Audit event scope attribute	@g.hickman
govern	User Management	Custom names for service accounts	@hsutor	Not planned	Would have to do complex logic comparing statically issued names vs users that modified the name - would not inform any future decisions, so not worth the effort
govern	System Access	Sign-in page improvements	@hsutor	Not planned	No way to instrument
manage	Integrations	Threaded notifications supported in Google Chat	@m_frankiewicz	Not Planned
manage	Webhooks	Custom payload template for webhooks	@m_frankiewicz	Not Planned
monitor	Service Desk	Create Service Desk tickets from the UI and API	@msaleiko	Instrumentation added	!139553 (merged)
package	Container Registry	List repository tags with the new container registry API	@trizzi
plan	Team Planning	Add an issue to the current or next iteration with a quick action	@gweaver
plan	Value Stream Management	Inherited filters in Value Stream Analytics for seamless and accurate workflow analysis	@hsnir1
plan	Pages	Support domain-level redirects for GitLab Pages	@mmacfarlane	Not Planned	This was an improvement to a pre-existing feature and lands in our free tier. As a result of this we don't plan on adding instrumentation to it.
plan	Value Stream Management	New ClickHouse integration for high-performance DevOps Analytics	@hsnir1
plan	Value Stream Management	New contributor count metric in the Value Streams Dashboard	@hsnir1
plan	Wiki	Wiki templates	@mmacfarlane	Planning (no timeline)	We have plans to add instrumentation to multiple areas of the Wiki.
platforms	GitLab Dedicated	GitLab Pages and Advanced Search available on GitLab Dedicated	@lbortins
secure	Software Composition Analysis	Continuous Vulnerability Scanning available by default for Container Scanning	@johncrowley	Instrumentation Available	See Tableau (not specific to Container Scanning, but this is what is available to us).
secure	DAST	DAST analyzer performance updates	@smeadzinger	Instrumentation added	https://gitlab.com/gitlab-org/security-products/analyzers/browserker/-/merge_requests/1288
secure	Software Composition Analysis	Improved Dependency Scanning support for sbt	@johncrowley	Not planned / Not necessary
systems	Geo replication	Offload CI traffic to Geo secondaries	@sranasinghe	Planned (milestone yet to be defined)
systems	Cloud Native Installation	GitLab chart improvements	@dorrino
systems	Omnibus Package	Omnibus improvements	@dorrino
verify	GitLab Runner Core	GitLab Runner 16.10	@deastman
verify	Pipeline Composition	Semantic versioning in the CI/CD catalog	@dhershkovitch	Instrumentation not planned	This is a functionality we enforce and there is no way/need to instrument it

Instrumentation wins this milestone

Kudos grouprunner groupacquisition for the highest number of instrumented metrics this milestone! 🎉 🏃 🎶

Link to Tableau dashboard

cc @stkerr @bastirehm @cbraza @mdrussell @iweeks @gitlab-org/analytics-section/analytics-instrumentation @steve-evangelista @timzallmann