<!-- The first three sections: "Problem to solve", "Intended users" and "Proposal", are strongly recommended, while the rest of the sections can be filled out during the problem validation or breakdown phase. However, keep in mind that providing complete and relevant information early helps our product team validate the problem and start working on a solution. --> ### Problem to solve <!-- What problem do we solve? Try to define the who/what/why of the opportunity as a user story. For example, "As a (who), I want (what), so I can (why/value)." --> Currently, if a pipeline has multiple jobs that output coverage, we calculate the average and use that as the coverage to report for the entire pipeline. This is almost guaranteed to be wrong, sometimes wildly so. If a developer wants more accurate coverage number, they have to [jump through a bunch of hoops](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5018). I don't know how to solve this generally. ### Intended users * [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer) - who wants to know from a glance at the pipeline widget on the MR page, jobs page or pipeline page how coverage has changed without having to download a report or visit another tool * [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead) - Who wants the project coverage badge to be an accurate reflection of coverage for the entire project. <!-- Who will use this feature? If known, include any of the following: types of users (e.g. Developer), personas, or specific company roles (e.g. Release Manager). It's okay to write "Unknown" and fill this field in later. Personas are described at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/ * [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager) * [Parker (Product Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#parker-product-manager) * [Presley (Product Designer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#presley-product-designer) * [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer) * [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator) * [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst) * [Dana (Data Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#dana-data-analyst) * [Simone (Software Engineer in Test)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#simone-software-engineer-in-test) * [Allison (Application Ops)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#allison-application-ops) --> ### Further details <!-- Include use cases, benefits, goals, or any other details that will help us understand the problem better. --> A use case from the gitlab project. * In the MR the pipeline shows coverage for the source branch and how it compares to the target branch.  * This value is made up of data from two jobs.   * Further complicating things is that the initial value is just the average of the data from the jobs, but there is much more ruby code than javascript in the repository. * Even further complicating things is that there are multiple ways coverage was calculated in the karma job, we picked one.  ### Proposal Modify the pipeline view that appears on the pipeline page to indicate how many jobs make up the calculated value and their coverage values in a tooltip. We may consider showing how the coverage diff from each job has changed but that may be a stretch. See the Designs below. ### Proposal options considered Some options we have considered: 1. Leave as-is 1. Use only the first (or last) coverage number generated in the pipeline 1. Tell people to manage their own coverage numbers and report a single, overall number. 1. Document how to report a single, overall number for some set of languages/frameworks. 1. Parse from various coverage reporters that know how to merge separate reports into a final number. How we will proceed: Leave the calculated value as is and add some context from the known jobs that make up the value. Why: None of the options considered are great ones nor meet everyone's needs and there is a distinct possibility we make things worse by changing the current method. Instead we believe that calling out when a coverage value has multiple inputs and providing the context about those inputs is the best option for users. To do this there are some changes coming in other issues: * [Custom text for coverage badges](https://gitlab.com/gitlab-org/gitlab/-/issues/17555) will allow for a better view of coverage numbers on the project view * [Graphing code coverage over time](https://gitlab.com/gitlab-org/gitlab/-/issues/33743) will allow a view of how the coverage value from each job is changing over time Other enhancements for integration with code coverage reports will be coming in the epic [Make code coverage more comprehensive](https://gitlab.com/groups/gitlab-org/-/epics/100) that we believe will further help add context to code coverage data for users. ### Permissions and Security <!-- What permissions are required to perform the described actions? Are they consistent with the existing permissions as documented for users, groups, and projects as appropriate? Is the proposed behavior consistent between the UI, API, and other access methods (e.g. email replies)?--> ### Documentation <!-- See the Feature Change Documentation Workflow https://docs.gitlab.com/ee/development/documentation/feature-change-workflow.html * Add all known Documentation Requirements in this section. See https://docs.gitlab.com/ee/development/documentation/feature-change-workflow.html#documentation-requirements * If this feature requires changing permissions, update the permissions document. See https://docs.gitlab.com/ee/user/permissions.html --> ### Availability & Testing <!-- This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier. What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing? Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance. * Unit test changes * Integration test changes * End-to-end test change See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning --> ### What does success look like, and how can we measure that? If this is successful more users will include code coverage calculations in their build jobs. We will calculate how many jobs include coverage data and think that within 90 days of shipping this and the other 2 issues mentioned in the proposal customer jobs with coverage reports will increase by 10%. <!-- Define both the success metrics and acceptance criteria. Note that success metrics indicate the desired business outcomes, while acceptance criteria indicate when the solution is working correctly. If there is no way to measure success, link to an issue that will implement a way to measure this. --> ### What is the type of buyer? This is a feature widely used by individuals and will be part of ~"GitLab Core" <!-- What is the buyer persona for this feature? See https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/buyer-persona/ In which enterprise tier should this feature go? See https://about.gitlab.com/handbook/product/pricing/#four-tiers --> ### Is this a cross-stage feature? <!-- Communicate if this change will affect multiple Stage Groups or product areas. We recommend always start with the assumption that a feature request will have an impact into another Group. Loop in the most relevant PM and Product Designer from that Group to provide strategic support to help align the Group's broader plan and vision, as well as to avoid UX and technical debt. https://about.gitlab.com/handbook/product/#cross-stage-features --> ### Links / references