<!-- The first four sections: "Problem to solve", "Intended users", "User experience goal", and "Proposal", are strongly recommended, while the rest of the sections can be filled out during the problem validation or breakdown phase. However, keep in mind that providing complete and relevant information early helps our product team validate the problem and start working on a solution. --> ### Problem to solve As a software engineer, when I configure the inline code coverage feature and upload a cobertura report generated by a common library in C# it should "just work" so that I can get back to coding and not worry about parsing a report with a program before I pass it to GitLab. Right now the [Test Coverage Visualization feature](https://docs.gitlab.com/ee/user/project/merge_requests/test_coverage_visualization.html) doesn't handle paths in the cobertura report in a robust way. Many different coverage report generating libraries have different approaches around: 1. How much of the path is included in the report in general 1. Whether the path is recorded as a relative or an absolute path 1. Where that path is in the cobertura file (the amount of the path that is in the `source` node as compared to the `class` node in the `filename` attribute varies This is particularly troublesome for GitLab CI as the report is typically generated inside of a container, and then processed inside of the main application. This means that the filesystem for the report generation is completely different than the filesystem used to process the report. The result of this is several notable report generating libraries will generate reports incompatible with this feature _some of the time_. It is unknown how often this is the case, but there have been a few issues raised: 1. C#/coverlet: https://gitlab.com/gitlab-org/gitlab/-/issues/213444 1. Python/coverage: https://forum.gitlab.com/t/enabling-cobertura/36918/5 <!-- What problem do we solve? Try to define the who/what/why of the opportunity as a user story. For example, "As a (who), I want (what), so I can (why/value)." --> ### Intended users * [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer) * [Simone (Software Engineer in Test)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#simone-software-engineer-in-test) <!-- Who will use this feature? If known, include any of the following: types of users (e.g. Developer), personas, or specific company roles (e.g. Release Manager). It's okay to write "Unknown" and fill this field in later. Personas are described at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/ * [Cameron (Compliance Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#cameron-compliance-manager) * [Parker (Product Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#parker-product-manager) * [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead) * [Presley (Product Designer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#presley-product-designer) * [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer) * [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer) * [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator) * [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst) * [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager) * [Alex (Security Operations Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#alex-security-operations-engineer) * [Simone (Software Engineer in Test)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#simone-software-engineer-in-test) * [Allison (Application Ops)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#allison-application-ops) * [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer) * [Dana (Data Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#dana-data-analyst) --> ### Proposal We can build some logic into how the paths are handled before we attempt to match them with files in the merge request. Ideally we could find a way to intelligently interpret the path so we can get to the project root path and not consider the rest of the path when attempting a match. We should look into implementing some logic around matching absolute paths and paths from the root of the git project, so that the cobertura report for C# can be parsed when it is generated in a container. @rickywiens has an example C# project, and both https://gitlab.com/gitlab-org/gitaly and https://gitlab.com/gitlab-org/gitlab-runner are using workarounds to fix the pathing issues, so they would be good samples to use for this. <!-- How are we going to solve the problem? Try to include the user journey! https://about.gitlab.com/handbook/journeys/#user-journey --> ### Further details <!-- Include use cases, benefits, goals, or any other details that will help us understand the problem better. --> ### Documentation <!-- See the Feature Change Documentation Workflow https://docs.gitlab.com/ee/development/documentation/workflow.html#for-a-product-change * Add all known Documentation Requirements in this section. See https://docs.gitlab.com/ee/development/documentation/feature-change-workflow.html#documentation-requirements * If this feature requires changing permissions, update the permissions document. See https://docs.gitlab.com/ee/user/permissions.html --> An issue has been created to document the process for working around this issue: https://gitlab.com/gitlab-org/gitlab/-/issues/217365 after this is fixed that documentation should be removed/modified. ### Availability & Testing <!-- This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier. What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing? Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance. * Unit test changes * Integration test changes * End-to-end test change See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning --> We should try and test all known combinations/permutations of the paths: Relative/Absolute, various splits between the path being in the `source` node and paths being in the `filename` attribute in the `class` node. We should likely generate example cobertura reports from popular libraries and include them in our integration tests for the feature. Some ideas about popular libraries: 1. ruby/rspec 1. python/coverage 1. Java/?? 1. C#/coverlet 1. probably more? ### What does success look like, and how can we measure that? <!-- Define both the success metrics and acceptance criteria. Note that success metrics indicate the desired business outcomes, while acceptance criteria indicate when the solution is working correctly. If there is no way to measure success, link to an issue that will implement a way to measure this. --> #### Acceptance Criteria * Test Coverage Visualization "works" (meets original feature AC) for projects using C# Libraries and generating a cobertura report. * Documentation is updated to detail any special setup / changes that need to be made to a project or report to make the feature work. #### Measures of success * Increase in page views of the feature documentation page by 10% Month over Month after release (so if this ships on Sept 22 we would measure Oct vs. Sept) to measure any increase in awareness of the feature. * There is no tracking of feature usage at this time. We should make this feature to "just work" with the dominant C# libraries that generate cobertura reports. ### What is the type of buyer? <!-- What is the buyer persona for this feature? See https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/buyer-persona/ In which enterprise tier should this feature go? See https://about.gitlab.com/handbook/product/pricing/#four-tiers --> ### Is this a cross-stage feature? <!-- Communicate if this change will affect multiple Stage Groups or product areas. We recommend always start with the assumption that a feature request will have an impact into another Group. Loop in the most relevant PM and Product Designer from that Group to provide strategic support to help align the Group's broader plan and vision, as well as to avoid UX and technical debt. https://about.gitlab.com/handbook/product/#cross-stage-features --> ### Links / references This will be behind a feature flag: `:smart_cobertura_parser`