<!--IssueSummary start--> <details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=283987) </details> <!--IssueSummary end--> <!-- The first section "Release notes" is required if you want to have your release post blog MR auto generated. Currently in BETA, details on the **release post item generator** can be found in the handbook: https://about.gitlab.com/handbook/marketing/blog/release-posts/#release-post-item-generator and this video: https://www.youtube.com/watch?v=rfn9ebgTwKg. The next four sections: "Problem to solve", "Intended users", "User experience goal", and "Proposal", are strongly recommended in your first draft, while the rest of the sections can be filled out during the problem validation or breakdown phase. However, keep in mind that providing complete and relevant information early helps our product team validate the problem and start working on a solution. --> ### Release notes <!-- What is the problem and solution you're proposing? This content sets the overall vision for the feature and serves as the release notes that will populate in various places, including the [release post blog](https://about.gitlab.com/releases/categories/releases/) and [Gitlab project releases](https://gitlab.com/gitlab-org/gitlab/-/releases). " --> Programmatically query the number of commits in a single project during a specified time period. ### Problem to solve <!-- What problem do we solve? Try to define the who/what/why of the opportunity as a user story. For example, "As a (who), I want (what), so I can (why/value)." --> As a developer or manager of developers, I want to be able to programmatically* query the number of commits made to a certain project within a specified time interval so that I can understand where developers are spending their time and identify trends in these results over time. (* - The goal is to be able to use the API to retrieve this information.) ### Intended users <!-- Who will use this feature? If known, include any of the following: types of users (e.g. Developer), personas, or specific company roles (e.g. Release Manager). It's okay to write "Unknown" and fill this field in later. Personas are described at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/ * [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead) * [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer) * [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator) * [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst) * [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager) * [Alex (Security Operations Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#alex-security-operations-engineer) * [Simone (Software Engineer in Test)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#simone-software-engineer-in-test) * [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer) * [Dana (Data Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#dana-data-analyst) --> * [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead) * [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer) * [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator) * [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst) * [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager) * [Alex (Security Operations Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#alex-security-operations-engineer) * [Simone (Software Engineer in Test)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#simone-software-engineer-in-test) * [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer) * [Dana (Data Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#dana-data-analyst) ### User experience goal <!-- What is the single user experience workflow this problem addresses? For example, "The user should be able to use the UI/API/.gitlab-ci.yml with GitLab to <perform a specific task>" https://about.gitlab.com/handbook/engineering/ux/ux-research-training/user-story-mapping/ --> The user should be able to use the API to get information about the number of commits that were made to a specific project during a specified time interval: 1. User makes authenticated call to API with these parameters: project ID, start date and time, end date and time 2. Successful response from the API includes the total number of commits made to the specified project between the specified start date and time and the specified end date and time 3. The user can implement custom code that consumes the API response, extract the number of commits and use that value for further processing. ### Proposal <!-- How are we going to solve the problem? Try to include the user journey! https://about.gitlab.com/handbook/journeys/#user-journey --> Update the [commits API](https://docs.gitlab.com/ee/api/commits.html) or the [projects API](https://docs.gitlab.com/ee/api/projects.html) to permit the user to make a request like the one described above. ### Further details <!-- Include use cases, benefits, goals, or any other details that will help us understand the problem better. --> It was previously possible to use the `X-Total` HTTP response header returned in a call like the following to solve the problem described above: ``` curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/repository/commits?since=2020-11-09T11:40:24.000-04:00&until=2020-11-10-T11:40:24.000-04" ``` This header was [excluded from the commits API](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/43159) as part of [an effort to improve Gitaly performance](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/470). I am filing this feature proposal on behalf of a customer who used to rely on `X-Total` for this functionality, noticed that it was missing after upgrading and would like some method of getting this functionality back. GitLab team members with access to ZenDesk can learn more by [viewing the ticket](https://gitlab.zendesk.com/agent/tickets/179540) that led to this feature proposal. NOTE: The `X-Total` approach worked with some limitations. Prior to the MR that removed the `X-Total` header altogether, the `X-Total` header was omitted when a project went beyond 10,000 commits. #### Alternative Approaches ##### statistics.commit_count The value in `statistics.commit_count` response from the projects API is not a viable solution because it is a sum over the lifetime of a project and does not allow for collecting a commit count during a specific date range. ##### Count commits ``` curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/repository/commits?since=2020-11-09T11:40:24.000-04:00&until=2020-11-10-T11:40:24.000-04" ``` Looping through the results of an API call like the one shown above would solve the problem but it has downsides, specifically: - This approach requires making multiple GET requests per repo in order to count commits where the previous solution only required one GET request per repository - As a result of the additional requests, the counting process is more inefficient. ##### Repository analytics Being able to query the data in the "Commits per day of month" chart in [Repository Analytics](https://docs.gitlab.com/ee/user/analytics/repository_analytics.html#charts) via the API would be a potentially useful approach. While this approach would not permit the user to specify a time alongside the date, it does get much closer to the desired functionality without requiring the user to resort to counting individual commits. ### Permissions and Security <!-- What permissions are required to perform the described actions? Are they consistent with the existing permissions as documented for users, groups, and projects as appropriate? Is the proposed behavior consistent between the UI, API, and other access methods (e.g. email replies)? Consider adding checkboxes and expectations of users with certain levels of membership https://docs.gitlab.com/ee/user/permissions.html * [ ] Add expected impact to members with no access (0) * [ ] Add expected impact to Guest (10) members * [ ] Add expected impact to Reporter (20) members * [ ] Add expected impact to Developer (30) members * [ ] Add expected impact to Maintainer (40) members * [ ] Add expected impact to Owner (50) members --> I suspect that the permissions required to perform the described actions should match who is permitted to make the [list repository commits](https://docs.gitlab.com/ee/api/commits.html#list-repository-commits) API call. ### Documentation <!-- See the Feature Change Documentation Workflow https://docs.gitlab.com/ee/development/documentation/workflow.html#for-a-product-change * Add all known Documentation Requirements in this section. See https://docs.gitlab.com/ee/development/documentation/feature-change-workflow.html#documentation-requirements * If this feature requires changing permissions, update the permissions document. See https://docs.gitlab.com/ee/user/permissions.html --> The documentation requirements will depend on how this is implemented. If this is implemented by adding to the response to the [list repository commits](https://docs.gitlab.com/ee/api/commits.html#list-repository-commits) request, that page will need to be updated. ### Availability & Testing <!-- This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier. What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing? Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance. * Unit test changes * Integration test changes * End-to-end test change See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning --> ### What does success look like, and how can we measure that? <!-- Define both the success metrics and acceptance criteria. Note that success metrics indicate the desired business outcomes, while acceptance criteria indicate when the solution is working correctly. If there is no way to measure success, link to an issue that will implement a way to measure this. Create tracking issue using the the Snowplow event tracking template. See https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Snowplow%20event%20tracking.md --> ### What is the type of buyer? <!-- What is the buyer persona for this feature? See https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/buyer-persona/ In which enterprise tier should this feature go? See https://about.gitlab.com/handbook/product/pricing/#four-tiers --> ### Is this a cross-stage feature? <!-- Communicate if this change will affect multiple Stage Groups or product areas. We recommend always start with the assumption that a feature request will have an impact into another Group. Loop in the most relevant PM and Product Designer from that Group to provide strategic support to help align the Group's broader plan and vision, as well as to avoid UX and technical debt. https://about.gitlab.com/handbook/product/#cross-stage-features --> ### Links / references Here are a few issues where we can see folks are relying on the presence of the `X-Total` header: - https://gitlab.com/gitlab-org/gitlab/-/issues/264375 - https://gitlab.com/gitlab-org/gitlab/-/issues/239165 <!-- Label reminders - you should have one of each of the following labels. Read the descriptions on https://gitlab.com/gitlab-org/gitlab/-/labels to find the correct ones --> ~feature ~customer