Huge number of duplicate webhook database entries created after setting a service template to active
Summary
Huge number of duplicate webhook database entries created after setting a service template to active. The database inserts continue to occur until GitLab is restarted. The number of entries causes database queries to begin to hang and eventually return error pages over time.
Steps to reproduce
- Go to the Service Templates section in the Admin Area
- Click on the Microsoft Teams Notification service template
- Click on the Active checkbox so that it is checked
- Uncheck all other checkboxes except Pipeline
- Enter webhook URL
- Click Save
Example Project
This was done with GitLab CE on premise.
What is the current bug behavior?
The services table in the database receives a huge number of repeat webhook entries per project.bug.txt
The file attached provides an example of what the database looks like during the duplicate entry insertion process. The insertions do not stop until the database begins to time connections out. The duplicate entries are all the exact same webhook entry corresponding to Microsoft Teams Notification.
What is the expected correct behavior?
Only a single entry is created per project. Essentially, auto-enabling the corresponding webhook on each project.
Relevant logs and/or screenshots
Output of checks
N/A
Results of GitLab environment info
Expand for output related to GitLab environment info
System information System: Current User: git Using RVM: no Ruby Version: 2.5.3p105 Gem Version: 2.7.9 Bundler Version:1.17.3 Rake Version: 12.3.2 Redis Version: 3.2.12 Git Version: 2.21.0 Sidekiq Version:5.2.7 Go Version: unknown
GitLab information Version: 11.11.3 Revision: e3eeb779d72 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: PostgreSQL DB Version: 9.6.11 URL: https://gitlab.. HTTP Clone URL: https://gitlab../some-group/some-project.git SSH Clone URL: git@gitlab.*******.*****:some-group/some-project.git Using LDAP: no Using Omniauth: yes Omniauth Providers:
GitLab Shell Version: 9.1.0 Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
Expand for output related to the GitLab application check
Checking GitLab subtasks ...
Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 9.1.0 ? ... OK (9.1.0) Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Redis available via internal API: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab App ...
Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... no Try fixing it: sudo chown -R git /var/opt/gitlab/gitlab-rails/uploads sudo find /var/opt/gitlab/gitlab-rails/uploads -type f -exec chmod 0644 {} ; sudo find /var/opt/gitlab/gitlab-rails/uploads -type d -not -path /var/opt/gitlab/gitlab-rails/uploads -exec chmod 0700 {} ; For more information see: doc/install/installation.md in section "GitLab" Please fix the error above and rerun the checks. Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 2/1 ... yes 30/2 ... yes 2/3 ... yes 2/4 ... yes 2/5 ... yes 2/6 ... yes 7/7 ... yes 2/8 ... yes 8/9 ... yes 8/10 ... yes 8/11 ... yes 8/12 ... yes 9/13 ... yes 9/14 ... yes 8/15 ... yes 10/17 ... yes 10/18 ... yes 8/19 ... yes 2/20 ... yes 11/21 ... yes 11/22 ... yes 11/23 ... yes 11/24 ... yes 11/25 ... yes 11/26 ... yes 11/27 ... yes 11/28 ... yes 11/29 ... yes 13/30 ... yes 11/31 ... yes 11/33 ... yes 11/34 ... yes 11/36 ... yes 15/37 ... yes 13/40 ... yes 13/41 ... yes 13/42 ... yes 13/43 ... yes 13/44 ... yes 13/45 ... yes 13/46 ... yes 13/47 ... yes 13/48 ... yes 13/49 ... yes 16/50 ... yes 16/51 ... yes 16/52 ... yes 17/54 ... yes 17/55 ... yes 17/56 ... yes 18/57 ... yes 18/58 ... yes 19/59 ... yes 18/60 ... yes 18/61 ... yes 18/62 ... yes 14/63 ... yes 18/64 ... yes 18/65 ... yes 17/66 ... yes 17/67 ... yes 17/68 ... yes 17/69 ... yes 14/70 ... yes 17/71 ... yes 21/72 ... yes 7/73 ... yes 14/74 ... yes 14/75 ... yes 22/76 ... yes 23/77 ... yes 23/78 ... yes 23/79 ... yes 23/80 ... yes 23/81 ... yes 17/82 ... yes 23/83 ... yes 14/85 ... yes 23/86 ... yes 25/88 ... yes 25/89 ... yes 23/90 ... yes 17/91 ... yes 26/92 ... yes 27/93 ... yes 28/94 ... yes 28/95 ... yes 29/96 ... yes 29/97 ... yes 30/98 ... yes 31/100 ... yes 71/101 ... yes 32/102 ... yes 7/103 ... yes 28/106 ... yes 23/107 ... yes 34/108 ... yes 72/109 ... yes 32/110 ... yes 22/111 ... yes 22/112 ... yes 31/113 ... yes 32/115 ... yes 71/117 ... yes 37/118 ... yes 37/119 ... yes 37/120 ... yes 37/121 ... yes 37/122 ... yes 37/123 ... yes 37/124 ... yes 37/125 ... yes 37/126 ... yes 37/127 ... yes 37/128 ... yes 37/129 ... yes 38/130 ... yes 38/131 ... yes 38/132 ... yes 38/133 ... yes 32/135 ... yes 38/136 ... yes 40/137 ... yes 41/138 ... yes 30/139 ... yes 30/140 ... yes 40/141 ... yes 38/142 ... yes 30/143 ... yes 31/144 ... yes 43/145 ... yes 43/146 ... yes 17/147 ... yes 31/148 ... yes 29/149 ... yes 17/150 ... yes 17/151 ... yes 17/152 ... yes 17/153 ... yes 17/154 ... yes 29/155 ... yes 31/156 ... yes 71/157 ... yes 35/158 ... yes 38/159 ... yes 38/160 ... yes 38/161 ... yes 35/162 ... yes 38/163 ... yes 38/164 ... yes 43/165 ... yes 74/166 ... yes 43/167 ... yes 71/168 ... yes 71/169 ... yes 32/170 ... yes 48/171 ... yes 31/172 ... yes 22/173 ... yes 43/174 ... yes 74/175 ... yes 30/176 ... yes 22/177 ... yes 49/178 ... yes 71/179 ... yes 48/180 ... yes 32/181 ... yes 51/182 ... yes 49/183 ... yes 30/184 ... yes 38/185 ... yes 53/186 ... yes 32/187 ... yes 31/188 ... yes 30/189 ... yes 38/190 ... yes 38/191 ... yes 54/192 ... yes 38/193 ... yes 38/194 ... yes 71/195 ... yes 30/196 ... yes 57/197 ... yes 56/198 ... yes 56/199 ... yes 56/200 ... yes 49/201 ... yes 30/202 ... yes 71/203 ... yes 30/204 ... yes 71/205 ... yes 71/206 ... yes 32/207 ... yes 14/208 ... yes 14/209 ... yes 59/210 ... yes 43/211 ... yes 40/212 ... yes 55/213 ... yes 48/214 ... yes 43/215 ... yes 14/216 ... yes 14/217 ... yes 49/219 ... yes 48/221 ... yes 22/222 ... yes 30/223 ... yes 35/225 ... yes 31/226 ... yes 61/227 ... yes 30/228 ... yes 22/229 ... yes 64/234 ... yes 64/238 ... yes 64/239 ... yes 64/240 ... yes 64/252 ... yes 7/253 ... yes 62/254 ... yes 30/263 ... yes 48/267 ... yes 64/269 ... yes 64/270 ... yes 64/271 ... yes 64/273 ... yes 64/274 ... yes 53/278 ... yes 22/294 ... yes 64/297 ... yes 64/299 ... yes 64/302 ... yes 64/303 ... yes 64/304 ... yes 22/305 ... yes 64/307 ... yes 7/309 ... yes 62/312 ... yes 64/316 ... yes 64/318 ... yes 62/319 ... yes 62/321 ... yes 64/322 ... yes 14/325 ... yes 64/326 ... yes 69/327 ... yes 64/328 ... yes 64/329 ... yes 14/330 ... yes 14/331 ... yes 77/332 ... yes 77/333 ... yes 62/334 ... yes 39/335 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.5.3 ? ... yes (2.5.3) Git version >= 2.21.0 ? ... yes (2.21.0) Git user has default SSH configuration? ... yes Active users: ... 16
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished
Possible fixes
(If you can, link to the line of code that might be responsible for the problem)