Missing stage error not explicit when using a CI template like SAST.gitlab-ci.yml
Summary
I am unable to use vendor-provided CI templates like SAST.gitlab-ci.yml
We use a self-hosted gitlab CE (v13.6.1). I tried to enable SAST scanning, by adding the following to my main .gitlab-ci.yml
include:
- template: Security/SAST.gitlab-ci.ymlHowever, the CI fails in early stage (i.e. at linting stage) with sast job: chosen stage does not exist; available stages are ... which seems to indicate that the CI does not find the template.
Looking at /api/v4/templates/gitlab_ci_ymls i only get the following response:
[{"key":"Android","name":"Android"},{"key":"Android-Fastlane","name":"Android-Fastlane"},{"key":"Auto-DevOps","name":"Auto-DevOps"},{"key":"Bash","name":"Bash"},{"key":"C++","name":"C++"},{"key":"Chef","name":"Chef"},{"key":"Clojure","name":"Clojure"},{"key":"Code-Quality","name":"Code-Quality"},{"key":"Composer","name":"Composer"},{"key":"Crystal","name":"Crystal"},{"key":"Dart","name":"Dart"},{"key":"Deploy-ECS","name":"Deploy-ECS"},{"key":"Django","name":"Django"},{"key":"Docker","name":"Docker"},{"key":"Elixir","name":"Elixir"},{"key":"Go","name":"Go"},{"key":"Gradle","name":"Gradle"},{"key":"Grails","name":"Grails"},{"key":"Julia","name":"Julia"},{"key":"LaTeX","name":"LaTeX"}]The server does have other templates though:
# ls -l /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/ci/templates
drwxr-xr-x 2 root root 4096 Nov 30 15:18 AWS
-rw-r--r-- 1 root root 3501 Nov 23 19:41 Android-Fastlane.gitlab-ci.yml
-rw-r--r-- 1 root root 3107 Nov 23 19:41 Android.gitlab-ci.yml
-rw-r--r-- 1 root root 7391 Nov 23 19:41 Auto-DevOps.gitlab-ci.yml
-rw-r--r-- 1 root root 806 Nov 23 19:41 Bash.gitlab-ci.yml
-rw-r--r-- 1 root root 702 Nov 23 19:41 C++.gitlab-ci.yml
-rw-r--r-- 1 root root 1229 Nov 23 19:41 Chef.gitlab-ci.yml
-rw-r--r-- 1 root root 720 Nov 23 19:41 Clojure.gitlab-ci.yml
-rw-r--r-- 1 root root 53 Nov 23 19:41 Code-Quality.gitlab-ci.yml
-rw-r--r-- 1 root root 858 Nov 23 19:41 Composer.gitlab-ci.yml
-rw-r--r-- 1 root root 1003 Nov 23 19:41 Crystal.gitlab-ci.yml
-rw-r--r-- 1 root root 554 Nov 23 19:41 Dart.gitlab-ci.yml
-rw-r--r-- 1 root root 776 Nov 23 19:41 Deploy-ECS.gitlab-ci.yml
-rw-r--r-- 1 root root 1463 Nov 23 19:41 Django.gitlab-ci.yml
-rw-r--r-- 1 root root 710 Nov 23 19:41 Docker.gitlab-ci.yml
-rw-r--r-- 1 root root 417 Nov 23 19:41 Elixir.gitlab-ci.yml
-rw-r--r-- 1 root root 1005 Nov 23 19:41 Go.gitlab-ci.yml
-rw-r--r-- 1 root root 807 Nov 23 19:41 Gradle.gitlab-ci.yml
-rw-r--r-- 1 root root 1602 Nov 23 19:41 Grails.gitlab-ci.yml
drwxr-xr-x 3 root root 4096 Nov 30 15:18 Jobs
-rw-r--r-- 1 root root 3034 Nov 23 19:41 Julia.gitlab-ci.yml
-rw-r--r-- 1 root root 289 Nov 23 19:41 LaTeX.gitlab-ci.yml
-rw-r--r-- 1 root root 2546 Nov 23 19:41 Laravel.gitlab-ci.yml
-rw-r--r-- 1 root root 538 Nov 23 19:41 Managed-Cluster-Applications.gitlab-ci.yml
-rw-r--r-- 1 root root 2680 Nov 23 19:41 Maven.gitlab-ci.yml
-rw-r--r-- 1 root root 1292 Nov 23 19:41 Mono.gitlab-ci.yml
-rw-r--r-- 1 root root 729 Nov 23 19:41 Nodejs.gitlab-ci.yml
-rw-r--r-- 1 root root 2226 Nov 23 19:41 OpenShift.gitlab-ci.yml
-rw-r--r-- 1 root root 1434 Nov 23 19:41 PHP.gitlab-ci.yml
-rw-r--r-- 1 root root 525 Nov 23 19:41 Packer.gitlab-ci.yml
drwxr-xr-x 2 root root 4096 Nov 30 15:18 Pages
-rw-r--r-- 1 root root 1223 Nov 23 19:41 Python.gitlab-ci.yml
-rw-r--r-- 1 root root 1493 Nov 23 19:41 Ruby.gitlab-ci.yml
-rw-r--r-- 1 root root 868 Nov 23 19:41 Rust.gitlab-ci.yml
-rw-r--r-- 1 root root 747 Nov 23 19:41 Scala.gitlab-ci.yml
drwxr-xr-x 2 root root 4096 Nov 30 15:18 Security
-rw-r--r-- 1 root root 720 Nov 23 19:41 Serverless.gitlab-ci.yml
-rw-r--r-- 1 root root 978 Nov 23 19:41 Swift.gitlab-ci.yml
drwxr-xr-x 2 root root 4096 Nov 30 15:18 Terraform
-rw-r--r-- 1 root root 1408 Nov 23 19:41 Terraform.gitlab-ci.yml
-rw-r--r-- 1 root root 373 Nov 23 19:41 Terraform.latest.gitlab-ci.yml
drwxr-xr-x 2 root root 4096 Nov 30 15:18 Verify
drwxr-xr-x 2 root root 4096 Nov 30 15:18 Workflows
-rw-r--r-- 1 root root 5009 Nov 23 19:41 dotNET-Core.yml
-rw-r--r-- 1 root root 3587 Nov 23 19:41 dotNET.gitlab-ci.yml
-rw-r--r-- 1 root root 1054 Nov 23 19:41 iOS-Fastlane.gitlab-ci.yml
-rw-r--r-- 1 root root 134 Nov 23 19:41 index.md
-rw-r--r-- 1 root root 2789 Nov 23 19:41 npm.gitlab-ci.ymlThis installation is quite aged (we use it for a few years now) so that could well be a migration or caching issue (?). As you can see, permissions seem identical for Android-Fastlane.gitlab-ci.yml (listed by the API) and unlisted ones (e.g. dotNET-Core.yml).
What is the current bug behavior?
The CI does not import the template.
What is the expected correct behavior?
I am expecting for the CI to be able to import the Security/SAST.gitlab-ci.yml template.
Results of GitLab environment info
debian 9 gitlab-ce 13.6.1-ce.0
gitlab-rake gitlab:env:info
root@git:~# gitlab-rake gitlab:env:infoSystem information System: Current User: git Using RVM: no Ruby Version: 2.7.2p137 Gem Version: 3.1.4 Bundler Version:2.1.4 Rake Version: 13.0.1 Redis Version: 5.0.9 Git Version: 2.29.0 Sidekiq Version:5.2.9 Go Version: unknown
GitLab information Version: 13.6.1 Revision: 1b6a590b197 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: PostgreSQL DB Version: 11.9 URL: https://git.ubicast.net HTTP Clone URL: https://git.ubicast.net/some-group/some-project.git SSH Clone URL: git@git.ubicast.net:some-group/some-project.git Using LDAP: yes Using Omniauth: yes Omniauth Providers:
GitLab Shell Version: 13.13.0 Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
gitlab-rake gitlab:env:info
root@git:~# gitlab-rake gitlab:env:infoSystem information System: Current User: git Using RVM: no Ruby Version: 2.7.2p137 Gem Version: 3.1.4 Bundler Version:2.1.4 Rake Version: 13.0.1 Redis Version: 5.0.9 Git Version: 2.29.0 Sidekiq Version:5.2.9 Go Version: unknown
GitLab information Version: 13.6.1 Revision: 1b6a590b197 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: PostgreSQL DB Version: 11.9 URL: https://git.ubicast.net HTTP Clone URL: https://git.ubicast.net/some-group/some-project.git SSH Clone URL: git@git.ubicast.net:some-group/some-project.git Using LDAP: yes Using Omniauth: yes Omniauth Providers:
GitLab Shell Version: 13.13.0 Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git root@git:~# gitlab-rake gitlab:check SANITIZE=true Checking GitLab subtasks ...
Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 13.13.0 ? ... OK (13.13.0) Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Internal API available: OK Redis available via internal API: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... Server: ldapmain not verifying SSL hostname of LDAPS server 'ldap.ubicast.net:389' LDAP authentication... Success LDAP users with access to your GitLab server (only showing the first 100 results) User output sanitized. Found 22 users of 100 limit.
Checking LDAP ... Finished
Checking GitLab App ...
Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 8/3 ... yes 8/4 ... yes 5/8 ... yes 5/9 ... yes 11/11 ... yes 7/13 ... yes 7/14 ... yes 7/15 ... yes 8/18 ... yes 8/21 ... yes 27/27 ... yes 27/28 ... yes 11/29 ... yes 11/30 ... yes 11/32 ... yes 11/34 ... yes 11/36 ... yes 11/37 ... yes 11/38 ... yes 11/39 ... yes 11/40 ... yes 11/41 ... yes 12/42 ... yes 12/43 ... yes 12/44 ... yes 12/45 ... yes 12/46 ... yes 12/47 ... yes 13/48 ... yes 13/49 ... yes 13/50 ... yes 14/52 ... yes 14/53 ... yes 2/54 ... yes 22/57 ... yes 5/58 ... yes 24/60 ... yes 24/61 ... yes 24/62 ... yes 12/63 ... yes 11/73 ... yes 11/76 ... yes 30/79 ... yes 12/80 ... yes 27/81 ... yes 11/83 ... yes 24/84 ... yes 11/85 ... yes 20/88 ... yes 8/89 ... yes 20/91 ... yes 31/92 ... yes 31/93 ... yes 12/95 ... yes 12/96 ... yes 11/98 ... yes 11/99 ... yes 11/100 ... yes 11/101 ... yes 11/102 ... yes 11/103 ... yes 11/105 ... yes 33/109 ... yes 30/111 ... yes 12/114 ... yes 31/115 ... yes 31/116 ... yes 8/117 ... yes 12/118 ... yes 2/119 ... yes 39/122 ... yes 39/124 ... yes 12/125 ... yes 12/126 ... yes 12/127 ... yes 8/128 ... yes 31/129 ... yes 31/130 ... yes 12/132 ... yes 12/133 ... yes 8/134 ... yes 36/135 ... yes 36/136 ... yes 24/137 ... yes 12/138 ... yes 36/139 ... yes 8/140 ... yes 12/141 ... yes 36/142 ... yes 39/143 ... yes 39/144 ... yes 8/145 ... yes 37/146 ... yes 37/147 ... yes 36/148 ... yes 36/149 ... yes 36/150 ... yes 36/151 ... yes 30/154 ... yes 7/156 ... yes 12/157 ... yes 11/158 ... yes 11/159 ... yes 36/161 ... yes 37/162 ... yes 36/163 ... yes 39/164 ... yes 36/165 ... yes 8/166 ... yes 8/167 ... yes 8/168 ... yes 37/169 ... yes 8/170 ... yes 53/171 ... yes 50/172 ... yes 50/173 ... yes 50/174 ... yes 36/175 ... yes 51/176 ... yes 50/177 ... yes 51/178 ... yes 51/179 ... yes 51/180 ... yes 8/181 ... yes 51/182 ... yes 39/183 ... yes 36/184 ... yes 5/185 ... yes 7/186 ... yes 50/187 ... yes 53/190 ... yes 5/191 ... yes 53/192 ... yes 53/193 ... yes 50/194 ... yes 51/195 ... yes 37/196 ... yes 37/197 ... yes 36/198 ... yes 50/199 ... yes 36/200 ... yes 5/201 ... yes 37/203 ... yes 8/204 ... yes 37/205 ... yes 27/206 ... yes 58/207 ... yes 12/208 ... yes 12/209 ... yes 36/210 ... yes 7/211 ... yes 51/212 ... yes 39/213 ... yes 8/214 ... yes 7/215 ... yes 8/216 ... yes 8/217 ... yes 8/218 ... yes 68/219 ... yes 61/224 ... yes 61/225 ... yes 51/226 ... yes 12/227 ... yes 58/228 ... yes 58/230 ... yes 8/231 ... yes 58/232 ... yes 58/233 ... yes 58/234 ... yes 58/235 ... yes 62/236 ... yes 58/238 ... yes Redis version >= 4.0.0? ... yes Ruby version >= 2.5.3 ? ... yes (2.7.2) Git version >= 2.29.0 ? ... yes (2.29.0) Git user has default SSH configuration? ... yes Active users: ... 19 Is authorized keys file accessible? ... yes GitLab configured to store new projects in hashed storage? ... yes All projects are in hashed storage? ... yes
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished