kics IaC scanner fails to run: open /tmp/kics.sarif: no such file or directory

👋 Read me first!

⚠ This log line can have a number of root causes.

We're tracking individual causes separately: see the Related issues section below.

First, make sure you have the latest version of the kics analyzer. The version is logged at the beginning of the job, like this:

$ /analyzer run
02:23
[INFO] [kics] [2022-04-23T07:06:06Z] ▶ GitLab kics analyzer v1.5.1

Your issue may be resolved if you update.

You can find the latest version number in the analyzer CHANGELOG. Your pipeline automatically updates to the latest version if you use the GitLab-managed CI/CD template. If you've explicitly pinned to a specific version or if you're running offline, you'll need to update.

If you're still having an issue, please run the job with the SECURE_LOG_LEVEL CI variable set to debug. This helps us take the next step to diagnose the underlying issue. See documentation for how to set this. If a related issue below matches your case, please comment on that issue. Otherwise, please open a new issue by clicking here. (This link pre-sets labels so it reaches the Static Analysis team directly.)

Original issue contents

Summary

The kics-iac-sast job fails to run:

[FATA] [kics] [2022-02-01T20:29:08Z] ▶ open /tmp/kics.sarif: no such file or directory

Steps to reproduce

  1. Have a self-hosted gitlab instance
  2. have a repository with some ansible scripts
  3. Add a new .gitlab-ci.yml with
include:
  template: Security/SAST-IaC.latest.gitlab-ci.yml
variables:
  SECURE_LOG_LEVEL: debug

Observe that the scanner fails.

What is the current bug behavior?

We see this in the job output:

Running with gitlab-runner 14.7.0 (98daeee0)
  on gitlab-runner-linux01 RfyVZW4z
Resolving secrets 00:00
Preparing the "docker" executor 00:01
Using Docker executor with image registry.gitlab.com/gitlab-org/security-products/analyzers/kics:1 ...
Pulling docker image registry.gitlab.com/gitlab-org/security-products/analyzers/kics:1 ...
Using docker image sha256:e3b44001f6b1aa0c30d6900b203d31f431048b69a7a3219481cafc3f634943a5 for registry.gitlab.com/gitlab-org/security-products/analyzers/kics:1 with digest registry.gitlab.com/gitlab-org/security-products/analyzers/kics@sha256:f5f9a05f3f2f3808564d01eabe7c1c5b1f046ba91f441c73392b5d3c9e589c62 ...
Preparing environment 00:01
Running on runner-rfyvzw4z-project-60-concurrent-0 via ip-10-250-83-52...
Getting source from Git repository 00:01
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /tmp/builds/RfyVZW4z/0/my/repo/here/.git/
Checking out fbbfa13b as feature/SOF-293-ultimate-scans...
Skipping Git submodules setup
Executing "step_script" stage of the job script 00:06
Using docker image sha256:e3b44001f6b1aa0c30d6900b203d31f431048b69a7a3219481cafc3f634943a5 for registry.gitlab.com/gitlab-org/security-products/analyzers/kics:1 with digest registry.gitlab.com/gitlab-org/security-products/analyzers/kics@sha256:f5f9a05f3f2f3808564d01eabe7c1c5b1f046ba91f441c73392b5d3c9e589c62 ...
$ /analyzer run
[INFO] [kics] [2022-02-07T17:05:13Z] ▶ GitLab kics analyzer v1.1.0
[INFO] [kics] [2022-02-07T17:05:13Z] ▶ Detecting project
[INFO] [kics] [2022-02-07T17:05:13Z] ▶ Found relevant files in project, analyzing entire repository
[INFO] [kics] [2022-02-07T17:05:13Z] ▶ Running analyzer
[DEBU] [kics] [2022-02-07T17:05:13Z] ▶ /tmp/builds/RfyVZW4z/0/my/repo/here/.gitlab/sast-ruleset.toml not found, ruleset support will be disabled.
[INFO] [kics] [2022-02-07T17:05:13Z] ▶ path /tmp/builds/RfyVZW4z/0/my/repo/here
[DEBU] [kics] [2022-02-07T17:05:22Z] ▶ Exit Status: 2
[DEBU] [kics] [2022-02-07T17:05:22Z] ▶ /usr/local/bin/kics scan --ci --path /tmp/builds/RfyVZW4z/0/my/repo/here --queries-path /usr/local/bin/assets/queries --log-level DEBUG --output-path /tmp --output-name kics --report-formats sarif
5:05PM DBG Could not find string flag ci
5:05PM DBG console.scan()
5:05PM INF Scanning with Keeping Infrastructure as Code Secure v1.4.5
5:05PM DBG storage.NewMemoryStorage()
5:05PM DBG Trying to load path (--queries-path) from /usr/local/bin/assets/queries
5:05PM INF Loading queries of type: ansible
5:05PM DBG source.NewFilesystemSource()
5:05PM DBG engine.NewInspector()
5:05PM DBG Custom library not provided. Loading embedded library instead
5:05PM DBG Custom library not provided. Loading embedded library instead
5:05PM DBG Could not open embedded library data for ansible platform
5:05PM INF Inspector initialized, number of queries=264
5:05PM INF Query execution timeout=1m0s
5:05PM DBG provider.NewFileSystemSourceProvider()
5:05PM DBG parser.NewBuilder()
5:05PM DBG resolver.Add()
5:05PM DBG resolver.Build()
panic: interface conversion: interface {} is string, not map[string]interface {}
goroutine 51 [running]:
github.com/Checkmarx/kics/pkg/parser/yaml.addExtraInfo({0xc00c90f250, 0x1, 0x1}, {0xc00c4906c0, 0x5a})
	/app/pkg/parser/yaml/parser.go:123 +0x1db
github.com/Checkmarx/kics/pkg/parser/yaml.(*Parser).Parse(0x250cee0, {0xc00c4906c0, 0x5a}, {0xc00c429300, 0xa, 0x10})
	/app/pkg/parser/yaml/parser.go:38 +0x42a
github.com/Checkmarx/kics/pkg/parser.(*Parser).Parse(0xc00c47c030, {0xc00c4906c0, 0x5a}, {0xc00c429300, 0xa, 0x10})
	/app/pkg/parser/parser.go:110 +0xb4
github.com/Checkmarx/kics/pkg/kics.(*Service).sink(0xc00c478a10, {0x254c0e0, 0xc0001be000}, {0xc00c4906c0, 0x5a}, {0x2122c17, 0x7}, {0x2511280, 0xc00c90f228})
	/app/pkg/kics/sink.go:33 +0xe5
github.com/Checkmarx/kics/pkg/kics.(*Service).PrepareSources.func1({0x254c0e0, 0xc0001be000}, {0xc00c4906c0, 0x5a}, {0x252e2c0, 0xc00c90f228})
	/app/pkg/kics/service.go:64 +0xa5
github.com/Checkmarx/kics/pkg/engine/provider.(*FileSystemSourceProvider).walkDir.func1({0xc00c4906c0, 0x5a}, {0x256ee28, 0xc00d54f1e0}, {0x0, 0x0})
	/app/pkg/engine/provider/filesystem.go:151 +0x3d9
path/filepath.walk({0xc00c4906c0, 0x5a}, {0x256ee28, 0xc00d54f1e0}, 0xc00c4b5e20)
	/usr/local/go/src/path/filepath/path.go:418 +0x125
path/filepath.walk({0xc00c460e10, 0x4b}, {0x256ee28, 0xc00d54f040}, 0xc00c4b5e20)
	/usr/local/go/src/path/filepath/path.go:442 +0x28f
path/filepath.walk({0xc0006a77c0, 0x41}, {0x256ee28, 0xc00d138750}, 0xc00c4b5e20)
	/usr/local/go/src/path/filepath/path.go:442 +0x28f
path/filepath.Walk({0xc0006a77c0, 0x41}, 0xc000850e20)
	/usr/local/go/src/path/filepath/path.go:505 +0x6c
github.com/Checkmarx/kics/pkg/engine/provider.(*FileSystemSourceProvider).walkDir(0xc0006a77c0, {0x254c0e0, 0xc0001be000}, {0xc0006a77c0, 0x7fddf83bc9a8}, 0x0, 0x20, 0x7fde2006cf18, 0x20)
	/app/pkg/engine/provider/filesystem.go:119 +0x9b
github.com/Checkmarx/kics/pkg/engine/provider.(*FileSystemSourceProvider).GetSources(0xc00cbd5960, {0x254c0e0, 0xc0001be000}, 0xc000850f98, 0xc00cbd5ae0, 0xc0000638f0)
	/app/pkg/engine/provider/filesystem.go:108 +0x1b0
github.com/Checkmarx/kics/pkg/kics.(*Service).PrepareSources(0xc00c478a10, {0x254c0e0, 0xc0001be000}, {0x2122c17, 0x7}, 0x0, 0x0)
	/app/pkg/kics/service.go:60 +0x1a7
created by github.com/Checkmarx/kics/pkg/scanner.PrepareAndScan
	/app/pkg/scanner/scanner.go:24 +0xe5
[FATA] [kics] [2022-02-07T17:05:22Z] ▶ open /tmp/kics.sarif: no such file or directory
Uploading artifacts for failed job 00:01
Uploading artifacts...
WARNING: gl-sast-report.json: no matching files    
ERROR: No files to upload                          
Cleaning up project directory and file based variables 00:01
ERROR: Job failed: exit code 1

Results of GitLab environment info

Self hosted 14.7

Expand for output related to GitLab environment info 

$ sudo gitlab-rake gitlab:env:info

System information
System:         Ubuntu 20.04
Proxy:          no
Current User:   git
Using RVM:      no
Ruby Version:   2.7.5p203
Gem Version:    3.1.4
Bundler Version:2.1.4
Rake Version:   13.0.6
Redis Version:  6.0.16
Git Version:    2.33.1.
Sidekiq Version:6.3.1
Go Version:     unknown

GitLab information
Version:        14.7.0-ee
Revision:       621e5984888
Directory:      /opt/gitlab/embedded/service/gitlab-rails
DB Adapter:     PostgreSQL
DB Version:     12.6
URL:            https://git.myurl.com
HTTP Clone URL: https://git.myurl.com/some-group/some-project.git
SSH Clone URL:  git@git.myurl.com:some-group/some-project.git
Elasticsearch:  no
Geo:            no
Using LDAP:     no
Using Omniauth: yes
Omniauth Providers: azure_oauth2

GitLab Shell
Version:        13.22.2
Repository storage paths:
- default:      /var/opt/gitlab/git-data/repositories
GitLab Shell path:              /opt/gitlab/embedded/service/gitlab-shell
Git:            /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check 
$ sudo gitlab-rake gitlab:check SANITIZE=true
Checking GitLab subtasks ...


Checking GitLab Shell ...


GitLab Shell: ... GitLab Shell version >= 13.22.2 ? ... OK (13.22.2)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful


Checking GitLab Shell ... Finished


Checking Gitaly ...


Gitaly: ... default ... OK


Checking Gitaly ... Finished


Checking Sidekiq ...


Sidekiq: ... Running? ... yes
Number of Sidekiq processes (cluster/worker) ... 1/1


Checking Sidekiq ... Finished


Checking Incoming Email ...


Incoming Email: ... Reply by email is disabled in config/gitlab.yml


Checking Incoming Email ... Finished


Checking LDAP ...


LDAP: ... LDAP is disabled in config/gitlab.yml


Checking LDAP ... Finished


Checking GitLab App ...


Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Systemd unit files or init script exist? ... skipped (omnibus-gitlab has neither init script nor systemd units)
Systemd unit files or init script up-to-date? ... skipped (omnibus-gitlab has neither init script nor systemd units)
Projects have namespace: ...
2/5 ... yes
10/6 ... yes
8/7 ... yes
8/13 ... yes
25/14 ... yes
3/15 ... yes
25/16 ... yes
3/17 ... yes
27/18 ... yes
27/19 ... yes
27/20 ... yes
27/21 ... yes
27/22 ... yes
36/24 ... yes
36/25 ... yes
35/26 ... yes
35/27 ... yes
35/28 ... yes
25/29 ... yes
3/30 ... yes
37/31 ... yes
25/32 ... yes
25/33 ... yes
35/34 ... yes
25/35 ... yes
37/38 ... yes
37/39 ... yes
37/40 ... yes
37/41 ... yes
37/42 ... yes
37/43 ... yes
37/44 ... yes
37/46 ... yes
37/48 ... yes
10/49 ... yes
10/50 ... yes
4/51 ... yes
169/52 ... yes
10/53 ... yes
25/54 ... yes
10/55 ... yes
169/57 ... yes
169/58 ... yes
169/59 ... yes
3/60 ... yes
26/61 ... yes
25/63 ... yes
71/66 ... yes
25/67 ... yes
26/68 ... yes
25/69 ... yes
25/70 ... yes
25/72 ... yes
25/73 ... yes
4/74 ... yes
25/75 ... yes
38/76 ... yes
48/77 ... yes
25/78 ... yes
25/79 ... yes
25/80 ... yes
25/81 ... yes
25/82 ... yes
25/83 ... yes
25/84 ... yes
34/86 ... yes
4/88 ... yes
27/89 ... yes
27/90 ... yes
27/91 ... yes
27/93 ... yes
81/94 ... yes
25/95 ... yes
27/96 ... yes
25/97 ... yes
79/99 ... yes
38/100 ... yes
25/101 ... yes
25/102 ... yes
25/133 ... yes
25/134 ... yes
25/135 ... yes
25/136 ... yes
48/137 ... yes
25/138 ... yes
65/139 ... yes
76/140 ... yes
26/141 ... yes
107/142 ... yes
107/143 ... yes
76/144 ... yes
27/145 ... yes
10/146 ... yes
25/147 ... yes
3/148 ... yes
76/149 ... yes
25/150 ... yes
27/151 ... yes
27/152 ... yes
64/154 ... yes
66/155 ... yes
27/156 ... yes
37/157 ... yes
25/159 ... yes
81/160 ... yes
5/161 ... yes
25/162 ... yes
35/163 ... yes
10/164 ... yes
26/165 ... yes
37/166 ... yes
26/167 ... yes
81/168 ... yes
25/169 ... yes
25/170 ... yes
9/171 ... yes
27/172 ... yes
27/173 ... yes
3/174 ... yes
25/175 ... yes
25/176 ... yes
25/177 ... yes
9/178 ... yes
3/179 ... yes
140/180 ... yes
10/181 ... yes
25/182 ... yes
10/184 ... yes
25/186 ... yes
140/187 ... yes
140/188 ... yes
140/189 ... yes
140/190 ... yes
140/191 ... yes
27/192 ... yes
10/193 ... yes
138/194 ... yes
29/196 ... yes
10/197 ... yes
25/199 ... yes
50/200 ... yes
10/201 ... yes
25/202 ... yes
10/203 ... yes
25/205 ... yes
25/207 ... yes
25/208 ... yes
25/211 ... yes
169/212 ... yes
25/214 ... yes
10/215 ... yes
10/216 ... yes
10/217 ... yes
10/218 ... yes
10/219 ... yes
10/220 ... yes
10/221 ... yes
10/224 ... yes
81/227 ... yes
27/228 ... yes
27/229 ... yes
155/230 ... yes
25/232 ... yes
25/233 ... yes
25/235 ... yes
81/236 ... yes
25/237 ... yes
158/238 ... yes
25/239 ... yes
25/240 ... yes
10/241 ... yes
25/242 ... yes
25/243 ... yes
25/244 ... yes
25/245 ... yes
37/246 ... yes
10/247 ... yes
37/248 ... yes
37/249 ... yes
25/250 ... yes
25/251 ... yes
25/252 ... yes
25/253 ... yes
25/254 ... yes
51/255 ... yes
35/256 ... yes
10/257 ... yes
25/258 ... yes
26/259 ... yes
27/262 ... yes
27/263 ... yes
10/264 ... yes
27/265 ... yes
27/266 ... yes
26/267 ... yes
25/268 ... yes
10/269 ... yes
162/271 ... yes
162/272 ... yes
164/274 ... yes
25/275 ... yes
25/276 ... yes
25/277 ... yes
19/278 ... yes
25/279 ... yes
50/281 ... yes
50/282 ... yes
10/283 ... yes
25/284 ... yes
147/286 ... yes
26/287 ... yes
26/288 ... yes
162/289 ... yes
162/290 ... yes
162/291 ... yes
138/293 ... yes
27/295 ... yes
27/296 ... yes
27/297 ... yes
181/298 ... yes
147/300 ... yes
147/301 ... yes
147/302 ... yes
164/303 ... yes
25/304 ... yes
Redis version >= 5.0.0? ... yes
Ruby version >= 2.7.2 ? ... yes (2.7.5)
Git version >= 2.33.0 ? ... yes (2.33.1)
Git user has default SSH configuration? ... yes
Active users: ... 111
Is authorized keys file accessible? ... yes
GitLab configured to store new projects in hashed storage? ... yes
All projects are in hashed storage? ... yes
Elasticsearch version 7.x (6.4 - 6.x deprecated to be removed in 13.8)? ... skipped (elasticsearch is disabled)


Checking GitLab App ... Finished


Checking GitLab subtasks ... Finished

Possible fixes

I think the bug is somewhere here: https://gitlab.com/gitlab-org/security-products/analyzers/kics/-/blob/main/analyze.go#L54-79

Edited by Connor Gilbert