KICS 4.1.4 has significantly slower scan times

Summary

A customer reported that with Kics Scanner 4.1.4, scan times are significantly worse. Scan times went from about 2 minutes to 20 minutes in one specific example. The user mentioned that these projects do have submodules and are larger projects in general. But pinning to an older version like 4.1.0 or 3.7.11 result in much faster times on the same project.

We've noted some areas of concern that seem to be taking a long time:

10:51AM DBG Starting to run query maximum_length_undefined
10:55AM DBG Finished to run query maximum_length_undefined after 4m54.051078304s

11:00AM DBG Starting to run query pattern_undefined
11:08AM DBG Finished to run query pattern_undefined after 8m18.968727555s

The latest update does not seem to point to any one particular issue, nor was I able to find any similar reports elsewhere.

[0;34m[DEBU] [kics] [2023-08-22T11:42:23Z] [/go/src/app/analyze.go:93] ▶ 11:40AM DBG console.scan()
11:40AM WRN KICS crash report disabled
11:40AM DBG console.scan()
11:40AM WRN KICS crash report disabled
11:40AM INF Scanning with Keeping Infrastructure as Code Secure v1.6.13
11:40AM INF Operating system: linux
11:40AM INF Total memory: 187.6G
11:40AM INF CPU: 40.0
11:40AM DBG storage.NewMemoryStorage()
11:40AM DBG Trying to load path (--queries-path) from /usr/local/bin/assets/queries
11:40AM INF Total files in the project: 1547
...
11:42AM INF Results saved to file /tmp/kics.sarif fileName=kics.sarif
11:42AM INF Scan duration: 115667ms
[0;34m[DEBU] [kics] [2023-08-22T11:10:17Z] [/go/src/app/analyze.go:93] ▶ 10:48AM DBG console.scan()
10:48AM WRN KICS crash report disabled
10:48AM DBG console.scan()
10:48AM WRN KICS crash report disabled
10:48AM INF Scanning with Keeping Infrastructure as Code Secure v1.7.5
10:48AM INF Operating system: linux
10:48AM INF Total memory: 187.6G
10:48AM INF CPU: 40.0
10:48AM DBG storage.NewMemoryStorage()
10:48AM DBG Trying to load path (--queries-path) from /usr/local/bin/assets/queries
10:48AM INF Total files in the project: 198
...
11:10AM INF Results saved to file /tmp/kics.sarif fileName=kics.sarif
11:10AM INF Scan duration: 1279766ms

We also ruled out that runners could be involved:

  1. Yes, both jobs were on the same runner. We do have runners on different machines but not for this project. My test for v3.7.11 was on the same runner.

The customer also tested older commits to rule out the potential of code change impacting the times:

  1. So, I’ve tracked when the Kics scanner was updated to v4.1.4. I’ve tested the commit before that was using v3.7.11 with v4.1.4 and it still runs with ~20mins.

Steps to reproduce

Unable to directly reproduce. I've tried to use Terragoat to test, but did not specifically account for submodules. In my testing on this project, scans all ran at about 1m20s regardless of the scanner version. Testing with submodules did not produce any significant difference.

Ticket Id

https://gitlab.zendesk.com/agent/tickets/442559

Configuration used

Architecture

N/A

Current behaviour

Running a IAC Kics scan on 4.1.4 results in significantly slower scan times.

Expected behaviour

IAC Kics scans on 4.1.4 should not be slow.

Versions

  • GitLab 16.2.4

  • GitLab Runner 16.2.0

  • Kics scanner v4.0.0 (Gives an error, likely unrelated)

  • Kics scanner v4.1.0 (Works, 2 minute run time)

  • Kics scanner v4.1.3 (Slow, 20 minute run time)

  • Kics scanner v4.1.4 (Slow, 20 minute run time)

Platforms

N/A

Relevant logs

See internal request for help issue

Workaround

You can pin to an older version of the scanner for now.