Semgrep error when the size of the rules is too large: Passthrough size should not exceed 1000000c bytes
Summary
When I want to customize the SAST ruleset, if the ruleset is too substantial, the SAST job fails with this error message:
Passthrough size should not exceed 1000000 bytes
Steps to reproduce
My tests were done on GitLab.com
- I want to use the Owasp Top Ten ruleset
- I updated
.gitlab/sast-ruleset.toml
with the following content:[[semgrep.passthrough]] type = "URL" value = "https://semgrep.dev/c/p/owasp-top-ten" target = "ot10.yml"
- The pipeline is triggered after the commit
- The job fails:
I did the test locally (I downloaded and committed the yaml file into the repository and used it like this:
[[semgrep.passthrough]]
type = "file"
value = ".gitlab/owasp-top-ten.yaml"
target = "ot10.yml"
The same result, the job failed.
some other rules cause the same error:
Remarks:
- Download the OWASP ruleset and split it into two files, then create 2 rules, It solves the problem, but it's painful, especially for the customer who needs something up to date.
- I tried with the SemGrep CLI:
semgrep ci --config "p/owasp-top-ten" --gitlab-sast > gl-sast-report.json
, I had no problem. - I try to change the
--max-memory
settings of SemGrep with this CI variable:SAST_SCANNER_ALLOWED_CLI_OPTS
, but the bug persists. Related issues:
Example Project
https://gitlab.com/tanuki-workshops/juice-shops/juice-shop-owasp-xp
What is the current bug behavior?
See the previous paragraph
What is the expected correct behavior?
The analyzer should support the large rulesets
Relevant logs and/or screenshots
See the previous paragraph
Output of checks
This bug happens on GitLab.com /label reproduced on GitLab.com
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true
)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
)(we will only investigate if the tests are passing)