Draft: Add local benchmarking
What does this MR do?
A typical use case of benchmarking is to run the benchmark suite locally for short cycle benchmarking and quick debugging. This MR adds a script to achieve this.
More specifically, the script is basically the same as the one in CI, except for some environment variables and gramma adjustment for zsh
. It traverses the files in a specified folder and use curl
command to trigger the SAST scanning in a specified url.
How to use?
-
Run
sh local_benchmarking.sh
-
Configuration: is minimal; configurable items include:
-
HOST
: the host/url of your image under test -
CURL_COMMAND
: the (curl) command you want to run to trigger the SAST service -
CURL_DIR
: the dir of the programs to be scanned -
MAX_CURL_TIMES
: the number of the programs to be scanned
-
-
Output:
-
Can be found in the console after running
sh local_benchmarking.sh
, like this:========Overall============== Average time taken per file: 1.70 seconds Average size of files: 20320.50 bytes
-
The bug reports of SAST is summarised in
result.html
- To see, just run
open result.html
- To see, just run
-
Related issues/MRs
gitlab-org/secure/sast-ide-integration!5
https://gitlab.com/gitlab-org/secure/static-analysis/sast-ide-benchmark/-/issues/3
Edited by Hua Yan