Skip to content

Allow Cleanup CI to be run in stages

Richard Kennedy requested to merge richken/fix_cleanup_ci into master

Description

This MR allows the cleanup compose command to be run in stages. With this change, if CLEANUP_NOT_RUN_IN_STAGES is set to a non-empty value, the cleanup compose command will return to its old behavior. The default behavior has been altered, so that blobs are first uploaded until the CAS size hits the high watermark. After this, upload will stop uploading blobs and cleanup will run. The test will succeed if the cleanup command brings the CAS size down to the low watermark (Or an expected value approximately in this range).

Validation

Run the following docker compose command:

CLEANUP_DELETE_BATCH_SIZE_IN_BYTES=500 \
CLEANUP_START_HIGH_WATERMARK=10000 \
CLEANUP_STOP_LOW_WATERMARK=8000 \
CLEANUP_MAX_UPLOAD_FILE_CONTENT_SIZE=100 \
CLEANUP_TEST_FOR_N_SECONDS=100 \
CLEANUP_RUN_IN_STAGES=true \
 docker-compose -f compose-files/docker-compose-cleanup.yml up \
 --abort-on-container-exit \
  --scale cleanup=5 --scale upload-random-blobs=1

Alternatively, select RUN_IN_STAGES when using the stress testing wizard.

Edited by Richard Kennedy

Merge request reports